Add Ruby language analyzer and related functionality

- Introduced global usings for Ruby analyzer. - Implemented RubyLockData, RubyLockEntry, and RubyLockParser for handling Gemfile.lock files. - Created RubyPackage and RubyPackageCollector to manage Ruby packages and vendor cache. - Developed RubyAnalyzerPlugin and RubyLanguageAnalyzer for analyzing Ruby projects. - Added tests for Ruby language analyzer with sample Gemfile.lock and expected output. - Included necessary project files and references for the Ruby analyzer. - Added third-party licenses for tree-sitter dependencies.
2025-11-03 01:15:43 +02:00
parent ff0eca3a51
commit bf2bf4b395
88 changed files with 6557 additions and 1568 deletions
--- a/docs/modules/issuer-directory/architecture.md
+++ b/docs/modules/issuer-directory/architecture.md
@@ -90,6 +90,11 @@ Payloads follow the contract in `Contracts/IssuerDtos.cs` and align with domain
 3. **SDK integration (ISSUER-30-004)** — supply cached issuer metadata to VEX Lens and Excititor clients.
 4. **Observability & Ops (ISSUER-30-005/006)** — metrics, dashboards, deployment automation, offline kit.

+## 9. Operations & runbooks
+- [Deployment guide](operations/deployment.md)
+- [Backup & restore](operations/backup-restore.md)
+- [Offline kit notes](operations/offline-kit.md)
+
 ---

 *Document owner: Issuer Directory Guild*
--- a/docs/modules/issuer-directory/operations/backup-restore.md
+++ b/docs/modules/issuer-directory/operations/backup-restore.md
@@ -0,0 +1,103 @@
+# Issuer Directory Backup & Restore
+
+## Scope
+- **Applies to:** Issuer Directory when deployed via Docker Compose (`deploy/compose/docker-compose.*.yaml`) or the Helm chart (`deploy/helm/stellaops`).
+- **Artifacts covered:** MongoDB database `issuer-directory`, service configuration (`etc/issuer-directory.yaml`), CSAF seed file (`data/csaf-publishers.json`), and secret material for the Mongo connection string.
+- **Frequency:** Take a hot backup before every upgrade and at least daily in production. Keep encrypted copies off-site/air-gapped according to your compliance program.
+
+## Inventory checklist
+| Component | Location (Compose default) | Notes |
+| --- | --- | --- |
+| Mongo data | `mongo-data` volume (`/var/lib/docker/volumes/.../mongo-data`) | Contains `issuers`, `issuer_keys`, `issuer_trust_overrides`, and `issuer_audit` collections. |
+| Configuration | `etc/issuer-directory.yaml` | Mounted read-only at `/etc/issuer-directory.yaml` inside the container. |
+| CSAF seed file | `src/IssuerDirectory/StellaOps.IssuerDirectory/data/csaf-publishers.json` | Ensure customised seeds are part of the backup; regenerate if you ship regional overrides. |
+| Mongo secret | `.env` entry `ISSUER_DIRECTORY_MONGO_CONNECTION_STRING` or secret store export | Required to restore connectivity; treat as sensitive. |
+
+> **Tip:** Export the secret via `kubectl get secret issuer-directory-secrets -o yaml` (sanitize before storage) or copy the Compose `.env` file into an encrypted vault.
+
+## Hot backup (no downtime)
+1. **Create output directory**
+   ```bash
+   BACKUP_DIR=backup/issuer-directory/$(date +%Y-%m-%dT%H%M%S)
+   mkdir -p "$BACKUP_DIR"
+   ```
+2. **Dump Mongo collections**
+   ```bash
+   docker compose -f deploy/compose/docker-compose.prod.yaml exec mongo \
+     mongodump --archive=/dump/issuer-directory-$(date +%Y%m%dT%H%M%SZ).gz \
+     --gzip --db issuer-directory
+
+   docker compose -f deploy/compose/docker-compose.prod.yaml cp \
+     mongo:/dump/issuer-directory-$(date +%Y%m%dT%H%M%SZ).gz "$BACKUP_DIR/"
+   ```
+   For Kubernetes, run the same `mongodump` command inside the `stellaops-mongo` pod and copy the archive via `kubectl cp`.
+3. **Capture configuration and seeds**
+   ```bash
+   cp etc/issuer-directory.yaml "$BACKUP_DIR/"
+   cp src/IssuerDirectory/StellaOps.IssuerDirectory/data/csaf-publishers.json "$BACKUP_DIR/"
+   ```
+4. **Capture secrets**
+   ```bash
+   grep '^ISSUER_DIRECTORY_MONGO_CONNECTION_STRING=' dev.env > "$BACKUP_DIR/issuer-directory.mongo.secret"
+   chmod 600 "$BACKUP_DIR/issuer-directory.mongo.secret"
+   ```
+5. **Generate checksums and encrypt**
+   ```bash
+   (cd "$BACKUP_DIR" && sha256sum * > SHA256SUMS)
+   tar czf "$BACKUP_DIR.tar.gz" -C "$BACKUP_DIR" .
+   age -r you@example.org "$BACKUP_DIR.tar.gz" > "$BACKUP_DIR.tar.gz.age"
+   ```
+
+## Cold backup (planned downtime)
+1. Notify stakeholders and pause automation calling the API.
+2. Stop services:
+   ```bash
+   docker compose -f deploy/compose/docker-compose.prod.yaml down issuer-directory
+   ```
+   (For Helm: `kubectl scale deploy stellaops-issuer-directory --replicas=0`.)
+3. Snapshot volumes:
+   ```bash
+   docker run --rm -v mongo-data:/data \
+     -v "$(pwd)":/backup busybox tar czf /backup/mongo-data-$(date +%Y%m%d).tar.gz -C /data .
+   ```
+4. Copy configuration, seeds, and secrets as in the hot backup.
+5. Restart services and confirm `/health/live` returns `200 OK`.
+
+## Restore procedure
+1. **Provision clean volumes**
+   - Compose: `docker volume rm mongo-data` (optional) then `docker compose up -d mongo`.
+   - Helm: delete the Mongo PVC or attach a fresh volume snapshot.
+2. **Restore Mongo**
+   ```bash
+   docker compose exec -T mongo \
+     mongorestore --archive \
+     --gzip --drop < issuer-directory-YYYYMMDDTHHMMSSZ.gz
+   ```
+3. **Restore configuration/secrets**
+   - Copy `issuer-directory.yaml` into `etc/`.
+   - Reapply the secret: `kubectl apply -f issuer-directory-secret.yaml` or repopulate `.env`.
+4. **Restore CSAF seeds** (optional)
+   - If you maintain a customised seed file, copy it back before starting the container. Otherwise the bundled file will be used.
+5. **Start services**
+   ```bash
+   docker compose up -d issuer-directory
+   # or
+   kubectl scale deploy stellaops-issuer-directory --replicas=1
+   ```
+6. **Validate**
+   - `curl -fsSL https://localhost:8447/health/live`
+   - Issue an access token and list issuers to confirm results.
+   - Check Mongo counts match expectations (`db.issuers.countDocuments()`, etc.).
+
+## Disaster recovery notes
+- **Retention:** Maintain 30 daily + 12 monthly archives. Store copies in geographically separate, access-controlled vaults.
+- **Audit reconciliation:** Ensure `issuer_audit` entries cover the restore window; export them for compliance.
+- **Seed replay:** If the CSAF seed file was lost, set `ISSUER_DIRECTORY_SEED_CSAF=true` for the first restart to rehydrate the global tenant.
+- **Testing:** Run quarterly restore drills in a staging environment to validate procedure drift.
+
+## Verification checklist
+- [ ] `/health/live` returns `200 OK`.
+- [ ] Mongo collections (`issuers`, `issuer_keys`, `issuer_trust_overrides`) have expected counts.
+- [ ] `issuer_directory_changes_total` and `issuer_directory_key_operations_total` metrics resume within 1 minute.
+- [ ] Audit entries exist for post-restore CRUD activity.
+- [ ] Client integrations (VEX Lens, Excititor) resolve issuers successfully.
--- a/docs/modules/issuer-directory/operations/deployment.md
+++ b/docs/modules/issuer-directory/operations/deployment.md
@@ -0,0 +1,100 @@
+# Issuer Directory Deployment Guide
+
+## Scope
+- **Applies to:** Issuer Directory WebService (`stellaops/issuer-directory-web`) running via the provided Docker Compose bundles (`deploy/compose/docker-compose.*.yaml`) or the Helm chart (`deploy/helm/stellaops`).
+- **Covers:** Environment prerequisites, secret handling, Compose + Helm rollout steps, and post-deploy verification.
+- **Audience:** Platform/DevOps engineers responsible for Identity & Signing sprint deliverables.
+
+## 1 · Prerequisites
+- Authority must be running and reachable at the issuer URL you configure (default Compose host: `https://authority:8440`).
+- MongoDB 4.2+ with credentials for the `issuer-directory` database (Compose defaults to the root user defined in `.env`).
+- Network access to Authority, MongoDB, and (optionally) Prometheus if you scrape metrics.
+- Issuer Directory configuration file `etc/issuer-directory.yaml` checked and customised for your environment (tenant header, audiences, telemetry level, CSAF seed path).
+
+> **Secrets:** Use `etc/secrets/issuer-directory.mongo.secret.example` as a template. Store the real connection string in an untracked file or secrets manager and reference it via environment variables (`ISSUER_DIRECTORY_MONGO_CONNECTION_STRING`) rather than committing credentials.
+
+## 2 · Deploy with Docker Compose
+1. **Prepare environment variables**
+   ```bash
+   cp deploy/compose/env/dev.env.example dev.env
+   cp etc/secrets/issuer-directory.mongo.secret.example issuer-directory.mongo.env
+   # Edit dev.env and issuer-directory.mongo.env with production-ready secrets.
+   ```
+
+2. **Inspect the merged configuration**
+   ```bash
+   docker compose \
+     --env-file dev.env \
+     --env-file issuer-directory.mongo.env \
+     -f deploy/compose/docker-compose.dev.yaml config
+   ```
+   The command confirms the new `issuer-directory` service resolves the port (`${ISSUER_DIRECTORY_PORT:-8447}`) and the Mongo connection string is in place.
+
+3. **Launch the stack**
+   ```bash
+   docker compose \
+     --env-file dev.env \
+     --env-file issuer-directory.mongo.env \
+     -f deploy/compose/docker-compose.dev.yaml up -d issuer-directory
+   ```
+   Compose automatically mounts `../../etc/issuer-directory.yaml` into the container at `/etc/issuer-directory.yaml`, seeds CSAF publishers, and exposes the API on `https://localhost:8447`.
+
+4. **Smoke test**
+   ```bash
+   curl -k https://localhost:8447/health/live
+   stellaops-cli issuer-directory issuers list \
+     --base-url https://localhost:8447 \
+     --tenant demo \
+     --access-token "$(stellaops-cli auth token issue --scope issuer-directory:read)"
+   ```
+
+5. **Upgrade & rollback**
+   - Update Compose images to the desired release manifest (`deploy/releases/*.yaml`), re-run `docker compose config`, then `docker compose up -d`.
+   - Rollbacks follow the same steps with the previous manifest. Mongo collections are backwards compatible within `2025.10.x`.
+
+## 3 · Deploy with Helm
+1. **Create or update the secret**
+   ```bash
+   kubectl create secret generic issuer-directory-secrets \
+     --from-literal=ISSUERDIRECTORY__MONGO__CONNECTIONSTRING='mongodb://stellaops:<password>@stellaops-mongo:27017' \
+     --dry-run=client -o yaml | kubectl apply -f -
+   ```
+   Add optional overrides (e.g. `ISSUERDIRECTORY__AUTHORITY__ISSUER`) if your Authority issuer differs from the default.
+
+2. **Template for validation**
+   ```bash
+   helm template issuer-directory deploy/helm/stellaops \
+     -f deploy/helm/stellaops/values-prod.yaml \
+     --set services.issuer-directory.env.ISSUERDIRECTORY__AUTHORITY__ISSUER=https://authority.prod.stella-ops.org \
+     > /tmp/issuer-directory.yaml
+   ```
+
+3. **Install / upgrade**
+   ```bash
+   helm upgrade --install stellaops deploy/helm/stellaops \
+     -f deploy/helm/stellaops/values-prod.yaml \
+     --set services.issuer-directory.env.ISSUERDIRECTORY__AUTHORITY__ISSUER=https://authority.prod.stella-ops.org
+   ```
+   The chart provisions:
+   - ConfigMap `stellaops-issuer-directory-config` with `IssuerDirectory` settings.
+   - Deployment `stellaops-issuer-directory` with readiness/liveness probes on `/health/live`.
+   - Service on port `8080` (ClusterIP by default).
+
+4. **Expose for operators (optional)**
+   - Use an Ingress/HTTPRoute to publish `https://issuer-directory.<env>.stella-ops.org`.
+   - Ensure the upstream includes DPoP headers if proxied through an API gateway.
+
+5. **Post-deploy validation**
+   ```bash
+   kubectl exec deploy/stellaops-issuer-directory -- \
+     curl -sf http://127.0.0.1:8080/health/live
+   kubectl logs deploy/stellaops-issuer-directory | grep 'IssuerDirectory Mongo connected'
+   ```
+   Prometheus should begin scraping `issuer_directory_changes_total` and related metrics (labels: `tenant`, `issuer`, `action`).
+
+## 4 · Operational checklist
+- **Secrets:** Connection strings live in `issuer-directory-secrets` (Helm) or an `.env` file stored in your secrets vault (Compose). Rotate credentials via secret update + pod restart.
+- **Audit streams:** Confirm `issuer_directory_audit` collection receives entries when CRUD operations run; export logs for compliance.
+- **Tenants:** The service enforces the `X-StellaOps-Tenant` header. For multi-tenant staging, configure the reverse proxy to inject the correct tenant or issue scoped tokens.
+- **CSAF seeds:** `ISSUER_DIRECTORY_SEED_CSAF=true` replays `data/csaf-publishers.json` on startup. Set to `false` once production tenants are fully managed, or override `csafSeedPath` with a curated bundle.
+- **Release alignment:** Before promotion, run `deploy/tools/validate-profiles.sh` to lint Compose/Helm bundles, then verify the new `issuer-directory-web` entry in `deploy/releases/2025.10-edge.yaml` (or the relevant manifest) matches the channel you intend to ship.
--- a/docs/modules/issuer-directory/operations/offline-kit.md
+++ b/docs/modules/issuer-directory/operations/offline-kit.md
@@ -0,0 +1,71 @@
+# Issuer Directory Offline Kit Notes
+
+## Purpose
+Operators bundling Stella Ops for fully disconnected environments must include the Issuer Directory service so VEX Lens, Excititor, and Policy Engine can resolve trusted issuers without reaching external registries.
+
+## 1 · Bundle contents
+Include the following artefacts in your Offline Update Kit staging tree:
+
+| Path (within kit) | Source | Notes |
+| --- | --- | --- |
+| `images/issuer-directory-web.tar` | `registry.stella-ops.org/stellaops/issuer-directory-web` (digest from `deploy/releases/<channel>.yaml`) | Export with `crane pull --format=tar` or `skopeo copy docker://... oci:...`. |
+| `config/issuer-directory/issuer-directory.yaml` | `etc/issuer-directory.yaml` (customised) | Replace Authority issuer, tenant header, and log level as required. |
+| `config/issuer-directory/csaf-publishers.json` | `src/IssuerDirectory/StellaOps.IssuerDirectory/data/csaf-publishers.json` or regional override | Operators can edit before import to add private publishers. |
+| `secrets/issuer-directory/connection.env` | Secure secret store export (`ISSUER_DIRECTORY_MONGO_CONNECTION_STRING=`) | Encrypt at rest; Offline Kit importer places it in the Compose/Helm secret. |
+| `docs/issuer-directory/deployment.md` | `docs/modules/issuer-directory/operations/deployment.md` | Ship alongside kit documentation for operators. |
+
+> **Image digests:** Update `deploy/releases/2025.10-edge.yaml` (or the relevant manifest) with the exact digest before building the kit so `offline-manifest.json` can assert integrity.
+
+## 2 · Compose (air-gapped) deployment
+1. Load images locally on the target:
+   ```bash
+   docker load < images/issuer-directory-web.tar
+   ```
+2. Copy Compose artefacts:
+   ```bash
+   cp deploy/compose/docker-compose.airgap.yaml .
+   cp deploy/compose/env/airgap.env.example airgap.env
+   cp secrets/issuer-directory/connection.env issuer-directory.mongo.env
+   ```
+3. Update `airgap.env` with site-specific values (Authority issuer, tenant, ports) and remove outbound endpoints.
+4. Bring up the service:
+   ```bash
+   docker compose \
+     --env-file airgap.env \
+     --env-file issuer-directory.mongo.env \
+     -f docker-compose.airgap.yaml up -d issuer-directory
+   ```
+5. Verify via `curl -k https://issuer-directory.airgap.local:8447/health/live`.
+
+## 3 · Kubernetes (air-gapped) deployment
+1. Pre-load the OCI image into your local registry mirror and update `values-airgap.yaml` to reference it.
+2. Apply the secret bundled in the kit:
+   ```bash
+   kubectl apply -f secrets/issuer-directory/connection-secret.yaml
+   ```
+   (Generate this file during packaging with `kubectl create secret generic issuer-directory-secrets ... --dry-run=client -o yaml`.)
+3. Install/upgrade the chart:
+   ```bash
+   helm upgrade --install stellaops deploy/helm/stellaops \
+     -f deploy/helm/stellaops/values-airgap.yaml \
+     --set services.issuer-directory.env.ISSUERDIRECTORY__AUTHORITY__ISSUER=https://authority.airgap.local/realms/stellaops
+   ```
+4. Confirm `issuer_directory_changes_total` is visible in your offline Prometheus stack.
+
+## 4 · Import workflow summary
+1. Run `ops/offline-kit/build_offline_kit.py` with the additional artefacts noted above.
+2. Sign the resulting tarball and manifest (Cosign) and record the SHA-256 in the release notes.
+3. At the destination:
+   ```bash
+   stellaops-cli offline kit import \
+     --bundle stella-ops-offline-kit-<version>-airgap.tar.gz \
+     --destination /opt/stellaops/offline-kit
+   ```
+4. Follow the Compose or Helm path depending on your topology.
+
+## 5 · Post-import validation
+- [ ] `docker images | grep issuer-directory` (Compose) or `kubectl get deploy stellaops-issuer-directory` (Helm) shows the expected version.
+- [ ] `csaf-publishers.json` in the container matches the offline bundle (hash check).
+- [ ] `/issuer-directory/issuers` returns global seed issuers (requires token with `issuer-directory:read` scope).
+- [ ] Audit collection receives entries when you create/update issuers offline.
+- [ ] Offline kit manifest (`offline-manifest.json`) lists `images/issuer-directory-web.tar` and `config/issuer-directory/issuer-directory.yaml` with SHA-256 values you recorded during packaging.