Implement ledger metrics for observability and add tests for Ruby packages endpoints
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Added `LedgerMetrics` class to record write latency and total events for ledger operations.
- Created comprehensive tests for Ruby packages endpoints, covering scenarios for missing inventory, successful retrieval, and identifier handling.
- Introduced `TestSurfaceSecretsScope` for managing environment variables during tests.
- Developed `ProvenanceMongoExtensions` for attaching DSSE provenance and trust information to event documents.
- Implemented `EventProvenanceWriter` and `EventWriter` classes for managing event provenance in MongoDB.
- Established MongoDB indexes for efficient querying of events based on provenance and trust.
- Added models and JSON parsing logic for DSSE provenance and trust information.
This commit is contained in:
master
2025-11-13 09:29:09 +02:00
parent 151f6b35cc
commit 61f963fd52
101 changed files with 5881 additions and 1776 deletions

View File

@@ -46,6 +46,17 @@
- Export Center profile with `attestations.bundle=true`.
- Rekor log snapshots mirrored (ORAS bundle or rsync of `/var/log/rekor`) for disconnected verification.
### 3.1 Configuration & env-var map
| Service | Key(s) | Env override | Notes |
|---------|--------|--------------|-------|
| Scanner WebService / Worker | `scanner.attestation.signerEndpoint`<br>`scanner.attestation.attestorEndpoint`<br>`scanner.attestation.requireDsse`<br>`scanner.attestation.uploadArtifacts` | `SCANNER__ATTESTATION__SIGNERENDPOINT`<br>`SCANNER__ATTESTATION__ATTESTORENDPOINT`<br>`SCANNER__ATTESTATION__REQUIREDSSE`<br>`SCANNER__ATTESTATION__UPLOADARTIFACTS` | Worker/WebService share the same config. Set `requireDsse=false` while observing, then flip to `true`. `uploadArtifacts=true` stores DSSE+Rekor bundles next to SBOM artefacts. |
| Signer | `signer.attestorEndpoint`<br>`signer.keyProvider`<br>`signer.fulcio.endpoint` | `SIGNER__ATTESTORENDPOINT` etc. | `attestorEndpoint` lets Signer push DSSE payloads downstream; key provider controls BYO KMS/HSM vs Fulcio. |
| Attestor | `attestor.rekor.api`<br>`attestor.rekor.publicKeyPath`<br>`attestor.rekor.offlineMirrorPath`<br>`attestor.retry.maxAttempts` | `ATTESTOR__REKOR__API`<br>`ATTESTOR__REKOR__PUBLICKEYPATH`<br>`ATTESTOR__REKOR__OFFLINEMIRRORPATH`<br>`ATTESTOR__RETRY__MAXATTEMPTS` | Mirror path points at the local snapshot directory used in sealed/air-gapped modes. |
| Export Center | `exportProfiles.<name>.includeAttestations`<br>`exportProfiles.<name>.includeRekorProofs` | `EXPORTCENTER__EXPORTPROFILES__SECURE-DEFAULT__INCLUDEATTESTATIONS` etc. | Use profiles to gate which bundles include DSSE/Reco r data; keep a “secure-default” profile enabled across tiers. |
> **Tip:** Every key above follows the ASP.NET Core double-underscore pattern. For Compose/Helm, add environment variables directly; for Offline Kit overrides, drop `appsettings.Offline.json` with the same sections.
---
## 4. Enablement workflow
@@ -161,6 +172,38 @@ Roll forward per environment; keep the previous phases toggles for hot rollba
---
## 8. Operational runbook & SLO guardrails
| Step | Owner | Target / Notes |
|------|-------|----------------|
| Health gate | Ops/SRE | `attestor_rekor_success_total` ≥ 99.5% rolling hour, `rekor_inclusion_latency_p95` ≤ 30s. Alert when retries spike or queue depth > 50. |
| Cutover dry-run | Scanner team | Set `SCANNER__ATTESTATION__REQUIREDSSE=false`, watch metrics + Attestor queue for 24h, capture Rekor proofs per environment. |
| Enforce | Platform | Flip `requireDsse=true`, promote Policy rule from `warn` → `deny`, notify AppSec + release managers. |
| Audit proof pack | Export Center | Run secure profile nightly; confirm `attestations/` + `rekor/` trees attached to Offline Kit. Store bundle hash in Evidence Locker. |
| Verification spot-check | AppSec | Weekly `stellaops-cli attest verify --bundle latest.tar --rekor-key rekor.pub --json` saved to ticket for auditors. |
| Rollback | Ops/SRE | If Rekor outage exceeds 15 min: set `requireDsse=false`, keep policy in `warn`, purge Attestor queue once log recovers, then re-enable. Document the waiver in the sprint log. |
**Dashboards & alerts**
- Grafana panel: Rekor inclusion latency (p50/p95) + Attestor retry rate.
- Alert when `attestationPending=true` events exceed 5 per minute for >5 minutes.
- Logs must include `rekorUuid`, `rekorLogIndex`, `attestationDigest` for SIEM correlation.
**Runbook snippets**
```bash
# test Rekor health + key mismatch
rekor-cli loginfo --rekor_server "${ATTESTOR__REKOR__API}" --format json | jq .rootHash
# replay stranded payloads after outage
stellaops-attestor replay --since "2025-11-13T00:00:00Z" \
--rekor ${ATTESTOR__REKOR__API} --rekor-key /etc/rekor/rekor.pub
# verify a single DSSE file against Rekor proof bundle
stellaops-cli attest verify --envelope artifacts/scan123/attest/sbom.dsse.json \
--rekor-proof artifacts/scan123/rekor/entry.json --rekor-key rekor.pub
```
---
## References
- Gap analysis: `docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md#dsse-rekor-operator-enablement-trivy-grype-snyk`
@@ -168,4 +211,3 @@ Roll forward per environment; keep the previous phases toggles for hot rollba
- Export Center profiles: `docs/modules/export-center/architecture.md`
- Policy Engine predicates: `docs/modules/policy/architecture.md`
- CLI reference: `docs/09_API_CLI_REFERENCE.md`