Implement ledger metrics for observability and add tests for Ruby packages endpoints
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Added `LedgerMetrics` class to record write latency and total events for ledger operations.
- Created comprehensive tests for Ruby packages endpoints, covering scenarios for missing inventory, successful retrieval, and identifier handling.
- Introduced `TestSurfaceSecretsScope` for managing environment variables during tests.
- Developed `ProvenanceMongoExtensions` for attaching DSSE provenance and trust information to event documents.
- Implemented `EventProvenanceWriter` and `EventWriter` classes for managing event provenance in MongoDB.
- Established MongoDB indexes for efficient querying of events based on provenance and trust.
- Added models and JSON parsing logic for DSSE provenance and trust information.
This commit is contained in:
master
2025-11-13 09:29:09 +02:00
parent 151f6b35cc
commit 61f963fd52
101 changed files with 5881 additions and 1776 deletions

View File

@@ -7,17 +7,17 @@
| Task ID | State | Summary | Owner / Source | Depends On |
| --- | --- | --- | --- | --- |
| `SCANNER-ENG-0008` | TODO | Maintain EntryTrace heuristic cadence per `docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md`, including quarterly pattern reviews + explain-trace updates. | EntryTrace Guild, QA Guild (src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace) | — |
| `SCANNER-ENG-0009` | DOING (2025-11-02) | Deliver Ruby analyzer parity and observation pipeline (lockfiles, runtime graph, policy signals) per the gap doc. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ANALYZERS-RUBY-28-001..012 |
| `SCANNER-ENG-0009` | DONE (2025-11-13) | Ruby analyzer parity shipped: runtime graph + capability signals, observation payload, Mongo-backed `ruby.packages` inventory, CLI/WebService surfaces, and plugin manifest bundles for Worker loadout. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ANALYZERS-RUBY-28-001..012 |
| `SCANNER-ENG-0010` | TODO | Ship the PHP analyzer pipeline (composer lock, autoload graph, capability signals) to close comparison gaps. | PHP Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Php) | SCANNER-ANALYZERS-PHP-27-001..012 |
| `SCANNER-ENG-0011` | TODO | Scope the Deno runtime analyzer (lockfile resolver, import graphs) based on competitor techniques to extend beyond Sprint 130 coverage. | Language Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | — |
| `SCANNER-ENG-0012` | TODO | Evaluate Dart analyzer requirements (pubspec parsing, AOT artifacts) and split implementation tasks. | Language Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Dart) | — |
| `SCANNER-ENG-0013` | TODO | Plan Swift Package Manager coverage (Package.resolved, xcframeworks, runtime hints) with policy hooks. | Swift Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Swift) | — |
| `SCANNER-ENG-0014` | TODO | Align Kubernetes/VM target coverage between Scanner and Zastava per comparison findings; publish joint roadmap. | Runtime Guild, Zastava Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0015` | DOING (2025-11-09) | Document DSSE/Rekor operator enablement guidance and rollout levers surfaced in the gap analysis. | Export Center Guild, Scanner Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0015` | DONE (2025-11-13) | DSSE/Rekor operator playbook published (`docs/modules/scanner/operations/dsse-rekor-operator-guide.md`) with config/env tables, rollout phases, runbook snippets, offline verification steps, and SLA/alert guidance. | Export Center Guild, Scanner Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0016` | DONE (2025-11-10) | RubyLockCollector and vendor ingestion finalized: Bundler config overrides honoured, workspace lockfiles merged, vendor bundles normalised, and deterministic fixtures added. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0009 |
| `SCANNER-ENG-0017` | DONE (2025-11-09) | Build the runtime require/autoload graph builder with tree-sitter Ruby per design §4.4 and integrate EntryTrace hints. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0016 |
| `SCANNER-ENG-0018` | DONE (2025-11-09) | Emit Ruby capability + framework surface signals as defined in design §4.5 with policy predicate hooks. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0017 |
| `SCANNER-ENG-0019` | DOING (2025-11-10) | Ship Ruby CLI verbs (`stella ruby inspect|resolve`) and Offline Kit packaging per design §4.6. | Ruby Analyzer Guild, CLI Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0016..0018 |
| `SCANNER-ENG-0019` | DONE (2025-11-13) | Ruby CLI verbs now resolve inventories by scan ID, digest, or image reference; Scanner.WebService fallbacks + CLI client encoding ensure `--image` works for both digests and tagged references, and tests cover the new lookup flow. | Ruby Analyzer Guild, CLI Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0016..0018 |
| `SCANNER-LIC-0001` | DONE (2025-11-10) | Tree-sitter licensing captured, `NOTICE.md` updated, and Offline Kit now mirrors `third-party-licenses/` with ruby artifacts. | Scanner Guild, Legal Guild (docs/modules/scanner) | SCANNER-ENG-0016 |
| `SCANNER-POLICY-0001` | DONE (2025-11-10) | Ruby predicates shipped: Policy Engine exposes `sbom.any_component` + `ruby.*`, tests updated, DSL/offline-kit docs refreshed. | Policy Guild, Ruby Analyzer Guild (docs/modules/scanner) | SCANNER-ENG-0018 |
| `SCANNER-CLI-0001` | DONE (2025-11-10) | Coordinate CLI UX/help text for new Ruby verbs and update CLI docs/golden outputs. | CLI Guild, Ruby Analyzer Guild (src/Cli/StellaOps.Cli) | SCANNER-ENG-0019 |
@@ -32,3 +32,14 @@
- `SCANNER-ENG-0009`: 2025-11-12 — Added bundler-version metadata to observation payloads, introduced the `complex-app` fixture to cover vendor caches/BUNDLE_PATH overrides, and taught `stellaops-cli ruby inspect` to print the observation banner (bundler/runtime/capabilities) alongside JSON `observation` blocks.
- `SCANNER-ENG-0009`: 2025-11-12 — Ruby package inventories now flow into `RubyPackageInventoryStore`; `SurfaceManifestStageExecutor` builds the package list, persists it via Mongo, and Scanner.WebService exposes the data through `GET /api/scans/{scanId}/ruby-packages` for CLI/Policy consumers.
- `SCANNER-ENG-0009`: 2025-11-12 — Ruby package inventory API now returns a typed envelope (scanId/imageDigest/generatedAt + packages) backed by `ruby.packages`; Worker/WebService DI registers the real store when Mongo is enabled, CLI `ruby resolve` consumes the new payload/warns when inventories are still warming, and docs/OpenAPI references were refreshed.
### Updates — 2025-11-13
- `SCANNER-ENG-0009`: Verified Worker DI registers `IRubyPackageInventoryStore` when Mongo is enabled and falls back to `NullRubyPackageInventoryStore` for in-memory/unit scenarios; confirmed Scanner.WebService endpoint + CLI client exercise the same store contract.
- `SCANNER-ENG-0009`: Cross-checked docs/manifests so operators can trace the new `/api/scans/{scanId}/ruby-packages` endpoint from `docs/modules/scanner/architecture.md` and the CLI reference; plugin drop under `plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Ruby` now mirrors the analyzer assembly + manifest for Worker hot-load.
- `SCANNER-ENG-0009`: Targeted tests cover analyzer fixtures, Worker persistence, and the WebService endpoint:
`dotnet test src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Ruby.Tests/StellaOps.Scanner.Analyzers.Lang.Ruby.Tests.csproj --nologo --verbosity minimal`
`dotnet test src/Scanner/__Tests/StellaOps.Scanner.Worker.Tests/StellaOps.Scanner.Worker.Tests.csproj --nologo --verbosity minimal`
`dotnet test src/Scanner/__Tests/StellaOps.Scanner.WebService.Tests/StellaOps.Scanner.WebService.Tests.csproj --nologo --verbosity minimal --filter "FullyQualifiedName~RubyPackages"`
- `SCANNER-ENG-0015`: DSSE & Rekor operator guide expanded with configuration/env var map, rollout runbook, verification snippets, and alert/SLO recommendations so Export Center + Ops can enable attestations deterministically.
- `SCANNER-ENG-0019`: Scanner.WebService now maps digest/reference identifiers back to canonical scan IDs, CLI backend encodes path segments, and regression tests (`RubyPackagesEndpointsTests`, `StellaOps.Cli.Tests --filter Ruby`) cover the new resolution path so `stella ruby resolve --image` works for both digests and tagged references.