Add tests and implement timeline ingestion options with NATS and Redis subscribers
- Introduced `BinaryReachabilityLifterTests` to validate binary lifting functionality. - Created `PackRunWorkerOptions` for configuring worker paths and execution persistence. - Added `TimelineIngestionOptions` for configuring NATS and Redis ingestion transports. - Implemented `NatsTimelineEventSubscriber` for subscribing to NATS events. - Developed `RedisTimelineEventSubscriber` for reading from Redis Streams. - Added `TimelineEnvelopeParser` to normalize incoming event envelopes. - Created unit tests for `TimelineEnvelopeParser` to ensure correct field mapping. - Implemented `TimelineAuthorizationAuditSink` for logging authorization outcomes.
This commit is contained in:
@@ -43,7 +43,10 @@ Define how external SBOM/scan outputs (Syft, Trivy, Clair) are normalized into S
|
||||
|
||||
## Regression + fixtures (CM4/CM5)
|
||||
- Fixtures under `docs/modules/scanner/fixtures/competitor-adapters/fixtures/` with golden hashes (BLAKE3/SHA256) and expected normalized output.
|
||||
- CI step runs adapter → normalized → hash compare; offline, no network.
|
||||
- `normalized-syft.json` BLAKE3=aa42c167d19535709a10df73dc39e6a50b8efbbb0ae596d17183ce62676fa85a SHA256=3f8684ff341808dcb92e97dd2c10acca727baaff05182e81a4364bb3dad0eaa7
|
||||
- `normalized-trivy.json` BLAKE3=0da216b49ebcf823d8d4aa3c9c1d2a1dcc579d836ba66bb2ae94dd781e214130 SHA256=c29aa6251d378c2aca1c3c6165e61bd2e16b6fa1227c976417b8a525ad7c1fc1
|
||||
- `normalized-clair.json` BLAKE3=92985f4cbdeecc8a0e585a70e07f17b07abdd866eecacaca9ba1b331f4b3af68 SHA256=bc232cc19885c53e4d801f5c830e3683a4031e42f6421739c4cc221f33f15e01
|
||||
- CI step runs adapter → normalized → hash compare; offline, no network. Hashes act as guardrails for deterministic ordering and mapping stability.
|
||||
|
||||
## Fallback hierarchy (CM6)
|
||||
1) Signed SBOM w/ valid provenance → accepted.
|
||||
@@ -53,7 +56,7 @@ Define how external SBOM/scan outputs (Syft, Trivy, Clair) are normalized into S
|
||||
|
||||
## Transparency & coverage (CM7–CM9)
|
||||
- Persist: `source.tool`, `source.version`, `source.hash`, `adapter.version`, `normalized_hash`.
|
||||
- Coverage matrix maintained in `docs/modules/scanner/fixtures/competitor-adapters/coverage.csv` (ecosystem yes/no, notes).
|
||||
- Coverage matrix maintained in `docs/modules/scanner/fixtures/competitor-adapters/coverage.csv` (ecosystem yes/no, notes). Current snapshot (2025-12-03): container/java/python/go/os rows populated; dotnet pending Syft/Clair support.
|
||||
- Bench parity (CM8): pin upstream versions; store run hashes/logs in fixtures folder.
|
||||
|
||||
## Error taxonomy (CM10)
|
||||
@@ -61,9 +64,13 @@ Define how external SBOM/scan outputs (Syft, Trivy, Clair) are normalized into S
|
||||
- Non-retryable: signature_invalid, schema_invalid, unsupported_version, no_evidence.
|
||||
- All errors must carry deterministic reason codes and be logged in normalized output metadata.
|
||||
|
||||
## Open Items
|
||||
- Decide minimal evidence set for accepting unsigned SBOMs (intermediate level before scan-only fallback).
|
||||
- Confirm which hash (BLAKE3/SHA256) is canonical for adapter outputs.
|
||||
## Offline kit (CM5)
|
||||
- Kit contents: adapter CSVs (one per tool), fixtures + hashes above, coverage matrix, trust roots, signature policy, retry taxonomy, and DSSE envelope referencing every file hash. Bundle path: `out/offline/competitor-ingest-kit-v1/`.
|
||||
|
||||
## Decisions (2025-12-03)
|
||||
- Minimal evidence for unsigned SBOM acceptance: must include tool metadata (name/version/hash), component list with purl + SHA256, and scan timestamp; otherwise fallback to scan-only path (CM6 step 3).
|
||||
- Canonical adapter output hash: BLAKE3 primary, SHA256 secondary; both recorded in fixture hash lists and surfaced in normalized metadata (`normalized_hash`).
|
||||
- Signature verification policy is strict fail-closed unless `--allow-unsigned` flag explicitly set; fallback hierarchy still applied but logged.
|
||||
|
||||
## Links
|
||||
- Sprint: `docs/implplan/SPRINT_0186_0001_0001_record_deterministic_execution.md` (CM1–CM10)
|
||||
|
||||
@@ -34,36 +34,41 @@ Define the concrete steps for adopting CVSS v4.0, CycloneDX 1.7 (incl. CBOM), an
|
||||
- timestamps UTC ISO-8601 without sub-ms; decimal rounding 4dp for ratios, 2dp for scores.
|
||||
- Adapter tables (SC4): mapping CSVs checked in under `docs/modules/scanner/fixtures/adapters/` with BLAKE3 + SHA256 hashes; adapters are pure, no net.
|
||||
|
||||
## Fixtures (SC5/SC8)
|
||||
- Add to `docs/modules/scanner/fixtures/cdx17-cbom/`:
|
||||
- `sample-cdx17-cbom.json` (golden), `sample-cdx16-downgraded.json`, `hashes.txt` (BLAKE3, SHA256).
|
||||
- Include CBOM ingress/egress example, CVSS v4 vector, SLSA Source Track fields, evidence properties.
|
||||
## Fixtures (SC2/SC5/SC8)
|
||||
- Golden payloads live in `docs/modules/scanner/fixtures/cdx17-cbom/`.
|
||||
- `sample-cdx17-cbom.json` (CDX 1.7 + CBOM + CVSS v4/v3.1 + SLSA Source Track + evidence).
|
||||
- `sample-cdx16.json` (downgraded CDX 1.6; CVSS v3.1 only; no CBOM channel properties).
|
||||
- `hashes.txt` records deterministic digests:
|
||||
- `sample-cdx17-cbom.json` BLAKE3=27c6de0ccd6adb8149c5521477fba8292aa119fb9e42b521cba6356b2308e761 SHA256=22d8f6f80f02be13f840b74b24b2eea769f108a225152695e1bf8d8a0577e6f6
|
||||
- `sample-cdx16.json` BLAKE3=da5b631a8cca865f929f8fd5d3b35adc512de1754fe2278cb8b415b01c81b3d3 SHA256=3cf6cb04aec97ec05fad0658f54b4ec099644176806f098897a9ba0bf1135cb0
|
||||
- CI step: `dotnet test` hook runs deterministic serializer + hash assertion; env `DOTNET_DISABLE_BUILTIN_GRAPH=1`, fixed `TZ=UTC`, `LC_ALL=C`.
|
||||
- Downgrade adapters (SC4) consume the CDX 1.7 fixture and emit the 1.6 fixture; verify hashes match the values above.
|
||||
|
||||
## Governance (SC1/SC9)
|
||||
- Propose RACI: Product (A), Scanner TL (R), Sbomer TL (C), Policy TL (C), Ops (I).
|
||||
- RACI: Product (A), Scanner TL (R), Sbomer TL (C), Policy TL (C), Ops (I).
|
||||
- Schema bump flow: draft → review → freeze → DSSE-sign schemas + fixtures → publish hash list → lock downgrade adapters.
|
||||
- Downgrade adapters cannot ship without approved mapping CSV + updated hashes.
|
||||
- Downgrade adapters cannot ship without approved mapping CSV + updated hashes; adapter CSVs live under `docs/modules/scanner/fixtures/adapters/` (hash list alongside CSVs).
|
||||
|
||||
## Offline (SC10)
|
||||
- Offline kit must include: schemas, adapter CSVs, fixtures, hash list, DSSE envelope, tool versions (Syft/Trivy pinned) and their hashes.
|
||||
- Bundle path: `out/offline/scanner-standards-kit-v1/`. DSSE envelope references manifest with all hashes.
|
||||
- Bundle path: `out/offline/scanner-standards-kit-v1/`. DSSE envelope references manifest with all hashes; include CBOM sample, downgrade sample, adapter CSVs, and their BLAKE3/SHA256 values.
|
||||
|
||||
## Milestones (proposed)
|
||||
1) Schema draft freeze (CDX 1.7/CBOM + CVSS v4 fields) — owners: Scanner Guild, due T+5d.
|
||||
2) Replay bundle field list for Source Track — owners: Scanner + Sbomer, due T+7d.
|
||||
3) Determinism harness upgrade (CI + fixtures) — owners: QA + Scanner, due T+10d.
|
||||
4) Downgrade adapter tables + hash tests — owners: Scanner, due T+12d.
|
||||
5) Offline-kit bundle update & DSSE signing — owners: Ops, due T+14d.
|
||||
## Milestones (locked for SC1 delivery)
|
||||
1) Schema draft freeze (CDX 1.7/CBOM + CVSS v4 fields) — owners: Scanner Guild, due 2025-12-08.
|
||||
2) Replay bundle field list for Source Track — owners: Scanner + Sbomer, due 2025-12-10.
|
||||
3) Determinism harness upgrade (CI + fixtures) — owners: QA + Scanner, due 2025-12-13.
|
||||
4) Downgrade adapter tables + hash tests — owners: Scanner, due 2025-12-15.
|
||||
5) Offline-kit bundle update & DSSE signing — owners: Ops, due 2025-12-17.
|
||||
|
||||
## Determinism & Offline requirements
|
||||
- Stable field ordering, culture-invariant formatting, UTC ISO-8601 timestamps.
|
||||
- No network calls during conversion/adapters; fixed seeds for any RNG.
|
||||
- All schemas/adapters/fixtures shipped in offline kit with DSSE envelope and recorded hashes.
|
||||
|
||||
## Open Items
|
||||
- Confirm CBOM section subset required for policy engine (ingredients vs evidence-only).
|
||||
- Decide default CVSS v4 vector precision and rounding rules.
|
||||
## Decisions (2025-12-03)
|
||||
- CBOM subset: include ingress + egress channel properties only; deeper data-flow capture deferred to policy/graph once schema stabilises.
|
||||
- CVSS v4 rounding: keep vendor vector precision; round scores to 2dp using `MidpointRounding.ToZero` for deterministic alignment with CVSS v3.1 sidecar values.
|
||||
- Evidence properties are mandatory for replay bundles and serialized CycloneDX 1.7 outputs; adapter must preserve them when downgrading.
|
||||
|
||||
## Links
|
||||
- Sprint: `docs/implplan/SPRINT_0186_0001_0001_record_deterministic_execution.md` (tasks SC1–SC10)
|
||||
|
||||
Reference in New Issue
Block a user