Add tests and implement timeline ingestion options with NATS and Redis subscribers

- Introduced `BinaryReachabilityLifterTests` to validate binary lifting functionality.
- Created `PackRunWorkerOptions` for configuring worker paths and execution persistence.
- Added `TimelineIngestionOptions` for configuring NATS and Redis ingestion transports.
- Implemented `NatsTimelineEventSubscriber` for subscribing to NATS events.
- Developed `RedisTimelineEventSubscriber` for reading from Redis Streams.
- Added `TimelineEnvelopeParser` to normalize incoming event envelopes.
- Created unit tests for `TimelineEnvelopeParser` to ensure correct field mapping.
- Implemented `TimelineAuthorizationAuditSink` for logging authorization outcomes.
This commit is contained in:
StellaOps Bot
2025-12-03 09:46:48 +02:00
parent e923880694
commit 35c8f9216f
520 changed files with 4416 additions and 31492 deletions

View File

@@ -20,6 +20,11 @@ Establish versioned spine API/DTO schemas with migration rules, determinism guar
- SP9: Pagination/ordering/perf budgets (stable sort keys, default page size limits, deterministic cursors).
- SP10: Crosswalk mappings SBOM ↔ VEX ↔ graph ↔ policy (table + sample payloads).
## Acceptance/verification artifacts
- Adapter CSV: `docs/modules/policy/fixtures/spine-adapters/v2-to-v1.csv` (hashes in `hashes.txt`).
- Crosswalk table: `docs/modules/policy/fixtures/spine-crosswalk/crosswalk.csv` (hashes in `hashes.txt`).
- Manifest signing proof: DSSE envelope must reference adapter + crosswalk hashes to anchor deterministic migrations.
## Migration & Determinism
- Version headers and DTO version fields must be required; rejects if missing or downgraded without declared adapter.
- Canonical JSON ordering for manifests; hashes computed over canonical form (UTF-8, no BOM).
@@ -29,27 +34,40 @@ Establish versioned spine API/DTO schemas with migration rules, determinism guar
- Header: `X-Spine-Version: v1` (required). DTO field mirror: `schemaVersion` (string, semver).
- Deprecation window: N-1 supported for 90 days; adapters required to downgrade v2→v1 (CSV in `docs/modules/policy/fixtures/spine-adapters/`).
- Hashing: canonical JSON, sorted properties, UTF-8 no BOM, normalized decimals (4dp), timestamps UTC ISO-8601.
- Hash anchors (2025-12-03): `v2-to-v1.csv` BLAKE3=f259a807fae1cac90c4d52223924d808eb52a4ab2cb0d314ab2d651bfdad4273 SHA256=f5f067bd7814bd65213610a5ae4a35ce98e70a990ca1eb86d275a8abd3659a1a.
### Evidence minima per edge (SP2, draft)
- `reachability`: state, confidence, score, method, evidenceRef (hash or URI), runtimeEvidence flag (bool).
- `package_identity`: purl, name, version, supplier, hashes[] (at least SHA256).
- `build_metadata`: buildId, sourceRepo, sourceRef, buildInvokerHash, provenanceHash (DSSE).
- Ordering: edges sorted by `subjectPurl`, `predicate`, `createdAt` for determinism; missing evidenceRef is invalid.
### Unknowns workflow (SP3, draft)
- States: `unknown`, `under_review`, `resolved`, `expired`.
- SLA: auto-review escalation after 7 days; decay to `expired` at 30 days unless refreshed.
- Surfacing: APIs must include `unknowns.count` and list endpoint with deterministic pagination; optional policy lattice flag to penalize unknowns.
- Determinism: cursors encode the last `subjectPurl` + `createdAt`; no random salts.
### Signing (SP4/SP7)
- Manifest structure: list of artifacts (type, id, hash, version, uri), signed using DSSE/ED25519 by default; Rekor optional online, mirrored checkpoints offline.
- Stage policy: compile → ingest → materialize → export; each stage produces DSSE, carries prior stage hash for chain-of-custody.
- Rekor/mirror matrix: online → Rekor+transparency required; offline → mirror checkpoints and DSSE only. PQ dual-sign optional but recorded in manifest metadata.
### Pagination/perf budgets (SP9)
- Default page size 200; max 500; stable sort: tenant asc, subjectPurl asc, advisoryId asc, createdAt asc.
- Cursors: base64-encoded tuple of sort keys; must round-trip deterministically.
- Perf budget: p95 response ≤250ms for page=200 on cached dataset; timeouts return deterministic error `spine_timeout`.
- Rate limits: 600 rpm per tenant; 429 payload includes retry-after seconds and last stable cursor.
### Crosswalk (SP10)
- Provide table mapping: SBOM component ↔ spine node ↔ graph node ↔ policy evaluation input; include sample payloads in `docs/modules/policy/fixtures/spine-crosswalk/`.
- Hash anchors (2025-12-03): `crosswalk.csv` BLAKE3=41926241c6d60bb856ceb4498e70381cdf54217435740f5fdf31ff8964044d78 SHA256=1e6644cdc00097b7e959e75f522335326b8f48fe1d05060d1c06ba660aac22a3.
## Decisions (2025-12-03)
- Evidence minima above are binding for SP2; missing hashes are fatal validation errors.
- Unknowns decay schedule adopted as written; extension requires policy-lattice approval.
- Stage DSSE is mandatory at every boundary; Rekor optional offline but checkpoints must be mirrored with manifest hash list.
- Pagination budgets and rate limits frozen until next version bump; adapters must preserve ordering when downgrading.
## Signing & Offline
- DSSE envelope mandatory for spine manifest; Rekor entry optional online, mirrored checkpoints for offline kits.

View File

@@ -1 +1 @@
v2-to-v1.csv: BLAKE3=<TBD> SHA256=<TBD>
v2-to-v1.csv: BLAKE3=f259a807fae1cac90c4d52223924d808eb52a4ab2cb0d314ab2d651bfdad4273 SHA256=f5f067bd7814bd65213610a5ae4a35ce98e70a990ca1eb86d275a8abd3659a1a

View File

@@ -0,0 +1,8 @@
field,v2,v1,rule
schemaVersion,schemaVersion,schemaVersion,copy
componentRef,component.ref,component.ref,copy
packageIdentity,edges.package_identity,edges.package_identity,copy
reachabilityEvidence,edges.reachability.evidence.hash,edges.reachability.evidence_hash,copy
unknownsState,unknowns.state,unknowns_state,enum_map:unknown=pending;under_review=review;resolved=resolved;expired=stale
paginationCursor,meta.page.cursor,meta.page_cursor,stable_base64(sort_keys)
createdAt,meta.createdAt,meta.created_at,iso8601_utc
1 field v2 v1 rule
1 field v2 v1 rule
2 schemaVersion schemaVersion schemaVersion copy
3 componentRef component.ref component.ref copy
4 packageIdentity edges.package_identity edges.package_identity copy
5 reachabilityEvidence edges.reachability.evidence.hash edges.reachability.evidence_hash copy
6 unknownsState unknowns.state unknowns_state enum_map:unknown=pending;under_review=review;resolved=resolved;expired=stale
7 paginationCursor meta.page.cursor meta.page_cursor stable_base64(sort_keys)
8 createdAt meta.createdAt meta.created_at iso8601_utc

View File

@@ -0,0 +1,4 @@
sbom_component,spine_node,graph_node,policy_input,evidence_ref
pkg:demo/lib-a@1.2.3,spine://component/lib-a,graph://node/lib-a,policy://input/component/lib-a,hash:blake3:27c6de0c
pkg:demo/lib-b@2.0.0,spine://component/lib-b,graph://node/lib-b,policy://input/component/lib-b,hash:blake3:da5b631a
vuln:CVE-0000-0001,spine://vuln/CVE-0000-0001,graph://advisory/CVE-0000-0001,policy://input/vuln/CVE-0000-0001,hash:sha256:22d8f6f8
1 sbom_component spine_node graph_node policy_input evidence_ref
1 sbom_component spine_node graph_node policy_input evidence_ref
2 pkg:demo/lib-a@1.2.3 spine://component/lib-a graph://node/lib-a policy://input/component/lib-a hash:blake3:27c6de0c
3 pkg:demo/lib-b@2.0.0 spine://component/lib-b graph://node/lib-b policy://input/component/lib-b hash:blake3:da5b631a
4 vuln:CVE-0000-0001 spine://vuln/CVE-0000-0001 graph://advisory/CVE-0000-0001 policy://input/vuln/CVE-0000-0001 hash:sha256:22d8f6f8

View File

@@ -1 +1 @@
crosswalk.csv: BLAKE3=<TBD> SHA256=<TBD>
crosswalk.csv: BLAKE3=41926241c6d60bb856ceb4498e70381cdf54217435740f5fdf31ff8964044d78 SHA256=1e6644cdc00097b7e959e75f522335326b8f48fe1d05060d1c06ba660aac22a3