feat: Add VEX compact fixture and implement offline verifier for Findings Ledger exports
- Introduced a new VEX compact fixture for testing purposes. - Implemented `verify_export.py` script to validate Findings Ledger exports, ensuring deterministic ordering and applying redaction manifests. - Added a lightweight stub `HarnessRunner` for unit tests to validate ledger hashing expectations. - Documented tasks related to the Mirror Creator. - Created models for entropy signals and implemented the `EntropyPenaltyCalculator` to compute penalties based on scanner outputs. - Developed unit tests for `EntropyPenaltyCalculator` to ensure correct penalty calculations and handling of edge cases. - Added tests for symbol ID normalization in the reachability scanner. - Enhanced console status service with comprehensive unit tests for connection handling and error recovery. - Included Cosign tool version 2.6.0 with checksums for various platforms.
This commit is contained in:
75
docs/modules/findings-ledger/schema-catalog.md
Normal file
75
docs/modules/findings-ledger/schema-catalog.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Findings Ledger Schema Catalog (FL1–FL3)
|
||||
|
||||
**Scope:** Versioned canonical schemas for ledger events, projections, and exports.
|
||||
**Status:** v1.0.0 sealed (2025-12-02) — breaking changes require new minor/major version tags.
|
||||
|
||||
## 1) Ledger event envelope — `ledger.event.v1`
|
||||
|
||||
| Field | Type | Notes |
|
||||
| --- | --- | --- |
|
||||
| `event.id` | `uuid` | V7 GUID allowed. |
|
||||
| `event.type` | `string` (`ledger_event_type`) | See `schema.md` §2.2. |
|
||||
| `event.tenant` | `string` | Partition key. |
|
||||
| `event.chainId` | `uuid` | Derived when absent (`tenant :: policyVersion`), see `workflow-inference.md`. |
|
||||
| `event.sequence` | `long` | Gapless per chain, starts at 1. |
|
||||
| `event.policyVersion` | `string` | SHA-256 digest of policy bundle; propagated into exports and DSSE. |
|
||||
| `event.finding` | object | `id`, `artifactId`, `vulnId`. |
|
||||
| `event.actor` | object | `id`, `type` (`system|operator|integration`). |
|
||||
| `event.occurredAt` | `string` (UTC ISO-8601 ms) | Domain clock. |
|
||||
| `event.recordedAt` | `string` (UTC ISO-8601 ms) | Service `TimeProvider`. |
|
||||
| `event.payload` | object | Mutation-specific body. |
|
||||
| `event.evidenceBundleRef` | `string?` | DSSE/capsule id (optional). |
|
||||
| `event.airgap.bundle` | object? | See `airgap-provenance.md`. |
|
||||
| `event_hash` | `char(64)` | `sha256(canonicalJson)` lower-hex. |
|
||||
| `previous_hash` | `char(64)` | All-zero for chain genesis. |
|
||||
| `merkle_leaf_hash` | `char(64)` | `sha256(event_hash || "-" || sequence)`. |
|
||||
|
||||
Canonicalisation: UTF-8, sorted keys, lower-case enums, ISO-8601 UTC with millisecond precision, arrays stable-order. Any field addition bumps minor version.
|
||||
|
||||
## 2) Finding projection — `ledger.projection.v1`
|
||||
|
||||
| Field | Type | Notes |
|
||||
| --- | --- | --- |
|
||||
| `tenantId` | `string` | Partition key. |
|
||||
| `findingId` | `string` | Stable identity. |
|
||||
| `policyVersion` | `string` | Hash of active policy bundle. |
|
||||
| `status` | `string` | `affected|triaged|accepted_risk|resolved|unknown`. |
|
||||
| `severity` | `number` | 0–10, 3 decimal places. |
|
||||
| `riskScore` | `number` | 0–10, 3 decimal places. |
|
||||
| `riskSeverity` | `string` | `low|medium|high|critical|unknown`. |
|
||||
| `riskProfileVersion` | `string` | Version/hash from Risk Engine. |
|
||||
| `riskExplanationId` | `uuid?` | Links to explain bundle. |
|
||||
| `labels` | `json` | KEV/runtime flags, sorted keys. |
|
||||
| `currentEventId` | `uuid` | Source ledger event. |
|
||||
| `explainRef` | `string?` | Object storage / DSSE reference. |
|
||||
| `policyRationale` | `json` | Array of rationale refs. |
|
||||
| `updatedAt` | `string` UTC | Projection timestamp. |
|
||||
| `cycleHash` | `char(64)` | `sha256(canonicalProjectionJson)`; used in exports. |
|
||||
|
||||
Projection deterministic hash recipe: serialize projection record with sorted keys (excluding `updatedAt` jitter) and hash via SHA-256. The replay harness recomputes and compares.
|
||||
|
||||
## 3) Export payloads — `export.v1`
|
||||
|
||||
Shapes share headers: `policyVersion`, `projectionVersion` (cycle hash), `filtersHash`, `pageToken`, `observedAt`, `provenance` (`ledgerRoot`, `projectorVersion`, `policyHash`, optional `dsseDigest`).
|
||||
|
||||
### Canonical vs compact
|
||||
- **Canonical (`export.v1.canonical`)** — full provenance fields, evidence refs, DSSE linkage.
|
||||
- **Compact (`export.v1.compact`)** — drops verbose fields (`policyRationale`, comments, actor ids), keeps `cycleHash` + `filtersHash` for determinism; redaction manifest enforced.
|
||||
|
||||
### Record fields
|
||||
- Findings: `findingId`, `eventSequence`, `status`, `severity`, `risk`, `advisories[]`, `evidenceBundleRef`, `cycleHash`.
|
||||
- VEX: `vexStatementId`, `product`, `status`, `justification`, `knownExploited`, `cycleHash`.
|
||||
- Advisories: `advisoryId`, `source`, `cvss{version,vector,baseScore}`, `epss`, `kev`, `cycleHash`.
|
||||
- SBOMs: `sbomId`, `subject{digest,mediaType}`, `sbomFormat`, `componentsCount`, `materials[]`, `cycleHash`.
|
||||
|
||||
Filters hash: `sha256(sortedQueryString)`; stored alongside fixtures for replayability.
|
||||
|
||||
## 4) Versioning rules
|
||||
- Patch: backward-compatible field additions (new optional key) — bump patch digit.
|
||||
- Minor: additive required fields or canonical rule tweaks — bump minor.
|
||||
- Major: breaking change (field removal/rename, hash recipe) — bump major and keep prior schema frozen.
|
||||
|
||||
## 5) Reference artefacts
|
||||
- Golden fixtures: `src/Findings/StellaOps.Findings.Ledger/fixtures/golden/*.ndjson`.
|
||||
- Checksum manifest: `docs/modules/findings-ledger/golden-checksums.json`.
|
||||
- Offline verifier: `tools/LedgerReplayHarness/scripts/verify_export.py`.
|
||||
Reference in New Issue
Block a user