feat: Add VEX compact fixture and implement offline verifier for Findings Ledger exports
- Introduced a new VEX compact fixture for testing purposes. - Implemented `verify_export.py` script to validate Findings Ledger exports, ensuring deterministic ordering and applying redaction manifests. - Added a lightweight stub `HarnessRunner` for unit tests to validate ledger hashing expectations. - Documented tasks related to the Mirror Creator. - Created models for entropy signals and implemented the `EntropyPenaltyCalculator` to compute penalties based on scanner outputs. - Developed unit tests for `EntropyPenaltyCalculator` to ensure correct penalty calculations and handling of edge cases. - Added tests for symbol ID normalization in the reachability scanner. - Enhanced console status service with comprehensive unit tests for connection handling and error recovery. - Included Cosign tool version 2.6.0 with checksums for various platforms.
This commit is contained in:
StellaOps Bot
@@ -9,6 +9,7 @@ _Frozen v1 (add-only) — approved 2025-11-17 for CONCELIER-LNM-21-001/002/101._
|
||||
|
||||
## Status
|
||||
- Frozen v1 as of 2025-11-17; further schema changes must go through ADR + sprint gating (CONCELIER-LNM-22x+).
|
||||
- Canonical JSON Schemas + signed manifest live in `docs/modules/concelier/schemas/` (advisory observation, linkset, offline bundle). Verify with `openssl dgst -sha256 -verify schema-signing-pub.pem -signature schema.manifest.sig schema.manifest.json`.
|
||||
|
||||
## Observation document (Mongo JSON Schema excerpt)
|
||||
```json
|
||||
|
||||
32
docs/modules/concelier/schemas/README.md
Normal file
32
docs/modules/concelier/schemas/README.md
Normal file
@@ -0,0 +1,32 @@
|
||||
# Concelier schema bundle (CI1–CI10 remediation)
|
||||
|
||||
This folder publishes the signed JSON Schemas for Link-Not-Merge ingestion artifacts and the offline bundle manifest used by Offline Kit builds.
|
||||
|
||||
- `advisory-observation.schema.json` — canonical observation shape (provenance + content hash enforced).
|
||||
- `advisory-linkset.schema.json` — linkset materialization with conflict reasons and deterministic IDs.
|
||||
- `offline-advisory-bundle.schema.json` — manifest for air-gapped advisory bundles, including staleness and signature metadata.
|
||||
- `schema.manifest.json` — digest manifest over all schemas.
|
||||
- `schema.manifest.sig` — detached ECDSA (P-256) signature over the manifest (public key: `schema-signing-pub.pem`).
|
||||
- `schema.manifest.sig.b64` — base64 view of the signature for air-gapped copy/paste.
|
||||
- `samples/` — deterministic sample payloads for CI fixtures (see `tests` notes below).
|
||||
|
||||
## Verify locally (deterministic, offline)
|
||||
|
||||
```bash
|
||||
# 1) Validate schemas are unchanged
|
||||
sha256sum -c schema.manifest.json
|
||||
|
||||
# 2) Verify detached signature with the published public key
|
||||
openssl dgst -sha256 -verify schema-signing-pub.pem \
|
||||
-signature schema.manifest.sig \
|
||||
schema.manifest.json
|
||||
```
|
||||
|
||||
## Test coverage
|
||||
|
||||
The fixtures in `samples/` are consumed by `StellaOps.Concelier.Core.Tests` to assert:
|
||||
- deterministic idempotency keys and conflict ordering (`Linksets/AdvisoryLinksetIdempotencyTests`),
|
||||
- tenant normalization and signature requirements for observations (`Aoc/AdvisoryObservationWriteGuardTests`),
|
||||
- offline bundle manifest validation (`Schemas/OfflineBundleSchemaTests`).
|
||||
|
||||
Keep the manifest and signature updated whenever schema files change. Keys are dev/test-only; production signing happens in the release pipeline.
|
||||
85
docs/modules/concelier/schemas/advisory-linkset.schema.json
Normal file
85
docs/modules/concelier/schemas/advisory-linkset.schema.json
Normal file
@@ -0,0 +1,85 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://stellaops.local/concelier/schemas/advisory-linkset.schema.json",
|
||||
"title": "Concelier Advisory Linkset",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"linksetId",
|
||||
"tenantId",
|
||||
"advisoryId",
|
||||
"source",
|
||||
"observationIds",
|
||||
"createdAt"
|
||||
],
|
||||
"properties": {
|
||||
"linksetId": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
|
||||
"tenantId": { "type": "string", "minLength": 1 },
|
||||
"source": { "type": "string", "minLength": 1 },
|
||||
"advisoryId": { "type": "string", "minLength": 1 },
|
||||
"observationIds": {
|
||||
"type": "array",
|
||||
"items": { "type": "string", "minLength": 1 },
|
||||
"uniqueItems": true,
|
||||
"minItems": 1
|
||||
},
|
||||
"normalized": {
|
||||
"type": ["object", "null"],
|
||||
"additionalProperties": true
|
||||
},
|
||||
"provenance": {
|
||||
"type": ["object", "null"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"observationHashes": {
|
||||
"type": "array",
|
||||
"items": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
|
||||
"uniqueItems": true
|
||||
},
|
||||
"toolVersion": { "type": "string" },
|
||||
"policyHash": { "type": "string" }
|
||||
}
|
||||
},
|
||||
"confidence": { "type": ["number", "null"], "minimum": 0, "maximum": 1 },
|
||||
"conflicts": {
|
||||
"type": ["array", "null"],
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["field", "reason"],
|
||||
"properties": {
|
||||
"field": { "type": "string" },
|
||||
"reason": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"severity-mismatch",
|
||||
"affected-range-divergence",
|
||||
"reference-clash",
|
||||
"alias-inconsistency",
|
||||
"metadata-gap",
|
||||
"statement-conflict"
|
||||
]
|
||||
},
|
||||
"sourceIds": {
|
||||
"type": ["array", "null"],
|
||||
"items": { "type": "string" },
|
||||
"uniqueItems": true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"aliases": {
|
||||
"type": ["object", "null"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"primary": { "type": "string" },
|
||||
"others": { "type": "array", "items": { "type": "string" }, "uniqueItems": true }
|
||||
}
|
||||
},
|
||||
"purls": { "type": ["array", "null"], "items": { "type": "string" }, "uniqueItems": true },
|
||||
"cpes": { "type": ["array", "null"], "items": { "type": "string" }, "uniqueItems": true },
|
||||
"createdAt": { "type": "string", "format": "date-time" },
|
||||
"updatedAt": { "type": ["string", "null"], "format": "date-time" },
|
||||
"builtByJobId": { "type": ["string", "null"] }
|
||||
}
|
||||
}
|
||||
163
docs/modules/concelier/schemas/advisory-observation.schema.json
Normal file
163
docs/modules/concelier/schemas/advisory-observation.schema.json
Normal file
@@ -0,0 +1,163 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://stellaops.local/concelier/schemas/advisory-observation.schema.json",
|
||||
"title": "Concelier Advisory Observation (Link-Not-Merge)",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"observationId",
|
||||
"tenant",
|
||||
"source",
|
||||
"upstream",
|
||||
"content",
|
||||
"linkset",
|
||||
"rawLinkset",
|
||||
"createdAt"
|
||||
],
|
||||
"properties": {
|
||||
"observationId": { "type": "string", "minLength": 1 },
|
||||
"tenant": { "type": "string", "minLength": 1, "pattern": "^[a-z0-9:-]+$" },
|
||||
"source": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["vendor", "stream", "api"],
|
||||
"properties": {
|
||||
"vendor": { "type": "string", "minLength": 1 },
|
||||
"stream": { "type": "string", "minLength": 1 },
|
||||
"api": { "type": "string", "format": "uri" },
|
||||
"collectorVersion": { "type": "string" }
|
||||
}
|
||||
},
|
||||
"upstream": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"upstreamId",
|
||||
"fetchedAt",
|
||||
"receivedAt",
|
||||
"contentHash",
|
||||
"signature"
|
||||
],
|
||||
"properties": {
|
||||
"upstreamId": { "type": "string", "minLength": 1 },
|
||||
"documentVersion": { "type": "string" },
|
||||
"fetchedAt": { "type": "string", "format": "date-time" },
|
||||
"receivedAt": { "type": "string", "format": "date-time" },
|
||||
"contentHash": {
|
||||
"type": "string",
|
||||
"pattern": "^sha256:[A-Fa-f0-9]{64}$"
|
||||
},
|
||||
"signature": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["present"],
|
||||
"properties": {
|
||||
"present": { "type": "boolean" },
|
||||
"format": { "type": "string" },
|
||||
"keyId": { "type": "string" },
|
||||
"signature": { "type": "string" }
|
||||
},
|
||||
"allOf": [
|
||||
{
|
||||
"if": { "properties": { "present": { "const": true } } },
|
||||
"then": {
|
||||
"required": ["format", "keyId", "signature"],
|
||||
"properties": {
|
||||
"format": { "minLength": 1 },
|
||||
"keyId": { "minLength": 1 },
|
||||
"signature": { "minLength": 1 }
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"if": { "properties": { "present": { "const": false } } },
|
||||
"then": {
|
||||
"properties": {
|
||||
"format": { "maxLength": 0 },
|
||||
"keyId": { "maxLength": 0 },
|
||||
"signature": { "maxLength": 0 }
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"metadata": {
|
||||
"type": "object",
|
||||
"additionalProperties": { "type": "string" },
|
||||
"propertyNames": { "pattern": "^[A-Za-z0-9_.:-]+$" }
|
||||
}
|
||||
}
|
||||
},
|
||||
"content": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["format", "raw"],
|
||||
"properties": {
|
||||
"format": { "type": "string", "minLength": 1 },
|
||||
"specVersion": { "type": "string" },
|
||||
"raw": { "type": ["object", "array"] },
|
||||
"metadata": {
|
||||
"type": "object",
|
||||
"additionalProperties": { "type": "string" },
|
||||
"propertyNames": { "pattern": "^[A-Za-z0-9_.:-]+$" }
|
||||
}
|
||||
}
|
||||
},
|
||||
"linkset": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"aliases": { "type": "array", "items": { "type": "string" }, "uniqueItems": true },
|
||||
"purls": { "type": "array", "items": { "type": "string" }, "uniqueItems": true },
|
||||
"cpes": { "type": "array", "items": { "type": "string" }, "uniqueItems": true },
|
||||
"references": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["type", "url"],
|
||||
"properties": {
|
||||
"type": { "type": "string" },
|
||||
"url": { "type": "string", "format": "uri" }
|
||||
}
|
||||
}
|
||||
},
|
||||
"reconciledFrom": { "type": "array", "items": { "type": "string" }, "uniqueItems": true }
|
||||
}
|
||||
},
|
||||
"rawLinkset": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"aliases": { "type": "array", "items": { "type": "string" }, "uniqueItems": true },
|
||||
"packageUrls": { "type": "array", "items": { "type": "string" } },
|
||||
"cpes": { "type": "array", "items": { "type": "string" } },
|
||||
"references": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"type": { "type": "string" },
|
||||
"url": { "type": "string" }
|
||||
},
|
||||
"required": ["type", "url"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"relationships": { "type": "array", "items": { "type": "object" } },
|
||||
"reconciledFrom": { "type": "array", "items": { "type": "string" } },
|
||||
"scopes": { "type": "array", "items": { "type": "string" } },
|
||||
"notes": {
|
||||
"type": "object",
|
||||
"additionalProperties": { "type": "string" }
|
||||
}
|
||||
}
|
||||
},
|
||||
"attributes": {
|
||||
"type": "object",
|
||||
"additionalProperties": { "type": "string" },
|
||||
"propertyNames": { "pattern": "^[A-Za-z0-9_.:-]+$" }
|
||||
},
|
||||
"createdAt": { "type": "string", "format": "date-time" }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://stellaops.local/concelier/schemas/offline-advisory-bundle.schema.json",
|
||||
"title": "Concelier Offline Advisory Bundle",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"bundleId",
|
||||
"tenant",
|
||||
"exportKind",
|
||||
"snapshot",
|
||||
"manifest",
|
||||
"hashes",
|
||||
"signatures",
|
||||
"createdAt"
|
||||
],
|
||||
"properties": {
|
||||
"bundleId": { "type": "string", "pattern": "^bundle:[A-Za-z0-9._:-]+$" },
|
||||
"tenant": { "type": "string", "minLength": 1 },
|
||||
"exportKind": { "type": "string", "enum": ["json", "trivydb"] },
|
||||
"createdAt": { "type": "string", "format": "date-time" },
|
||||
"snapshot": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["windowStart", "windowEnd", "sources"],
|
||||
"properties": {
|
||||
"windowStart": { "type": "string", "format": "date-time" },
|
||||
"windowEnd": { "type": "string", "format": "date-time" },
|
||||
"stalenessHours": { "type": "integer", "minimum": 0 },
|
||||
"sources": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["name", "cursor", "hash"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"name": { "type": "string" },
|
||||
"cursor": { "type": "string" },
|
||||
"hash": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
|
||||
"snapshotUri": { "type": "string", "format": "uri" }
|
||||
}
|
||||
},
|
||||
"uniqueItems": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"manifest": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["path", "sha256", "size"],
|
||||
"properties": {
|
||||
"path": { "type": "string" },
|
||||
"sha256": { "type": "string", "pattern": "^[A-Fa-f0-9]{64}$" },
|
||||
"size": { "type": "integer", "minimum": 0 },
|
||||
"contentType": { "type": "string" }
|
||||
}
|
||||
},
|
||||
"uniqueItems": true
|
||||
},
|
||||
"hashes": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"patternProperties": {
|
||||
"^sha256$": { "type": "string", "pattern": "^[A-Fa-f0-9]{64}$" }
|
||||
}
|
||||
},
|
||||
"signatures": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["type", "keyId", "signature"],
|
||||
"properties": {
|
||||
"type": { "type": "string", "enum": ["dsse-inline", "detached"] },
|
||||
"keyId": { "type": "string" },
|
||||
"signature": { "type": "string" },
|
||||
"envelopeDigest": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
|
||||
"rekor": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"logIndex": { "type": "integer", "minimum": 0 },
|
||||
"uuid": { "type": "string" },
|
||||
"integratedTime": { "type": "integer", "minimum": 0 }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"determinism": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"contentHash": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
|
||||
"idempotencyKey": { "type": "string", "pattern": "^[a-f0-9]{64}$" },
|
||||
"canonVersion": { "type": "string", "default": "1" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
{
|
||||
"$schema": "../offline-advisory-bundle.schema.json",
|
||||
"bundleId": "bundle:concelier:offline:2025-12-02T00-00Z",
|
||||
"tenant": "default",
|
||||
"exportKind": "json",
|
||||
"createdAt": "2025-12-02T00:00:00Z",
|
||||
"snapshot": {
|
||||
"windowStart": "2025-11-25T00:00:00Z",
|
||||
"windowEnd": "2025-12-01T23:59:59Z",
|
||||
"stalenessHours": 168,
|
||||
"sources": [
|
||||
{
|
||||
"name": "osv",
|
||||
"cursor": "2025-12-01T23:50:00Z",
|
||||
"hash": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcd",
|
||||
"snapshotUri": "https://mirror.example/offline/osv-2025-12-01.zip"
|
||||
},
|
||||
{
|
||||
"name": "redhat",
|
||||
"cursor": "2025-12-01T23:45:00Z",
|
||||
"hash": "sha256:abcd456789abcdef0123456789abcdef0123456789abcdef0123456789abcd"
|
||||
}
|
||||
]
|
||||
},
|
||||
"manifest": [
|
||||
{
|
||||
"path": "export/index.json",
|
||||
"sha256": "89abcdef0123456789abcdef0123456789abcdef0123456789abcdef01234567",
|
||||
"size": 482192,
|
||||
"contentType": "application/json"
|
||||
},
|
||||
{
|
||||
"path": "export/db/trivy.db",
|
||||
"sha256": "fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210",
|
||||
"size": 1289932,
|
||||
"contentType": "application/octet-stream"
|
||||
}
|
||||
],
|
||||
"hashes": {
|
||||
"sha256": "0f0e0d0c0b0a09080706050403020100ffeeddccbbaa99887766554433221100"
|
||||
},
|
||||
"signatures": [
|
||||
{
|
||||
"type": "dsse-inline",
|
||||
"keyId": "schema-offline-pub",
|
||||
"signature": "MEUCIQDkexampleSignedDigestx+deterministicSig==",
|
||||
"envelopeDigest": "sha256:aa55aa55aa55aa55aa55aa55aa55aa55aa55aa55aa55aa55aa55aa55aa55aa55"
|
||||
}
|
||||
],
|
||||
"determinism": {
|
||||
"contentHash": "sha256:d3c3f6c75c6a3f0906bcee457cc77a2d6d7c0f9d1a1d7da78c0d2ab8e0dba111",
|
||||
"idempotencyKey": "29d58b9fdc5c4e65b26c03f3bd9f442ff0c7f8514b8a9225f8b6417ffabc0101",
|
||||
"canonVersion": "1"
|
||||
}
|
||||
}
|
||||
4
docs/modules/concelier/schemas/schema-signing-pub.pem
Normal file
4
docs/modules/concelier/schemas/schema-signing-pub.pem
Normal file
@@ -0,0 +1,4 @@
|
||||
-----BEGIN PUBLIC KEY-----
|
||||
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEyi7gVscxgRXQzX5ErNuQFN3dPjVw
|
||||
YzU0JE3PGhjSinBwpODxtweLfP6zw2N6f0H9z25t8HwTpFeuk1PWqTX7Gg==
|
||||
-----END PUBLIC KEY-----
|
||||
22
docs/modules/concelier/schemas/schema.manifest.json
Normal file
22
docs/modules/concelier/schemas/schema.manifest.json
Normal file
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"version": 1,
|
||||
"generatedAt": "2025-12-02T00:00:00Z",
|
||||
"files": [
|
||||
{
|
||||
"path": "advisory-observation.schema.json",
|
||||
"sha256": "e3f40aea09794f72f2722c46657377518489e2ca7e3122cfbb65655c3296c083"
|
||||
},
|
||||
{
|
||||
"path": "advisory-linkset.schema.json",
|
||||
"sha256": "e3b40a0cca5aff85be2fbc5af9a96f00f5b7a20f6740a3f32947fae56bd599e5"
|
||||
},
|
||||
{
|
||||
"path": "offline-advisory-bundle.schema.json",
|
||||
"sha256": "9b64af7c2e5fa0c071af7dc04b7984fd1787b4f9e2082cb47174610097e2dc51"
|
||||
},
|
||||
{
|
||||
"path": "samples/offline-advisory-bundle.sample.json",
|
||||
"sha256": "15874bbafe5b2ead5ec9a853c32d715a4b48d41107ff2887d6ccdc222e462f45"
|
||||
}
|
||||
]
|
||||
}
|
||||
BIN
docs/modules/concelier/schemas/schema.manifest.sig
Normal file
BIN
docs/modules/concelier/schemas/schema.manifest.sig
Normal file
Binary file not shown.
2
docs/modules/concelier/schemas/schema.manifest.sig.b64
Normal file
2
docs/modules/concelier/schemas/schema.manifest.sig.b64
Normal file
@@ -0,0 +1,2 @@
|
||||
MEUCIBDcyrpqWmYQUrkWLTwMs6QyG2YWCFTxte10/7TobThlAiEAvqOSESmIxNFQ6pDtHlhpfL1K
|
||||
1SZrDM+PhdAMSOMwoU4=
|
||||
Reference in New Issue
Block a user