Add new features and tests for AirGap and Time modules
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Introduced `SbomService` tasks documentation.
- Updated `StellaOps.sln` to include new projects: `StellaOps.AirGap.Time` and `StellaOps.AirGap.Importer`.
- Added unit tests for `BundleImportPlanner`, `DsseVerifier`, `ImportValidator`, and other components in the `StellaOps.AirGap.Importer.Tests` namespace.
- Implemented `InMemoryBundleRepositories` for testing bundle catalog and item repositories.
- Created `MerkleRootCalculator`, `RootRotationPolicy`, and `TufMetadataValidator` tests.
- Developed `StalenessCalculator` and `TimeAnchorLoader` tests in the `StellaOps.AirGap.Time.Tests` namespace.
- Added `fetch-sbomservice-deps.sh` script for offline dependency fetching.
This commit is contained in:
master
2025-11-20 23:29:54 +02:00
parent 65b1599229
commit 79b8e53441
182 changed files with 6660 additions and 1242 deletions

View File

@@ -0,0 +1,57 @@
# Console Consumption of LNM Linksets (Sprint 110)
Goal: unblock CONCELIER-CONSOLE-23-001..003 by defining how Console reads Link-Not-Merge (LNM) data now that the schema is frozen (2025-11-17) and cache plan exists.
## Data sources
- Primary collection: `advisory_linksets` (see `docs/modules/concelier/link-not-merge-schema.md`).
- Cache: `advisory_linkset_cache` (see `docs/modules/concelier/operations/lnm-cache-plan.md`). Console should read from cache when present; fall back to live rebuild otherwise.
## API shape (WebService)
- Endpoint to expose via Console: `GET /v1/lnm/linksets/{advisoryId}`
- Query: `source` (required), `tenant`, optional `includeConflicts=true|false` (default true).
- Response JSON (deterministic key order):
```json
{
"advisoryId": "GHSA-123",
"source": "ghsa",
"observations": ["<mongoId>", "<mongoId2>"],
"normalized": {
"purls": ["pkg:npm/foo@1.0.0"],
"versions": ["1.0.0"],
"ranges": [],
"severities": [{"system":"cvssv3","score":7.5}]
},
"conflicts": [
{"field":"severities","reason":"disagreement","values":["7.5","9.8"]}
],
"provenance": {
"observationHashes": ["..."],
"toolVersion": "lnm-1.0"
},
"createdAt": "2025-11-20T00:00:00Z"
}
```
- Sorting: observations sorted by `source, advisoryId, fetchedAt` before hashing; response preserves that order.
- Caching: when cache hit, return cached document; when miss, rebuild, write cache, then return.
## Console rendering contract
- Fields to surface:
- `normalized.purls` (list) and `normalized.versions`
- `conflicts` if non-empty: show field + reason text; keep order as returned.
- `provenance.observationHashes` as “evidence anchors” (list); do not prettify.
- Tenant header required; no cross-tenant leakage. Scope `concelier:lnm.read`.
## Error handling
- 404 when linkset missing for advisory/source.
- 409 when tenant header missing/invalid.
- Deterministic error body: `{ "error": "<code>", "message": "..." }`.
## Telemetry (reuse existing meters)
- Increment `lnm.cache.hit_total` / `lnm.cache.write_total` as defined in cache plan.
- Log template on returns: `lnm_console_success tenant={Tenant} advisoryId={AdvisoryId} source={Source} cached={Cached}`.
## Owners
- Concelier Console Guild (primary)
- Concelier Core Guild (review)
This document is authoritative for CONCELIER-CONSOLE-23-001..003 in Sprint 110.

View File

@@ -0,0 +1,33 @@
# Concelier LNM Cache Plan (Sprint 110)
Purpose: finalize structured caching fields now that Link-Not-Merge (LNM) schema is frozen (2025-11-17) and Evidence Locker contract is published.
## Cache payload shape
- Source: `advisory_linksets` collection (see `docs/modules/concelier/link-not-merge-schema.md`).
- Cache entry key: `{tenant}:{advisoryId}:{source}`.
- Cached fields (deterministic order):
- `observations` (ObjectId list, already sorted by source, advisoryId, fetchedAt)
- `normalized.purls`, `normalized.versions`, `normalized.ranges`, `normalized.severities`
- `conflicts` array (field, reason, values)
- `provenance.observationHashes`
- `builtByJobId`, `createdAt`
- Exclude mutable/derived fields; cache is rebuilt on ingestion replay, not updated in place.
## Storage & eviction
- Collection: `advisory_linkset_cache` (per-tenant index `{tenantId:1, advisoryId:1, source:1}` unique).
- Eviction: TTL on `createdAt` disabled by default; cache is replaced when a newer `provenance.observationHashes` differs.
- Determinism: cache documents written via canonical JSON writer (sorted keys); timestamps UTC.
## API exposure
- WebService: surface cached linksets via existing `/v1/lnm/linksets/{advisoryId}?source=...` (read-through); if cache miss, rebuild synchronously and store.
- Console: may rely on this cache for Advisory AI surfaces; no new payload fields required.
## Telemetry
- Meter `StellaOps.Concelier.Linksets` (existing) to add counters: `lnm.cache.write_total{result}`, `lnm.cache.hit_total{source}`; histogram `lnm.cache.rebuild_ms` for synchronous rebuilds. (To be wired in subsequent sprint.)
## Alignment
- Schema source of truth: `docs/modules/concelier/link-not-merge-schema.md` (frozen v1).
- Evidence Locker contract: `docs/modules/evidence-locker/attestation-contract.md` informs provenance fields; no shape changes required.
## Ownership
- Concelier Core + WebService guilds.

View File

@@ -0,0 +1,20 @@
# Concelier AirGap Prep — PREP-CONCELIER-AIRGAP-56-001..58-001
Status: Draft (2025-11-20)
Owners: Concelier Core · AirGap Guilds
Scope: Capture mirror bundle/staleness requirements for Concelier ingestion under sealed mode.
## Dependencies
- Mirror thin bundle milestones (bundle_id, provenance, staleness_budget) from Mirror sprint 56-001.
- AirGap controller staleness/time anchor fields.
## Needed contract
- Ingestion must accept `bundle_id`, `provenance`, `staleness_seconds_remaining` on advisory/linkset endpoints.
- Reject non-mirror sources when sealed; surface `AIRGAP_EGRESS_BLOCKED` per Concelier AirGap response contract.
## Open decisions
- Exact header names for bundle/staleness metadata.
- Whether to cache bundle provenance per tenant.
## Handoff
Use as PREP artefact for 56-001..58-001 chain; update when mirror bundle schema and controller staleness fields are finalized.

View File

@@ -0,0 +1,16 @@
# Concelier Attestation Prep — PREP-CONCELIER-ATTEST-73-001/002
Status: Draft (2025-11-20)
Owners: Concelier Core · Evidence Locker Guild
Scope: Define attestation scope/signoff pending for Evidence Locker integration.
## Needs
- Evidence Locker attestation scope and DSSE profile.
- Endpoint contract for attestation verification of Concelier exports.
## Open decisions
- Signer identity and Rekor usage in sealed mode.
- What evidence hashes to include (bundle_id, merkle_root).
## Handoff
Use as PREP artefact; update once EvidenceLocker publishes scope and profile.

View File

@@ -0,0 +1,16 @@
# Concelier Console Schema Prep — PREP-CONCELIER-CONSOLE-23-001..003
Status: Draft (2025-11-20)
Owners: Concelier Console Guild
Scope: Provide schema samples for Console evidence bundles and identifiers.
## Needed artefacts
- Sample schema for console evidence bundle IDs and fields (linkset refs, advisory ids, staleness metadata).
- Example payloads for CONCELIER-CONSOLE-23-001..003.
## Open decisions
- ID format for console evidence bundle (ulid vs hash).
- Required fields for linkage to LNM outputs.
## Handoff
Use as PREP artefact; fill once LNM schema freeze and console bundle id rules are provided.

View File

@@ -0,0 +1,12 @@
# Feed Remediation Prep — PREP-FEEDCONN-ICSCISA-02-012 / KISA-02-008
Status: Draft (2025-11-20)
Owners: Concelier Feed Owners
Scope: Capture remediation plan for problematic feeds.
## Items to collect
- Current ingestion endpoints and auth for ICSCISA-02-012, KISA-02-008.
- Known issues and required remediation steps (validation fixes, throttling, schema tweaks).
## Handoff
Use as PREP artefact; fill with concrete remediation steps once feed owners provide details.

View File

@@ -0,0 +1,42 @@
# Concelier PREP Notes — 2025-11-20
Owner: Concelier Core Guild · Scheduler Guild · Data Science Guild
Scope: Provide traceable prep outputs for PREP-CONCELIER-GRAPH-21-002-PLATFORM-EVENTS-S and PREP-CONCELIER-LNM-21-002-WAITING-ON-FINALIZE so downstream tasks can proceed without blocking on missing contracts.
## 1) `sbom.observation.updated` platform event (Graph-21-002)
- Goal: publish deterministic, facts-only observation updates for graph overlays; no derived judgments.
- Proposed envelope (draft for Scheduler/Platform Events review):
- `event_type`: `sbom.observation.updated`
- `tenant_id` (string, required)
- `advisory_ids` (array of strings; upstream IDs as-ingested)
- `observation_ids` (array of stable per-observation IDs emitted by LNM storage)
- `source` (string; advisory source slug)
- `version_range` (string; original upstream semantics)
- `occurred_at` (ISO-8601 UTC, produced by Concelier at write time; deterministic)
- `trace` (object; optional provenance pointers, DSSE envelope digest with alg/id fields)
- Delivery and wiring expectations:
- Publisher lives in `StellaOps.Concelier.Core` after linkset/observation persistence.
- Scheduler binding: NATS/Redis topic `concelier.sbom.observation.updated`; ack + idempotent replay friendly; max delivery once semantics via message ID = `<tenant>:<observation_id>::<digest>`.
- Telemetry: counter `concelier_events_observation_updated_total{tenant,source,result}`; log template includes `tenant`, `advisory_id`, `observation_id`, `event_id`.
- Offline posture: allow emitting into local bus, enqueue to file-backed spool when offline; retry with deterministic ordering by `(tenant, observation_id)`.
- Open questions to resolve in impl task:
- Final topic naming and DSSE requirement (optional vs required per deployment).
- Whether to include component alias list in the event payload or expect consumers to join via API.
## 2) LNM fixtures + precedence markers (LNM-21-002)
- Goal: unblock correlation pipelines and downstream linkset tasks by defining required fixture shape and precedence rules.
- Fixture requirements (additive to frozen LNM v1 schema):
- Provide at least three sources with conflicting severity/CVSS to exercise conflict markers.
- Include overlapping version ranges to validate precedence tie-breakers.
- Each fixture must include `provenance` (source, fetch_time, collector) and `confidence` hints.
- Precedence rule proposal for review:
1. Prefer explicit source ranking table (to be agreed) over recency.
2. If ranking ties, prefer narrower version ranges, then higher confidence, then stable lexical order of `(source, advisory_id)`.
3. Never collapse conflicting fields; emit `conflicts[]` entries with reason codes `severity-disagree`, `cvss-disagree`, `reference-disagree`.
- Delivery path for fixtures once agreed: `src/Concelier/seed-data/lnm/v1/fixtures/*.json` with deterministic ordering; wire into `StellaOps.Concelier.Core.Tests` harness.
- Next actions captured for implementation task:
- Confirm ranking table and conflict reason code list with Cartographer/Data Science.
- Drop initial fixtures into the above path and reference them from the implementation tasks tests.
## Handoff
- This document is the published prep artefact requested by PREP-CONCELIER-GRAPH-21-002-PLATFORM-EVENTS-S and PREP-CONCELIER-LNM-21-002-WAITING-ON-FINALIZE. Downstream tasks should cite this file until the final schemas/fixtures are merged.

View File

@@ -0,0 +1,44 @@
# Concelier Web AirGap Prep — PREP-CONCELIER-WEB-AIRGAP-57-001
Status: Draft (2025-11-20)
Owners: Concelier WebService Guild · AirGap Policy Guild
Scope: Define remediation payloads and staleness plumbing for sealed-mode violations, dependent on WEB-AIRGAP-56-002.
## Dependencies
- WEB-AIRGAP-56-001: mirror bundle registration + sealed-mode enforcement.
- WEB-AIRGAP-56-002: staleness + bundle provenance metadata surfaces.
- AirGap controller scopes (seal/unseal) and time anchor semantics from AirGap Controller/Time guilds.
## Proposed payload mapping (EGRESS blocked)
- Error code: `AIRGAP_EGRESS_BLOCKED`.
- Shape:
```json
{
"error": "AIRGAP_EGRESS_BLOCKED",
"message": "Direct internet fetches disabled in sealed mode; use mirror bundle sources only.",
"bundle_required": true,
"staleness_seconds": 0,
"remediation": [
"Import mirror bundle via /airgap/import or offline kit",
"Ensure sealed mode is set with valid time anchor",
"Retry with cached/mirrored sources enabled"
]
}
```
- Determinism: fixed ordering of fields, remediation list sorted.
## Staleness surfacing
- Staleness derived from bundle metadata supplied by 56-002 (`bundle_id`, `provenance`, `staleness_budget_seconds`).
- Responses include `staleness_seconds_remaining` and `bundle_id` when available.
## Observability
- Emit timeline event `concelier.airgap.egress_blocked` with `{tenant_id, bundle_id?, endpoint, request_id}`.
- Metric: `concelier_airgap_egress_blocked_total` (counter) tagged by endpoint.
## Open decisions
- Final error envelope format (depends on WEB-OAS-61-002 standard envelope).
- Exact header name for staleness metadata (suggest `x-concelier-bundle-staleness`).
- Whether to include advisory key/linkset ids in the blocked response.
## Handoff
Use this as the PREP artefact for WEB-AIRGAP-57-001. Update once 56-002 and error envelope standard are finalized.