Add new features and tests for AirGap and Time modules
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Introduced `SbomService` tasks documentation. - Updated `StellaOps.sln` to include new projects: `StellaOps.AirGap.Time` and `StellaOps.AirGap.Importer`. - Added unit tests for `BundleImportPlanner`, `DsseVerifier`, `ImportValidator`, and other components in the `StellaOps.AirGap.Importer.Tests` namespace. - Implemented `InMemoryBundleRepositories` for testing bundle catalog and item repositories. - Created `MerkleRootCalculator`, `RootRotationPolicy`, and `TufMetadataValidator` tests. - Developed `StalenessCalculator` and `TimeAnchorLoader` tests in the `StellaOps.AirGap.Time.Tests` namespace. - Added `fetch-sbomservice-deps.sh` script for offline dependency fetching.
This commit is contained in:
57
docs/modules/concelier/operations/console-lnm-consumption.md
Normal file
57
docs/modules/concelier/operations/console-lnm-consumption.md
Normal file
@@ -0,0 +1,57 @@
|
||||
# Console Consumption of LNM Linksets (Sprint 110)
|
||||
|
||||
Goal: unblock CONCELIER-CONSOLE-23-001..003 by defining how Console reads Link-Not-Merge (LNM) data now that the schema is frozen (2025-11-17) and cache plan exists.
|
||||
|
||||
## Data sources
|
||||
- Primary collection: `advisory_linksets` (see `docs/modules/concelier/link-not-merge-schema.md`).
|
||||
- Cache: `advisory_linkset_cache` (see `docs/modules/concelier/operations/lnm-cache-plan.md`). Console should read from cache when present; fall back to live rebuild otherwise.
|
||||
|
||||
## API shape (WebService)
|
||||
- Endpoint to expose via Console: `GET /v1/lnm/linksets/{advisoryId}`
|
||||
- Query: `source` (required), `tenant`, optional `includeConflicts=true|false` (default true).
|
||||
- Response JSON (deterministic key order):
|
||||
```json
|
||||
{
|
||||
"advisoryId": "GHSA-123",
|
||||
"source": "ghsa",
|
||||
"observations": ["<mongoId>", "<mongoId2>"],
|
||||
"normalized": {
|
||||
"purls": ["pkg:npm/foo@1.0.0"],
|
||||
"versions": ["1.0.0"],
|
||||
"ranges": [],
|
||||
"severities": [{"system":"cvssv3","score":7.5}]
|
||||
},
|
||||
"conflicts": [
|
||||
{"field":"severities","reason":"disagreement","values":["7.5","9.8"]}
|
||||
],
|
||||
"provenance": {
|
||||
"observationHashes": ["..."],
|
||||
"toolVersion": "lnm-1.0"
|
||||
},
|
||||
"createdAt": "2025-11-20T00:00:00Z"
|
||||
}
|
||||
```
|
||||
- Sorting: observations sorted by `source, advisoryId, fetchedAt` before hashing; response preserves that order.
|
||||
- Caching: when cache hit, return cached document; when miss, rebuild, write cache, then return.
|
||||
|
||||
## Console rendering contract
|
||||
- Fields to surface:
|
||||
- `normalized.purls` (list) and `normalized.versions`
|
||||
- `conflicts` if non-empty: show field + reason text; keep order as returned.
|
||||
- `provenance.observationHashes` as “evidence anchors” (list); do not prettify.
|
||||
- Tenant header required; no cross-tenant leakage. Scope `concelier:lnm.read`.
|
||||
|
||||
## Error handling
|
||||
- 404 when linkset missing for advisory/source.
|
||||
- 409 when tenant header missing/invalid.
|
||||
- Deterministic error body: `{ "error": "<code>", "message": "..." }`.
|
||||
|
||||
## Telemetry (reuse existing meters)
|
||||
- Increment `lnm.cache.hit_total` / `lnm.cache.write_total` as defined in cache plan.
|
||||
- Log template on returns: `lnm_console_success tenant={Tenant} advisoryId={AdvisoryId} source={Source} cached={Cached}`.
|
||||
|
||||
## Owners
|
||||
- Concelier Console Guild (primary)
|
||||
- Concelier Core Guild (review)
|
||||
|
||||
This document is authoritative for CONCELIER-CONSOLE-23-001..003 in Sprint 110.
|
||||
33
docs/modules/concelier/operations/lnm-cache-plan.md
Normal file
33
docs/modules/concelier/operations/lnm-cache-plan.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# Concelier LNM Cache Plan (Sprint 110)
|
||||
|
||||
Purpose: finalize structured caching fields now that Link-Not-Merge (LNM) schema is frozen (2025-11-17) and Evidence Locker contract is published.
|
||||
|
||||
## Cache payload shape
|
||||
- Source: `advisory_linksets` collection (see `docs/modules/concelier/link-not-merge-schema.md`).
|
||||
- Cache entry key: `{tenant}:{advisoryId}:{source}`.
|
||||
- Cached fields (deterministic order):
|
||||
- `observations` (ObjectId list, already sorted by source, advisoryId, fetchedAt)
|
||||
- `normalized.purls`, `normalized.versions`, `normalized.ranges`, `normalized.severities`
|
||||
- `conflicts` array (field, reason, values)
|
||||
- `provenance.observationHashes`
|
||||
- `builtByJobId`, `createdAt`
|
||||
- Exclude mutable/derived fields; cache is rebuilt on ingestion replay, not updated in place.
|
||||
|
||||
## Storage & eviction
|
||||
- Collection: `advisory_linkset_cache` (per-tenant index `{tenantId:1, advisoryId:1, source:1}` unique).
|
||||
- Eviction: TTL on `createdAt` disabled by default; cache is replaced when a newer `provenance.observationHashes` differs.
|
||||
- Determinism: cache documents written via canonical JSON writer (sorted keys); timestamps UTC.
|
||||
|
||||
## API exposure
|
||||
- WebService: surface cached linksets via existing `/v1/lnm/linksets/{advisoryId}?source=...` (read-through); if cache miss, rebuild synchronously and store.
|
||||
- Console: may rely on this cache for Advisory AI surfaces; no new payload fields required.
|
||||
|
||||
## Telemetry
|
||||
- Meter `StellaOps.Concelier.Linksets` (existing) to add counters: `lnm.cache.write_total{result}`, `lnm.cache.hit_total{source}`; histogram `lnm.cache.rebuild_ms` for synchronous rebuilds. (To be wired in subsequent sprint.)
|
||||
|
||||
## Alignment
|
||||
- Schema source of truth: `docs/modules/concelier/link-not-merge-schema.md` (frozen v1).
|
||||
- Evidence Locker contract: `docs/modules/evidence-locker/attestation-contract.md` informs provenance fields; no shape changes required.
|
||||
|
||||
## Ownership
|
||||
- Concelier Core + WebService guilds.
|
||||
@@ -0,0 +1,20 @@
|
||||
# Concelier AirGap Prep — PREP-CONCELIER-AIRGAP-56-001..58-001
|
||||
|
||||
Status: Draft (2025-11-20)
|
||||
Owners: Concelier Core · AirGap Guilds
|
||||
Scope: Capture mirror bundle/staleness requirements for Concelier ingestion under sealed mode.
|
||||
|
||||
## Dependencies
|
||||
- Mirror thin bundle milestones (bundle_id, provenance, staleness_budget) from Mirror sprint 56-001.
|
||||
- AirGap controller staleness/time anchor fields.
|
||||
|
||||
## Needed contract
|
||||
- Ingestion must accept `bundle_id`, `provenance`, `staleness_seconds_remaining` on advisory/linkset endpoints.
|
||||
- Reject non-mirror sources when sealed; surface `AIRGAP_EGRESS_BLOCKED` per Concelier AirGap response contract.
|
||||
|
||||
## Open decisions
|
||||
- Exact header names for bundle/staleness metadata.
|
||||
- Whether to cache bundle provenance per tenant.
|
||||
|
||||
## Handoff
|
||||
Use as PREP artefact for 56-001..58-001 chain; update when mirror bundle schema and controller staleness fields are finalized.
|
||||
16
docs/modules/concelier/prep/2025-11-20-attest-73-001-prep.md
Normal file
16
docs/modules/concelier/prep/2025-11-20-attest-73-001-prep.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# Concelier Attestation Prep — PREP-CONCELIER-ATTEST-73-001/002
|
||||
|
||||
Status: Draft (2025-11-20)
|
||||
Owners: Concelier Core · Evidence Locker Guild
|
||||
Scope: Define attestation scope/signoff pending for Evidence Locker integration.
|
||||
|
||||
## Needs
|
||||
- Evidence Locker attestation scope and DSSE profile.
|
||||
- Endpoint contract for attestation verification of Concelier exports.
|
||||
|
||||
## Open decisions
|
||||
- Signer identity and Rekor usage in sealed mode.
|
||||
- What evidence hashes to include (bundle_id, merkle_root).
|
||||
|
||||
## Handoff
|
||||
Use as PREP artefact; update once EvidenceLocker publishes scope and profile.
|
||||
@@ -0,0 +1,16 @@
|
||||
# Concelier Console Schema Prep — PREP-CONCELIER-CONSOLE-23-001..003
|
||||
|
||||
Status: Draft (2025-11-20)
|
||||
Owners: Concelier Console Guild
|
||||
Scope: Provide schema samples for Console evidence bundles and identifiers.
|
||||
|
||||
## Needed artefacts
|
||||
- Sample schema for console evidence bundle IDs and fields (linkset refs, advisory ids, staleness metadata).
|
||||
- Example payloads for CONCELIER-CONSOLE-23-001..003.
|
||||
|
||||
## Open decisions
|
||||
- ID format for console evidence bundle (ulid vs hash).
|
||||
- Required fields for linkage to LNM outputs.
|
||||
|
||||
## Handoff
|
||||
Use as PREP artefact; fill once LNM schema freeze and console bundle id rules are provided.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Feed Remediation Prep — PREP-FEEDCONN-ICSCISA-02-012 / KISA-02-008
|
||||
|
||||
Status: Draft (2025-11-20)
|
||||
Owners: Concelier Feed Owners
|
||||
Scope: Capture remediation plan for problematic feeds.
|
||||
|
||||
## Items to collect
|
||||
- Current ingestion endpoints and auth for ICSCISA-02-012, KISA-02-008.
|
||||
- Known issues and required remediation steps (validation fixes, throttling, schema tweaks).
|
||||
|
||||
## Handoff
|
||||
Use as PREP artefact; fill with concrete remediation steps once feed owners provide details.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Concelier PREP Notes — 2025-11-20
|
||||
|
||||
Owner: Concelier Core Guild · Scheduler Guild · Data Science Guild
|
||||
Scope: Provide traceable prep outputs for PREP-CONCELIER-GRAPH-21-002-PLATFORM-EVENTS-S and PREP-CONCELIER-LNM-21-002-WAITING-ON-FINALIZE so downstream tasks can proceed without blocking on missing contracts.
|
||||
|
||||
## 1) `sbom.observation.updated` platform event (Graph-21-002)
|
||||
- Goal: publish deterministic, facts-only observation updates for graph overlays; no derived judgments.
|
||||
- Proposed envelope (draft for Scheduler/Platform Events review):
|
||||
- `event_type`: `sbom.observation.updated`
|
||||
- `tenant_id` (string, required)
|
||||
- `advisory_ids` (array of strings; upstream IDs as-ingested)
|
||||
- `observation_ids` (array of stable per-observation IDs emitted by LNM storage)
|
||||
- `source` (string; advisory source slug)
|
||||
- `version_range` (string; original upstream semantics)
|
||||
- `occurred_at` (ISO-8601 UTC, produced by Concelier at write time; deterministic)
|
||||
- `trace` (object; optional provenance pointers, DSSE envelope digest with alg/id fields)
|
||||
- Delivery and wiring expectations:
|
||||
- Publisher lives in `StellaOps.Concelier.Core` after linkset/observation persistence.
|
||||
- Scheduler binding: NATS/Redis topic `concelier.sbom.observation.updated`; ack + idempotent replay friendly; max delivery once semantics via message ID = `<tenant>:<observation_id>::<digest>`.
|
||||
- Telemetry: counter `concelier_events_observation_updated_total{tenant,source,result}`; log template includes `tenant`, `advisory_id`, `observation_id`, `event_id`.
|
||||
- Offline posture: allow emitting into local bus, enqueue to file-backed spool when offline; retry with deterministic ordering by `(tenant, observation_id)`.
|
||||
- Open questions to resolve in impl task:
|
||||
- Final topic naming and DSSE requirement (optional vs required per deployment).
|
||||
- Whether to include component alias list in the event payload or expect consumers to join via API.
|
||||
|
||||
## 2) LNM fixtures + precedence markers (LNM-21-002)
|
||||
- Goal: unblock correlation pipelines and downstream linkset tasks by defining required fixture shape and precedence rules.
|
||||
- Fixture requirements (additive to frozen LNM v1 schema):
|
||||
- Provide at least three sources with conflicting severity/CVSS to exercise conflict markers.
|
||||
- Include overlapping version ranges to validate precedence tie-breakers.
|
||||
- Each fixture must include `provenance` (source, fetch_time, collector) and `confidence` hints.
|
||||
- Precedence rule proposal for review:
|
||||
1. Prefer explicit source ranking table (to be agreed) over recency.
|
||||
2. If ranking ties, prefer narrower version ranges, then higher confidence, then stable lexical order of `(source, advisory_id)`.
|
||||
3. Never collapse conflicting fields; emit `conflicts[]` entries with reason codes `severity-disagree`, `cvss-disagree`, `reference-disagree`.
|
||||
- Delivery path for fixtures once agreed: `src/Concelier/seed-data/lnm/v1/fixtures/*.json` with deterministic ordering; wire into `StellaOps.Concelier.Core.Tests` harness.
|
||||
- Next actions captured for implementation task:
|
||||
- Confirm ranking table and conflict reason code list with Cartographer/Data Science.
|
||||
- Drop initial fixtures into the above path and reference them from the implementation tasks’ tests.
|
||||
|
||||
## Handoff
|
||||
- This document is the published prep artefact requested by PREP-CONCELIER-GRAPH-21-002-PLATFORM-EVENTS-S and PREP-CONCELIER-LNM-21-002-WAITING-ON-FINALIZE. Downstream tasks should cite this file until the final schemas/fixtures are merged.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Concelier Web AirGap Prep — PREP-CONCELIER-WEB-AIRGAP-57-001
|
||||
|
||||
Status: Draft (2025-11-20)
|
||||
Owners: Concelier WebService Guild · AirGap Policy Guild
|
||||
Scope: Define remediation payloads and staleness plumbing for sealed-mode violations, dependent on WEB-AIRGAP-56-002.
|
||||
|
||||
## Dependencies
|
||||
- WEB-AIRGAP-56-001: mirror bundle registration + sealed-mode enforcement.
|
||||
- WEB-AIRGAP-56-002: staleness + bundle provenance metadata surfaces.
|
||||
- AirGap controller scopes (seal/unseal) and time anchor semantics from AirGap Controller/Time guilds.
|
||||
|
||||
## Proposed payload mapping (EGRESS blocked)
|
||||
- Error code: `AIRGAP_EGRESS_BLOCKED`.
|
||||
- Shape:
|
||||
```json
|
||||
{
|
||||
"error": "AIRGAP_EGRESS_BLOCKED",
|
||||
"message": "Direct internet fetches disabled in sealed mode; use mirror bundle sources only.",
|
||||
"bundle_required": true,
|
||||
"staleness_seconds": 0,
|
||||
"remediation": [
|
||||
"Import mirror bundle via /airgap/import or offline kit",
|
||||
"Ensure sealed mode is set with valid time anchor",
|
||||
"Retry with cached/mirrored sources enabled"
|
||||
]
|
||||
}
|
||||
```
|
||||
- Determinism: fixed ordering of fields, remediation list sorted.
|
||||
|
||||
## Staleness surfacing
|
||||
- Staleness derived from bundle metadata supplied by 56-002 (`bundle_id`, `provenance`, `staleness_budget_seconds`).
|
||||
- Responses include `staleness_seconds_remaining` and `bundle_id` when available.
|
||||
|
||||
## Observability
|
||||
- Emit timeline event `concelier.airgap.egress_blocked` with `{tenant_id, bundle_id?, endpoint, request_id}`.
|
||||
- Metric: `concelier_airgap_egress_blocked_total` (counter) tagged by endpoint.
|
||||
|
||||
## Open decisions
|
||||
- Final error envelope format (depends on WEB-OAS-61-002 standard envelope).
|
||||
- Exact header name for staleness metadata (suggest `x-concelier-bundle-staleness`).
|
||||
- Whether to include advisory key/linkset ids in the blocked response.
|
||||
|
||||
## Handoff
|
||||
Use this as the PREP artefact for WEB-AIRGAP-57-001. Update once 56-002 and error envelope standard are finalized.
|
||||
Reference in New Issue
Block a user