Add new features and tests for AirGap and Time modules

- Introduced `SbomService` tasks documentation. - Updated `StellaOps.sln` to include new projects: `StellaOps.AirGap.Time` and `StellaOps.AirGap.Importer`. - Added unit tests for `BundleImportPlanner`, `DsseVerifier`, `ImportValidator`, and other components in the `StellaOps.AirGap.Importer.Tests` namespace. - Implemented `InMemoryBundleRepositories` for testing bundle catalog and item repositories. - Created `MerkleRootCalculator`, `RootRotationPolicy`, and `TufMetadataValidator` tests. - Developed `StalenessCalculator` and `TimeAnchorLoader` tests in the `StellaOps.AirGap.Time.Tests` namespace. - Added `fetch-sbomservice-deps.sh` script for offline dependency fetching.
2025-11-20 23:29:54 +02:00
parent 65b1599229
commit 79b8e53441
182 changed files with 6660 additions and 1242 deletions
--- a/docs/modules/policy/TASKS.md
+++ b/docs/modules/policy/TASKS.md
@@ -2,5 +2,8 @@

 | Task ID | State | Notes |
 | --- | --- | --- |
+| `PREP-EXPORT-CONSOLE-23-001` | DOING (2025-11-20) | Drafted export bundle + scheduler job contract (see `docs/modules/policy/design/export-console-bundle-contract.md`); waiting on DSSE/storage decisions from Console/Scheduler/Authority. |
+| `PREP-POLICY-AIRGAP-56-001` | DOING (2025-11-20) | Drafted mirror bundle schema for air-gap/ sealed mode (see `docs/modules/policy/design/policy-mirror-bundle-schema.md`); waiting on trust-root and retention policy decisions. |
+| `PREP-POLICY-ENGINE-30-001` | DOING (2025-11-20) | Drafted overlay projection contract (see `docs/modules/policy/design/policy-overlay-projection.md`); waiting on 29-004 metrics/log schema from Platform/Observability. |
 | `SCANNER-POLICY-0001` | DONE (2025-11-10) | Ruby component predicates implemented in engine/tests, DSL docs updated, offline kit verifies `seed-data/analyzers/ruby/git-sources`. |
 | `DOCS-AIAI-31-006` | DONE (2025-11-13) | Published `docs/policy/assistant-parameters.md` capturing Advisory AI configuration knobs (inference/guardrails/cache/queue) and linked it from the module architecture dossier. |
--- a/docs/modules/policy/design/export-console-bundle-contract.md
+++ b/docs/modules/policy/design/export-console-bundle-contract.md
@@ -0,0 +1,55 @@
+# Export Console Bundle Contract (Draft) — PREP-EXPORT-CONSOLE-23-001
+
+Status: Draft (2025-11-20)
+Owners: Policy Guild · Scheduler Guild · Observability Guild
+Scope: Define the evidence bundle exported by Policy Engine for Console replay/verification and the scheduler job contract that produces it.
+
+## 1) Manifest format
+- Container: tar.gz (deterministic ordering, UTC mtime `1970-01-01T00:00:00Z`), content-addressed by SHA-256 of archive bytes.
+- Root manifest: `bundle.manifest.json` (unsigned) and `bundle.manifest.dsse` (signed, optional until Rekor/Authority hookup).
+- `bundle.manifest.json` fields:
+  - `bundle_id` (string, required): GUID/ULID assigned by Policy Engine.
+  - `policy_run_id` (string, required): ID of originating policy run.
+  - `tenant_id` (string, required): tenant scope.
+  - `policy_version` (string, required): semantic version or commit hash of the policy set evaluated.
+  - `inputs_hash` (hex): SHA-256 of normalized inputs (SBOM, advisories, VEX) used for the run.
+  - `generated_at` (string, RFC3339 UTC): timestamp of bundle creation.
+  - `schema_version` (string): `policy.export.console.v1`.
+  - `artifacts`: array of artefact descriptors with `{path, media_type, sha256, purpose}`; expected purposes: `policy-run-summary`, `effective-findings`, `inputs`, `logs`, `metrics`.
+  - `signatures`: optional array referencing DSSE statement(s) with signer identity and key hint.
+
+## 2) Artefact layout
+- `summary/policy-run.json`: minimal run summary (status, duration, rule counts, decision stats).
+- `findings/effective.ndjson`: deterministic NDJSON list of effective findings emitted by the run (sorted by `finding_id`).
+- `inputs/sbom/*.json`: normalized SBOM slices (content-addressed references pointing back to source digests).
+- `inputs/advisories/*.json`: advisories snapshot used during evaluation.
+- `inputs/vex/*.json`: VEX statements snapshot.
+- `telemetry/logs.ndjson`: structured logs with correlation IDs and runner metadata (no secrets, scrub file paths).
+- `telemetry/metrics.json`: counters/histograms for run (p50/p95 latency, evaluated rules, facts ingested).
+
+## 3) Scheduler job contract
+- Job type: `policy.export.console`.
+- Inputs:
+  - `policy_run_id` (required)
+  - `tenant_id` (required)
+  - `priority` (enum: background|interactive; default background)
+  - `include_logs` (bool, default true)
+- Outputs:
+  - `bundle_pointer` (URI/path) to stored bundle in Evidence Locker/Export Center bucket.
+  - `bundle_id`, `sha256`.
+- Events:
+  - Progress events to Observability stream `{job_id, state, percent, bundle_id?, error?}`.
+  - Completion emits `policy.export.console.completed` with bundle metadata for Console ingestion.
+
+## 4) Determinism & security
+- All NDJSON/JSON sorted by stable keys; floats avoided; timestamps UTC RFC3339.
+- No inline PII; tenant-scoped file paths must be redacted or normalized.
+- Offline-friendly: no external references; all URIs either `bundle://` or content hashes.
+
+## 5) Open items / decisions needed
+- Signer identity and DSSE profile (Rekor optional) — awaiting Authority/Attestor alignment.
+- Exact location for bundle storage (Evidence Locker vs Export Center) — propose Evidence Locker namespace `policy-console/` with retention 30d.
+- Metrics schema: align with Observability guild’s latest counters/histogram names.
+
+## 6) Handoff
+Cite this file in sprint trackers for PREP-EXPORT-CONSOLE-23-001. When upstream decisions land, update schema_version and finalize DSSE profile.
--- a/docs/modules/policy/design/policy-deterministic-evaluator.md
+++ b/docs/modules/policy/design/policy-deterministic-evaluator.md
@@ -0,0 +1,27 @@
+# Deterministic Evaluator Spec (Draft) — PREP-POLICY-ENGINE-20-002
+
+Status: Draft (2025-11-20)
+Owners: Policy Guild
+Scope: Define deterministic evaluator requirements for Policy Engine to unblock POLICY-ENGINE-20-002.
+
+## Determinism rules
+- Evaluation order: lexical by rule ID within policy set; stable tie-breaker by condition hash.
+- Data types: disallow host clock/network; only allow provided inputs; random seeded with fixed seed per run.
+- Outputs: sorted by finding_id; numeric outputs with fixed precision (3 decimals) and invariant culture.
+- Time handling: all timestamps in inputs are treated as UTC; now() not permitted.
+
+## Engine contract
+- Input envelope: `{policy_set_id, policy_version, inputs_hash, tenant_id, run_id}`.
+- Execution context: immutable; no mutations to global state; cache keyed by inputs_hash + policy_version.
+- Result envelope: `{run_id, policy_set_id, policy_version, findings[], metrics{duration_ms, rules_evaluated}}`.
+
+## Testing strategy
+- Golden runs: same inputs produce identical outputs/hashes across runs and machines.
+- Property tests: altering rule order must not change result ordering; injecting clock skew should be rejected.
+
+## Open decisions
+- Whether to allow parallel rule evaluation if output ordering remains deterministic (requires stable merge logic).
+- Exact numeric tolerance/rounding strategy.
+
+## Handoff
+Use this as the PREP artefact for PREP-POLICY-ENGINE-20-002. Update once numeric rounding and parallelism decisions are finalized.
--- a/docs/modules/policy/design/policy-mirror-bundle-schema.md
+++ b/docs/modules/policy/design/policy-mirror-bundle-schema.md
@@ -0,0 +1,46 @@
+# Policy Mirror Bundle Schema (Draft) — PREP-POLICY-AIRGAP-56-001
+
+Status: Draft (2025-11-20)
+Owners: Policy Guild · Cartographer Guild
+Scope: Define sealed/offline-ready mirror bundle schema for policy packs to unblock air-gap ingestion (Mirror Bundles v1).
+
+## 1) Bundle container
+- Format: tar.gz with deterministic ordering, UTC mtime `1970-01-01T00:00:00Z`.
+- Root: `mirror.manifest.json` plus optional `mirror.manifest.dsse` (post-quantum ready profile TBD).
+
+## 2) Manifest fields
+- `bundle_id` (string, required): ULID.
+- `schema_version` (string): `policy.mirror.v1`.
+- `created_at` (RFC3339 UTC).
+- `producer` (object): `{name, version, build}` of the mirror tool.
+- `provenance` (object): `{source_registry, source_digest, fetch_time, trust_roots[]}`.
+- `policies` (array) of policy pack descriptors:
+  - `pack_id` (string): content-addressed ID of the pack archive.
+  - `version` (string): semver or commit hash.
+  - `sha256` (string): hash of pack bytes.
+  - `signatures[]` (optional): detached signatures (Sigstore/DSSE) with key hints.
+  - `metadata` (object): `{rules_count, dependencies, min_engine_version}`.
+- `attestations` (optional array): references to DSSE statements covering manifest or pack artefacts.
+- `integrity` (object): `{merkle_root, entries[]}` mirroring file tree for audit.
+
+## 3) File layout
+- `packs/{pack_id}.tar.gz`: raw policy pack.
+- `signatures/{pack_id}.sig`: detached signature per pack (optional).
+- `notes/compliance.md`: optional compliance notes per pack.
+
+## 4) Sealed-mode rules
+- Whole-bundle DSSE signature required when `sealed=true` (header flag in manifest).
+- No post-fetch mutation; ingest must verify merkle root and DSSE before unpack.
+- Trust roots pinned per bundle (`provenance.trust_roots[]`), defaulting to org-configured roots when absent.
+
+## 5) Determinism & validation
+- All arrays sorted; use lowercase hex for hashes.
+- Validation checklist: bundle hash matches manifest; merkle root recomputes; DSSE signer authorized; engine version compatibility enforced before import.
+
+## 6) Open decisions
+- Final DSSE profile (FIPS/eIDAS/GOST) depending on deployment region.
+- Retention/GC policy for stale mirror bundles in sealed mode.
+- Whether to embed policy docs or keep external references; draft assumes external docs are excluded for air-gap safety.
+
+## 7) Handoff
+Treat this file as the publishable artefact for PREP-POLICY-AIRGAP-56-001. Update when Authority/Platform finalize DSSE and trust-root policies.
--- a/docs/modules/policy/design/policy-overlay-projection.md
+++ b/docs/modules/policy/design/policy-overlay-projection.md
@@ -0,0 +1,55 @@
+# Policy Overlay Projection Contract (Draft) — PREP-POLICY-ENGINE-30-001
+
+Status: Draft (2025-11-20)
+Owners: Policy Guild · Cartographer Guild · Platform/Observability Guild
+Scope: Define the overlay projection output that depends on metrics/logging outputs from POLICY-ENGINE-29-004. Intended to unblock POLICY-ENGINE-30-001 and downstream 30-00x tasks.
+
+## 1) Inputs
+- `policy_run_id` (required)
+- `tenant_id` (required)
+- Metrics/logging envelope from 29-004 (pending): expected fields include run duration, rule evaluation counts, fact ingest counts, cache hit/miss, scheduler job metadata.
+- Optional: advisory/KB versions, SBOM/VEX digests, risk profile version.
+
+## 2) Overlay projection shape (proposed)
+```json
+{
+  "overlay_id": "ulid",
+  "policy_run_id": "...",
+  "tenant_id": "...",
+  "generated_at": "2025-11-20T00:00:00Z",
+  "schema_version": "policy.overlay.v1",
+  "metrics": {
+    "duration_ms": 1234,
+    "rules_evaluated": 4200,
+    "facts_ingested": 98765,
+    "cache_hit_rate": 0.92,
+    "p95_rule_latency_ms": 8
+  },
+  "logs_pointer": "bundle://telemetry/logs.ndjson",
+  "inputs": {
+    "sbom_digest": "sha256:...",
+    "advisories_digest": "sha256:...",
+    "vex_digest": "sha256:..."
+  },
+  "provenance": {
+    "engine_version": "x.y.z",
+    "profile": "policy-default",
+    "scheduler_job_id": "..."
+  }
+}
+```
+- Determinism: sorted keys; timestamps UTC; numeric metrics fixed to 3 decimal places where fractional.
+- Overlay acts as the query surface for simulation/change events (30-002/30-003) and UI overlays.
+
+## 3) Storage & API
+- Stored as NDJSON under `overlays/{tenant_id}/{policy_run_id}.ndjson` in policy engine store; referenced by Export/Console bundle.
+- API (proposed): `GET /policy-runs/{policy_run_id}/overlay` with ETag = sha256 of payload; `POST /policy-runs/{policy_run_id}/overlay/rebuild` for re-projection when metrics contract changes.
+
+## 4) Open dependencies / decisions
+- Need final metrics/logging schema from 29-004 to lock `metrics` section (owner: Platform/Observability).
+- Confirm cache metrics naming and units.
+- Confirm whether overlay should embed inline logs vs pointer.
+- Clarify retention/GC policy for overlays (suggest 30d, aligned with export bundles).
+
+## 5) Handoff
+Use this document as the PREP artefact for POLICY-ENGINE-30-001. Update once 29-004 publishes metrics/logging outputs; then fix schema_version to `overlay.v1` and add JSON Schema under `docs/modules/policy/schemas/`.
--- a/docs/modules/policy/prep/2025-11-20-riskprofile-66-001-prep.md
+++ b/docs/modules/policy/prep/2025-11-20-riskprofile-66-001-prep.md
@@ -0,0 +1,16 @@
+# Risk Profile Library Prep — PREP-POLICY-RISK-66-001
+
+Status: Draft (2025-11-20)
+Owners: Risk Profile Schema Guild
+Scope: Scaffold requirements for `StellaOps.Policy.RiskProfile` project and storage contract.
+
+## Needs
+- Project skeleton (net10.0) under `src/Policy/StellaOps.Policy.RiskProfile` with schema/validators.
+- Storage contract: collections/fields for risk profiles; deterministic hashing rules.
+
+## Open decisions
+- Exact schema fields and validation rules.
+- Versioning strategy (semver vs commit hash) for profiles.
+
+## Handoff
+Use as PREP artefact; update once schema fields and storage strategy are agreed.