feat: Add UI benchmark driver and scenarios for graph interactions

- Introduced `ui_bench_driver.mjs` to read scenarios and fixture manifest, generating a deterministic run plan. - Created `ui_bench_plan.md` outlining the purpose, scope, and next steps for the benchmark. - Added `ui_bench_scenarios.json` containing various scenarios for graph UI interactions. - Implemented tests for CLI commands, ensuring bundle verification and telemetry defaults. - Developed schemas for orchestrator components, including replay manifests and event envelopes. - Added mock API for risk management, including listing and statistics functionalities. - Implemented models for risk profiles and query options to support the new API.
2025-12-02 01:28:17 +02:00
parent 909d9b6220
commit 44171930ff
94 changed files with 3606 additions and 271 deletions
--- a/docs/modules/cli/contracts/cli-spec-v1.yaml
+++ b/docs/modules/cli/contracts/cli-spec-v1.yaml
@@ -0,0 +1,63 @@
+version: 1
+generated: 2025-12-01T00:00:00Z
+compatibility:
+  policy: "SemVer-like: commands/flags/exitCodes are backwards compatible within major version."
+  deprecation:
+    noticeMinimumDays: 90
+    channels:
+      - release-notes
+      - --compat-report
+commands:
+  - name: advise
+    subcommands:
+      - name: summarize
+        formats: [json, markdown, table]
+        exitCodes:
+          0: success
+          2: validation-error
+          3: backend-unavailable
+      - name: explain
+        formats: [json, markdown, table]
+        exitCodes:
+          0: success
+          2: validation-error
+          3: backend-unavailable
+      - name: remediate
+        flags:
+          - name: strategy
+            required: false
+            values: [minimal, defense-in-depth, fast-track]
+        exitCodes:
+          0: success
+          2: validation-error
+          3: backend-unavailable
+  - name: auth
+    subcommands:
+      - name: doctor
+        exitCodes:
+          0: success
+          4: auth-misconfigured
+          5: token-invalid
+telemetry:
+  defaultEnabled: false
+  envVars:
+    optIn: STELLAOPS_TELEMETRY=1
+    optOut: STELLAOPS_TELEMETRY=0
+  persistField: telemetryEnabled
+install:
+  checksumRequired: true
+  cosignVerifyDefault: true
+  exitCodes:
+    21: checksum-file-missing
+    22: checksum-mismatch
+buildxPlugin:
+  imageDigest: "sha256:0000000000000000000000000000000000000000000000000000000000000000"
+  rollbackCommand: "stella tool buildx rollback --to <digest>"
+determinism:
+  locale: "en-US"
+  timezone: "UTC"
+  jsonFormatting: "stable-sort-keys"
+  tableWidth: 80
+tests:
+  - name: cli-compatibility-regression
+    description: "Ensure commands/flags/exit codes match spec and telemetry defaults are enforced."
--- a/docs/modules/cli/contracts/install-integrity.md
+++ b/docs/modules/cli/contracts/install-integrity.md
@@ -0,0 +1,15 @@
+# CLI Install & Update Integrity (v1) — 2025-12-01
+
+Requirements
+- Checksums: Every release publishes `stellaops-cli-$version.tar.zst` with `SHA256SUMS` + detached `.sig`.
+- Verification: `stella install` and `stella self-update` run `cosign verify` by default against pinned public key fingerprint; `--skip-verify` prohibited.
+- Offline: Provide `install-offline.sh` that reads from kit directory with checksum + signature checks only; no network fetches.
+- Buildx plugin: pin image digest (see `cli-spec-v1.yaml`); rollback command included in help.
+
+Failure modes
+- Missing checksum/signature → command fails with exit code 21 and structured error.
+- Digest mismatch → command fails with exit code 22; log path to offending file.
+
+Artifacts
+- Public key fingerprints recorded in `cli-spec-v1.yaml`.
+- Example verify script to be bundled in release kit: `scripts/cli/verify-install.sh`.
--- a/docs/modules/cli/contracts/output-determinism.md
+++ b/docs/modules/cli/contracts/output-determinism.md
@@ -0,0 +1,19 @@
+# CLI Output Determinism Policy (v1) — 2025-12-01
+
+Scope: `StellaOps.Cli` JSON/table/markdown outputs for advisory verbs and auth doctor.
+
+Rules
+- Time: All timestamps UTC; no local timezone conversion.
+- Locale: `en-US`, `InvariantCulture` for number/date formatting.
+- Ordering: Sort collections by stable key (id/name) before rendering; JSON keys stable-sorted.
+- Width: Table renderer clamps to width 80; no ANSI when `--output json` or non-TTY.
+- Seeds: Randomness forbidden; no wall-clock in hashes; use provided deterministic IDs.
+
+Tests
+- Golden fixtures stored under `src/Cli/__Tests/StellaOps.Cli.Tests/Fixtures/output-determinism/`.
+- Hash check: two consecutive runs of the same command with identical inputs must produce identical SHA256 of stdout.
+- Locale guard: integration test forces `CultureInfo("fr-FR")` and asserts output matches fixtures.
+
+Failure handling
+- Any drift fails CI; diff is printed with unified format.
+- Add new fields behind explicit versioned spec entry in `cli-spec-v1.yaml`.
--- a/docs/modules/scanner/readiness-checkpoints.md
+++ b/docs/modules/scanner/readiness-checkpoints.md
@@ -0,0 +1,26 @@
+# Scanner Readiness Checkpoints (as of 2025-12-01)
+
+## Snapshot
+- Scope: scanner/surface sprints 0131–0138.
+- Status legend: **Green** = shipped and validated; **Amber** = shipped but validation blocked or partial; **Red** = not shipped/blocked upstream.
+
+## Phase Readiness
+| Phase / Sprint | Status | Evidence | Gaps / Actions |
+| --- | --- | --- | --- |
+| Phase II · Sprint 0131 (Deno/Java/.NET bootstrap) | Amber/Red | Deno runtime capture shipped and tested; Java chain 21-005..011 blocked on Concelier build + CI runner; .NET Lang 11-001 blocked awaiting clean runner; PHP VFS 27-001 blocked pending bootstrap spec. | Need CI slice (DEVOPS-SCANNER-CI-11-001) for Java/.NET; define PHP bootstrap spec and fixtures to unblock 27-001. |
+| Phase III · Sprint 0132 (Native + Node foundations) | Amber | Native analyzers 20-001..010 shipped with tests; Node 22-001..005 shipped; Node isolated/CI tests pending due to build graph bloat; .NET Lang 11-002..005 blocked on upstream design 11-001 outputs. | Trim Node test graph or run on clean runner to record pass; unblock .NET analyzer design to proceed with runtime/export/fixtures. |
+| Phase IV · Sprint 0133 (Node bundle/source-map) | Amber | Phase22 bundle/native/WASM observation implemented and fixtures hashed; validation tests pending (SDK resolver cancels build on current runner). | Execute `scripts/run-node-phase22-smoke.sh` on clean runner; capture TRX/binlog to close. |
+| Phase V · Sprint 0134 (PHP fixtures/runtime/package) | Green | PHP analyzer fixtures, runtime evidence, and packaging shipped; docs updated. | Keep fixture hashes stable; rerun benchmarks when dependencies change. |
+| Phase VI · Sprint 0135 (Python container + Ruby VFS/edges) | Green | Python container/zipapp adapters shipped; Ruby VFS/dependency edges/observations/runtime capture packaged; EntryTrace 18-502/503 delivered. | Maintain determinism; re-run EntryTrace suite in CI. |
+| Phase VII · Sprint 0136 (EntryTrace surface/CLI) | Green | EntryTrace phase VII tasks 18-504/505/506 completed; CLI/WebService surfaces show best-terminal metadata and confidence. | Keep NDJSON schema stable; rerun worker payload tests in CI. |
+| Sprint 0138 (Ruby parity & future analyzers) | Amber/Red | Ruby parity shipped; Mongo package inventory live. PHP pipeline SCANNER-ENG-0010 blocked on composer/autoload design + restore stability; Deno/Dart/Swift analyzer scopes blocked awaiting design; Kubernetes/VM roadmap pending. | Resolve PHP restore/design, produce Deno/Dart/Swift scopes, schedule Zastava/Runtime alignment. |
+
+## Overall
+- Green areas: native analyzers, PHP fixtures/runtime packaging, Ruby analyzer, Python container adapters, EntryTrace phases VI–VII.
+- Amber/Red drivers: Java/.NET analyzer chains (CI/design dependencies), PHP pipeline (0138), Node validation on clean runner, design gaps for Deno/Dart/Swift, PHP VFS bootstrap (0131).
+
+## Recommended Next Actions
+1) Secure clean CI slice for Java/.NET and Node Phase22 smoke tests; store binlogs/TRX.
+2) Finalise PHP analyzer design (composer/autoload graph) and stabilise restore pipeline to unblock SCANNER-ENG-0010/27-001.
+3) Publish Deno/Dart/Swift analyzer scopes with fixtures to unblock 0138 tasks and roadmap alignment with Zastava/Runtime.
+4) Re-run EntryTrace and Native suites in CI to lock deterministic hashes before downstream release.
--- a/docs/modules/signals/SHA256SUMS
+++ b/docs/modules/signals/SHA256SUMS
@@ -0,0 +1,8 @@
+170892f6a48b0aef6f426ea97a86f6cd4420bc52634f12a92f72e20f0fa12e29  decay/confidence_decay_config.yaml
+450675035928e4771cca1b9e5f9e42035dbe10b3de7b66a4077a7b729b2c5b13  unknowns/unknowns_scoring_manifest.json
+e33fa0963493252a5ac379a12f820f6b356ea94310afd1db9ad7394e8307000e  heuristics/heuristics.catalog.json
+6e2e6dfeeb4ae016b7ae881d4653ab79a3babba28a4c6d072266e81c61366e2c  heuristics/heuristics.schema.json
+99b1a4abc941bea5d4ee6b1dbd6faea37de9c67474bb1bac5f23570a44beff17  heuristics/fixtures/heur.callgraph.hotpath/input.json
+851452aeac775e71d7507b70e9b7eb05bc619b37ba656a31b53fb3a504fb3b4a  heuristics/fixtures/heur.callgraph.hotpath/expected.json
+4d0f69b6064df2014f20673feda7881f815b489ab22e0a219301514b766b29d1  heuristics/fixtures/heur.pkg.sbom_age/input.json
+bd9e5b2081e7fe9d947e7a44b0c493efd4d8e97d4c13ec4efd577869559fb1c2  heuristics/fixtures/heur.pkg.sbom_age/expected.json
--- a/docs/modules/signals/decay/2025-12-01-confidence-decay.md
+++ b/docs/modules/signals/decay/2025-12-01-confidence-decay.md
@@ -0,0 +1,79 @@
+# Confidence Decay Controls · Signals Runtime
+
+**Compiled:** 2025-12-01 (UTC)
+**Scope:** Close U1–U10 gaps from `docs/product-advisories/31-Nov-2025 FINDINGS.md` for confidence decay of unknowns/signals.
+**Status:** Draft for review on 2025-12-03; to be signed (DSSE) after sign-off.
+
+## Decisions (U1–U10)
+- **τ governance (U1):** All τ values live in `confidence_decay_config.yaml`, change-controlled via DSSE-signed PRs; allowable τ range 1–90 days. Changes require dual approval (Signals + Policy), recorded in history.
+- **Floor / freeze (U2):** `confidence_floor` per severity; `is_confidence_frozen=true` when SLA-bound or manual pin. Floors: Critical 0.60, High 0.45, Medium 0.30, Low 0.20. Freeze auto-expires at `freeze_until`.
+- **Weighted signals (U3):** Signal taxonomy with weights: exploit=1.0, customer_incident=0.9, threat_intel=0.7, code_change=0.4, artifact_refresh=0.3, metadata_touch=0.1. `last_signal_weighted_at` uses max(weighted timestamp).
+- **Time source / drift (U4):** All timestamps in UTC; decay uses monotonic clock fallback; reject events >5 minutes in the future or >30 days backdated, log corrections.
+- **Deterministic recompute (U5):** Nightly job at 03:00 UTC recomputes decay for all items; emits `decay_snapshot_YYYY-MM-DD.ndjson` with SHA256 and checksum record. On-read recompute only if snapshot is older than 24h.
+- **SLA coupling (U6):** Items with active SLA clamp to `sla_floor` (0.60 Critical, 0.50 High) until SLA met. SLA flag and floor are emitted in API.
+- **Uncertainty linkage (U7):** Confidence is capped by `(1 - uncertainty_score)`; if uncertainty_score ≥0.4, band forced to "under_review" and alerts fire.
+- **Backfill & migration (U8):** Initial migration seeds `last_signal_at` from latest activity; default τ from entity profile; dry-run impact report required; backfill script outputs before/after bands.
+- **API/UX surfacing (U9):** New fields: `confidence`, `confidence_band` (critical/high/medium/low/under_review), `tau_days`, `is_frozen`, `confidence_floor`, `uncertainty_score`, `last_signal_weighted_at`. Sort default: `priority * confidence`.
+- **Observability & alerts (U10):** Counters/gauges: `confidence_recalc_latency`, `items_below_floor`, `signals_weighted_by_type{type}`, `decay_snapshots_age_hours`, `uncertainty_forced_under_review`. Alerts on missing nightly snapshot, decay drift >1 band, or SLA items below floor.
+
+## Reference Config (draft)
+```yaml
+version: 1
+updated_at: 2025-12-01T00:00:00Z
+entities:
+  vulnerability:
+    tau_days: 21
+    tau_min: 7
+    tau_max: 90
+    confidence_floor: {critical: 0.60, high: 0.45, medium: 0.30, low: 0.20}
+    sla_floor: {critical: 0.60, high: 0.50}
+    freeze_default_days: 30
+  incident:
+    tau_days: 14
+    tau_min: 3
+    tau_max: 60
+signals_taxonomy:
+  exploit: 1.0
+  customer_incident: 0.9
+  threat_intel: 0.7
+  code_change: 0.4
+  artifact_refresh: 0.3
+  metadata_touch: 0.1
+time:
+  reject_future_minutes: 5
+  reject_backdated_days: 30
+recompute:
+  schedule_utc: "03:00"
+  snapshot_retention_days: 30
+observability:
+  alerts:
+    missing_snapshot_hours: 26
+    sla_floor_breach: true
+    uncertainty_band_force: 0.4
+signing:
+  predicate: stella.ops/confidenceDecayConfig@v1
+  dsse_required: true
+```
+
+## Operational Rules
+- Config changes must produce a new DSSE envelope and update the checksum in the nightly snapshot header.
+- Nightly job writes `decay_snapshot_<date>.ndjson` (sorted by `item_id`) plus `SHA256SUMS`; both stored in Evidence Locker.
+- Any on-read recompute must emit an audit log with reasons (stale snapshot or forced recalculation).
+
+## Migration Playbook
+1) Run dry-run backfill: compute bands with proposed config; write `decay_backfill_diff.ndjson` (before/after bands, delta) and checksum.
+2) Get dual approval; sign `confidence_decay_config.yaml` with DSSE predicate above.
+3) Apply config, execute full recompute, publish snapshot + checksums, update observability dashboard baselines.
+
+## API Notes
+- Add fields to Signals API and CLI responses; ensure canonical serialization (sorted keys, UTC timestamps, fixed decimals 3dp) to avoid hash drift.
+- Bands map: `>=0.75 critical`, `>=0.55 high`, `>=0.35 medium`, `>=0.20 low`, else `under_review`.
+
+## Evidence & Storage
+- Store config DSSE, snapshots, and backfill reports in Evidence Locker with retention class `signals-decay-config`.
+- For offline kits, include latest config DSSE + last 3 snapshots and checksums.
+
+## Open Items for Review (12-03)
+- Confirm weights for threat_intel vs exploit; adjust if customer data suggests different ordering.
+- Confirm `under_review` threshold (currently uncertainty ≥0.4).
+- Align with Policy on SLA floors for High severity (0.50 proposed).
--- a/docs/modules/signals/decay/confidence_decay_config.yaml
+++ b/docs/modules/signals/decay/confidence_decay_config.yaml
@@ -0,0 +1,41 @@
+version: 1
+updated_at: 2025-12-01T00:00:00Z
+entities:
+  vulnerability:
+    tau_days: 21
+    tau_min: 7
+    tau_max: 90
+    confidence_floor:
+      critical: 0.60
+      high: 0.45
+      medium: 0.30
+      low: 0.20
+    sla_floor:
+      critical: 0.60
+      high: 0.50
+    freeze_default_days: 30
+  incident:
+    tau_days: 14
+    tau_min: 3
+    tau_max: 60
+signals_taxonomy:
+  exploit: 1.0
+  customer_incident: 0.9
+  threat_intel: 0.7
+  code_change: 0.4
+  artifact_refresh: 0.3
+  metadata_touch: 0.1
+time:
+  reject_future_minutes: 5
+  reject_backdated_days: 30
+recompute:
+  schedule_utc: "03:00"
+  snapshot_retention_days: 30
+observability:
+  alerts:
+    missing_snapshot_hours: 26
+    sla_floor_breach: true
+    uncertainty_band_force: 0.4
+signing:
+  predicate: stella.ops/confidenceDecayConfig@v1
+  dsse_required: true
--- a/docs/modules/signals/heuristics/2025-12-01-heuristic-catalog.md
+++ b/docs/modules/signals/heuristics/2025-12-01-heuristic-catalog.md
@@ -0,0 +1,66 @@
+# Signals Heuristic Catalog · Deterministic Scoring
+
+**Compiled:** 2025-12-01 (UTC)
+**Scope:** Close UT1–UT10 gaps from `docs/product-advisories/31-Nov-2025 FINDINGS.md` by publishing a signed heuristic catalog and golden outputs.
+**Status:** Draft; target publish 2025-12-05 with DSSE signature.
+
+## Decisions (UT1–UT10)
+- **Signed catalog/schema (UT1):** Catalog lives at `heuristics.catalog.json` with schema versioned `heuristics.schema.json`; DSSE predicate `stella.ops/heuristicCatalog@v1` required.
+- **Deterministic scoring formula (UT2):** Each heuristic defines `inputs`, `weights`, and `normalization`; scoring outputs canonicalized (sorted keys, fixed 3dp). Engine must be pure/deterministic; randomization forbidden.
+- **Quality bands (UT3):** Bands: `gold` (precision≥0.9, recall≥0.8), `silver` (≥0.8/0.7), `bronze` (≥0.7/0.6). Bands recorded in catalog and enforced in admission checks.
+- **Waiver policy with DSSE (UT4):** Waivers require DSSE envelope `stella.ops/heuristicWaiver@v1`, include reason, scope, expiry; dual approval (Signals+Policy).
+- **SLA coupling (UT5):** SLA-tagged items cannot use heuristics below `silver`; SLA enforcement checks band before accepting results.
+- **Offline kit packaging (UT6):** Catalog, schema, golden fixtures, and DSSE envelopes bundled in offline kits with `SHA256SUMS`.
+- **Observability/alerts (UT7):** Metrics: `heuristics_eval_latency`, `heuristics_band_usage`, `heuristics_waivers_total`, `heuristics_score_drift`. Alerts when drift >1 band vs golden fixtures or when waivers exceed threshold.
+- **Backfill plan (UT8):** Backfill job recomputes heuristic scores with current catalog; outputs `heuristics_backfill.ndjson` + checksum; mismatches raise alerts.
+- **Explainability fields/exports (UT9):** Outputs must include `explanation` block: contributing signals with weights, normalized scores, and rule IDs. CLI/API export supports `--explain` and deterministic ordering.
+- **Fixtures with golden outputs (UT10):** Golden set per heuristic under `fixtures/<heuristic>/` containing `input.json`, `expected.json`, and `README`; used in CI for determinism.
+
+## Catalog Structure (draft)
+```json
+{
+  "version": "1.0.0",
+  "updatedAt": "2025-12-01T00:00:00Z",
+  "heuristics": [
+    {
+      "id": "heur.callgraph.hotpath",
+      "band": "gold",
+      "inputs": ["callgraph.depth", "callgraph.betweenness"],
+      "formula": "0.6*depth_norm + 0.4*betweenness_norm",
+      "normalization": "minmax",
+      "evidence": ["signals/callgraph"]
+    },
+    {
+      "id": "heur.pkg.sbom_age",
+      "band": "silver",
+      "inputs": ["sbom.age_days", "release_channel"],
+      "formula": "if release_channel=='stable' then age_norm else 0.8*age_norm",
+      "normalization": "log1p"
+    }
+  ],
+  "signing": {
+    "predicate": "stella.ops/heuristicCatalog@v1",
+    "dsse_required": true
+  }
+}
+```
+
+## Golden Fixtures (layout)
+- `docs/modules/signals/heuristics/fixtures/heur.callgraph.hotpath/{input.json,expected.json}`
+- `docs/modules/signals/heuristics/fixtures/heur.pkg.sbom_age/{input.json,expected.json}`
+- `expected.json` must be canonicalized (sorted keys, fixed 3dp) and include explanation block.
+
+## CI / Determinism Checks
+- Lint: reject heuristics without band or DSSE signature.
+- Determinism test: run golden fixtures; fail if output hash differs.
+- Drift alert: compare live scores vs golden baselines; trigger if >1 band difference for same input hash.
+
+## Publish Steps (12-05)
+1) Finalize catalog + schema; canonicalize via JCS; sign DSSE envelope.
+2) Populate fixtures and compute `SHA256SUMS` for all files.
+3) Update sprint doc status and Evidence Locker with catalog + fixtures + signatures.
+4) Enable observability dashboards and waiver policy checks.
+
+## Open Items
+- Confirm minimum band allowed for non-SLA items (proposal: bronze acceptable, but not for SLA).
+- Decide on additional heuristics for runtime traces vs SBOM freshness.
--- a/docs/modules/signals/heuristics/fixtures/heur.callgraph.hotpath/expected.json
+++ b/docs/modules/signals/heuristics/fixtures/heur.callgraph.hotpath/expected.json
@@ -0,0 +1,16 @@
+{
+  "heuristicId": "heur.callgraph.hotpath",
+  "score": 0.472,
+  "band": "gold",
+  "explanation": {
+    "inputs": {
+      "callgraph.depth": 7,
+      "callgraph.betweenness": 0.12
+    },
+    "normalized": {
+      "depth_norm": 0.700,
+      "betweenness_norm": 0.120
+    },
+    "formula": "0.6*depth_norm + 0.4*betweenness_norm"
+  }
+}
--- a/docs/modules/signals/heuristics/fixtures/heur.callgraph.hotpath/input.json
+++ b/docs/modules/signals/heuristics/fixtures/heur.callgraph.hotpath/input.json
@@ -0,0 +1,4 @@
+{
+  "callgraph.depth": 7,
+  "callgraph.betweenness": 0.12
+}
--- a/docs/modules/signals/heuristics/fixtures/heur.pkg.sbom_age/expected.json
+++ b/docs/modules/signals/heuristics/fixtures/heur.pkg.sbom_age/expected.json
@@ -0,0 +1,16 @@
+{
+  "heuristicId": "heur.pkg.sbom_age",
+  "score": 0.450,
+  "band": "silver",
+  "explanation": {
+    "inputs": {
+      "sbom.age_days": 45,
+      "release_channel": "stable"
+    },
+    "normalized": {
+      "age_norm": 0.450,
+      "release_channel": "stable"
+    },
+    "formula": "case release_channel=='stable' then age_norm else 0.8*age_norm"
+  }
+}
--- a/docs/modules/signals/heuristics/fixtures/heur.pkg.sbom_age/input.json
+++ b/docs/modules/signals/heuristics/fixtures/heur.pkg.sbom_age/input.json
@@ -0,0 +1,4 @@
+{
+  "sbom.age_days": 45,
+  "release_channel": "stable"
+}
--- a/docs/modules/signals/heuristics/heuristics.catalog.json
+++ b/docs/modules/signals/heuristics/heuristics.catalog.json
@@ -0,0 +1,26 @@
+{
+  "version": "1.0.0",
+  "updatedAt": "2025-12-01T00:00:00Z",
+  "heuristics": [
+    {
+      "id": "heur.callgraph.hotpath",
+      "band": "gold",
+      "inputs": ["callgraph.depth", "callgraph.betweenness"],
+      "formula": "0.6*depth_norm + 0.4*betweenness_norm",
+      "normalization": "minmax",
+      "evidence": ["signals/callgraph"]
+    },
+    {
+      "id": "heur.pkg.sbom_age",
+      "band": "silver",
+      "inputs": ["sbom.age_days", "release_channel"],
+      "formula": "case release_channel=='stable' then age_norm else 0.8*age_norm",
+      "normalization": "log1p",
+      "evidence": ["sbom/age"]
+    }
+  ],
+  "signing": {
+    "predicate": "stella.ops/heuristicCatalog@v1",
+    "dsse_required": true
+  }
+}
--- a/docs/modules/signals/heuristics/heuristics.schema.json
+++ b/docs/modules/signals/heuristics/heuristics.schema.json
@@ -0,0 +1,32 @@
+{
+  "$id": "https://stella-ops.org/schemas/heuristics.schema.json",
+  "type": "object",
+  "required": ["version", "updatedAt", "heuristics", "signing"],
+  "properties": {
+    "version": {"type": "string"},
+    "updatedAt": {"type": "string", "format": "date-time"},
+    "heuristics": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["id", "band", "inputs", "formula", "normalization"],
+        "properties": {
+          "id": {"type": "string"},
+          "band": {"enum": ["gold", "silver", "bronze"]},
+          "inputs": {"type": "array", "items": {"type": "string"}},
+          "formula": {"type": "string"},
+          "normalization": {"type": "string"},
+          "evidence": {"type": "array", "items": {"type": "string"}}
+        }
+      }
+    },
+    "signing": {
+      "type": "object",
+      "required": ["predicate", "dsse_required"],
+      "properties": {
+        "predicate": {"type": "string"},
+        "dsse_required": {"type": "boolean"}
+      }
+    }
+  }
+}
--- a/docs/modules/signals/unknowns/2025-12-01-unknowns-registry.md
+++ b/docs/modules/signals/unknowns/2025-12-01-unknowns-registry.md
@@ -0,0 +1,86 @@
+# Unknowns Registry & Scoring Manifest
+
+**Compiled:** 2025-12-01 (UTC)
+**Scope:** Close UN1–UN10 gaps from `docs/product-advisories/31-Nov-2025 FINDINGS.md` for Unknowns Registry.
+**Status:** Draft; review 2025-12-04; DSSE signing required before adoption.
+
+## Decisions (UN1–UN10)
+- **Canonical schema/enums (UN1):** Unknown types: `vulnerability`, `asset`, `signal`, `evidence-gap`, `policy-gap`. Status enums: `new`, `triaging`, `under_review`, `validated`, `dismissed`. Severity: `critical/high/medium/low/none`.
+- **Deterministic scoring manifest (UN2):** Manifest `unknowns_scoring_manifest.json` defines inputs, weights, and canonical serialization (JCS, sorted keys, UTC timestamps, fixed 3dp). Hash used as `scoringManifestHash` in API/DSSE.
+- **Decay policy catalog (UN3):** Unknowns reuse `confidence_decay_config` but may override τ by type (see table). Overrides stored in manifest; DSSE-signed.
+- **Evidence/provenance capture (UN4):** Each unknown must reference Evidence Locker URIs with DSSE envelopes; minimal evidence: `{source, observedAt, evidenceType, hash}`. Provenance includes tool identity and policy hash.
+- **SBOM/VEX linkage (UN5):** Unknown links: `sbomDigest`, `vexDecisionId` (if present), `reachabilityGraphHash`. If absent, status forced to `under_review`.
+- **SLA / suppression rules (UN6):** SLA timers mirror severity; suppression requires dual sign-off and DSSE note with expiry. Suppressed items emit `suppression_reason`, `expiresAt`.
+- **API/CLI contracts (UN7):** New endpoints `/unknowns` support filter by `status`, `type`, `confidence_band`, `uncertainty_score`, `suppressed`. CLI mirrors with `--format ndjson` and `--include-provenance` flags. Output sorted deterministically by `createdAt, id`.
+- **Observability/reporting (UN8):** Metrics: `unknowns_total{type,status}`, `unknowns_suppressed_total`, `unknowns_without_sbom`, `unknowns_without_vex`, `unknowns_confidence_band`, `unknowns_manifest_hash_mismatch`. Alerts on manifest hash mismatch, >1% unknowns missing SBOM/VEX, or suppression expiry.
+- **Offline bundle inclusion (UN9):** Include latest manifest, schema, and NDJSON export in offline kit; bundle hashes recorded in kit manifest; verify against DSSE signatures.
+- **Migration/backfill (UN10):** Backfill script `backfill_unknowns_v1` seeds `scoringManifestHash`, `sbomDigest`, and `vexDecisionId` from existing records; produces `unknowns_backfill_report.ndjson` with before/after status/bands and checksum.
+
+## Schema (draft)
+```json
+{
+  "$id": "https://stella-ops.org/schemas/unknown.json",
+  "type": "object",
+  "required": ["id", "type", "status", "severity", "createdAt", "confidence", "confidenceBand"],
+  "properties": {
+    "id": {"type": "string"},
+    "type": {"enum": ["vulnerability", "asset", "signal", "evidence-gap", "policy-gap"]},
+    "status": {"enum": ["new", "triaging", "under_review", "validated", "dismissed"]},
+    "severity": {"enum": ["critical", "high", "medium", "low", "none"]},
+    "confidence": {"type": "number"},
+    "confidenceBand": {"enum": ["critical", "high", "medium", "low", "under_review"]},
+    "uncertaintyScore": {"type": "number", "minimum": 0, "maximum": 1},
+    "tauDays": {"type": "integer"},
+    "sbomDigest": {"type": "string"},
+    "vexDecisionId": {"type": "string"},
+    "reachabilityGraphHash": {"type": "string"},
+    "scoringManifestHash": {"type": "string"},
+    "suppression": {
+      "type": "object",
+      "properties": {
+        "isSuppressed": {"type": "boolean"},
+        "reason": {"type": "string"},
+        "expiresAt": {"type": "string", "format": "date-time"},
+        "signedBy": {"type": "string"}
+      }
+    },
+    "evidence": {"type": "array", "items": {"$ref": "#/definitions/evidenceRef"}},
+    "createdAt": {"type": "string", "format": "date-time"},
+    "updatedAt": {"type": "string", "format": "date-time"}
+  },
+  "definitions": {
+    "evidenceRef": {
+      "type": "object",
+      "required": ["uri", "hash", "observedAt", "evidenceType"],
+      "properties": {
+        "uri": {"type": "string"},
+        "hash": {"type": "string"},
+        "observedAt": {"type": "string", "format": "date-time"},
+        "evidenceType": {"type": "string"},
+        "provenance": {"type": "string"}
+      }
+    }
+  }
+}
+```
+
+## Scoring Manifest (summary)
+- Inputs: severity weight, decay factor (τ), uncertainty cap, SLA floor, suppression flag, weighted signals timestamp.
+- Formula (deterministic): `confidence = max(floor, min((exp(-Δt/τ) * weight_signal), 1 - uncertainty))`, then clamp by SLA floor if SLA active.
+- Canonicalization: JSON Canonicalization Scheme (JCS); decimals fixed 3dp; UTC ISO-8601 timestamps.
+- Hash: SHA256 over canonical manifest; published as `scoringManifestHash` and signed via DSSE `stella.ops/unknownsScoringManifest@v1`.
+
+## Offline & Evidence
+- Bundle schema, manifest, and latest NDJSON export with `SHA256SUMS` and DSSE envelope for each artifact.
+- Evidence Locker class: `signals-unknowns-manifest` (30d retention minimum).
+
+## Migration Checklist (UN10)
+1) Generate `unknowns_scoring_manifest.json` and sign (DSSE).
+2) Run `backfill_unknowns_v1 --manifest <hash>`; produce report and checksums.
+3) Update API/CLI serializers to include new fields and canonical ordering.
+4) Enable observability dashboards and alerts; verify thresholds.
+
+## Review Questions (12-04)
+- Confirm suppression expiry default (proposal: 30 days).
+- Validate `under_review` trigger when SBOM/VEX missing—keep or allow grace period?
+- Align SLA floors with decay config (Critical 0.60, High 0.50).
--- a/docs/modules/signals/unknowns/unknowns_scoring_manifest.json
+++ b/docs/modules/signals/unknowns/unknowns_scoring_manifest.json
@@ -0,0 +1,48 @@
+{
+  "version": "1.0.0",
+  "updatedAt": "2025-12-01T00:00:00Z",
+  "canonicalization": {
+    "scheme": "JCS",
+    "decimals": 3,
+    "timezone": "UTC"
+  },
+  "inputs": {
+    "severityWeight": {
+      "critical": 1.0,
+      "high": 0.8,
+      "medium": 0.6,
+      "low": 0.4,
+      "none": 0.2
+    },
+    "uncertaintyCap": 1.0,
+    "slaFloor": {
+      "critical": 0.60,
+      "high": 0.50
+    },
+    "tauOverrides": {
+      "vulnerability": 21,
+      "asset": 28,
+      "signal": 14,
+      "evidence-gap": 21,
+      "policy-gap": 30
+    }
+  },
+  "formula": "confidence = max(floor, min(exp(-deltaDays/tau) * severityWeight, 1 - uncertainty))",
+  "floor": 0.20,
+  "uncertaintyThreshold": 0.4,
+  "bands": {
+    "critical": 0.75,
+    "high": 0.55,
+    "medium": 0.35,
+    "low": 0.20,
+    "under_review": 0.0
+  },
+  "hash": {
+    "algorithm": "SHA-256",
+    "value": "TO_BE_SIGNED"
+  },
+  "signing": {
+    "predicate": "stella.ops/unknownsScoringManifest@v1",
+    "dsse_required": true
+  }
+}
--- a/docs/modules/telemetry/AGENTS.md
+++ b/docs/modules/telemetry/AGENTS.md
@@ -1,7 +1,7 @@
 # Telemetry agent guide

 ## Mission
-Telemetry module captures deployment and operations guidance for the shared observability stack (collectors, storage, dashboards).
+Telemetry module captures deployment and operations guidance for the shared observability stack (collectors, storage, dashboards). Telemetry outputs must be deterministic, sealed-mode safe, and verifiable via DSSE/offline bundles.

 ## Advisory Handling
 - Any new/updated advisory triggers immediate doc + sprint updates; no approval.
@@ -16,6 +16,8 @@ Telemetry module captures deployment and operations guidance for the shared obse
 - [Implementation plan](./implementation_plan.md)
 - [Task board](./TASKS.md)
 - [Observability runbook](./operations/observability.md) (offline import friendly)
+- [Telemetry gaps remediation](./contracts/telemetry-gaps-remediation.md)
+- Schemas: `./schemas/telemetry-config.schema.json`, `./schemas/telemetry-bundle.schema.json`

 ## How to get started
 1. Open sprint file `/docs/implplan/SPRINT_*.md` and locate the stories referencing this module.
--- a/docs/modules/telemetry/contracts/telemetry-gaps-remediation.md
+++ b/docs/modules/telemetry/contracts/telemetry-gaps-remediation.md
@@ -0,0 +1,29 @@
+# Telemetry Gap Remediation (TO1–TO10) — v1 · 2025-12-01
+
+Source: `docs/product-advisories/31-Nov-2025 FINDINGS.md` (Telemetry gaps TO1–TO10).
+Scope: telemetry core (collectors/SDK defaults/bundles) across services; applicable to default/forensic/airgap profiles.
+
+## Decisions (mapped to gaps)
+- **TO1 Canonical schemas & hashing**: Published versioned schemas
+  - `telemetry-config.schema.json` for collector/SDK profile configs (signed, canonical JSON, stable ordering)
+  - `telemetry-bundle.schema.json` for offline bundle manifests
+  - Hash recipe: SHA-256 over normalized (UTF-8, LF, sorted keys) JSON; test vectors to follow.
+- **TO2 Provenance & DSSE**: Bundles and profile activations must include DSSE envelope (`*.dsse.json`) with predicate fields: profileHash, collectorVersion, exporters, redactionPolicyUri, cryptoProfile.
+- **TO3 Determinism & sampling stability**: Sampling policies must declare deterministic seed, ordered rules, and backpressure policy. Logs/traces ordered by (timestamp, traceId). Multi-run hash check recommended in CI.
+- **TO4 Sealed mode / egress guards**: Sealed mode blocks all non-loopback exporters unless explicitly allowlisted; DNS pinning required; failure is fail-closed. Seal status recorded as DSSE event.
+- **TO5 Redaction policy & PII tests**: Redaction catalog/allowlist required; bundle must include `redaction-manifest.json` listing rules applied and violations=0. CI must run PII/secret test suite before export.
+- **TO6 Tenant isolation & quotas**: OTLP signals include `tenant.id` and `project.id`; collector routes by tenant pipeline; per-tenant quotas/limits enforced with counters and alerts.
+- **TO7 Forensic triggers governance**: Forensic mode requires dual approval, DSSE activation record, expiry timestamp, and auto-rollback; alert if forensic mode active > configured window.
+- **TO8 Offline bundle schema & verify**: Bundles must follow `telemetry-bundle.schema.json`, created with deterministic tar flags, include hash manifest + DSSE + RFC3161 time-anchor; verifier script provided (`ops/devops/telemetry/verify-telemetry-bundle.sh`).
+- **TO9 Observability of observability**: Add SLOs + alerts for collector/exporter health, queue backpressure, bundle success rate; scheduled self-test emits DSSE result.
+- **TO10 CLI/pack contracts**: CLI/pack contract tracked in `cli-spec-v1.yaml`; telemetry exports must respect exit codes and checksum policy (reuse 21/22 for checksum missing/mismatch).
+
+## Artifacts
+- Schemas: `docs/modules/telemetry/schemas/telemetry-config.schema.json`, `telemetry-bundle.schema.json`.
+- Hash recipe: in-line within schemas (canonical JSON, SHA-256).
+- Verify script: `ops/devops/telemetry/verify-telemetry-bundle.sh`.
+
+## Adoption notes
+- Profile and bundle producers must validate against schemas and sign DSSE envelopes before distribution.
+- Air-gap/forensic profiles MUST set sealed mode and include redaction manifest.
+- CI should add a multi-run hash test for telemetry exporter output and fail on drift.
--- a/docs/modules/telemetry/schemas/telemetry-bundle.schema.json
+++ b/docs/modules/telemetry/schemas/telemetry-bundle.schema.json
@@ -0,0 +1,46 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "title": "StellaOps Telemetry Bundle Manifest",
+  "version": "1.0.0",
+  "type": "object",
+  "required": ["schemaVersion", "bundleId", "createdAt", "artifacts", "manifestHashAlgorithm", "timeAnchor"],
+  "properties": {
+    "schemaVersion": { "type": "string", "const": "1.0.0" },
+    "bundleId": { "type": "string", "format": "uuid" },
+    "createdAt": { "type": "string", "format": "date-time" },
+    "profileHash": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+    "collectorVersion": { "type": "string" },
+    "sealedMode": { "type": "boolean" },
+    "redactionManifest": { "type": "string" },
+    "manifestHashAlgorithm": { "type": "string", "enum": ["sha256"] },
+    "timeAnchor": {
+      "type": "object",
+      "required": ["type", "value"],
+      "properties": {
+        "type": { "type": "string", "enum": ["rfc3161", "roughtime"] },
+        "value": { "type": "string" }
+      }
+    },
+    "artifacts": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["path", "sha256", "mediaType"],
+        "properties": {
+          "path": { "type": "string" },
+          "sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+          "mediaType": { "type": "string" },
+          "size": { "type": "integer", "minimum": 0 }
+        }
+      }
+    },
+    "dsseEnvelope": {
+      "type": "object",
+      "required": ["hash"],
+      "properties": {
+        "hash": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+        "location": { "type": "string" }
+      }
+    }
+  }
+}
--- a/docs/modules/telemetry/schemas/telemetry-config.schema.json
+++ b/docs/modules/telemetry/schemas/telemetry-config.schema.json
@@ -0,0 +1,75 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "title": "StellaOps Telemetry Config",
+  "version": "1.0.0",
+  "type": "object",
+  "required": ["schemaVersion", "profiles"],
+  "properties": {
+    "schemaVersion": { "type": "string", "const": "1.0.0" },
+    "hashAlgorithm": { "type": "string", "enum": ["sha256"] },
+    "profiles": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["name", "collectorVersion", "exporters", "redactionPolicyUri", "sampling"],
+        "properties": {
+          "name": { "type": "string" },
+          "description": { "type": "string" },
+          "collectorVersion": { "type": "string" },
+          "cryptoProfile": { "type": "string" },
+          "sealedMode": { "type": "boolean" },
+          "allowlistedEndpoints": {
+            "type": "array",
+            "items": { "type": "string", "format": "uri" }
+          },
+          "exporters": {
+            "type": "array",
+            "items": {
+              "type": "object",
+              "required": ["type", "endpoint"],
+              "properties": {
+                "type": { "type": "string", "enum": ["otlp", "file", "stdout", "null"] },
+                "endpoint": { "type": "string" },
+                "protocol": { "type": "string", "enum": ["grpc", "http"] },
+                "compression": { "type": "string", "enum": ["none", "gzip"] },
+                "enabled": { "type": "boolean", "default": true }
+              }
+            }
+          },
+          "redactionPolicyUri": { "type": "string", "format": "uri" },
+          "sampling": {
+            "type": "object",
+            "required": ["strategy", "seed"],
+            "properties": {
+              "strategy": { "type": "string", "enum": ["always_on", "always_off", "traceidratio", "tail"] },
+              "seed": { "type": "string", "pattern": "^[0-9a-fA-F]{16}$" },
+              "rules": {
+                "type": "array",
+                "items": {
+                  "type": "object",
+                  "required": ["match", "priority"],
+                  "properties": {
+                    "match": { "type": "string" },
+                    "priority": { "type": "integer", "minimum": 0 },
+                    "sampleRate": { "type": "number", "minimum": 0, "maximum": 1 }
+                  }
+                }
+              }
+            }
+          },
+          "tenantRouting": {
+            "type": "object",
+            "required": ["attribute"],
+            "properties": {
+              "attribute": { "type": "string", "const": "tenant.id" },
+              "quotasPerTenant": {
+                "type": "object",
+                "additionalProperties": { "type": "integer", "minimum": 0 }
+              }
+            }
+          }
+        }
+      }
+    }
+  }
+}