git.stella-ops.org/docs/modules/signals/unknowns/2025-12-01-unknowns-registry.md

# Unknowns Registry & Scoring Manifest

**Compiled:** 2025-12-01 (UTC)
**Scope:** Close UN1–UN10 gaps from `docs/product-advisories/31-Nov-2025 FINDINGS.md` for Unknowns Registry.
**Status:** Draft; review 2025-12-04; DSSE signing required before adoption.

## Decisions (UN1–UN10)
- **Canonical schema/enums (UN1):** Unknown types: `vulnerability`, `asset`, `signal`, `evidence-gap`, `policy-gap`. Status enums: `new`, `triaging`, `under_review`, `validated`, `dismissed`. Severity: `critical/high/medium/low/none`.
- **Deterministic scoring manifest (UN2):** Manifest `unknowns_scoring_manifest.json` defines inputs, weights, and canonical serialization (JCS, sorted keys, UTC timestamps, fixed 3dp). Hash used as `scoringManifestHash` in API/DSSE.
- **Decay policy catalog (UN3):** Unknowns reuse `confidence_decay_config` but may override τ by type (see table). Overrides stored in manifest; DSSE-signed.
- **Evidence/provenance capture (UN4):** Each unknown must reference Evidence Locker URIs with DSSE envelopes; minimal evidence: `{source, observedAt, evidenceType, hash}`. Provenance includes tool identity and policy hash.
- **SBOM/VEX linkage (UN5):** Unknown links: `sbomDigest`, `vexDecisionId` (if present), `reachabilityGraphHash`. If absent, status forced to `under_review`.
- **SLA / suppression rules (UN6):** SLA timers mirror severity; suppression requires dual sign-off and DSSE note with expiry. Suppressed items emit `suppression_reason`, `expiresAt`.
- **API/CLI contracts (UN7):** New endpoints `/unknowns` support filter by `status`, `type`, `confidence_band`, `uncertainty_score`, `suppressed`. CLI mirrors with `--format ndjson` and `--include-provenance` flags. Output sorted deterministically by `createdAt, id`.
- **Observability/reporting (UN8):** Metrics: `unknowns_total{type,status}`, `unknowns_suppressed_total`, `unknowns_without_sbom`, `unknowns_without_vex`, `unknowns_confidence_band`, `unknowns_manifest_hash_mismatch`. Alerts on manifest hash mismatch, >1% unknowns missing SBOM/VEX, or suppression expiry.
- **Offline bundle inclusion (UN9):** Include latest manifest, schema, and NDJSON export in offline kit; bundle hashes recorded in kit manifest; verify against DSSE signatures.
- **Migration/backfill (UN10):** Backfill script `backfill_unknowns_v1` seeds `scoringManifestHash`, `sbomDigest`, and `vexDecisionId` from existing records; produces `unknowns_backfill_report.ndjson` with before/after status/bands and checksum.

## Schema (draft)
```json
{
  "$id": "https://stella-ops.org/schemas/unknown.json",
  "type": "object",
  "required": ["id", "type", "status", "severity", "createdAt", "confidence", "confidenceBand"],
  "properties": {
    "id": {"type": "string"},
    "type": {"enum": ["vulnerability", "asset", "signal", "evidence-gap", "policy-gap"]},
    "status": {"enum": ["new", "triaging", "under_review", "validated", "dismissed"]},
    "severity": {"enum": ["critical", "high", "medium", "low", "none"]},
    "confidence": {"type": "number"},
    "confidenceBand": {"enum": ["critical", "high", "medium", "low", "under_review"]},
    "uncertaintyScore": {"type": "number", "minimum": 0, "maximum": 1},
    "tauDays": {"type": "integer"},
    "sbomDigest": {"type": "string"},
    "vexDecisionId": {"type": "string"},
    "reachabilityGraphHash": {"type": "string"},
    "scoringManifestHash": {"type": "string"},
    "suppression": {
      "type": "object",
      "properties": {
        "isSuppressed": {"type": "boolean"},
        "reason": {"type": "string"},
        "expiresAt": {"type": "string", "format": "date-time"},
        "signedBy": {"type": "string"}
      }
    },
    "evidence": {"type": "array", "items": {"$ref": "#/definitions/evidenceRef"}},
    "createdAt": {"type": "string", "format": "date-time"},
    "updatedAt": {"type": "string", "format": "date-time"}
  },
  "definitions": {
    "evidenceRef": {
      "type": "object",
      "required": ["uri", "hash", "observedAt", "evidenceType"],
      "properties": {
        "uri": {"type": "string"},
        "hash": {"type": "string"},
        "observedAt": {"type": "string", "format": "date-time"},
        "evidenceType": {"type": "string"},
        "provenance": {"type": "string"}
      }
    }
  }
}
```

## Scoring Manifest (summary)
- Inputs: severity weight, decay factor (τ), uncertainty cap, SLA floor, suppression flag, weighted signals timestamp.
- Formula (deterministic): `confidence = max(floor, min((exp(-Δt/τ) * weight_signal), 1 - uncertainty))`, then clamp by SLA floor if SLA active.
- Canonicalization: JSON Canonicalization Scheme (JCS); decimals fixed 3dp; UTC ISO-8601 timestamps.
- Hash: SHA256 over canonical manifest; published as `scoringManifestHash` and signed via DSSE `stella.ops/unknownsScoringManifest@v1`.

## Offline & Evidence
- Bundle schema, manifest, and latest NDJSON export with `SHA256SUMS` and DSSE envelope for each artifact.
- Evidence Locker class: `signals-unknowns-manifest` (30d retention minimum).

## Migration Checklist (UN10)
1) Generate `unknowns_scoring_manifest.json` and sign (DSSE).
2) Run `backfill_unknowns_v1 --manifest <hash>`; produce report and checksums.
3) Update API/CLI serializers to include new fields and canonical ordering.
4) Enable observability dashboards and alerts; verify thresholds.

## Review Questions (12-04)
- Confirm suppression expiry default (proposal: 30 days).
- Validate `under_review` trigger when SBOM/VEX missing—keep or allow grace period?
- Align SLA floors with decay config (Critical 0.60, High 0.50).