87 lines
5.4 KiB
Markdown
87 lines
5.4 KiB
Markdown
# Unknowns Registry & Scoring Manifest
|
||
|
||
**Compiled:** 2025-12-01 (UTC)
|
||
**Scope:** Close UN1–UN10 gaps from `docs/product/advisories/31-Nov-2025 FINDINGS.md` for Unknowns Registry.
|
||
**Status:** Draft; review 2025-12-04; DSSE signing required before adoption.
|
||
|
||
## Decisions (UN1–UN10)
|
||
- **Canonical schema/enums (UN1):** Unknown types: `vulnerability`, `asset`, `signal`, `evidence-gap`, `policy-gap`. Status enums: `new`, `triaging`, `under_review`, `validated`, `dismissed`. Severity: `critical/high/medium/low/none`.
|
||
- **Deterministic scoring manifest (UN2):** Manifest `unknowns_scoring_manifest.json` defines inputs, weights, and canonical serialization (JCS, sorted keys, UTC timestamps, fixed 3dp). Hash used as `scoringManifestHash` in API/DSSE.
|
||
- **Decay policy catalog (UN3):** Unknowns reuse `confidence_decay_config` but may override τ by type (see table). Overrides stored in manifest; DSSE-signed.
|
||
- **Evidence/provenance capture (UN4):** Each unknown must reference Evidence Locker URIs with DSSE envelopes; minimal evidence: `{source, observedAt, evidenceType, hash}`. Provenance includes tool identity and policy hash.
|
||
- **SBOM/VEX linkage (UN5):** Unknown links: `sbomDigest`, `vexDecisionId` (if present), `reachabilityGraphHash`. If absent, status forced to `under_review`.
|
||
- **SLA / suppression rules (UN6):** SLA timers mirror severity; suppression requires dual sign-off and DSSE note with expiry. Suppressed items emit `suppression_reason`, `expiresAt`.
|
||
- **API/CLI contracts (UN7):** New endpoints `/unknowns` support filter by `status`, `type`, `confidence_band`, `uncertainty_score`, `suppressed`. CLI mirrors with `--format ndjson` and `--include-provenance` flags. Output sorted deterministically by `createdAt, id`.
|
||
- **Observability/reporting (UN8):** Metrics: `unknowns_total{type,status}`, `unknowns_suppressed_total`, `unknowns_without_sbom`, `unknowns_without_vex`, `unknowns_confidence_band`, `unknowns_manifest_hash_mismatch`. Alerts on manifest hash mismatch, >1% unknowns missing SBOM/VEX, or suppression expiry.
|
||
- **Offline bundle inclusion (UN9):** Include latest manifest, schema, and NDJSON export in offline kit; bundle hashes recorded in kit manifest; verify against DSSE signatures.
|
||
- **Migration/backfill (UN10):** Backfill script `backfill_unknowns_v1` seeds `scoringManifestHash`, `sbomDigest`, and `vexDecisionId` from existing records; produces `unknowns_backfill_report.ndjson` with before/after status/bands and checksum.
|
||
|
||
## Schema (draft)
|
||
```json
|
||
{
|
||
"$id": "https://stella-ops.org/schemas/unknown.json",
|
||
"type": "object",
|
||
"required": ["id", "type", "status", "severity", "createdAt", "confidence", "confidenceBand"],
|
||
"properties": {
|
||
"id": {"type": "string"},
|
||
"type": {"enum": ["vulnerability", "asset", "signal", "evidence-gap", "policy-gap"]},
|
||
"status": {"enum": ["new", "triaging", "under_review", "validated", "dismissed"]},
|
||
"severity": {"enum": ["critical", "high", "medium", "low", "none"]},
|
||
"confidence": {"type": "number"},
|
||
"confidenceBand": {"enum": ["critical", "high", "medium", "low", "under_review"]},
|
||
"uncertaintyScore": {"type": "number", "minimum": 0, "maximum": 1},
|
||
"tauDays": {"type": "integer"},
|
||
"sbomDigest": {"type": "string"},
|
||
"vexDecisionId": {"type": "string"},
|
||
"reachabilityGraphHash": {"type": "string"},
|
||
"scoringManifestHash": {"type": "string"},
|
||
"suppression": {
|
||
"type": "object",
|
||
"properties": {
|
||
"isSuppressed": {"type": "boolean"},
|
||
"reason": {"type": "string"},
|
||
"expiresAt": {"type": "string", "format": "date-time"},
|
||
"signedBy": {"type": "string"}
|
||
}
|
||
},
|
||
"evidence": {"type": "array", "items": {"$ref": "#/definitions/evidenceRef"}},
|
||
"createdAt": {"type": "string", "format": "date-time"},
|
||
"updatedAt": {"type": "string", "format": "date-time"}
|
||
},
|
||
"definitions": {
|
||
"evidenceRef": {
|
||
"type": "object",
|
||
"required": ["uri", "hash", "observedAt", "evidenceType"],
|
||
"properties": {
|
||
"uri": {"type": "string"},
|
||
"hash": {"type": "string"},
|
||
"observedAt": {"type": "string", "format": "date-time"},
|
||
"evidenceType": {"type": "string"},
|
||
"provenance": {"type": "string"}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
## Scoring Manifest (summary)
|
||
- Inputs: severity weight, decay factor (τ), uncertainty cap, SLA floor, suppression flag, weighted signals timestamp.
|
||
- Formula (deterministic): `confidence = max(floor, min((exp(-Δt/τ) * weight_signal), 1 - uncertainty))`, then clamp by SLA floor if SLA active.
|
||
- Canonicalization: JSON Canonicalization Scheme (JCS); decimals fixed 3dp; UTC ISO-8601 timestamps.
|
||
- Hash: SHA256 over canonical manifest; published as `scoringManifestHash` and signed via DSSE `stella.ops/unknownsScoringManifest@v1`.
|
||
|
||
## Offline & Evidence
|
||
- Bundle schema, manifest, and latest NDJSON export with `SHA256SUMS` and DSSE envelope for each artifact.
|
||
- Evidence Locker class: `signals-unknowns-manifest` (30d retention minimum).
|
||
|
||
## Migration Checklist (UN10)
|
||
1) Generate `unknowns_scoring_manifest.json` and sign (DSSE).
|
||
2) Run `backfill_unknowns_v1 --manifest <hash>`; produce report and checksums.
|
||
3) Update API/CLI serializers to include new fields and canonical ordering.
|
||
4) Enable observability dashboards and alerts; verify thresholds.
|
||
|
||
## Review Questions (12-04)
|
||
- Confirm suppression expiry default (proposal: 30 days).
|
||
- Validate `under_review` trigger when SBOM/VEX missing—keep or allow grace period?
|
||
- Align SLA floors with decay config (Critical 0.60, High 0.50).
|