feat: add PolicyPackSelectorComponent with tests and integration

- Implemented PolicyPackSelectorComponent for selecting policy packs.
- Added unit tests for component behavior, including API success and error handling.
- Introduced monaco-workers type declarations for editor workers.
- Created acceptance tests for guardrails with stubs for AT1–AT10.
- Established SCA Failure Catalogue Fixtures for regression testing.
- Developed plugin determinism harness with stubs for PL1–PL10.
- Added scripts for evidence upload and verification processes.
This commit is contained in:
StellaOps Bot
2025-12-05 21:24:34 +02:00
parent 347c88342c
commit 18d87c64c5
220 changed files with 7700 additions and 518 deletions

45
docs/risk/api.md Normal file
View File

@@ -0,0 +1,45 @@
# Risk API (draft outline)
> Draft scaffold; populate once 67-001 explainability outputs and API publishing workflow are available. Keep examples deterministic; include ETags and error payloads when provided.
## Purpose
- Document risk-related endpoints for profile management, simulation, scoring results, explainability retrieval, and export.
## Scope & Audience
- Audience: API consumers, SDK authors, platform integrators.
- In scope: endpoint list, methods, request/response schemas, auth/tenancy headers, rate limits, feature flags, error model.
- Out of scope: console/UI workflow details (see `explainability.md`).
## Endpoint Outline (placeholders)
- `GET /api/risk/profiles` — list profiles (filters by tenant, status).
- `POST /api/risk/profiles` — create/update; includes DSSE/attestation fields.
- `POST /api/risk/simulations` — run simulation with fixture set; supports dry-run.
- `GET /api/risk/results/{id}` — retrieve scored results + explainability link.
- `GET /api/risk/explain/{id}` — fetch explainability payload.
- `GET /api/risk/export/{id}` — export bundle (JSON/CSV) with hash manifest.
- Feature flags: `<pending>`
## Auth & Tenancy
- Required headers: `X-Stella-Tenant`, `X-Stella-Scope`, auth tokens (PAT/OAuth2) — confirm once schema published.
- Imposed rule reminder must be present on every page.
## Error Model (pending)
- Standard error envelope: code, message, correlation_id, severity, remediation.
- Rate limit headers and retry guidance.
## Determinism & Offline Posture
- Provide sample requests/responses under `docs/risk/samples/`; include SHA256 table.
- No live dependencies; use frozen fixtures.
## Open Items
- API publishing workflow outputs
- Final endpoint list and field names
- Error/code catalog
- SDK generator targets and examples
## References
- `docs/risk/overview.md`
- `docs/risk/profiles.md`
- `docs/risk/factors.md`
- `docs/risk/formulas.md`
- `docs/risk/explainability.md`

View File

@@ -0,0 +1,36 @@
# Risk Explainability (draft outline)
> Draft scaffold; fill once 66-004 outputs and UI telemetry captures arrive. Keep fixtures deterministic (frozen payloads, stable ordering, SHA tables).
## Purpose
- Show how the scoring engine produces per-factor contributions and traces that UI/CLI/export surfaces render for auditors and operators.
## Scope & Audience
- Audience: Console/CLI users, auditors, SREs.
- In scope: explainability payload shape, field meanings, provenance, UI/CLI mapping, offline/export behavior.
- Out of scope: formula math (see `formulas.md`), API specifics (see `api.md`).
## Payload Shape (pending)
- Envelope fields: score, severity, factors[], provenance, timestamps (UTC), profile version, environment.
- Factor entry fields: id, type, input_value, normalized_value, weight, contribution, source, evidence_refs[].
- UI/CLI expectations: stable ordering, highlight top contributors, include attestation status.
## UI/CLI Views (to fill)
- Console panels and charts (needs telemetry captures)
- CLI `stella risk explain` output (deterministic table examples)
- Export Center bundles (JSON + CSV + hash manifests)
## Determinism & Offline Posture
- Store example payloads under `docs/risk/samples/`; record `SHA256SUMS`.
- No live calls; all captures from frozen fixtures.
## Open Items
- Capture UI telemetry (Console Guild) and CLI sample outputs.
- Finalize explainability JSON schema once 66-004 is approved.
## References
- `docs/risk/overview.md`
- `docs/risk/profiles.md`
- `docs/risk/factors.md`
- `docs/risk/formulas.md`
- `docs/risk/api.md`

44
docs/risk/factors.md Normal file
View File

@@ -0,0 +1,44 @@
# Risk Factors (draft outline)
> Draft scaffold; fill once 66-002/66-003 inputs (engine contract + sample payloads) arrive. Keep fixtures deterministic and offline-friendly.
## Purpose
- Catalog supported factors (exploit likelihood, VEX state, reachability, runtime facts, fix availability, asset criticality, provenance trust, tenant overrides) and how they normalize into risk math.
## Scope & Audience
- Audience: risk engineers, policy authors, platform SREs.
- In scope: factor definitions, required/optional fields, normalization rules, TTLs, provenance expectations.
- Out of scope: full formula math (see `formulas.md`), API wiring (see `api.md`).
## Factor Catalog (to fill with schema-backed tables)
- Exploit likelihood — fields: source, score, last_seen, confidence
- VEX status — fields: status, justification, impact_statement
- Reachability — fields: entrypoint, callgraph evidence, runtime observation
- Runtime facts — fields: host, container, signal type, timestamp (UTC), provenance attestation
- Fix availability — fields: advisory id, patch released at, mitigation guidance
- Asset criticality — fields: business tier, data class, tenancy scope
- Provenance trust — fields: signature status, key id, chain of custody
- Custom/tenant overrides — fields: override reason, reviewer, expiry/TTL
## Normalization Rules (outline)
- Input validation + schema versioning
- Unit ranges (01) and clamping
- Time decay / TTL handling
- Precedence rules when multiple sources disagree
Interim notes: follow legacy profile guidance — preserve provenance, never mutate source evidence, and keep ordering stable so explainability hashes are repeatable across UI/CLI/exports.
## Determinism & Ordering
- Sort factors by type then source; stable hashing rules for fixtures.
- Record SHA256 for sample payloads once provided.
## Open Items
- Engine contract and sample payloads for each factor
- TTL/decay parameters from Risk Engine Guild
- Provenance attestation examples
## References
- `docs/risk/overview.md`
- `docs/risk/profiles.md`
- `docs/risk/formulas.md`
- `docs/risk/api.md`

50
docs/risk/formulas.md Normal file
View File

@@ -0,0 +1,50 @@
# Risk Formulas (draft outline)
> Draft scaffold; fill once 66-003/66-004 inputs (engine rollout notes + factor contract) are available. Keep math examples deterministic with fixed fixtures.
## Purpose
- Describe how normalized factors combine into a 0100 risk score with severity bands.
- Capture gating, weighting, normalization, and override rules.
## Scope & Audience
- Audience: risk engineers, policy authors, auditors.
- In scope: weighting strategies, aggregation functions, severity thresholds, gating rules, tie-breakers.
- Out of scope: full API payloads (see `api.md`), factor definitions (see `factors.md`).
## Formula Building Blocks (to fill)
- Weighted sum / capped contribution
- Max/min guards per factor family
- Threshold gates (e.g., block if exploitability + reachability high)
- Decay/time weighting
- Tenant/asset overrides and imposed rules
## Severity Mapping (outline)
- Proposed bands (example placeholder):
- Critical: 90100
- High: 7089
- Medium: 4069
- Low: 139
- Info: 0
- Final bands pending governance approval; update once PLLG0104 confirms.
## Determinism
- Stable ordering of factors before aggregation.
- Use fixed precision (e.g., 4 decimals) before severity mapping.
- Hash fixtures and record SHA256 for every example payload.
Interim notes: mirror legacy rule — simulation and production must share the exact evaluation codepath; no per-environment divergences. Severity buckets must be deterministic and governed by Authority scopes.
## Examples (placeholders)
- TBD sample JSON: input factors + output score + contributions table.
- TBD CLI/Console screenshots once telemetry assets provided.
## Open Items
- Engine rollout notes for gating/weighting defaults
- Severity band approval
- Sample payloads and UI traces
## References
- `docs/risk/overview.md`
- `docs/risk/profiles.md`
- `docs/risk/factors.md`
- `docs/risk/api.md`

49
docs/risk/overview.md Normal file
View File

@@ -0,0 +1,49 @@
# Risk Overview (draft outline)
> Draft scaffold only. Populate content after PLLG0104 risk profile schema approval and risk engine/API samples land. Keep all fixtures deterministic (UTC timestamps, stable ordering, sealed sample payloads) and avoid external assets.
## Purpose
- Explain the risk model at a glance: factors, formulas, scoring semantics (0100), and severity bands.
- Show how risk flows through StellaOps services (ingest → evaluate → explain → export) and how provenance is preserved.
## Scope & Audience
- Audience: policy authors, risk engineers, auditors, and SREs consuming risk outputs.
- In scope: concepts, glossary, lifecycle, artifacts, cross-module data flow diagrams (add after schema approval).
- Out of scope: detailed factor math (goes to `formulas.md`), API specifics (goes to `api.md`).
## Core Concepts (to fill)
- Risk factor vs. evidence vs. signal
- Profile vs. formula vs. severity mapping
- Provenance and attestations
- Explainability payloads and UI/CLI displays
- Determinism expectations (ordering, timestamps, hashing)
Interim notes (from legacy doc and sprint context): profiles take normalized factors (exploit likelihood, VEX status, reachability, runtime evidence, fix availability, asset criticality, provenance trust) and output 0100 scores with severity buckets; same code path for simulation and production to ensure determinism.
## Lifecycle (outline)
1. Evidence ingestion (signals, VEX, reachability, runtime)
2. Factor normalization
3. Profile evaluation
4. Severity assignment + gating
5. Explainability + observability
6. Export/archival paths
## Artifacts & Schemas (pending)
- Risk profile schema: `<pending PLLG0104>`
- Risk factor catalog: shared shapes reused by `factors.md`
- Explainability envelope: shared with UI/CLI; add JSON examples once provided.
## Determinism & Offline Posture
- Use frozen fixture sets with SHA256 tables.
- Document regeneration steps (no live network calls) once payloads arrive.
## Open Items
- PLLG0104 schema approval
- Risk engine API payload samples
- UI telemetry captures for explainability walkthroughs
## References (to link once available)
- `docs/risk/profiles.md`
- `docs/risk/factors.md`
- `docs/risk/formulas.md`
- `docs/risk/api.md`

52
docs/risk/profiles.md Normal file
View File

@@ -0,0 +1,52 @@
# Risk Profiles (draft outline)
> Draft scaffold pending PLLG0104 risk profile schema approval. Do not publish externally until schemas and sample payloads arrive. Mirrors existing `docs/risk/risk-profiles.md`; this file will supersede it once populated.
## Purpose
- Define how profiles group factors, weights, thresholds, and severity bands.
- Describe authoring, simulation, promotion, rollback, and provenance for profiles.
## Scope & Audience
- Audience: policy authors, risk engineers, platform SREs.
- Coverage: profile schema, lifecycle, governance, promotion paths, rollback, and observability hooks.
## Schema (placeholder)
- Profile schema reference: `<pending PLLG0104>`
- Required fields: id, versioning, factors list, weights, thresholds, severity mapping, metadata, provenance.
- Optional fields: tenant overrides, imposed rules, time-to-live.
## Lifecycle (outline)
1. Authoring in Policy Studio (draft state)
2. Simulation against fixtures (deterministic inputs)
3. Review/approval workflow
4. Promotion to environments (dev → staging → prod)
5. Rollback hooks and audit trail
## Governance & Determinism
- Profiles stored with DSSE/signatures; record SHA256 for fixtures.
- Same evaluation codepath for simulation and production; note required feature flags.
- Offline posture: include profiles and fixtures inside mirror bundles.
## Explainability & Observability
- Per-factor contribution outputs (JSON) with stable ordering.
- Metrics to log: evaluation latency, cache hit ratio, factor coverage.
- Dashboards/alerts to enumerate once telemetry payloads are supplied.
## Open Items
- PLLG0104 schema approval and sample JSON payloads
- Feature-flag list for registry alignment
- Telemetry field list for dashboards/alerts
## References
- `docs/risk/overview.md`
- `docs/risk/factors.md`
- `docs/risk/formulas.md`
- `docs/risk/explainability.md`
- `docs/risk/api.md`
- Existing context: `docs/risk/risk-profiles.md` (to reconcile once schema lands)
## Interim Notes (carried from legacy `docs/risk/risk-profiles.md`)
- Profiles define how evidence (CVSS/EPSS-like exploit likelihood, KEV flags, VEX status, reachability, runtime evidence, fix availability, asset criticality, provenance trust) normalizes into a 0100 score with severity buckets.
- Workflow highlights: author in Policy Studio → simulate with fixtures → activate in Policy Engine → explain outputs in CLI/Console → export for auditors via Export Center.
- Governance: draft/review/approval with DSSE/signatures; rollback hooks and promotion gates enforced by Authority scopes; determinism required (same codepath for simulation and production).
- Observability: record scoring latency, factor distribution, and profile usage; offline posture via mirror bundles with fixtures and hash manifests.

View File

@@ -0,0 +1,8 @@
# Risk Samples Ingest Checklist (use when payloads arrive)
1) Drop payloads into the correct folder (`profiles/`, `factors/`, `explain/`, `api/`).
2) Normalize JSON deterministically (e.g., `jq -S .`) before hashing; keep UTC timestamps.
3) Run `sha256sum * > SHA256SUMS` in the target folder; keep file sorted.
4) Verify hashes: `sha256sum -c SHA256SUMS`.
5) Add a short README snippet in the sprint Execution Log noting files added and hashes updated.
6) Keep fixtures offline-only; no external calls or redactions after hashing.

View File

@@ -0,0 +1,26 @@
# Risk Samples (fixtures layout)
Use this folder for frozen, deterministic fixtures once schemas and payloads arrive.
Structure (proposed):
- `profiles/` — profile JSON (DSSE-wrapped where applicable) + `SHA256SUMS`
- `factors/` — factor input payloads grouped by source (epss/, kev/, reachability/, runtime/), each with `SHA256SUMS`
- `explain/` — explainability outputs paired with inputs; include `SHA256SUMS`
- `api/` — request/response examples for risk endpoints; include `SHA256SUMS`
Rules:
- UTC timestamps; stable ordering of arrays/objects.
- No live calls; fixtures only.
- Record hashes via `sha256sum` and keep manifests alongside samples.
Quick receipt checklist (see `INGEST_CHECKLIST.md` for detail):
1) Normalize JSON with `jq -S .`
2) Update `SHA256SUMS` in the target folder
3) Verify with `sha256sum -c`
4) Log files + hashes in the sprint Execution Log
Manifests created:
- `profiles/SHA256SUMS`
- `factors/SHA256SUMS`
- `explain/SHA256SUMS`
- `api/SHA256SUMS`

View File

@@ -0,0 +1,3 @@
Use the root `INGEST_CHECKLIST.md`.
Place request/response examples here; normalize with `jq -S .`, update `SHA256SUMS`, verify with `sha256sum -c`.
Include required headers; redact secrets; UTC timestamps only.

View File

View File

@@ -0,0 +1,3 @@
Use the root `INGEST_CHECKLIST.md`.
Store explainability outputs paired with their inputs; normalize with `jq -S .`, update `SHA256SUMS`, verify with `sha256sum -c`.
Maintain ordering and UTC timestamps; no live data.

View File

View File

@@ -0,0 +1,3 @@
Use the root `INGEST_CHECKLIST.md`.
Drop factor payloads by source (epss/, kev/, reachability/, runtime/), normalize with `jq -S .`, update `SHA256SUMS`, verify with `sha256sum -c`.
Keep UTC timestamps and no live data.

View File

View File

@@ -0,0 +1,8 @@
| Date (UTC) | Folder | Files added | SHA256SUMS updated | Notes |
| --- | --- | --- | --- | --- |
| 2025-__-__ | profiles/ | | yes/no | source + checklist step refs |
| 2025-__-__ | factors/ | | yes/no | source + checklist step refs |
| 2025-__-__ | explain/ | | yes/no | source + checklist step refs |
| 2025-__-__ | api/ | | yes/no | source + checklist step refs |
Instructions: copy a row per drop, fill actual date, list filenames, mark whether `SHA256SUMS` was updated, and note evidence source. Keep this file sorted by date for determinism.

View File

@@ -0,0 +1,3 @@
Use the root `INGEST_CHECKLIST.md`.
Place profile JSON/DSSE here, normalize with `jq -S .`, update `SHA256SUMS`, and verify with `sha256sum -c`.
UTC timestamps only; no live data.

View File