partly or unimplemented features - now implemented
This commit is contained in:
@@ -115,8 +115,10 @@ All context references include `content_hash` and `source_id` enabling verifiabl
|
||||
|
||||
## 8) APIs
|
||||
|
||||
- `POST /api/v1/advisory/{task}` — executes Summary/Conflict/Remediation pipeline (`task` ∈ `summary|conflict|remediation`). Requests accept `{advisoryKey, artifactId?, policyVersion?, profile, preferredSections?, forceRefresh}` and return sanitized prompt payloads, citations, guardrail metadata, provenance hash, and cache hints.
|
||||
- `GET /api/v1/advisory/outputs/{cacheKey}?taskType=SUMMARY&profile=default` — retrieves cached artefacts for downstream consumers (Console, CLI, Export Center). Guardrail state and provenance hash accompany results.
|
||||
- `POST /api/v1/advisory/{task}` - executes Summary/Conflict/Remediation pipeline (`task` in `summary|conflict|remediation`). Requests accept `{advisoryKey, artifactId?, policyVersion?, profile, preferredSections?, forceRefresh}` and return sanitized prompt payloads, citations, guardrail metadata, provenance hash, and cache hints.
|
||||
- `GET /api/v1/advisory/outputs/{cacheKey}?taskType=SUMMARY&profile=default` - retrieves cached artifacts for downstream consumers (Console, CLI, Export Center). Guardrail state and provenance hash accompany results.
|
||||
- `POST /v1/advisory-ai/companion/explain` - composes explanation output with deterministic runtime signals from Zastava-compatible observers. Request payload extends explain fields with `runtimeSignals[]`; response returns `companionId`, `companionHash`, composed summary lines, and normalized runtime highlights.
|
||||
- Companion endpoint authorization accepts `advisory:run`, `advisory:explain`, or `advisory:companion` scopes and maps companion validation failures to HTTP 400 without leaking internal state.
|
||||
|
||||
All endpoints accept `profile` parameter (default `fips-local`) and return `output_hash`, `input_digest`, and `citations` for verification.
|
||||
|
||||
|
||||
63
docs/modules/advisory-lens/architecture.md
Normal file
63
docs/modules/advisory-lens/architecture.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# Advisory Lens Architecture
|
||||
|
||||
## Purpose
|
||||
|
||||
StellaOps.AdvisoryLens is a deterministic, offline-first library for semantic case matching of vulnerability advisories. It produces ranked suggestions and contextual hints without AI/LLM inference.
|
||||
|
||||
## Scope
|
||||
|
||||
- Working directory: `src/__Libraries/StellaOps.AdvisoryLens/`
|
||||
- Tests: `src/__Libraries/__Tests/StellaOps.AdvisoryLens.Tests/`
|
||||
- Integration entry point: `services.AddAdvisoryLens(...)`
|
||||
|
||||
## Models
|
||||
|
||||
| Type | Purpose |
|
||||
|------|---------|
|
||||
| `AdvisoryCase` | Advisory input including CVE, PURL, severity, and metadata |
|
||||
| `LensContext` | Evaluation envelope (advisory case, tenant id, evidence refs, optional timestamp) |
|
||||
| `CasePattern` | Matching rule with severity/ecosystem/CVE conditions and default suggestion payload |
|
||||
| `LensSuggestion` | Ranked operator-facing recommendation with confidence and action |
|
||||
| `LensHint` | Contextual evidence hint grouped by deterministic categories |
|
||||
| `LensResult` | Evaluation output containing suggestions, hints, matched pattern ids, timestamp, and input hash |
|
||||
|
||||
## Matching Algorithm
|
||||
|
||||
1. `CaseMatcher` evaluates each `CasePattern` against the input `AdvisoryCase`
|
||||
2. Scoring factors are severity range match, PURL ecosystem match, and CVE pattern match
|
||||
3. Disqualifying mismatches (severity out of range, wrong ecosystem) return score `0.0`
|
||||
4. If no factors are configured for a pattern, score defaults to `0.5`
|
||||
5. Positive-score matches are sorted by score descending, then `PatternId` ascending for deterministic tie-breaking
|
||||
6. `AdvisoryLensService` maps sorted matches into suggestions with rank = position + 1
|
||||
|
||||
## Hint Generation
|
||||
|
||||
Hints are derived from `LensContext` and sorted by category ordinal then text:
|
||||
|
||||
- Severity: `High` or `Critical` advisories emit a priority remediation hint
|
||||
- Reachability: non-empty reachability evidence emits code-path guidance
|
||||
- VEX: non-empty VEX references emit a count-based hint
|
||||
- Policy: non-empty policy traces emit a count-based hint
|
||||
|
||||
## Integration
|
||||
|
||||
```csharp
|
||||
services.AddAdvisoryLens(patterns, timeProvider);
|
||||
```
|
||||
|
||||
- Registers `IAdvisoryLensService` as a singleton
|
||||
- Uses empty patterns when none are provided
|
||||
- Uses `TimeProvider.System` when no provider is injected
|
||||
|
||||
## Determinism Guarantees
|
||||
|
||||
- Stable ordering for matches and hints
|
||||
- Input hash computed as `sha256:` + SHA-256 over canonical JSON (`camelCase`, no indentation, nulls ignored)
|
||||
- Timestamp comes from `LensContext.EvaluationTimestampUtc` or injected `TimeProvider`
|
||||
- Identical inputs and clock source produce identical `LensResult`
|
||||
|
||||
## Offline Posture
|
||||
|
||||
- No network dependencies in library behavior
|
||||
- In-process, side-effect-free evaluation and scoring
|
||||
- Tests validate execution with no HTTP or external service setup
|
||||
File diff suppressed because it is too large
Load Diff
48
docs/modules/authority/timestamping-ci-cd.md
Normal file
48
docs/modules/authority/timestamping-ci-cd.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Authority CI/CD Timestamping
|
||||
|
||||
This document describes the CI/CD timestamping orchestration added in Sprint `SPRINT_20260208_025_Authority_rfc_3161_tsa_client_for_ci_cd_timestamping`.
|
||||
|
||||
## Scope
|
||||
- Automatically request RFC-3161 timestamps for pipeline artifacts (SBOMs, attestations, logs, or other digest-addressed artifacts).
|
||||
- Persist deterministic artifact-to-token mappings for replay, lookup, and audit.
|
||||
- Support pipeline-scoped and environment-scoped timestamp policies without requiring network access in tests.
|
||||
|
||||
## Implementation
|
||||
- Orchestration service:
|
||||
- `src/Authority/__Libraries/StellaOps.Authority.Timestamping/CiCdTimestampingService.cs`
|
||||
- `src/Authority/__Libraries/StellaOps.Authority.Timestamping/ICiCdTimestampingService.cs`
|
||||
- Artifact timestamp registry:
|
||||
- `src/Authority/__Libraries/StellaOps.Authority.Timestamping/IArtifactTimestampRegistry.cs`
|
||||
- `src/Authority/__Libraries/StellaOps.Authority.Timestamping/InMemoryArtifactTimestampRegistry.cs`
|
||||
- Policy models:
|
||||
- `src/Authority/__Libraries/StellaOps.Authority.Timestamping/PipelineTimestampingPolicyOptions.cs`
|
||||
- `src/Authority/__Libraries/StellaOps.Authority.Timestamping/CiCdTimestampingModels.cs`
|
||||
- DI registration:
|
||||
- `src/Authority/__Libraries/StellaOps.Authority.Timestamping/TimestampingServiceCollectionExtensions.cs`
|
||||
|
||||
## Policy behavior
|
||||
- `DefaultPolicy` applies when no pipeline override exists.
|
||||
- `Pipelines[<pipelineId>]` overrides the default policy.
|
||||
- `Pipelines[<pipelineId>].Environments[<environment>]` overrides the pipeline policy.
|
||||
- Core controls:
|
||||
- `Enabled`
|
||||
- `RequiredSuccessCount`
|
||||
- `MaxAttemptsPerArtifact`
|
||||
- `RequireDistinctProviders`
|
||||
- `IncludeNonce`
|
||||
- `CertificateRequired`
|
||||
- `HashAlgorithm`
|
||||
- `PolicyOid`
|
||||
|
||||
## Determinism and offline posture
|
||||
- Artifact processing is deterministic: artifacts are sorted by digest and type before orchestration.
|
||||
- Digest normalization is deterministic (`algo:hex-lowercase`).
|
||||
- Nonce generation is deterministic when `IncludeNonce=true` (derived from pipeline/artifact identity and attempt index).
|
||||
- Tests use in-memory fakes only and run without network access.
|
||||
|
||||
## Test coverage
|
||||
- `src/Authority/__Tests/StellaOps.Authority.Timestamping.Tests/CiCdTimestampingServiceTests.cs`
|
||||
- `src/Authority/__Tests/StellaOps.Authority.Timestamping.Tests/InMemoryArtifactTimestampRegistryTests.cs`
|
||||
|
||||
Validation command used:
|
||||
- `dotnet test src/Authority/__Tests/StellaOps.Authority.Timestamping.Tests/StellaOps.Authority.Timestamping.Tests.csproj --no-restore -p:BuildProjectReferences=false -v minimal`
|
||||
@@ -19,6 +19,16 @@ Bench provides performance benchmark infrastructure for StellaOps modules. Measu
|
||||
- `StellaOps.Bench.PolicyEngine` - Policy evaluation performance
|
||||
- `StellaOps.Bench.ScannerAnalyzers` - Language analyzer performance
|
||||
|
||||
## Scanner Vendor Parity Tracking
|
||||
|
||||
`StellaOps.Bench.ScannerAnalyzers` now supports vendor parity tracking for offline benchmark runs:
|
||||
- Scenario-level vendor ingestion from JSON or SARIF artifacts (`vendorResults[]` in benchmark config).
|
||||
- Optional Stella finding ingestion (`stellaFindingsPath`) for exact overlap comparisons.
|
||||
- Deterministic parity outputs in benchmark JSON and Prometheus exports:
|
||||
- overlap counts and percentages
|
||||
- scanner-only / vendor-only counts
|
||||
- parity score (Jaccard-style overlap over union)
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
|
||||
@@ -1592,5 +1592,152 @@ Offline verification bundles include tile proofs for air-gapped environments.
|
||||
|
||||
---
|
||||
|
||||
*Document Version: 1.3.0*
|
||||
*Last Updated: 2026-01-28*
|
||||
## 13. Cross-Distro Coverage Matrix for Backport Validation
|
||||
|
||||
Manages a curated set of high-impact CVEs with per-distribution backport
|
||||
status tracking, enabling systematic validation of backport detection
|
||||
accuracy across Alpine, Debian, and RHEL.
|
||||
|
||||
### 13.1 Architecture
|
||||
|
||||
1. **CuratedCveEntry** — One row per CVE (e.g., Heartbleed, Baron Samedit)
|
||||
with cross-distro `DistroCoverageEntry` array tracking backport status
|
||||
per distro-version
|
||||
2. **CrossDistroCoverageService** — In-memory coverage matrix with upsert,
|
||||
query, summary, and validation marking operations
|
||||
3. **SeedBuiltInEntries** — Idempotent seeding of 5 curated high-impact CVEs
|
||||
(CVE-2014-0160, CVE-2021-3156, CVE-2015-0235, CVE-2023-38545, CVE-2024-6387)
|
||||
with pre-populated backport status across Alpine, Debian, and RHEL versions
|
||||
|
||||
### 13.2 Distro Families & Backport Status
|
||||
|
||||
| Enum | Values |
|
||||
|---|---|
|
||||
| `DistroFamily` | Alpine, Debian, Rhel |
|
||||
| `BackportStatus` | NotPatched, Backported, NotApplicable, Unknown |
|
||||
|
||||
### 13.3 Models
|
||||
|
||||
| Type | Description |
|
||||
|---|---|
|
||||
| `DistroCoverageEntry` | Per distro-version: package name/version, backport status, validated flag |
|
||||
| `CuratedCveEntry` | CVE with CommonName, CvssScore, CweIds, Coverage array, computed CoverageRatio |
|
||||
| `CrossDistroCoverageSummary` | Aggregated counts: TotalCves, TotalEntries, ValidatedEntries, ByDistro breakdown |
|
||||
| `DistroBreakdown` | Per-distro EntryCount, ValidatedCount, BackportedCount |
|
||||
| `CuratedCveQuery` | Component/Distro/Status/OnlyUnvalidated filters with Limit/Offset paging |
|
||||
|
||||
### 13.4 Built-in Curated CVEs
|
||||
|
||||
| CVE | Component | Common Name | CVSS |
|
||||
|---|---|---|---|
|
||||
| CVE-2014-0160 | openssl | Heartbleed | 7.5 |
|
||||
| CVE-2021-3156 | sudo | Baron Samedit | 7.8 |
|
||||
| CVE-2015-0235 | glibc | GHOST | 10.0 |
|
||||
| CVE-2023-38545 | curl | SOCKS5 heap overflow | 9.8 |
|
||||
| CVE-2024-6387 | openssh | regreSSHion | 8.1 |
|
||||
|
||||
### 13.5 DI Registration
|
||||
|
||||
`ICrossDistroCoverageService` → `CrossDistroCoverageService` registered via
|
||||
TryAddSingleton in `GoldenSetServiceCollectionExtensions.AddGoldenSetServices()`.
|
||||
|
||||
### 13.6 OTel Metrics
|
||||
|
||||
Meter: `StellaOps.BinaryIndex.GoldenSet.CrossDistro`
|
||||
|
||||
| Counter | Description |
|
||||
|---|---|
|
||||
| `crossdistro.upsert.total` | CVE entries upserted |
|
||||
| `crossdistro.query.total` | Coverage queries executed |
|
||||
| `crossdistro.seed.total` | Built-in entries seeded |
|
||||
| `crossdistro.validated.total` | Entries marked as validated |
|
||||
|
||||
### 13.7 Source Files
|
||||
|
||||
- Models: `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GoldenSet/Models/CrossDistroCoverageModels.cs`
|
||||
- Interface: `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GoldenSet/Services/ICrossDistroCoverageService.cs`
|
||||
- Implementation: `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GoldenSet/Services/CrossDistroCoverageService.cs`
|
||||
|
||||
### 13.8 Test Coverage (37 tests)
|
||||
|
||||
- Models: DistroFamily/BackportStatus enum counts, DistroCoverageEntry roundtrips/defaults,
|
||||
CuratedCveEntry coverage ratio/empty, CuratedCveQuery defaults, Summary coverage/empty
|
||||
- Service: SeedBuiltInEntries population/idempotency/heartbleed/baron-samedit/distro coverage,
|
||||
UpsertAsync store-retrieve/overwrite/null/empty, GetByCveIdAsync unknown/case-insensitive/null,
|
||||
QueryAsync all/component/distro/status/unvalidated/limit-offset/ordering,
|
||||
GetSummaryAsync counts/empty, SetValidatedAsync mark/unknown-cve/unknown-version/summary/null,
|
||||
CreateBuiltInEntries deterministic/distro-coverage
|
||||
|
||||
---
|
||||
|
||||
## 14. ELF Segment Normalization for Delta Hashing
|
||||
|
||||
### 14.1 Purpose
|
||||
|
||||
The existing instruction-level normalization (X64/Arm64 pipelines) operates on
|
||||
disassembled instruction streams. ELF Segment Normalization fills the gap for
|
||||
**raw binary bytes** — zeroing position-dependent data (relocation entries,
|
||||
GOT/PLT displacements, alignment padding) and canonicalizing NOP sleds
|
||||
*before* disassembly, enabling deterministic delta hashing across builds
|
||||
compiled at different base addresses or link orders.
|
||||
|
||||
### 14.2 Key Types
|
||||
|
||||
| Type | Location | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `ElfNormalizationStep` | `Normalization/ElfSegmentNormalizer.cs` | Enum of normalization passes (RelocationZeroing, GotPltCanonicalization, NopCanonicalization, JumpTableRewriting, PaddingZeroing) |
|
||||
| `ElfSegmentNormalizationOptions` | same | Options record with `Default` and `Minimal` presets |
|
||||
| `ElfSegmentNormalizationResult` | same | Result with NormalizedBytes, DeltaHash (SHA-256), ModifiedBytes, AppliedSteps, StepCounts, computed ModificationRatio |
|
||||
| `IElfSegmentNormalizer` | same | Interface: `Normalize`, `ComputeDeltaHash` |
|
||||
| `ElfSegmentNormalizer` | same | Implementation with 5 internal passes and 2 OTel counters |
|
||||
|
||||
### 14.3 Normalization Passes
|
||||
|
||||
1. **RelocationZeroing** — Scans for ELF64 RELA-shaped entries (heuristic:
|
||||
info field encodes valid x86-64 relocation types 1–42 with symbol index
|
||||
≤100 000); zeros the offset and addend fields (16 bytes per entry).
|
||||
2. **GotPltCanonicalization** — Detects `FF 25` (JMP [rip+disp32]) and
|
||||
`FF 35` (PUSH [rip+disp32]) PLT stub patterns; zeros the 4-byte
|
||||
displacement to remove position-dependent indirect jump targets.
|
||||
3. **NopCanonicalization** — Matches 7 multi-byte x86-64 NOP variants
|
||||
(2–7 bytes each, per Intel SDM) and replaces with canonical single-byte
|
||||
NOPs (0x90).
|
||||
4. **JumpTableRewriting** — Identifies sequences of 4+ consecutive 8-byte
|
||||
entries sharing the same upper 32 bits (switch-statement jump tables);
|
||||
zeros the entries.
|
||||
5. **PaddingZeroing** — Detects runs of 4+ alignment padding bytes (0xCC or
|
||||
0x00) between code regions and zeros them.
|
||||
|
||||
### 14.4 Delta Hashing
|
||||
|
||||
`ComputeDeltaHash` produces a lowercase SHA-256 hex string of the normalized
|
||||
byte buffer. Two builds of the same source compiled at different addresses
|
||||
will produce the same delta hash after normalization.
|
||||
|
||||
### 14.5 OTel Instrumentation
|
||||
|
||||
Meter: `StellaOps.BinaryIndex.Normalization.ElfSegment`
|
||||
|
||||
| Counter | Description |
|
||||
| --- | --- |
|
||||
| `elfsegment.normalize.total` | Segments normalized |
|
||||
| `elfsegment.bytes.modified` | Total bytes modified across all passes |
|
||||
|
||||
### 14.6 DI Registration
|
||||
|
||||
`IElfSegmentNormalizer` is registered as `TryAddSingleton<ElfSegmentNormalizer>`
|
||||
inside `AddNormalizationPipelines()` in `ServiceCollectionExtensions.cs`.
|
||||
|
||||
### 14.7 Test Coverage (35 tests)
|
||||
|
||||
- Models: DefaultOptions (all enabled), MinimalOptions (relocations only), ModificationRatio zero/computed, enum values
|
||||
- Service: Constructor null guard, empty input result + SHA-256, ComputeDeltaHash determinism/distinct,
|
||||
NOP canonicalization (3-byte, 2-byte, 4-byte, no-NOP, 7-byte, single-byte),
|
||||
GOT/PLT (JMP disp32, PUSH disp32), alignment padding (INT3 run, zero run, short run),
|
||||
relocation zeroing (valid RELA, invalid entry), jump table (consecutive addresses, random data),
|
||||
full pipeline (deterministic hash, default vs minimal, all-disabled, step-count consistency)
|
||||
|
||||
---
|
||||
|
||||
*Document Version: 1.5.0*
|
||||
*Last Updated: 2026-02-08*
|
||||
|
||||
@@ -142,7 +142,76 @@ See [migration-v3.md](./guides/migration-v3.md) for user-facing migration instru
|
||||
* `export sbom <digest> [--view ... --format ... --out file]` — download artifact.
|
||||
* `sbom upload --file <path> --artifact <ref> [--format cyclonedx|spdx]` - BYOS upload into the scanner analysis pipeline (ledger join uses the SBOM digest).
|
||||
* `report final <digest> [--policy-revision ... --attest]` — request PASS/FAIL report from backend (policy+vex) and optional attestation.
|
||||
### 2.3.1 Compare Commands & Baseline Selection (SPRINT_20260208_029)
|
||||
|
||||
The `compare` command group supports diffing scan snapshots with automatic baseline resolution.
|
||||
|
||||
#### Commands
|
||||
|
||||
* `compare diff --base <digest> --target <digest>` â€" Full comparison showing detailed diff.
|
||||
* `compare summary --base <digest> --target <digest>` â€" Quick summary of changes.
|
||||
* `compare can-ship --base <digest> --target <digest>` â€" Check if target passes policy (exit code: 0=pass, 1=fail).
|
||||
* `compare vulns --base <digest> --target <digest>` â€" List vulnerability changes only.
|
||||
|
||||
#### Baseline Selection Strategies
|
||||
|
||||
All compare commands support `--baseline-strategy` for automatic baseline resolution:
|
||||
|
||||
| Strategy | Description | Requirements |
|
||||
|----------|-------------|--------------|
|
||||
| `explicit` (default) | Uses the digest provided via `--base` | `--base` required |
|
||||
| `last-green` | Selects most recent passing snapshot | `--artifact` required |
|
||||
| `previous-release` | Selects previous release tag from registry metadata | `--artifact` required |
|
||||
|
||||
**Options:**
|
||||
|
||||
* `--baseline-strategy <explicit|last-green|previous-release>` â€" Strategy for baseline selection
|
||||
* `--artifact <purl|oci-ref>` â€" Artifact identifier for auto-resolution strategies
|
||||
* `--current-version <tag>` â€" Current version (helps `previous-release` find older releases)
|
||||
* `--verification-report <path>` - Attach `bundle verify --output json` checks to compare output (hash/signature overlay)
|
||||
* `--reverify-bundle <directory>` - Recompute artifact hash and DSSE-sidecar status from local evidence bundle for live re-verification
|
||||
* `--determinism-manifest <path>` - Attach determinism manifest score/threshold summary to compare output
|
||||
|
||||
**Examples:**
|
||||
|
||||
```bash
|
||||
# Explicit baseline (traditional)
|
||||
stella compare can-ship --base sha256:abc123 --target sha256:def456
|
||||
|
||||
# Auto-select last green baseline
|
||||
stella compare can-ship --target sha256:def456 \
|
||||
--baseline-strategy last-green \
|
||||
--artifact pkg:oci/myapp
|
||||
|
||||
# Use previous release as baseline
|
||||
stella compare can-ship --target sha256:def456 \
|
||||
--baseline-strategy previous-release \
|
||||
--artifact pkg:oci/myapp \
|
||||
--current-version v2.0.0
|
||||
|
||||
# Compare diff with inline verification overlay and determinism context
|
||||
stella compare diff --base sha256:abc123 --target sha256:def456 \
|
||||
--verification-report ./verify-report.json \
|
||||
--reverify-bundle ./evidence-bundle \
|
||||
--determinism-manifest ./determinism.json
|
||||
```
|
||||
|
||||
**Resolution Behavior:**
|
||||
|
||||
* `last-green`: Queries forensic snapshot store for latest artifact snapshot with `verdict:pass` tag.
|
||||
* `previous-release`: Queries for release-tagged snapshots, excludes `--current-version`, returns most recent.
|
||||
* Both strategies show suggestions when resolution fails.
|
||||
* Verification overlay: `compare diff` can now include per-artifact `hash`/`signature` status plus determinism score context in table and JSON outputs.
|
||||
|
||||
**Service Interface:**
|
||||
|
||||
```csharp
|
||||
public interface IBaselineResolver
|
||||
{
|
||||
Task<BaselineResolutionResult> ResolveAsync(BaselineResolutionRequest request, CancellationToken ct);
|
||||
Task<IReadOnlyList<BaselineSuggestion>> GetSuggestionsAsync(string artifactId, CancellationToken ct);
|
||||
}
|
||||
```
|
||||
### 2.4 Policy & data
|
||||
|
||||
* `policy get/set/apply` — fetch active policy, apply staged policy, compute digest.
|
||||
@@ -243,6 +312,12 @@ Both subcommands honour offline-first expectations (no network access) and norma
|
||||
|
||||
* Calls advisory chat endpoints, returns a cited answer with evidence refs.
|
||||
* `--no-action` disables action proposals; `--evidence` forces evidence chips in output.
|
||||
* `--file <queries.jsonl>` processes newline-delimited JSON batch requests (`{"query":"..."}` or JSON string lines) and emits deterministic per-line results in `json|table|markdown` format.
|
||||
|
||||
* `advise export [--conversation-id <id>] [--tenant <tenant>] [--user <user>] [--limit <n>] [--format <json|table|markdown>] [--output <file>]`
|
||||
|
||||
* Exports advisory conversation history through the existing AdvisoryAI conversation endpoints (`/v1/advisory-ai/conversations`).
|
||||
* When no `--conversation-id` is provided, the CLI lists conversations for the scope and fetches each conversation deterministically by `conversationId` before rendering.
|
||||
|
||||
### 2.12 Decision evidence (new)
|
||||
|
||||
@@ -268,6 +343,15 @@ All verbs require scopes `policy.findings:read`, `signer.verify`, and (for Rekor
|
||||
|
||||
- CLI outbound HTTP flows (Authority auth, backend APIs, advisory downloads) route through `StellaOps.AirGap.Policy`. When sealed mode is active the CLI refuses commands that would require external egress and surfaces the shared `AIRGAP_EGRESS_BLOCKED` remediation guidance instead of attempting the request.
|
||||
|
||||
### 2.14 Unknowns export artifacts
|
||||
|
||||
- `unknowns export [--band <hot|warm|cold|all>] [--format <json|csv|ndjson>] [--schema-version <value>] [--output <path>]`
|
||||
|
||||
* `json` now emits a deterministic export envelope with `schemaVersion`, `exportedAt`, `itemCount`, and sorted `items`.
|
||||
* `csv` prepends a schema metadata comment (`schema_version`, `exported_at`, `item_count`) before the column header.
|
||||
* `ndjson` emits a metadata header line followed by schema-scoped item lines.
|
||||
* The formal contract artifact for the JSON envelope is at `src/Cli/StellaOps.Cli/Commands/Schemas/unknowns-export.schema.json`.
|
||||
|
||||
---
|
||||
|
||||
## 3) AuthN: Authority + DPoP
|
||||
@@ -553,7 +637,43 @@ script:
|
||||
|
||||
## 20) Test matrix (OS/arch)
|
||||
|
||||
* Linux: ubuntu‑20.04/22.04/24.04 (x64, arm64), alpine (musl).
|
||||
* macOS: 13–15 (x64, arm64).
|
||||
* Linux: ubuntu‑20.04/22.04/24.04 (x64, arm64), alpine (musl).
|
||||
* macOS: 13–15 (x64, arm64).
|
||||
* Windows: 10/11, Server 2019/2022 (x64, arm64).
|
||||
* Docker engines: Docker Desktop, containerd‑based runners.
|
||||
* Docker engines: Docker Desktop, containerd‑based runners.
|
||||
|
||||
## 21) OCI Referrers for Evidence Storage
|
||||
|
||||
### 21.1 Overview
|
||||
|
||||
Two new evidence sub-commands enable native OCI Referrers API integration:
|
||||
|
||||
| Command | Purpose |
|
||||
| --- | --- |
|
||||
| `stella evidence push-referrer` | Push an evidence artifact as an OCI referrer attached to a subject digest |
|
||||
| `stella evidence list-referrers` | List all OCI referrers for a given artifact, with optional artifact-type filter |
|
||||
|
||||
### 21.2 Push Referrer
|
||||
|
||||
Options: `--image` (required), `--artifact-type` (required), `--file` (required), `--annotation` (repeatable), `--offline`.
|
||||
|
||||
Builds an OCI image manifest v2 with `subject` field pointing to the target
|
||||
digest. The evidence file becomes a single layer. Config is the OCI empty
|
||||
descriptor. Annotations are passed through to the manifest.
|
||||
|
||||
`--offline` mode simulates the push locally without network, producing the
|
||||
manifest JSON on stdout for auditing.
|
||||
|
||||
### 21.3 List Referrers
|
||||
|
||||
Options: `--image` (required), `--digest` (optional), `--artifact-type` (filter), `--format` (table|json), `--offline`.
|
||||
|
||||
Uses `IOciRegistryClient.GetReferrersAsync()` (already implemented) to query
|
||||
the registry's Referrers API. `--offline` returns simulated data for testing.
|
||||
|
||||
### 21.4 Implementation
|
||||
|
||||
- `EvidenceReferrerCommands.cs` — static command builder class following existing pattern
|
||||
- Wired into `EvidenceCommandGroup.BuildEvidenceCommand()` alongside existing sub-commands
|
||||
- Reuses `IOciRegistryClient` and OCI models from `StellaOps.Cli.Services`
|
||||
- 25 unit tests in `EvidenceReferrerCommandTests.cs`
|
||||
|
||||
@@ -14,7 +14,7 @@ This index lists Concelier connectors, their status, authentication expectations
|
||||
| Ubuntu USN | `ubuntu` | stable | none | [docs/modules/concelier/operations/connectors/ubuntu.md](docs/modules/concelier/operations/connectors/ubuntu.md) |
|
||||
| Red Hat OVAL/CSAF | `redhat` | stable | none | [docs/modules/concelier/operations/connectors/redhat.md](docs/modules/concelier/operations/connectors/redhat.md) |
|
||||
| SUSE OVAL/CSAF | `suse` | stable | none | [docs/modules/concelier/operations/connectors/suse.md](docs/modules/concelier/operations/connectors/suse.md) |
|
||||
| Astra Linux | `astra` | beta | none | [docs/modules/concelier/operations/connectors/astra.md](docs/modules/concelier/operations/connectors/astra.md) |
|
||||
| Astra Linux | `astra` | stable | none | [docs/modules/concelier/operations/connectors/astra.md](docs/modules/concelier/operations/connectors/astra.md) |
|
||||
| CISA KEV | `kev` | stable | none | [docs/modules/concelier/operations/connectors/cve-kev.md](docs/modules/concelier/operations/connectors/cve-kev.md) |
|
||||
| CISA ICS-CERT | `ics-cisa` | stable | none | [docs/modules/concelier/operations/connectors/ics-cisa.md](docs/modules/concelier/operations/connectors/ics-cisa.md) |
|
||||
| CERT-CC | `cert-cc` | stable | none | [docs/modules/concelier/operations/connectors/cert-cc.md](docs/modules/concelier/operations/connectors/cert-cc.md) |
|
||||
|
||||
@@ -496,3 +496,89 @@ DSSE envelope contains:
|
||||
4. **Rotate signing keys periodically**
|
||||
5. **Audit import events**
|
||||
6. **Monitor for duplicate bundle imports**
|
||||
## Snapshot Pinning and Rollback
|
||||
|
||||
> **Sprint:** SPRINT_20260208_035_Concelier_feed_snapshot_coordinator
|
||||
|
||||
### Overview
|
||||
|
||||
Snapshot pinning provides cross-instance coordination for federated deployments. It ensures that:
|
||||
|
||||
- All federated sites can synchronize to a common snapshot version
|
||||
- Failed imports are automatically rolled back to the previous stable state
|
||||
- Concurrent snapshot operations are detected and prevented
|
||||
|
||||
### Services
|
||||
|
||||
The following services are registered by `AddConcelierFederationServices()`:
|
||||
|
||||
| Service | Description |
|
||||
|---------|-------------|
|
||||
| `IFeedSnapshotPinningService` | Low-level snapshot pinning using SyncLedgerRepository |
|
||||
| `ISnapshotIngestionOrchestrator` | High-level orchestration with automatic rollback |
|
||||
|
||||
### Automatic Rollback on Import Failure
|
||||
|
||||
When importing a snapshot bundle, the `ISnapshotIngestionOrchestrator` provides:
|
||||
|
||||
1. **Lock acquisition** - Prevents concurrent operations on the same source
|
||||
2. **Conflict detection** - Checks for cursor conflicts before proceeding
|
||||
3. **Pin-before-import** - Pins the snapshot ID before import begins
|
||||
4. **Automatic rollback** - On import failure, automatically reverts to previous state
|
||||
|
||||
```csharp
|
||||
// Example usage in application code
|
||||
var result = await orchestrator.ImportWithRollbackAsync(
|
||||
inputStream,
|
||||
importOptions,
|
||||
sourceId,
|
||||
cancellationToken);
|
||||
|
||||
if (!result.Success)
|
||||
{
|
||||
if (result.WasRolledBack)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"Import failed but rolled back to {SnapshotId}",
|
||||
result.RolledBackToSnapshotId);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### API Endpoints
|
||||
|
||||
The snapshot pinning service is available through the existing feed snapshot endpoints:
|
||||
|
||||
```
|
||||
POST /api/v1/feeds/snapshot/import
|
||||
```
|
||||
|
||||
When the orchestrator is used, the response includes rollback information:
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"error": "Import failed: invalid bundle format",
|
||||
"was_rolled_back": true,
|
||||
"rolled_back_to_snapshot_id": "snapshot-2024-001"
|
||||
}
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Snapshot pinning uses the same `FederationOptions` as other federation features:
|
||||
|
||||
```yaml
|
||||
Federation:
|
||||
Enabled: true
|
||||
SiteId: "site-us-west-1" # Required for pinning coordination
|
||||
```
|
||||
|
||||
### Monitoring
|
||||
|
||||
Key metrics for snapshot pinning:
|
||||
|
||||
- `snapshot_pin_success_total` - Successful pin operations
|
||||
- `snapshot_pin_failure_total` - Failed pin operations
|
||||
- `snapshot_rollback_total` - Rollback operations triggered
|
||||
- `snapshot_conflict_total` - Conflict detections
|
||||
@@ -1,27 +1,167 @@
|
||||
# Concelier Astra Linux Connector - Operations Runbook
|
||||
|
||||
_Last updated: 2026-01-16_
|
||||
_Last updated: 2026-02-09_
|
||||
|
||||
## 1. Overview
|
||||
The Astra Linux connector ingests regional Astra advisories and maps them to Astra package versions.
|
||||
|
||||
The Astra Linux connector ingests security advisories from the Astra Linux OVAL database and maps them to canonical Advisory records for use in policy decisions and vulnerability management.
|
||||
|
||||
### 1.1 Data Source
|
||||
|
||||
- **Format**: OVAL XML (Open Vulnerability and Assessment Language)
|
||||
- **Source**: Astra Linux official OVAL repository
|
||||
- **Coverage**: Astra Linux SE (Special Edition) packages
|
||||
- **Versioning**: Debian EVR (Epoch:Version-Release) format
|
||||
|
||||
### 1.2 Trust Vector
|
||||
|
||||
| Dimension | Score | Rationale |
|
||||
| --- | --- | --- |
|
||||
| Provenance | 0.95 | Official FSTEC-certified source, government-backed |
|
||||
| Coverage | 0.90 | Comprehensive for Astra-specific packages |
|
||||
| Replayability | 0.85 | OVAL XML is structured and deterministic |
|
||||
|
||||
## 2. Authentication
|
||||
- No authentication required for public feeds unless a mirrored source enforces access controls.
|
||||
|
||||
- No authentication required for public OVAL feeds.
|
||||
- Mirror deployments may require access controls configured at the mirror level.
|
||||
|
||||
## 3. Configuration (`concelier.yaml`)
|
||||
|
||||
```yaml
|
||||
concelier:
|
||||
sources:
|
||||
astra:
|
||||
baseUri: "<astra-advisory-base>"
|
||||
maxDocumentsPerFetch: 20
|
||||
fetchTimeout: "00:00:45"
|
||||
requestDelay: "00:00:00"
|
||||
bulletinBaseUri: "https://astra.ru/en/support/security-bulletins/"
|
||||
ovalRepositoryUri: "https://download.astralinux.ru/astra/stable/oval/"
|
||||
maxDefinitionsPerFetch: 100
|
||||
requestTimeout: "00:02:00"
|
||||
requestDelay: "00:00:00.500"
|
||||
failureBackoff: "00:15:00"
|
||||
initialBackfill: "365.00:00:00"
|
||||
resumeOverlap: "7.00:00:00"
|
||||
userAgent: "StellaOps.Concelier.Astra/1.0 (+https://stella-ops.org)"
|
||||
```
|
||||
|
||||
## 4. Offline and air-gapped deployments
|
||||
- Mirror Astra advisories into the Offline Kit and repoint `baseUri` to the mirror.
|
||||
### 3.1 Configuration Options
|
||||
|
||||
## 5. Common failure modes
|
||||
- Regional mirror availability.
|
||||
- Non-standard versioning metadata.
|
||||
| Option | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `bulletinBaseUri` | - | Base URL for Astra security bulletin pages |
|
||||
| `ovalRepositoryUri` | - | Base URL for OVAL database downloads |
|
||||
| `maxDefinitionsPerFetch` | 100 | Maximum definitions to process per fetch cycle |
|
||||
| `requestTimeout` | 2 min | HTTP request timeout for OVAL downloads |
|
||||
| `requestDelay` | 500ms | Delay between requests to avoid rate limiting |
|
||||
| `failureBackoff` | 15 min | Backoff period after fetch failures |
|
||||
| `initialBackfill` | 365 days | How far back to look on initial sync |
|
||||
| `resumeOverlap` | 7 days | Overlap window when resuming after interruption |
|
||||
|
||||
## 4. OVAL Parsing Pipeline
|
||||
|
||||
### 4.1 Pipeline Stages
|
||||
|
||||
1. **Fetch**: Download OVAL XML database from repository
|
||||
2. **Parse**: Extract vulnerability definitions, tests, objects, and states
|
||||
3. **Map**: Convert OVAL definitions to canonical Advisory records
|
||||
|
||||
### 4.2 OVAL Structure Mapping
|
||||
|
||||
| OVAL Element | Advisory Field | Notes |
|
||||
| --- | --- | --- |
|
||||
| `definition/@id` | fallback `advisoryKey` | Used when no CVE ID present |
|
||||
| `definition/metadata/title` | `title` | |
|
||||
| `definition/metadata/description` | `description` | |
|
||||
| `definition/metadata/reference[@source='CVE']/@ref_id` | `advisoryKey`, `aliases` | First CVE is key, rest are aliases |
|
||||
| `definition/metadata/advisory/severity` | `severity` | |
|
||||
| `definition/metadata/advisory/issued/@date` | `published` | |
|
||||
| `dpkginfo_object/name` | `AffectedPackage.identifier` | |
|
||||
| `dpkginfo_state/evr` | `AffectedVersionRange` | Version constraints |
|
||||
|
||||
### 4.3 Version Comparison
|
||||
|
||||
- Astra Linux is Debian-based and uses **Debian EVR** (Epoch:Version-Release) versioning
|
||||
- Version ranges use `rangeKind: evr` in the canonical model
|
||||
- Comparison follows dpkg version comparison rules
|
||||
|
||||
## 5. Offline and Air-gapped Deployments
|
||||
|
||||
### 5.1 Mirror Setup
|
||||
|
||||
1. Download OVAL databases: `astra-linux-1.7-oval.xml`, etc.
|
||||
2. Place in offline mirror directory
|
||||
3. Update `ovalRepositoryUri` to point to local mirror
|
||||
|
||||
### 5.2 Offline Kit Structure
|
||||
|
||||
```
|
||||
offline-kit/
|
||||
├── concelier/
|
||||
│ └── astra/
|
||||
│ ├── oval/
|
||||
│ │ ├── astra-linux-1.7-oval.xml
|
||||
│ │ └── astra-linux-1.8-oval.xml
|
||||
│ └── manifest.json
|
||||
```
|
||||
|
||||
### 5.3 Configuration for Offline
|
||||
|
||||
```yaml
|
||||
concelier:
|
||||
sources:
|
||||
astra:
|
||||
ovalRepositoryUri: "file:///opt/stella-ops/offline/concelier/astra/oval/"
|
||||
```
|
||||
|
||||
## 6. Common Failure Modes
|
||||
|
||||
### 6.1 Network Issues
|
||||
|
||||
| Symptom | Cause | Resolution |
|
||||
| --- | --- | --- |
|
||||
| Timeout errors | Large OVAL files | Increase `requestTimeout` |
|
||||
| Connection refused | Regional blocking | Use mirror or VPN |
|
||||
| Certificate errors | Proxy/firewall | Configure trusted roots |
|
||||
|
||||
### 6.2 Parsing Errors
|
||||
|
||||
| Error | Cause | Resolution |
|
||||
| --- | --- | --- |
|
||||
| `OvalParseException: Invalid OVAL document` | Wrong namespace or malformed XML | Validate OVAL file manually |
|
||||
| Empty definitions | Missing `definitions` element | Check file is complete |
|
||||
| Missing packages | No linked tests/objects/states | Check OVAL structure |
|
||||
|
||||
### 6.3 Rate Limiting
|
||||
|
||||
- Default `requestDelay: 500ms` should prevent rate limiting
|
||||
- Increase delay if 429 errors occur
|
||||
|
||||
## 7. Monitoring and Alerting
|
||||
|
||||
### 7.1 Key Metrics
|
||||
|
||||
| Metric | Alert Threshold | Description |
|
||||
| --- | --- | --- |
|
||||
| `concelier_fetch_duration_seconds{source="distro-astra"}` | > 300s | Fetch taking too long |
|
||||
| `concelier_parse_errors_total{source="distro-astra"}` | > 0 | Parsing failures |
|
||||
| `concelier_definitions_parsed{source="distro-astra"}` | < 10 | Unusually few definitions |
|
||||
|
||||
### 7.2 Health Check
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:5000/health/sources/distro-astra | jq
|
||||
```
|
||||
|
||||
## 8. Evidence Artifacts
|
||||
|
||||
- Parsed OVAL definitions stored in `DtoStore`
|
||||
- Mapped advisories stored in `AdvisoryStore`
|
||||
- Provenance records include:
|
||||
- Source: `distro-astra`
|
||||
- Kind: `oval-definition`
|
||||
- Original definition ID
|
||||
|
||||
## 9. Related Documentation
|
||||
|
||||
- [Connector Architecture](../../architecture.md)
|
||||
- [Concelier Implementation Notes](../../../../src/Concelier/__Connectors/StellaOps.Concelier.Connector.Astra/IMPLEMENTATION_NOTES.md)
|
||||
- [OVAL Schema Reference](https://oval.mitre.org/language/version5.11/)
|
||||
|
||||
@@ -223,3 +223,37 @@ Capabilities are cached per registry host with a 1-hour TTL.
|
||||
- [ ] Retention policies and pruning jobs configured for staged bundles.
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
## Client Surfacing of Hidden Backend Capabilities
|
||||
|
||||
The `ExportSurfacingClient` extends the existing `ExportCenterClient` by
|
||||
exposing backend capabilities that were previously not surfaced to CLI/UI
|
||||
consumers.
|
||||
|
||||
### Surfaced Capabilities
|
||||
|
||||
| Capability | Interface Method | Route |
|
||||
| --- | --- | --- |
|
||||
| Profile CRUD | `CreateProfileAsync`, `UpdateProfileAsync`, `ArchiveProfileAsync` | `POST/PUT/DELETE /v1/exports/profiles` |
|
||||
| Run Lifecycle | `StartRunAsync`, `CancelRunAsync` | `POST /profiles/{id}/runs`, `POST /runs/{id}/cancel` |
|
||||
| Artifact Browsing | `ListArtifactsAsync`, `GetArtifactAsync`, `DownloadArtifactAsync` | `GET /runs/{id}/artifacts` |
|
||||
| Verification | `VerifyRunAsync`, `GetManifestAsync`, `GetAttestationStatusAsync` | `POST /runs/{id}/verify`, `GET .../manifest`, `GET .../attestation` |
|
||||
| Discovery | `DiscoverCapabilitiesAsync` | Local (15 known capabilities) |
|
||||
|
||||
### Key Types
|
||||
|
||||
| Type | Location | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `IExportSurfacingClient` | `Client/IExportSurfacingClient.cs` | Interface for extended operations |
|
||||
| `ExportSurfacingClient` | `Client/ExportSurfacingClient.cs` | HTTP implementation |
|
||||
| `ExportSurfacingModels.cs` | `Client/Models/` | DTOs for profile CRUD, artifacts, verification, attestation status, capability discovery |
|
||||
|
||||
### DI Registration
|
||||
|
||||
`AddExportSurfacingClient(Action<ExportCenterClientOptions>)` in
|
||||
`ServiceCollectionExtensions.cs` — reuses the same `ExportCenterClientOptions`.
|
||||
|
||||
### Test Coverage (37 tests)
|
||||
|
||||
- Models: CreateExportProfileRequest defaults, UpdateExportProfileRequest nulls, StartExportRunRequest defaults, ExportArtifact roundtrip, empty list, VerifyExportRunRequest defaults, ExportVerificationResult, HashVerificationEntry match/mismatch, SignatureVerificationEntry, ExportManifest, ExportAttestationStatus, ExportCapability, ExportCapabilitySummary, StartExportRunResponse
|
||||
- Client: Constructor null guards (2), DiscoverCapabilities (all/profiles/verification/audit-bundles/openapi-anonymous), argument validation (8 — CreateProfile/ArchiveProfile/CancelRun/StartRun/ListArtifacts/GetArtifact/VerifyRun/GetManifest/GetAttestationStatus)
|
||||
|
||||
@@ -44,8 +44,10 @@ src/Gateway/
|
||||
│ ├── Middleware/
|
||||
│ │ ├── TenantMiddleware.cs # Tenant context extraction
|
||||
│ │ ├── RequestRoutingMiddleware.cs # HTTP → binary routing
|
||||
│ │ ├── AuthenticationMiddleware.cs # DPoP/mTLS validation
|
||||
│ │ └── RateLimitingMiddleware.cs # Per-tenant throttling
|
||||
│ │ ├── SenderConstraintMiddleware.cs # DPoP/mTLS validation
|
||||
│ │ ├── IdentityHeaderPolicyMiddleware.cs # Identity header sanitization
|
||||
│ │ ├── CorrelationIdMiddleware.cs # Request correlation
|
||||
│ │ └── HealthCheckMiddleware.cs # Health probe handling
|
||||
│ ├── Services/
|
||||
│ │ ├── GatewayHostedService.cs # Transport lifecycle
|
||||
│ │ ├── OpenApiAggregationService.cs # Spec aggregation
|
||||
@@ -329,9 +331,37 @@ gateway:
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
- Per-tenant: Configurable requests/minute
|
||||
- Per-identity: Burst protection
|
||||
- Global: Circuit breaker for overload
|
||||
Gateway uses the Router's dual-window rate limiting middleware with circuit breaker:
|
||||
|
||||
- **Instance-level** (in-memory): Per-router-instance limits using sliding window counters
|
||||
- High-precision sub-second buckets for fair rate distribution
|
||||
- No external dependencies; always available
|
||||
- **Environment-level** (Valkey-backed): Cross-instance limits for distributed deployments
|
||||
- Atomic Lua scripts for consistent counting across instances
|
||||
- Circuit breaker pattern for fail-open behavior when Valkey is unavailable
|
||||
- **Activation gate**: Environment-level checks only activate above traffic threshold (configurable)
|
||||
- **Response headers**: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, Retry-After
|
||||
|
||||
Configuration via `appsettings.yaml`:
|
||||
```yaml
|
||||
rate_limiting:
|
||||
process_back_pressure_when_more_than_per_5min: 5000
|
||||
for_instance:
|
||||
rules:
|
||||
- max_requests: 100
|
||||
per_seconds: 1
|
||||
- max_requests: 1000
|
||||
per_seconds: 60
|
||||
for_environment:
|
||||
valkey_connection: "localhost:6379"
|
||||
rules:
|
||||
- max_requests: 10000
|
||||
per_seconds: 60
|
||||
circuit_breaker:
|
||||
failure_threshold: 3
|
||||
timeout_seconds: 30
|
||||
half_open_timeout: 10
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -443,12 +473,80 @@ spec:
|
||||
| Feature | Sprint | Status |
|
||||
|---------|--------|--------|
|
||||
| Core implementation | 3600.0001.0001 | TODO |
|
||||
| Performance Testing Pipeline | 038 | DONE |
|
||||
| WebSocket support | Future | Planned |
|
||||
| gRPC passthrough | Future | Planned |
|
||||
| GraphQL aggregation | Future | Exploration |
|
||||
|
||||
---
|
||||
|
||||
## 14) Performance Testing Pipeline (k6 + Prometheus + Correlation IDs)
|
||||
|
||||
### Overview
|
||||
|
||||
The Gateway includes a comprehensive performance testing pipeline with k6 load tests,
|
||||
Prometheus metric instrumentation, and Grafana dashboards for performance curve modelling.
|
||||
|
||||
### k6 Scenarios (A–G)
|
||||
|
||||
| Scenario | Purpose | VUs | Duration | Key Metric |
|
||||
|----------|---------|-----|----------|------------|
|
||||
| A — Health Baseline | Sub-ms health probe overhead | 10 | 1 min | P95 < 10 ms |
|
||||
| B — OpenAPI Aggregation | Spec cache under concurrent readers | 50 | 75 s | P95 < 200 ms |
|
||||
| C — Routing Throughput | Mixed-method routing at target RPS | 200 | 2 min | P50 < 2 ms, P99 < 5 ms |
|
||||
| D — Correlation ID | Propagation overhead measurement | 20 | 1 min | P95 < 5 ms overhead |
|
||||
| E — Rate Limit Boundary | Enforcement correctness at boundary | 100 | 1 min | Retry-After header |
|
||||
| F — Connection Ramp | Transport saturation (ramp to 1000 VUs) | 1000 | 2 min | No 503 responses |
|
||||
| G — Steady-State Soak | Memory leak / resource exhaustion | 50 | 10 min | Stable memory |
|
||||
|
||||
Run all scenarios:
|
||||
```bash
|
||||
k6 run --env BASE_URL=https://gateway.stella-ops.local src/Gateway/__Tests/load/gateway_performance.k6.js
|
||||
```
|
||||
|
||||
Run a single scenario:
|
||||
```bash
|
||||
k6 run --env BASE_URL=https://gateway.stella-ops.local --env SCENARIO=scenario_c_routing_throughput src/Gateway/__Tests/load/gateway_performance.k6.js
|
||||
```
|
||||
|
||||
### Performance Metrics (GatewayPerformanceMetrics)
|
||||
|
||||
Meter: `StellaOps.Gateway.Performance`
|
||||
|
||||
| Instrument | Type | Unit | Description |
|
||||
|------------|------|------|-------------|
|
||||
| `gateway.requests.total` | Counter | — | Total requests processed |
|
||||
| `gateway.errors.total` | Counter | — | Errors (4xx/5xx) |
|
||||
| `gateway.ratelimit.total` | Counter | — | Rate-limited requests (429) |
|
||||
| `gateway.request.duration` | Histogram | ms | Full request duration |
|
||||
| `gateway.auth.duration` | Histogram | ms | Auth middleware duration |
|
||||
| `gateway.transport.duration` | Histogram | ms | TCP/TLS transport duration |
|
||||
| `gateway.routing.duration` | Histogram | ms | Instance selection duration |
|
||||
|
||||
### Grafana Dashboard
|
||||
|
||||
Dashboard: `devops/telemetry/dashboards/stella-ops-gateway-performance.json`
|
||||
UID: `stella-ops-gateway-performance`
|
||||
|
||||
Panels:
|
||||
1. **Overview row** — P50/P99 gauges, error rate, RPS
|
||||
2. **Latency Distribution** — Percentile time series (overall + per-service)
|
||||
3. **Throughput & Rate Limiting** — RPS by service, rate-limited requests by route
|
||||
4. **Pipeline Breakdown** — Auth/Routing/Transport P95 breakdown, errors by status
|
||||
5. **Connections & Resources** — Active connections, endpoints, memory usage
|
||||
|
||||
### C# Models
|
||||
|
||||
| Type | Purpose |
|
||||
|------|---------|
|
||||
| `GatewayPerformanceObservation` | Single request observation (all pipeline phases) |
|
||||
| `PerformanceScenarioConfig` | Scenario definition with SLO thresholds |
|
||||
| `PerformanceCurvePoint` | Aggregated window data with computed RPS/error rate |
|
||||
| `PerformanceTestSummary` | Complete test run result with threshold violations |
|
||||
| `GatewayPerformanceMetrics` | OTel service emitting Prometheus-compatible metrics |
|
||||
|
||||
---
|
||||
|
||||
## 14) References
|
||||
|
||||
- Router Architecture: `docs/modules/router/architecture.md`
|
||||
|
||||
@@ -31,8 +31,24 @@
|
||||
- `POST /graph/diff` — compares `snapshotA` vs `snapshotB`, streaming node/edge added/removed/changed tiles plus stats; budget enforcement mirrors `/graph/query`.
|
||||
- `POST /graph/export` — async job producing deterministic manifests (`sha256`, size, format) for `ndjson/csv/graphml/png/svg`; download via `/graph/export/{jobId}`.
|
||||
- `POST /graph/lineage` - returns SBOM lineage nodes/edges anchored by `artifactDigest` or `sbomDigest`, with optional relationship filters and depth limits.
|
||||
- **Edge Metadata API** (added 2025-01):
|
||||
- `POST /graph/edges/metadata` — batch query for edge explanations; request contains `EdgeIds[]`, response includes `EdgeTileWithMetadata[]` with full provenance.
|
||||
- `GET /graph/edges/{edgeId}/metadata` — single edge metadata with explanation, via, provenance, and evidence references.
|
||||
- `GET /graph/edges/path/{sourceNodeId}/{targetNodeId}` — returns all edges on the shortest path between two nodes, each with metadata.
|
||||
- `GET /graph/edges/by-reason/{reason}` — query edges by `EdgeReason` enum (e.g., `SbomDependency`, `AdvisoryAffects`, `VexStatement`, `RuntimeTrace`).
|
||||
- `GET /graph/edges/by-evidence?evidenceType=&evidenceRef=` — query edges by evidence reference.
|
||||
- Legacy: `GET /graph/nodes/{id}`, `POST /graph/query/saved`, `GET /graph/impact/{advisoryKey}`, `POST /graph/overlay/policy` remain in spec but should align to the NDJSON surfaces above as they are brought forward.
|
||||
|
||||
### 3.1) Edge Metadata Contracts
|
||||
|
||||
The edge metadata system provides explainability for graph relationships:
|
||||
|
||||
- **EdgeReason** enum: `Unknown`, `SbomDependency`, `StaticSymbol`, `RuntimeTrace`, `PackageManifest`, `Lockfile`, `BuildArtifact`, `ImageLayer`, `AdvisoryAffects`, `VexStatement`, `PolicyOverlay`, `AttestationRef`, `OperatorAnnotation`, `TransitiveInference`, `Provenance`.
|
||||
- **EdgeVia** record: Describes how the edge was discovered (method, version, timestamp, confidence in basis points, evidence reference).
|
||||
- **EdgeExplanationPayload** record: Full explanation including reason, via, human-readable summary, evidence list, provenance reference, and tags.
|
||||
- **EdgeProvenanceRef** record: Source system, collection timestamp, SBOM digest, scan digest, attestation ID, event offset.
|
||||
- **EdgeTileWithMetadata** record: Extends `EdgeTile` with `Explanation` property containing the full metadata.
|
||||
|
||||
## 4) Storage considerations
|
||||
|
||||
- Backed by either:
|
||||
|
||||
@@ -58,3 +58,50 @@ The `StellaOpsMirror` connector in Concelier handles:
|
||||
* Concelier: `../concelier/architecture.md`
|
||||
* AirGap: `../airgap/architecture.md`
|
||||
* Provenance observers: `./provenance/observers.md`
|
||||
|
||||
---
|
||||
|
||||
## 3) Mirror Creator Core (2026-02-08)
|
||||
|
||||
Sprint `SPRINT_20260208_041_Mirror_mirror_creator` adds a deterministic core library at:
|
||||
|
||||
- `src/Mirror/StellaOps.Mirror.Creator/StellaOps.Mirror.Creator.Core.csproj`
|
||||
|
||||
### Implemented Contracts
|
||||
|
||||
- `IMirrorCreatorService`
|
||||
- `UpsertSourceAsync(MirrorSourceConfiguration source, CancellationToken cancellationToken = default)`
|
||||
- `GetSourcesAsync(string tenantId, CancellationToken cancellationToken = default)`
|
||||
- `CreateSyncPlanAsync(MirrorSyncRequest request, CancellationToken cancellationToken = default)`
|
||||
- `RecordSyncResultAsync(MirrorSyncResult result, CancellationToken cancellationToken = default)`
|
||||
- Model types in `MirrorModels.cs`:
|
||||
- `MirrorSourceConfiguration`
|
||||
- `MirrorSyncRequest`
|
||||
- `MirrorSyncPlan` and `MirrorSyncPlanItem`
|
||||
- `MirrorSyncResult`
|
||||
- `MirrorContentKind` and `MirrorSyncMode`
|
||||
- Options in `MirrorCreatorOptions` with configurable `OutputRoot`.
|
||||
- DI registration in `MirrorServiceCollectionExtensions.AddMirrorCreator(...)`.
|
||||
|
||||
### Determinism Guarantees
|
||||
|
||||
- Tenant and source IDs are normalized to lowercase-trimmed values.
|
||||
- Source ordering is stable (ordinal sort by source ID per tenant).
|
||||
- Plan IDs are generated from canonical plan content using SHA-256.
|
||||
- Output bundle path format is stable:
|
||||
- `<outputRoot>/<tenant>/<source>/<yyyyMMddHHmmss>.bundle.json`
|
||||
- Sync mode behavior:
|
||||
- `Full` when no prior cursor exists.
|
||||
- `Incremental` after successful cursor recording via `RecordSyncResultAsync`.
|
||||
|
||||
### Test Evidence
|
||||
|
||||
- Test project: `src/Mirror/__Tests/StellaOps.Mirror.Creator.Core.Tests/`
|
||||
- Executed: `dotnet test src/Mirror/__Tests/StellaOps.Mirror.Creator.Core.Tests/StellaOps.Mirror.Creator.Core.Tests.csproj`
|
||||
- Result on 2026-02-08: Passed `4/4` tests.
|
||||
|
||||
### Current Boundaries
|
||||
|
||||
- Implementation is currently in-memory and does not persist checkpoints to a backing store.
|
||||
- No dedicated HTTP endpoints or CLI command group are added in this sprint.
|
||||
- Runtime mirror transport/execution remains the responsibility of future integration work.
|
||||
|
||||
@@ -29,14 +29,93 @@
|
||||
- Circuit breakers automatically pause job types when failure rate > configured threshold; incidents generated via Notify and Observability stack.
|
||||
- Control plane quota updates require Authority scope `orch:quota` (issued via `Orch.Admin` role). Historical rebuilds/backfills additionally require `orch:backfill` and must supply `backfill_reason` and `backfill_ticket` alongside the operator metadata. Authority persists all four fields (`quota_reason`, `quota_ticket`, `backfill_reason`, `backfill_ticket`) for audit replay.
|
||||
|
||||
### 3.1) Quota governance service
|
||||
|
||||
The `QuotaGovernanceService` provides cross-tenant quota allocation with configurable policies:
|
||||
|
||||
**Allocation strategies:**
|
||||
- `Equal` — Divide total capacity equally among all active tenants.
|
||||
- `Proportional` — Allocate based on tenant weight/priority tier.
|
||||
- `Priority` — Higher priority tenants get allocation first, with preemption.
|
||||
- `ReservedWithFairShare` — Reserved minimum per tenant, remainder distributed fairly.
|
||||
- `Fixed` — Static allocation per tenant regardless of demand.
|
||||
|
||||
**Key operations:**
|
||||
- `CalculateAllocationAsync` — Compute quota for a tenant based on active policies.
|
||||
- `RequestQuotaAsync` — Request quota from shared pool; returns granted amount with burst usage.
|
||||
- `ReleaseQuotaAsync` — Return quota to shared pool after job completion.
|
||||
- `CanScheduleAsync` — Check scheduling eligibility combining quota and circuit breaker state.
|
||||
|
||||
**Quota allocation policy properties:**
|
||||
- `TotalCapacity` — Pool size to allocate from (for proportional/fair strategies).
|
||||
- `MinimumPerTenant` / `MaximumPerTenant` — Allocation bounds.
|
||||
- `ReservedCapacity` — Guaranteed capacity for high-priority tenants.
|
||||
- `AllowBurst` / `BurstMultiplier` — Allow temporary overallocation when capacity exists.
|
||||
- `Priority` — Policy evaluation order (higher = first).
|
||||
- `JobType` — Optional job type filter (null = applies to all).
|
||||
|
||||
### 3.2) Circuit breaker service
|
||||
|
||||
The `CircuitBreakerService` implements the circuit breaker pattern for downstream services:
|
||||
|
||||
**States:**
|
||||
- `Closed` — Normal operation; requests pass through. Failures are tracked.
|
||||
- `Open` — Circuit tripped; requests are blocked for `OpenDuration`. Prevents cascade failures.
|
||||
- `HalfOpen` — After open duration, limited test requests allowed. Success → Closed; Failure → Open.
|
||||
|
||||
**Thresholds:**
|
||||
- `FailureThreshold` (0.0–1.0) — Failure rate that triggers circuit open.
|
||||
- `WindowDuration` — Sliding window for failure rate calculation.
|
||||
- `MinimumSamples` — Minimum requests before circuit can trip.
|
||||
- `OpenDuration` — How long circuit stays open before half-open transition.
|
||||
- `HalfOpenTestCount` — Number of test requests allowed in half-open state.
|
||||
|
||||
**Key operations:**
|
||||
- `CheckAsync` — Verify if request is allowed; returns `CircuitBreakerCheckResult`.
|
||||
- `RecordSuccessAsync` / `RecordFailureAsync` — Update circuit state after request.
|
||||
- `ForceOpenAsync` / `ForceCloseAsync` — Manual operator intervention (audited).
|
||||
- `ListAsync` — View all circuit breakers for a tenant with optional state filter.
|
||||
|
||||
**Downstream services protected:**
|
||||
- Scanner
|
||||
- Attestor
|
||||
- Policy Engine
|
||||
- Registry clients
|
||||
- External integrations
|
||||
|
||||
## 4) APIs
|
||||
|
||||
### 4.1) Job management
|
||||
- `GET /api/jobs?status=` — list jobs with filters (tenant, jobType, status, time window).
|
||||
- `GET /api/jobs/{id}` — job detail (payload digest, attempts, worker, lease history, metrics).
|
||||
- `POST /api/jobs/{id}/cancel` — cancel running/pending job with audit reason.
|
||||
- `POST /api/jobs/{id}/replay` — schedule replay.
|
||||
- `POST /api/limits/throttle` — apply throttle (requires elevated scope).
|
||||
- `GET /api/dashboard/metrics` — aggregated metrics for Console dashboards.
|
||||
|
||||
### 4.2) Circuit breaker endpoints (`/api/v1/orchestrator/circuit-breakers`)
|
||||
- `GET /` — List all circuit breakers for tenant (optional `?state=` filter).
|
||||
- `GET /{serviceId}` — Get circuit breaker state for specific downstream service.
|
||||
- `GET /{serviceId}/check` — Check if requests are allowed; returns `IsAllowed`, `State`, `FailureRate`, `TimeUntilRetry`.
|
||||
- `POST /{serviceId}/success` — Record successful request to downstream service.
|
||||
- `POST /{serviceId}/failure` — Record failed request (body: `failureReason`).
|
||||
- `POST /{serviceId}/force-open` — Manually open circuit (body: `reason`; audited).
|
||||
- `POST /{serviceId}/force-close` — Manually close circuit (audited).
|
||||
|
||||
### 4.3) Quota governance endpoints (`/api/v1/orchestrator/quota-governance`)
|
||||
- `GET /policies` — List quota allocation policies (optional `?enabled=` filter).
|
||||
- `GET /policies/{policyId}` — Get specific policy.
|
||||
- `POST /policies` — Create new policy.
|
||||
- `PUT /policies/{policyId}` — Update policy.
|
||||
- `DELETE /policies/{policyId}` — Delete policy.
|
||||
- `GET /allocation` — Calculate allocation for current tenant (optional `?jobType=`).
|
||||
- `POST /request` — Request quota from pool (body: `jobType`, `requestedAmount`).
|
||||
- `POST /release` — Release quota back to pool (body: `jobType`, `releasedAmount`).
|
||||
- `GET /status` — Get tenant quota status (optional `?jobType=`).
|
||||
- `GET /summary` — Get quota governance summary across all tenants (optional `?policyId=`).
|
||||
- `GET /can-schedule` — Check if job can be scheduled (optional `?jobType=`).
|
||||
|
||||
### 4.4) Discovery and documentation
|
||||
- Event envelope draft (`docs/modules/orchestrator/event-envelope.md`) defines notifier/webhook/SSE payloads with idempotency keys, provenance, and task runner metadata for job/pack-run events.
|
||||
- OpenAPI discovery: `/.well-known/openapi` exposes `/openapi/orchestrator.json` (OAS 3.1) with pagination/idempotency/error-envelope examples; legacy job detail/summary endpoints now ship `Deprecation` + `Link` headers that point to their replacements.
|
||||
|
||||
|
||||
@@ -177,6 +177,70 @@ Determinization scores are exposed to SPL policies via the `signals.trust.*` and
|
||||
|
||||
EWS weights are externalized to versioned JSON manifests in `etc/weights/`. The unified score facade (`IUnifiedScoreService`) loads weights from these manifests rather than using compiled defaults, enabling auditable weight changes without code modifications. See [Unified Score Architecture](../../technical/scoring-algebra.md) §4 for manifest schema and versioning rules.
|
||||
|
||||
### 3.1.1 · Trust Score Algebra Facade
|
||||
|
||||
The **TrustScoreAlgebraFacade** (`ITrustScoreAlgebraFacade`) provides a unified entry point composing TrustScoreAggregator + K4Lattice + ScorePolicy into a single deterministic scoring pipeline.
|
||||
|
||||
```csharp
|
||||
public interface ITrustScoreAlgebraFacade
|
||||
{
|
||||
Task<TrustScoreResult> ComputeTrustScoreAsync(TrustScoreRequest request, CancellationToken ct);
|
||||
TrustScoreResult ComputeTrustScore(TrustScoreRequest request);
|
||||
}
|
||||
```
|
||||
|
||||
**Pipeline steps:**
|
||||
1. Calculate uncertainty entropy from signal snapshot
|
||||
2. Aggregate weighted signal scores via TrustScoreAggregator
|
||||
3. Compute K4 lattice verdict (Unknown/True/False/Conflict)
|
||||
4. Extract dimension scores (BaseSeverity, Reachability, Evidence, Provenance)
|
||||
5. Compute weighted final score in basis points (0-10000)
|
||||
6. Determine risk tier (Info/Low/Medium/High/Critical)
|
||||
7. Produce Score.v1 predicate for DSSE attestation
|
||||
|
||||
**Score.v1 Predicate Format:**
|
||||
|
||||
All numeric scores use **basis points (0-10000)** for bit-exact determinism:
|
||||
|
||||
```json
|
||||
{
|
||||
"predicateType": "https://stella-ops.org/predicates/score/v1",
|
||||
"artifactId": "pkg:maven/com.example/mylib@1.0.0",
|
||||
"vulnerabilityId": "CVE-2024-1234",
|
||||
"trustScoreBps": 7250,
|
||||
"tier": "High",
|
||||
"latticeVerdict": "True",
|
||||
"uncertaintyBps": 2500,
|
||||
"dimensions": {
|
||||
"baseSeverityBps": 5000,
|
||||
"reachabilityBps": 10000,
|
||||
"evidenceBps": 6000,
|
||||
"provenanceBps": 8000,
|
||||
"epssBps": 3500,
|
||||
"vexBps": 10000
|
||||
},
|
||||
"weightsUsed": {
|
||||
"baseSeverity": 1000,
|
||||
"reachability": 4500,
|
||||
"evidence": 3000,
|
||||
"provenance": 1500
|
||||
},
|
||||
"policyDigest": "sha256:abc123...",
|
||||
"computedAt": "2026-01-15T12:00:00Z",
|
||||
"tenantId": "tenant-123"
|
||||
}
|
||||
```
|
||||
|
||||
**Risk Tier Mapping:**
|
||||
|
||||
| Score (bps) | Tier |
|
||||
|-------------|------|
|
||||
| ≥ 9000 | Critical |
|
||||
| ≥ 7000 | High |
|
||||
| ≥ 4000 | Medium |
|
||||
| ≥ 1000 | Low |
|
||||
| < 1000 | Info |
|
||||
|
||||
### 3.2 - License compliance configuration
|
||||
|
||||
License compliance evaluation runs during SBOM evaluation when enabled in
|
||||
@@ -871,6 +935,7 @@ The Interop Layer provides bidirectional policy exchange between Stella's native
|
||||
| Format | Schema | Direction | Notes |
|
||||
|--------|--------|-----------|-------|
|
||||
| **PolicyPack v2 (JSON)** | `policy.stellaops.io/v2` | Import + Export | Canonical format with typed gates, environment overrides, remediation hints |
|
||||
| **PolicyPack v2 (YAML)** | `policy.stellaops.io/v2` | Import + Export | Deterministic YAML with sorted keys; YAML→JSON roundtrip for validation |
|
||||
| **OPA/Rego** | `package stella.release` | Export (+ Import with pattern matching) | Deny-by-default pattern, `remediation` output rules |
|
||||
|
||||
### 13.2 · Architecture
|
||||
@@ -878,8 +943,9 @@ The Interop Layer provides bidirectional policy exchange between Stella's native
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Interop["StellaOps.Policy.Interop"]
|
||||
Exporter[JsonPolicyExporter / RegoPolicyExporter]
|
||||
Importer[JsonPolicyImporter / RegoPolicyImporter]
|
||||
Exporter[JsonPolicyExporter / YamlPolicyExporter / RegoPolicyExporter]
|
||||
Importer[JsonPolicyImporter / YamlPolicyImporter / RegoPolicyImporter]
|
||||
DiffMerge[PolicyDiffMergeEngine]
|
||||
Validator[PolicySchemaValidator]
|
||||
Generator[RegoCodeGenerator]
|
||||
Resolver[RemediationResolver]
|
||||
@@ -887,7 +953,7 @@ graph TD
|
||||
Detector[FormatDetector]
|
||||
end
|
||||
subgraph Consumers
|
||||
CLI[stella policy export/import/validate/evaluate]
|
||||
CLI[stella policy export/import/validate/evaluate/diff/merge]
|
||||
API[Platform API /api/v1/policy/interop]
|
||||
UI[Policy Editor UI]
|
||||
end
|
||||
@@ -895,9 +961,11 @@ graph TD
|
||||
CLI --> Exporter
|
||||
CLI --> Importer
|
||||
CLI --> Validator
|
||||
CLI --> DiffMerge
|
||||
API --> Exporter
|
||||
API --> Importer
|
||||
API --> Validator
|
||||
API --> DiffMerge
|
||||
UI --> API
|
||||
|
||||
Exporter --> Generator
|
||||
@@ -946,7 +1014,51 @@ All exports and evaluations are deterministic:
|
||||
- No time-dependent logic in deterministic mode
|
||||
- `outputDigest` in evaluation results enables replay verification
|
||||
|
||||
### 13.6 · Implementation Reference
|
||||
### 13.6 · YAML Format Support
|
||||
|
||||
> **Sprint:** SPRINT_20260208_048_Policy_policy_interop_framework
|
||||
|
||||
YAML export/import operates on the same `PolicyPackDocument` model as JSON. The YAML format is useful for human-editable policy files and GitOps workflows.
|
||||
|
||||
**Export** (`YamlPolicyExporter : IPolicyYamlExporter`):
|
||||
- Converts `PolicyPackDocument` to a `SortedDictionary` intermediate for deterministic key ordering
|
||||
- Serializes via YamlDotNet (CamelCaseNamingConvention, DisableAliases, OmitNull)
|
||||
- Produces SHA-256 digest for replay verification
|
||||
- Supports environment filtering and remediation stripping (same options as JSON)
|
||||
|
||||
**Import** (`YamlPolicyImporter`):
|
||||
- Deserializes YAML via YamlDotNet, then re-serializes as JSON
|
||||
- Delegates to `JsonPolicyImporter` for validation (apiVersion, kind, duplicate gates/rules)
|
||||
- Errors: `YAML_PARSE_ERROR`, `YAML_EMPTY`, `YAML_CONVERSION_ERROR`
|
||||
|
||||
**Format Detection** (`FormatDetector`):
|
||||
- Content-based: detects `apiVersion:`, `---`, `kind:` patterns
|
||||
- Extension-based: `.yaml`, `.yml` → `PolicyFormats.Yaml`
|
||||
|
||||
### 13.7 · Policy Diff/Merge Engine
|
||||
|
||||
> **Sprint:** SPRINT_20260208_048_Policy_policy_interop_framework
|
||||
|
||||
The `PolicyDiffMergeEngine` (`IPolicyDiffMerge`) compares and merges `PolicyPackDocument` instances structurally.
|
||||
|
||||
**Diff** produces `PolicyDiffResult` containing:
|
||||
- Changes to metadata (name, version, description)
|
||||
- Changes to settings (defaultAction, unknownsThreshold, stopOnFirstFailure, deterministicMode)
|
||||
- Gate changes (by ID): added, removed, modified (action, type, config diffs)
|
||||
- Rule changes (by Name): added, removed, modified (action, match diffs)
|
||||
- Summary with counts of added/removed/modified and `HasChanges` flag
|
||||
|
||||
**Merge** applies one of three strategies via `PolicyMergeStrategy`:
|
||||
|
||||
| Strategy | Behavior |
|
||||
|----------|----------|
|
||||
| `OverlayWins` | Overlay values take precedence on conflict |
|
||||
| `BaseWins` | Base values take precedence on conflict |
|
||||
| `FailOnConflict` | Returns error with conflict details |
|
||||
|
||||
Merge output includes the merged `PolicyPackDocument` and a list of `PolicyMergeConflict` items (path, base value, overlay value).
|
||||
|
||||
### 13.8 · Implementation Reference
|
||||
|
||||
| Component | Source File |
|
||||
|-----------|-------------|
|
||||
@@ -954,18 +1066,22 @@ All exports and evaluations are deterministic:
|
||||
| Remediation Models | `src/Policy/__Libraries/StellaOps.Policy.Interop/Contracts/RemediationModels.cs` |
|
||||
| Interfaces | `src/Policy/__Libraries/StellaOps.Policy.Interop/Abstractions/` |
|
||||
| JSON Exporter | `src/Policy/__Libraries/StellaOps.Policy.Interop/Export/JsonPolicyExporter.cs` |
|
||||
| YAML Exporter | `src/Policy/__Libraries/StellaOps.Policy.Interop/Export/YamlPolicyExporter.cs` |
|
||||
| JSON Importer | `src/Policy/__Libraries/StellaOps.Policy.Interop/Import/JsonPolicyImporter.cs` |
|
||||
| YAML Importer | `src/Policy/__Libraries/StellaOps.Policy.Interop/Import/YamlPolicyImporter.cs` |
|
||||
| Rego Generator | `src/Policy/__Libraries/StellaOps.Policy.Interop/Rego/RegoCodeGenerator.cs` |
|
||||
| Rego Importer | `src/Policy/__Libraries/StellaOps.Policy.Interop/Import/RegoPolicyImporter.cs` |
|
||||
| Diff/Merge Engine | `src/Policy/__Libraries/StellaOps.Policy.Interop/DiffMerge/PolicyDiffMergeEngine.cs` |
|
||||
| Embedded OPA | `src/Policy/__Libraries/StellaOps.Policy.Interop/Evaluation/EmbeddedOpaEvaluator.cs` |
|
||||
| Remediation Resolver | `src/Policy/__Libraries/StellaOps.Policy.Interop/Evaluation/RemediationResolver.cs` |
|
||||
| Format Detector | `src/Policy/__Libraries/StellaOps.Policy.Interop/Import/FormatDetector.cs` |
|
||||
| Schema Validator | `src/Policy/__Libraries/StellaOps.Policy.Interop/Validation/PolicySchemaValidator.cs` |
|
||||
| DI Registration | `src/Policy/__Libraries/StellaOps.Policy.Interop/DependencyInjection/PolicyInteropServiceCollectionExtensions.cs` |
|
||||
| CLI Commands | `src/Cli/StellaOps.Cli/Commands/Policy/PolicyInteropCommandGroup.cs` |
|
||||
| Platform API | `src/Platform/StellaOps.Platform.WebService/Endpoints/PolicyInteropEndpoints.cs` |
|
||||
| JSON Schema | `docs/schemas/policy-pack-v2.schema.json` |
|
||||
|
||||
### 13.7 · CLI Interface
|
||||
### 13.9 · CLI Interface
|
||||
|
||||
```bash
|
||||
# Export to Rego
|
||||
@@ -983,7 +1099,7 @@ stella policy evaluate --policy baseline.json --input evidence.json --environmen
|
||||
|
||||
Exit codes: `0` = success/allow, `1` = warn, `2` = block/errors, `10` = input-error, `12` = policy-error.
|
||||
|
||||
### 13.8 · Platform API
|
||||
### 13.10 · Platform API
|
||||
|
||||
Group: `/api/v1/policy/interop` with tag `PolicyInterop`
|
||||
|
||||
@@ -995,7 +1111,7 @@ Group: `/api/v1/policy/interop` with tag `PolicyInterop`
|
||||
| POST | `/evaluate` | `platform.policy.evaluate` | Evaluate policy against input |
|
||||
| GET | `/formats` | `platform.policy.read` | List supported formats |
|
||||
|
||||
### 13.9 · OPA Supply Chain Evidence Input
|
||||
### 13.11 · OPA Supply Chain Evidence Input
|
||||
|
||||
> **Sprint:** SPRINT_0129_001_Policy_supply_chain_evidence_input
|
||||
|
||||
@@ -1061,4 +1177,107 @@ allow {
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2026-01-29 (Sprint 0129_001).*
|
||||
*Last updated: 2026-02-09 (Sprint 049 — Proof Studio UX).*
|
||||
|
||||
## 14 · Proof Studio (Explainable Confidence Scoring)
|
||||
|
||||
The Proof Studio UX provides visual, auditable evidence chains for every verdict decision. It bridges existing verdict rationale, score explanation, and counterfactual simulation data into composable views.
|
||||
|
||||
### 14.1 · Library Layout
|
||||
|
||||
```
|
||||
StellaOps.Policy.Explainability/
|
||||
├── VerdictRationale.cs # 4-line structured rationale model
|
||||
├── VerdictRationaleRenderer.cs # Content-addressed render (text/md/JSON)
|
||||
├── IVerdictRationaleRenderer.cs # Renderer interface
|
||||
├── ProofGraphModels.cs # Proof graph DAG types
|
||||
├── ProofGraphBuilder.cs # Deterministic graph builder
|
||||
├── ScoreBreakdownDashboard.cs # Score breakdown dashboard model
|
||||
├── ProofStudioService.cs # Composition + counterfactual integration
|
||||
├── ServiceCollectionExtensions.cs # DI registration
|
||||
└── GlobalUsings.cs
|
||||
```
|
||||
|
||||
### 14.2 · Proof Graph
|
||||
|
||||
A proof graph is a directed acyclic graph (DAG) that visualizes the complete evidence chain from source artifacts to a final verdict decision.
|
||||
|
||||
| Node Type | Depth | Purpose |
|
||||
|---|---|---|
|
||||
| `Verdict` | 0 | Root: the final verdict + composite score |
|
||||
| `PolicyRule` | 1 | Policy clause that triggered the decision |
|
||||
| `Guardrail` | 1 | Score guardrail (cap/floor) that modified the score |
|
||||
| `ScoreComputation` | 2 | Per-factor score contribution |
|
||||
| `ReachabilityAnalysis` | 3 | Reachability evidence leaf |
|
||||
| `VexStatement` | 3 | VEX attestation leaf |
|
||||
| `Provenance` | 3 | Provenance attestation leaf |
|
||||
| `SbomEvidence` | 3 | SBOM evidence leaf |
|
||||
| `RuntimeSignal` | 3 | Runtime detection signal leaf |
|
||||
| `AdvisoryData` | 3 | Advisory data leaf |
|
||||
| `Counterfactual` | 0 | What-if hypothesis (overlay) |
|
||||
|
||||
Edge relations: `ProvidesEvidence`, `ContributesScore`, `Gates`, `Attests`, `Overrides`, `GuardrailApplied`.
|
||||
|
||||
Graph IDs are content-addressed (`pg:sha256:...`) from deterministic sorting of node and edge identifiers.
|
||||
|
||||
### 14.3 · Score Breakdown Dashboard
|
||||
|
||||
The `ScoreBreakdownDashboard` exposes per-factor contributions with weighted contributions and percentages:
|
||||
|
||||
```
|
||||
ScoreBreakdownDashboard
|
||||
├── CompositeScore (int)
|
||||
├── ActionBucket (string)
|
||||
├── Factors[] → FactorContribution
|
||||
│ ├── FactorId / FactorName
|
||||
│ ├── RawScore, Weight → WeightedContribution (computed)
|
||||
│ ├── Confidence, IsSubtractive
|
||||
│ └── PercentageOfTotal
|
||||
├── GuardrailsApplied[] → GuardrailApplication
|
||||
│ ├── ScoreBefore → ScoreAfter
|
||||
│ └── Reason, Conditions
|
||||
├── PreGuardrailScore
|
||||
├── Entropy
|
||||
└── NeedsReview
|
||||
```
|
||||
|
||||
### 14.4 · Counterfactual Explorer
|
||||
|
||||
The `AddCounterfactualOverlay()` method on `IProofGraphBuilder` adds hypothetical nodes to an existing proof graph. A `CounterfactualScenario` specifies factor overrides (factorId → hypothetical score) and an optional resulting composite score. The overlay:
|
||||
|
||||
1. Creates a `Counterfactual` node at depth 0 with the scenario label.
|
||||
2. Connects overridden factor score nodes to the counterfactual node via `Overrides` edges.
|
||||
3. Recomputes the content-addressed graph ID, making each scenario distinctly identifiable.
|
||||
|
||||
### 14.5 · Proof Studio Service (Integration)
|
||||
|
||||
The `IProofStudioService` is the primary integration surface:
|
||||
|
||||
| Method | Input | Output |
|
||||
|---|---|---|
|
||||
| `Compose(request)` | `ProofStudioRequest` (rationale + optional score factors + guardrails) | `ProofStudioView` (proof graph + optional score breakdown) |
|
||||
| `ApplyCounterfactual(view, scenario)` | Existing view + `CounterfactualScenario` | Updated view with overlay |
|
||||
|
||||
The service bridges `ScoreFactorInput` (from scoring engine) to `FactorContribution` models and formats factor names for UI display.
|
||||
|
||||
### 14.6 · DI Registration
|
||||
|
||||
```csharp
|
||||
services.AddVerdictExplainability();
|
||||
// Registers:
|
||||
// IVerdictRationaleRenderer → VerdictRationaleRenderer
|
||||
// IProofGraphBuilder → ProofGraphBuilder
|
||||
// IProofStudioService → ProofStudioService
|
||||
```
|
||||
|
||||
### 14.7 · OTel Metrics
|
||||
|
||||
| Metric | Type | Description |
|
||||
|---|---|---|
|
||||
| `stellaops.proofstudio.views_composed_total` | Counter | Proof studio views composed |
|
||||
| `stellaops.proofstudio.counterfactuals_applied_total` | Counter | Counterfactual scenarios applied |
|
||||
|
||||
### 14.8 · Tests
|
||||
|
||||
- `ProofGraphBuilderTests.cs` — 18 tests (graph construction, determinism, depth hierarchy, critical paths, counterfactual overlay, edge cases)
|
||||
- `ProofStudioServiceTests.cs` — 10 tests (compose, score breakdown, guardrails, counterfactual, DI resolution)
|
||||
|
||||
@@ -429,3 +429,151 @@ Policy:
|
||||
MinConfidence: 0.75
|
||||
MaxEntropy: 0.3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13. Delta-If-Present Calculations (TSF-004)
|
||||
|
||||
> **Sprint:** SPRINT_20260208_043_Policy_delta_if_present_calculations_for_missing_signals
|
||||
|
||||
The Delta-If-Present API provides "what-if" analysis for missing signals, showing hypothetical score changes if specific evidence were obtained.
|
||||
|
||||
### 13.1 Purpose
|
||||
|
||||
When making release decisions with incomplete evidence, operators need to understand:
|
||||
- **Gap prioritization:** Which missing signals would have the most impact?
|
||||
- **Score bounds:** What is the possible range of trust scores given current gaps?
|
||||
- **Risk simulation:** What would the score be if a missing signal had a specific value?
|
||||
|
||||
### 13.2 API Endpoints
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/api/v1/policy/delta-if-present/signal` | POST | Calculate delta for a single signal |
|
||||
| `/api/v1/policy/delta-if-present/analysis` | POST | Full gap analysis with prioritization |
|
||||
| `/api/v1/policy/delta-if-present/bounds` | POST | Calculate min/max score bounds |
|
||||
|
||||
### 13.3 Single Signal Delta
|
||||
|
||||
Calculate hypothetical score change for one missing signal:
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"snapshot": {
|
||||
"cve": "CVE-2024-1234",
|
||||
"purl": "pkg:maven/org.example/lib@1.0.0",
|
||||
"vex": { "state": "not_queried" },
|
||||
"epss": { "state": "not_queried" },
|
||||
"reachability": {
|
||||
"state": "queried",
|
||||
"value": { "status": "Reachable", "analyzed_at": "2026-01-15T00:00:00Z" }
|
||||
},
|
||||
"runtime": { "state": "not_queried" },
|
||||
"backport": { "state": "not_queried" },
|
||||
"sbom": {
|
||||
"state": "queried",
|
||||
"value": { "sbom_digest": "sha256:abc", "format": "SPDX" }
|
||||
}
|
||||
},
|
||||
"signal_name": "VEX",
|
||||
"assumed_value": 0.0
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"signal": "VEX",
|
||||
"current_score": 0.65,
|
||||
"hypothetical_score": 0.52,
|
||||
"score_delta": -0.13,
|
||||
"assumed_value": 0.0,
|
||||
"signal_weight": 0.25,
|
||||
"current_entropy": 0.60,
|
||||
"hypothetical_entropy": 0.35,
|
||||
"entropy_delta": -0.25
|
||||
}
|
||||
```
|
||||
|
||||
### 13.4 Full Gap Analysis
|
||||
|
||||
Analyze all missing signals with best/worst/prior cases:
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"cve": "CVE-2024-1234",
|
||||
"purl": "pkg:maven/org.example/lib@1.0.0",
|
||||
"current_score": 0.65,
|
||||
"current_entropy": 0.60,
|
||||
"gap_analysis": [
|
||||
{
|
||||
"signal": "VEX",
|
||||
"gap_reason": "NotQueried",
|
||||
"best_case": {
|
||||
"assumed_value": 0.0,
|
||||
"hypothetical_score": 0.52,
|
||||
"score_delta": -0.13
|
||||
},
|
||||
"worst_case": {
|
||||
"assumed_value": 1.0,
|
||||
"hypothetical_score": 0.77,
|
||||
"score_delta": 0.12
|
||||
},
|
||||
"prior_case": {
|
||||
"assumed_value": 0.5,
|
||||
"hypothetical_score": 0.64,
|
||||
"score_delta": -0.01
|
||||
},
|
||||
"max_impact": 0.25
|
||||
}
|
||||
],
|
||||
"prioritized_gaps": ["VEX", "Reachability", "EPSS", "Runtime", "Backport", "SBOMLineage"],
|
||||
"computed_at": "2026-01-15T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### 13.5 Score Bounds
|
||||
|
||||
Calculate the possible range of trust scores:
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"cve": "CVE-2024-1234",
|
||||
"purl": "pkg:maven/org.example/lib@1.0.0",
|
||||
"current_score": 0.65,
|
||||
"current_entropy": 0.60,
|
||||
"minimum_score": 0.35,
|
||||
"maximum_score": 0.85,
|
||||
"range": 0.50,
|
||||
"gap_count": 4,
|
||||
"missing_weight_percentage": 65.0,
|
||||
"computed_at": "2026-01-15T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### 13.6 Signal Weights
|
||||
|
||||
Default signal weights used in delta calculations:
|
||||
|
||||
| Signal | Weight | Default Prior |
|
||||
|--------|--------|---------------|
|
||||
| VEX | 0.25 | 0.5 |
|
||||
| Reachability | 0.25 | 0.5 |
|
||||
| EPSS | 0.15 | 0.3 |
|
||||
| Runtime | 0.15 | 0.3 |
|
||||
| Backport | 0.10 | 0.5 |
|
||||
| SBOMLineage | 0.10 | 0.5 |
|
||||
|
||||
Custom weights can be passed in requests to override defaults.
|
||||
|
||||
### 13.7 Use Cases
|
||||
|
||||
1. **Evidence Prioritization:** Determine which signals to acquire first based on maximum impact
|
||||
2. **Risk Bounding:** Understand worst-case score before making release decisions
|
||||
3. **Sensitivity Analysis:** Explore how different evidence values would affect outcomes
|
||||
4. **Operator Guidance:** Help operators focus collection efforts on high-impact signals
|
||||
5. **Audit Trail:** Document "what-if" analysis as part of release decision rationale
|
||||
|
||||
|
||||
@@ -69,7 +69,37 @@ src/Policy/__Libraries/StellaOps.Policy.Determinization/
|
||||
│ ├── IDecayedConfidenceCalculator.cs
|
||||
│ ├── DecayedConfidenceCalculator.cs # Half-life decay application
|
||||
│ ├── SignalWeights.cs # Configurable signal weights
|
||||
│ └── PriorDistribution.cs # Default priors for missing signals
|
||||
│ ├── PriorDistribution.cs # Default priors for missing signals
|
||||
│ ├── EvidenceWeightedScoring/ # 6-dimension EWS model
|
||||
│ │ ├── EwsDimension.cs # RCH/RTS/BKP/XPL/SRC/MIT enum
|
||||
│ │ ├── IEwsDimensionNormalizer.cs # Pluggable normalizer interface
|
||||
│ │ ├── EwsSignalInput.cs # Raw signal inputs
|
||||
│ │ ├── EwsModels.cs # Scores, weights, guardrails
|
||||
│ │ ├── IGuardrailsEngine.cs # Guardrails enforcement interface
|
||||
│ │ ├── GuardrailsEngine.cs # Caps/floors (KEV, backport, etc.)
|
||||
│ │ ├── IEwsCalculator.cs # Unified calculator interface
|
||||
│ │ ├── EwsCalculator.cs # Orchestrates normalizers + guardrails
|
||||
│ │ └── Normalizers/
|
||||
│ │ ├── ReachabilityNormalizer.cs
|
||||
│ │ ├── RuntimeSignalsNormalizer.cs
|
||||
│ │ ├── BackportEvidenceNormalizer.cs
|
||||
│ │ ├── ExploitabilityNormalizer.cs
|
||||
│ │ ├── SourceConfidenceNormalizer.cs
|
||||
│ │ └── MitigationStatusNormalizer.cs
|
||||
│ └── Triage/ # Decay-based triage queue (Sprint 050)
|
||||
│ ├── TriageModels.cs # TriagePriority, TriageItem, TriageQueueSnapshot, TriageQueueOptions
|
||||
│ ├── ITriageQueueEvaluator.cs # Batch + single evaluation interface
|
||||
│ ├── TriageQueueEvaluator.cs # Priority classification, days-until-stale, OTel metrics
|
||||
│ ├── ITriageObservationSource.cs # Source for observation candidates
|
||||
│ ├── ITriageReanalysisSink.cs # Sink interface for re-analysis queue
|
||||
│ ├── InMemoryTriageReanalysisSink.cs # ConcurrentQueue-based default sink
|
||||
│ └── UnknownTriageQueueService.cs # Fetch→evaluate→enqueue cycle orchestrator
|
||||
│ └── WeightManifest/ # Versioned weight manifests (Sprint 051)
|
||||
│ ├── WeightManifestModels.cs # WeightManifestDocument, weights, guardrails, buckets, diff models
|
||||
│ ├── WeightManifestHashComputer.cs # Deterministic SHA-256 with canonical JSON (excludes contentHash)
|
||||
│ ├── IWeightManifestLoader.cs # Interface: list, load, select, validate, diff
|
||||
│ ├── WeightManifestLoader.cs # File-based discovery, effectiveFrom selection, OTel metrics
|
||||
│ └── WeightManifestCommands.cs # CLI backing: list, validate, diff, activate, hash
|
||||
├── Policies/
|
||||
│ ├── IDeterminizationPolicy.cs
|
||||
│ ├── DeterminizationPolicy.cs # Allow/quarantine/escalate rules
|
||||
@@ -913,6 +943,158 @@ public static class DeterminizationMetrics
|
||||
}
|
||||
```
|
||||
|
||||
## Evidence-Weighted Score (EWS) Model
|
||||
|
||||
The EWS model extends the Determinization subsystem with a **6-dimension scoring
|
||||
pipeline** that replaces ad-hoc signal weighting with a unified, pluggable, and
|
||||
guardrail-enforced composite score.
|
||||
|
||||
### Dimensions
|
||||
|
||||
Each dimension maps a family of raw signals to a **normalised risk score 0–100**
|
||||
(higher = riskier) and a **confidence 0.0–1.0**:
|
||||
|
||||
| Code | Dimension | Key signals | Score semantics |
|
||||
|------|-----------|-------------|-----------------|
|
||||
| RCH | Reachability | Call-graph tier R0–R4, runtime trace | Higher = more reachable |
|
||||
| RTS | RuntimeSignals | Instrumentation coverage, invocation count, APM | Higher = more actively exercised |
|
||||
| BKP | BackportEvidence | Vendor confirmation, binary-analysis confidence | Higher = no backport / low confidence |
|
||||
| XPL | Exploitability | EPSS, KEV, exploit-kit availability, PoC age, CVSS | Higher = more exploitable |
|
||||
| SRC | SourceConfidence | SBOM completeness, signatures, attestation count | **Inverted**: high confidence = low risk |
|
||||
| MIT | MitigationStatus | VEX status, workarounds, network controls | Higher = less mitigated |
|
||||
|
||||
### Default Weights
|
||||
|
||||
```
|
||||
RCH 0.25 XPL 0.20 RTS 0.15
|
||||
BKP 0.15 SRC 0.15 MIT 0.10
|
||||
─── Total: 1.00 ───
|
||||
```
|
||||
|
||||
A **Legacy** preset preserves backward-compatible weights aligned with the
|
||||
original `SignalWeights` record.
|
||||
|
||||
### Guardrails
|
||||
|
||||
After weighted scoring, a `GuardrailsEngine` enforces hard caps and floors:
|
||||
|
||||
| Guardrail | Default | Trigger condition |
|
||||
|-----------|---------|-------------------|
|
||||
| `kev_floor` | 70 | `IsInKev == true` — floor the score |
|
||||
| `backported_cap` | 20 | `BackportDetected && Confidence ≥ 0.8` — cap the score |
|
||||
| `not_affected_cap` | 25 | `VexStatus == not_affected` — cap the score |
|
||||
| `runtime_floor` | 30 | `RuntimeTraceConfirmed == true` — floor the score |
|
||||
| `speculative_cap` | 60 | Overall confidence < `MinConfidenceThreshold` (0.3) — cap |
|
||||
|
||||
Guardrails are applied in priority order (KEV first). The resulting
|
||||
`EwsCompositeScore` records which guardrails fired and whether the score was
|
||||
adjusted up or down.
|
||||
|
||||
### Calculator API
|
||||
|
||||
```csharp
|
||||
// Via DI
|
||||
IEwsCalculator calculator = serviceProvider.GetRequiredService<IEwsCalculator>();
|
||||
|
||||
// Or standalone
|
||||
IEwsCalculator calculator = EwsCalculator.CreateDefault();
|
||||
|
||||
var signal = new EwsSignalInput
|
||||
{
|
||||
CveId = "CVE-2025-1234",
|
||||
ReachabilityTier = 3, // R3
|
||||
EpssProbability = 0.42,
|
||||
IsInKev = false,
|
||||
VexStatus = "under_investigation",
|
||||
SbomCompleteness = 0.85,
|
||||
};
|
||||
|
||||
EwsCompositeScore result = calculator.Calculate(signal);
|
||||
// result.Score → 0-100 composite
|
||||
// result.BasisPoints → 0-10000 (fine-grained)
|
||||
// result.Confidence → weighted confidence
|
||||
// result.RiskTier → Critical/High/Medium/Low/Negligible
|
||||
// result.AppliedGuardrails → list of guardrail names that fired
|
||||
// result.NeedsReview → true when confidence < threshold
|
||||
```
|
||||
|
||||
### Normalizer Interface
|
||||
|
||||
Each dimension is implemented as an `IEwsDimensionNormalizer`:
|
||||
|
||||
```csharp
|
||||
public interface IEwsDimensionNormalizer
|
||||
{
|
||||
EwsDimension Dimension { get; }
|
||||
int Normalize(EwsSignalInput signal); // 0-100
|
||||
double GetConfidence(EwsSignalInput signal); // 0.0-1.0
|
||||
string GetExplanation(EwsSignalInput signal, int score);
|
||||
}
|
||||
```
|
||||
|
||||
Normalizers are registered via DI as `IEnumerable<IEwsDimensionNormalizer>`.
|
||||
Custom normalizers can be added by registering additional implementations.
|
||||
|
||||
### Observability
|
||||
|
||||
The calculator emits two OTel metrics:
|
||||
|
||||
- **`stellaops_ews_score`** (Histogram) — score distribution 0–100
|
||||
- **`stellaops_ews_guardrails_applied`** (Counter) — number of guardrail applications
|
||||
|
||||
## Unknowns Decay Triage Queue
|
||||
|
||||
> **Sprint:** SPRINT_20260208_050_Policy_unknowns_decay_and_triage_queue
|
||||
|
||||
The triage queue automatically identifies unknowns whose evidence has decayed past staleness thresholds and queues them for re-analysis, closing the gap between passive `ObservationDecay.CheckIsStale()` tracking and active re-analysis triggering.
|
||||
|
||||
### Triage Priority Classification
|
||||
|
||||
| Priority | Decay Multiplier Range | Action |
|
||||
|----------|----------------------|--------|
|
||||
| **None** | > 0.70 | No action — fresh |
|
||||
| **Low** | 0.50 – 0.70 | Monitor — approaching staleness |
|
||||
| **Medium** | 0.30 – 0.50 | Schedule re-analysis — stale |
|
||||
| **High** | 0.10 – 0.30 | Re-analyse soon — heavily decayed |
|
||||
| **Critical** | ≤ 0.10 | URGENT — evidence at floor |
|
||||
|
||||
Thresholds are configurable via `TriageQueueOptions` (section: `Determinization:TriageQueue`).
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
UnknownTriageQueueService (orchestrator)
|
||||
├── ITriageObservationSource → fetch candidates
|
||||
├── ITriageQueueEvaluator → classify priority, compute days-until-stale
|
||||
└── ITriageReanalysisSink → enqueue Medium+ items for re-analysis
|
||||
```
|
||||
|
||||
- **`TriageQueueEvaluator`**: Deterministic evaluator. Given the same observations and reference time, produces identical output. Calculates days-until-stale using the formula: `d = -halfLife × ln(threshold) / ln(2) - currentAgeDays`.
|
||||
- **`UnknownTriageQueueService`**: Orchestrates fetch→evaluate→enqueue cycles. Designed for periodic invocation by a background host, timer, or scheduler. Also supports on-demand evaluation (CLI/API) without auto-enqueue.
|
||||
- **`InMemoryTriageReanalysisSink`**: Default `ConcurrentQueue<TriageItem>` implementation for single-node and offline scenarios. Host-level can replace with message bus or database-backed sink.
|
||||
|
||||
### OTel Metrics
|
||||
|
||||
- **`stellaops_triage_items_evaluated_total`** (Counter) — observations evaluated per cycle
|
||||
- **`stellaops_triage_items_queued_total`** (Counter) — items added to triage queue
|
||||
- **`stellaops_triage_decay_multiplier`** (Histogram) — decay multiplier distribution
|
||||
- **`stellaops_triage_cycles_total`** (Counter) — evaluation cycles executed
|
||||
- **`stellaops_triage_reanalysis_enqueued_total`** (Counter) — items sent to re-analysis sink
|
||||
- **`stellaops_triage_cycle_duration_seconds`** (Histogram) — cycle duration
|
||||
|
||||
### Configuration
|
||||
|
||||
```yaml
|
||||
Determinization:
|
||||
TriageQueue:
|
||||
ApproachingThreshold: 0.70 # Multiplier below which Low priority starts
|
||||
HighPriorityThreshold: 0.30 # Below this → High
|
||||
CriticalPriorityThreshold: 0.10 # Below this → Critical
|
||||
MaxSnapshotItems: 500 # Max items per snapshot
|
||||
IncludeApproaching: true # Include Low priority in snapshots
|
||||
MinEvaluationIntervalMinutes: 60
|
||||
```
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
| Test Category | Focus Area | Example |
|
||||
@@ -934,6 +1116,90 @@ public static class DeterminizationMetrics
|
||||
4. **Escalation Path:** Runtime evidence always escalates regardless of other signals
|
||||
5. **Tamper Detection:** Signal snapshots hashed for integrity verification
|
||||
|
||||
## Versioned Weight Manifests
|
||||
|
||||
Weight manifests (Sprint 051) provide versioned, content-addressed configuration for
|
||||
all scoring weights, guardrails, buckets, and determinization thresholds. Manifests
|
||||
live in `etc/weights/` as JSON files with a `*.weights.json` extension.
|
||||
|
||||
### Manifest Schema (v1.0.0)
|
||||
|
||||
| Field | Type | Description |
|
||||
| --- | --- | --- |
|
||||
| `schemaVersion` | string | Must be `"1.0.0"` |
|
||||
| `version` | string | Manifest version identifier (e.g. `"v2026-01-22"`) |
|
||||
| `effectiveFrom` | ISO-8601 | UTC date from which this manifest is active |
|
||||
| `profile` | string | Environment profile (`production`, `staging`, etc.) |
|
||||
| `contentHash` | string | `sha256:<hex>` content hash or `sha256:auto` placeholder |
|
||||
| `weights.legacy` | dict | 6-dimension EWS weights (must sum to 1.0) |
|
||||
| `weights.advisory` | dict | Advisory-profile weights |
|
||||
| `guardrails` | object | Guardrail rules (notAffectedCap, runtimeFloor, speculativeCap) |
|
||||
| `buckets` | object | Action tier boundaries (actNowMin, scheduleNextMin, investigateMin) |
|
||||
| `determinizationThresholds` | object | Entropy thresholds for triage |
|
||||
| `signalWeightsForEntropy` | dict | Signal weights for uncertainty calculation (sum to 1.0) |
|
||||
| `metadata` | object | Provenance: createdBy, createdAt, changelog, notes |
|
||||
|
||||
### Content Hash Computation
|
||||
|
||||
The `WeightManifestHashComputer` computes a deterministic SHA-256 hash over
|
||||
canonical JSON (alphabetically sorted properties, `contentHash` field excluded):
|
||||
|
||||
```
|
||||
Input JSON → parse → remove contentHash → sort keys recursively → UTF-8 → SHA-256 → "sha256:<hex>"
|
||||
```
|
||||
|
||||
This enables tamper detection and content-addressed references. The `sha256:auto`
|
||||
placeholder is replaced by `stella weights hash --write-back` or at build time.
|
||||
|
||||
### CLI Commands (backing services)
|
||||
|
||||
| Command | Service Method | Description |
|
||||
| --- | --- | --- |
|
||||
| `stella weights list` | `WeightManifestCommands.ListAsync()` | List all manifests with version, profile, hash status |
|
||||
| `stella weights validate` | `WeightManifestCommands.ValidateAsync()` | Validate schema, weight normalization, hash integrity |
|
||||
| `stella weights diff` | `WeightManifestCommands.DiffAsync()` | Compare two manifests field-by-field |
|
||||
| `stella weights activate` | `WeightManifestCommands.ActivateAsync()` | Select effective manifest for a reference date |
|
||||
| `stella weights hash` | `WeightManifestCommands.HashAsync()` | Compute/verify content hash, optionally write back |
|
||||
|
||||
### EffectiveFrom Selection
|
||||
|
||||
`WeightManifestLoader.SelectEffectiveAsync(referenceDate)` picks the most recent
|
||||
manifest where `effectiveFrom ≤ referenceDate`, enabling time-travel replay:
|
||||
|
||||
```
|
||||
Manifests: v2026-01-01 v2026-01-22 v2026-03-01
|
||||
Reference: 2026-02-15
|
||||
Selected: v2026-01-22 (most recent ≤ reference date)
|
||||
```
|
||||
|
||||
### OTel Metrics
|
||||
|
||||
| Metric | Type | Description |
|
||||
| --- | --- | --- |
|
||||
| `stellaops.weight_manifest.loaded_total` | Counter | Manifests loaded from disk |
|
||||
| `stellaops.weight_manifest.validated_total` | Counter | Manifests validated |
|
||||
| `stellaops.weight_manifest.hash_mismatch_total` | Counter | Content hash mismatches |
|
||||
| `stellaops.weight_manifest.validation_error_total` | Counter | Validation errors |
|
||||
|
||||
### DI Registration
|
||||
|
||||
```csharp
|
||||
services.AddDeterminization(); // Registers WeightManifestLoaderOptions,
|
||||
// IWeightManifestLoader → WeightManifestLoader,
|
||||
// WeightManifestCommands
|
||||
```
|
||||
|
||||
### YAML Configuration
|
||||
|
||||
```yaml
|
||||
Determinization:
|
||||
WeightManifest:
|
||||
ManifestDirectory: "etc/weights"
|
||||
FilePattern: "*.weights.json"
|
||||
RequireComputedHash: true # Reject sha256:auto in production
|
||||
StrictHashVerification: true # Fail on hash mismatch
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- Product Advisory: "Unknown CVEs: graceful placeholders, not blockers"
|
||||
|
||||
283
docs/modules/policy/dsl-grammar-specification.md
Normal file
283
docs/modules/policy/dsl-grammar-specification.md
Normal file
@@ -0,0 +1,283 @@
|
||||
# Stella Policy DSL Grammar Specification
|
||||
|
||||
**Version**: stella-dsl@1.0
|
||||
**Status**: Implemented
|
||||
**Last Updated**: 2026-02-15
|
||||
|
||||
## Overview
|
||||
|
||||
The Stella Policy DSL is a domain-specific language for defining release policies that control software deployment decisions. Policies are evaluated against signal contexts to produce deterministic verdicts.
|
||||
|
||||
## File Extension
|
||||
|
||||
Policy files use the `.stella` extension.
|
||||
|
||||
## Lexical Structure
|
||||
|
||||
### Comments
|
||||
|
||||
```
|
||||
// Single-line comment
|
||||
|
||||
/*
|
||||
* Multi-line comment
|
||||
*/
|
||||
```
|
||||
|
||||
### Keywords
|
||||
|
||||
Reserved keywords (case-sensitive):
|
||||
|
||||
| Keyword | Description |
|
||||
|---------|-------------|
|
||||
| `policy` | Policy declaration |
|
||||
| `syntax` | Syntax version declaration |
|
||||
| `metadata` | Policy metadata block |
|
||||
| `settings` | Policy settings block |
|
||||
| `profile` | Profile declaration |
|
||||
| `rule` | Rule declaration |
|
||||
| `when` | Rule condition |
|
||||
| `then` | Rule action (condition true) |
|
||||
| `else` | Rule action (condition false) |
|
||||
| `and` | Logical AND |
|
||||
| `or` | Logical OR |
|
||||
| `not` | Logical NOT |
|
||||
| `true` | Boolean true |
|
||||
| `false` | Boolean false |
|
||||
| `null` | Null literal |
|
||||
| `in` | Membership operator |
|
||||
| `map` | Map literal |
|
||||
| `env` | Environment binding |
|
||||
|
||||
### Operators
|
||||
|
||||
| Operator | Description |
|
||||
|----------|-------------|
|
||||
| `==` | Equality |
|
||||
| `!=` | Inequality |
|
||||
| `<` | Less than |
|
||||
| `<=` | Less than or equal |
|
||||
| `>` | Greater than |
|
||||
| `>=` | Greater than or equal |
|
||||
| `:=` | Definition |
|
||||
| `=>` | Arrow (lambda/map) |
|
||||
| `.` | Member access |
|
||||
| `,` | Separator |
|
||||
| `:` | Key-value separator |
|
||||
| `=` | Assignment |
|
||||
|
||||
### Literals
|
||||
|
||||
#### Strings
|
||||
```
|
||||
"hello world"
|
||||
"escaped \"quotes\""
|
||||
```
|
||||
|
||||
#### Numbers
|
||||
```
|
||||
42
|
||||
3.14
|
||||
-1
|
||||
0.5
|
||||
```
|
||||
|
||||
#### Booleans
|
||||
```
|
||||
true
|
||||
false
|
||||
```
|
||||
|
||||
#### Arrays
|
||||
```
|
||||
[1, 2, 3]
|
||||
["a", "b", "c"]
|
||||
```
|
||||
|
||||
### Identifiers
|
||||
|
||||
Identifiers start with a letter or underscore, followed by letters, digits, or underscores:
|
||||
|
||||
```
|
||||
identifier
|
||||
_private
|
||||
signal_name
|
||||
cvss_score
|
||||
```
|
||||
|
||||
## Grammar (EBNF)
|
||||
|
||||
```ebnf
|
||||
document = policy-header "{" body "}" ;
|
||||
|
||||
policy-header = "policy" string-literal "syntax" string-literal ;
|
||||
|
||||
body = { metadata-block | settings-block | profile | rule } ;
|
||||
|
||||
metadata-block = "metadata" "{" { key-value } "}" ;
|
||||
|
||||
settings-block = "settings" "{" { key-value } "}" ;
|
||||
|
||||
key-value = identifier ":" literal ;
|
||||
|
||||
profile = "profile" identifier "{" { profile-item } "}" ;
|
||||
|
||||
profile-item = map-item | env-item | scalar-item ;
|
||||
|
||||
map-item = "map" identifier "=>" expression ;
|
||||
|
||||
env-item = "env" identifier "=>" string-literal ;
|
||||
|
||||
scalar-item = identifier ":=" expression ;
|
||||
|
||||
rule = "rule" identifier [ "(" priority ")" ] "{" rule-body "}" ;
|
||||
|
||||
priority = number-literal ;
|
||||
|
||||
rule-body = when-clause then-clause [ else-clause ] ;
|
||||
|
||||
when-clause = "when" expression ;
|
||||
|
||||
then-clause = "then" "{" { action } "}" ;
|
||||
|
||||
else-clause = "else" "{" { action } "}" ;
|
||||
|
||||
action = action-name [ "(" { argument } ")" ] ;
|
||||
|
||||
action-name = identifier ;
|
||||
|
||||
argument = expression | key-value ;
|
||||
|
||||
expression = or-expression ;
|
||||
|
||||
or-expression = and-expression { "or" and-expression } ;
|
||||
|
||||
and-expression = unary-expression { "and" unary-expression } ;
|
||||
|
||||
unary-expression = [ "not" ] primary-expression ;
|
||||
|
||||
primary-expression = literal
|
||||
| identifier
|
||||
| member-access
|
||||
| "(" expression ")"
|
||||
| comparison ;
|
||||
|
||||
comparison = primary-expression comparison-op primary-expression ;
|
||||
|
||||
comparison-op = "==" | "!=" | "<" | "<=" | ">" | ">=" | "in" ;
|
||||
|
||||
member-access = identifier { "." identifier } ;
|
||||
|
||||
literal = string-literal
|
||||
| number-literal
|
||||
| boolean-literal
|
||||
| array-literal
|
||||
| null-literal ;
|
||||
|
||||
string-literal = '"' { character } '"' ;
|
||||
|
||||
number-literal = [ "-" | "+" ] digit { digit } [ "." digit { digit } ] ;
|
||||
|
||||
boolean-literal = "true" | "false" ;
|
||||
|
||||
array-literal = "[" [ expression { "," expression } ] "]" ;
|
||||
|
||||
null-literal = "null" ;
|
||||
```
|
||||
|
||||
## Example Policy
|
||||
|
||||
```stella
|
||||
policy "Production Release Policy" syntax "stella-dsl@1" {
|
||||
metadata {
|
||||
author: "security-team@example.com"
|
||||
version: "1.2.0"
|
||||
description: "Governs production releases"
|
||||
}
|
||||
|
||||
settings {
|
||||
default_action: "block"
|
||||
audit_mode: false
|
||||
}
|
||||
|
||||
profile production {
|
||||
env target => "prod"
|
||||
map severity_threshold := 7.0
|
||||
}
|
||||
|
||||
rule critical_cve_block (100) {
|
||||
when cvss.score >= 9.0 and cve.reachable == true
|
||||
then {
|
||||
block("Critical CVE is reachable")
|
||||
notify("security-oncall")
|
||||
}
|
||||
}
|
||||
|
||||
rule high_cve_warn (50) {
|
||||
when cvss.score >= 7.0 and cvss.score < 9.0
|
||||
then {
|
||||
warn("High severity CVE detected")
|
||||
}
|
||||
else {
|
||||
allow()
|
||||
}
|
||||
}
|
||||
|
||||
rule sbom_required (80) {
|
||||
when not sbom.present
|
||||
then {
|
||||
block("SBOM attestation required")
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Signal Context
|
||||
|
||||
Policies are evaluated against a signal context containing runtime values:
|
||||
|
||||
| Signal | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `cvss.score` | number | CVSS score of vulnerability |
|
||||
| `cve.reachable` | boolean | Whether CVE is reachable |
|
||||
| `cve.id` | string | CVE identifier |
|
||||
| `sbom.present` | boolean | SBOM attestation exists |
|
||||
| `sbom.format` | string | SBOM format (cyclonedx, spdx) |
|
||||
| `artifact.digest` | string | Artifact content digest |
|
||||
| `artifact.tag` | string | Container image tag |
|
||||
| `environment` | string | Target environment |
|
||||
| `attestation.signed` | boolean | Has signed attestation |
|
||||
|
||||
## Compilation
|
||||
|
||||
The DSL compiles to a content-addressed IR (Intermediate Representation):
|
||||
|
||||
1. **Tokenize**: Source → Token stream
|
||||
2. **Parse**: Tokens → AST
|
||||
3. **Compile**: AST → PolicyIrDocument
|
||||
4. **Serialize**: IR → Canonical JSON
|
||||
5. **Hash**: JSON → SHA-256 checksum
|
||||
|
||||
The checksum provides deterministic policy identity for audit and replay.
|
||||
|
||||
## CLI Commands
|
||||
|
||||
```bash
|
||||
# Lint a policy file
|
||||
stella policy lint policy.stella
|
||||
|
||||
# Compile to IR JSON
|
||||
stella policy compile policy.stella --output policy.ir.json
|
||||
|
||||
# Get deterministic checksum
|
||||
stella policy compile policy.stella --checksum-only
|
||||
|
||||
# Simulate with signals
|
||||
stella policy simulate policy.stella --signals context.json
|
||||
```
|
||||
|
||||
## See Also
|
||||
|
||||
- [Policy Module Architecture](architecture.md)
|
||||
- [PolicyDsl Implementation](../../../src/Policy/StellaOps.PolicyDsl/)
|
||||
- [Signal Context Reference](signal-context-reference.md)
|
||||
@@ -112,6 +112,17 @@ When policy bundles change:
|
||||
WHERE policy_hash = ?
|
||||
```
|
||||
|
||||
### Invalidation DI Wiring and Lifecycle
|
||||
|
||||
`AddProvcacheInvalidators()` registers the event-driven invalidation pipeline in dependency injection:
|
||||
|
||||
- Creates `IEventStream<SignerRevokedEvent>` using `IEventStreamFactory.Create<T>(new EventStreamOptions { StreamName = SignerRevokedEvent.StreamName })`
|
||||
- Creates `IEventStream<FeedEpochAdvancedEvent>` using `IEventStreamFactory.Create<T>(new EventStreamOptions { StreamName = FeedEpochAdvancedEvent.StreamName })`
|
||||
- Registers `SignerSetInvalidator` and `FeedEpochInvalidator` as singleton `IProvcacheInvalidator` implementations
|
||||
- Registers `InvalidatorHostedService` as `IHostedService` to own invalidator startup/shutdown
|
||||
|
||||
`InvalidatorHostedService` starts all registered invalidators during host startup and stops them in reverse order during host shutdown. Each invalidator subscribes from `StreamPosition.End`, so only new events are consumed after process start.
|
||||
|
||||
### Invalidation Recording
|
||||
|
||||
All invalidation events are recorded in the revocation ledger for audit and replay:
|
||||
|
||||
@@ -226,6 +226,191 @@ All artifacts are identified by BLAKE3-256 digest:
|
||||
- [PoE Predicate Spec](../../../src/Attestor/POE_PREDICATE_SPEC.md)
|
||||
- [Module AGENTS.md](../../../src/__Libraries/StellaOps.ReachGraph/AGENTS.md)
|
||||
|
||||
## Unified Query Interface
|
||||
|
||||
The ReachGraph module exposes a **Unified Reachability Query API** that provides a single facade for static, runtime, and hybrid queries.
|
||||
|
||||
### API Endpoints
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/v1/reachability/static` | POST | Query static reachability from call graph analysis |
|
||||
| `/v1/reachability/runtime` | POST | Query runtime reachability from observed execution facts |
|
||||
| `/v1/reachability/hybrid` | POST | Combine static and runtime for best-effort verdict |
|
||||
| `/v1/reachability/batch` | POST | Batch query for CVE vulnerability analysis |
|
||||
|
||||
### Adapters
|
||||
|
||||
The unified query interface is backed by two adapters:
|
||||
|
||||
1. **ReachGraphStoreAdapter**: Implements `IReachGraphAdapter` from `StellaOps.Reachability.Core`
|
||||
- Queries static reachability from stored call graphs
|
||||
- Uses BFS from entrypoints to target symbols
|
||||
- Returns `StaticReachabilityResult` with distance, path, and evidence URIs
|
||||
|
||||
2. **InMemorySignalsAdapter**: Implements `ISignalsAdapter` from `StellaOps.Reachability.Core`
|
||||
- Queries runtime observation facts
|
||||
- Supports observation window filtering
|
||||
- Returns `RuntimeReachabilityResult` with hit count, contexts, and evidence URIs
|
||||
- Note: Production deployments should integrate with the actual Signals runtime service
|
||||
|
||||
### Hybrid Query Flow
|
||||
|
||||
```
|
||||
┌────────────────┐
|
||||
│ Hybrid Query │
|
||||
│ Request │
|
||||
└───────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────────────────────┐
|
||||
│ ReachabilityIndex Facade │
|
||||
│ (StellaOps.Reachability.Core) │
|
||||
└───────┬───────────────────────┬───────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌───────────────┐ ┌───────────────┐
|
||||
│ ReachGraph │ │ Signals │
|
||||
│ StoreAdapter │ │ Adapter │
|
||||
└───────┬───────┘ └───────┬───────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌───────────────┐ ┌───────────────┐
|
||||
│ PostgreSQL + │ │ Runtime Facts │
|
||||
│ Valkey Cache │ │ (In-Memory) │
|
||||
└───────────────┘ └───────────────┘
|
||||
```
|
||||
|
||||
### Query Models
|
||||
|
||||
**SymbolRef** - Identifies a code symbol:
|
||||
```json
|
||||
{
|
||||
"namespace": "System.Net.Http",
|
||||
"typeName": "HttpClient",
|
||||
"memberName": "GetAsync"
|
||||
}
|
||||
```
|
||||
|
||||
**StaticReachabilityResult**:
|
||||
```json
|
||||
{
|
||||
"symbol": { "namespace": "...", "typeName": "...", "memberName": "..." },
|
||||
"artifactDigest": "sha256:abc123...",
|
||||
"isReachable": true,
|
||||
"distanceFromEntrypoint": 3,
|
||||
"path": ["entry -> A -> B -> target"],
|
||||
"evidenceUris": ["stella:evidence/reachgraph/sha256:abc123/symbol:..."]
|
||||
}
|
||||
```
|
||||
|
||||
**RuntimeReachabilityResult**:
|
||||
```json
|
||||
{
|
||||
"symbol": { ... },
|
||||
"artifactDigest": "sha256:abc123...",
|
||||
"wasObserved": true,
|
||||
"hitCount": 1250,
|
||||
"firstSeen": "2025-06-10T08:00:00Z",
|
||||
"lastSeen": "2025-06-15T12:00:00Z",
|
||||
"contexts": [{ "environment": "production", "service": "api-gateway" }],
|
||||
"evidenceUris": ["stella:evidence/signals/sha256:abc123/symbol:..."]
|
||||
}
|
||||
```
|
||||
|
||||
**HybridReachabilityResult**:
|
||||
```json
|
||||
{
|
||||
"symbol": { ... },
|
||||
"artifactDigest": "sha256:abc123...",
|
||||
"staticResult": { ... },
|
||||
"runtimeResult": { ... },
|
||||
"confidence": 0.92,
|
||||
"verdict": "reachable",
|
||||
"reasoning": "Static analysis shows 3-hop path; runtime confirms 1250 observations"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
_Last updated: 2025-12-27_
|
||||
## 14. Lattice Triage Service
|
||||
|
||||
### Overview
|
||||
|
||||
The Lattice Triage Service provides a workflow-oriented surface on top of the
|
||||
8-state reachability lattice, enabling operators to visualise lattice states,
|
||||
apply evidence, perform manual overrides, and maintain a full audit trail of
|
||||
every state transition.
|
||||
|
||||
Library: `StellaOps.Reachability.Core`
|
||||
Namespace: `StellaOps.Reachability.Core`
|
||||
|
||||
### Models
|
||||
|
||||
| Type | Purpose |
|
||||
|------|---------|
|
||||
| `LatticeTriageEntry` | Per-(component, CVE) snapshot: current state, confidence, VEX status, full transition history. Content-addressed `EntryId` (`triage:sha256:…`). Computed `RequiresReview` / `HasOverride`. |
|
||||
| `LatticeTransitionRecord` | Immutable log entry per state change: from/to state, confidence before/after, trigger, reason, actor, evidence digests, timestamp. Computed `IsManualOverride`. |
|
||||
| `LatticeTransitionTrigger` | Enum: `StaticAnalysis`, `RuntimeObservation`, `ManualOverride`, `SystemReset`, `AutomatedRule`. Serialised as `JsonStringEnumConverter`. |
|
||||
| `LatticeOverrideRequest` | Operator request to force a target state with reason, actor, and evidence digests. |
|
||||
| `LatticeOverrideResult` | Outcome of an override: applied flag, updated entry, transition, optional warning. |
|
||||
| `LatticeTriageQuery` | Filters: `State?`, `RequiresReview?`, `ComponentPurlPrefix?`, `Cve?`, `Limit` (default 100), `Offset`. |
|
||||
|
||||
### Service Interface (`ILatticeTriageService`)
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `GetOrCreateEntryAsync(purl, cve)` | Returns existing entry or creates one at `Unknown` state. |
|
||||
| `ApplyEvidenceAsync(purl, cve, evidenceType, digests, actor, reason)` | Delegates to `ReachabilityLattice.ApplyEvidence`, records transition. |
|
||||
| `OverrideStateAsync(request)` | Forces target state via Reset + ForceState sequence. Warns when overriding `Confirmed*` states. |
|
||||
| `ListAsync(query)` | Filters + pages entries; ordered by `UpdatedAt` descending. |
|
||||
| `GetHistoryAsync(purl, cve)` | Returns full transition log for an entry. |
|
||||
| `ResetAsync(purl, cve, actor, reason)` | Resets entry to `Unknown`, records `SystemReset` transition. |
|
||||
|
||||
### VEX Status Mapping
|
||||
|
||||
| Lattice State | VEX Status |
|
||||
|---------------|------------|
|
||||
| `Unknown`, `StaticReachable`, `Contested` | `under_investigation` |
|
||||
| `StaticUnreachable`, `RuntimeUnobserved`, `ConfirmedUnreachable` | `not_affected` |
|
||||
| `RuntimeObserved`, `ConfirmedReachable` | `affected` |
|
||||
|
||||
### Manual Override Behaviour
|
||||
|
||||
When an operator overrides state, the service:
|
||||
1. Resets the lattice to `Unknown`.
|
||||
2. Applies the minimal evidence sequence to reach the target state (e.g., `ConfirmedReachable` = `StaticReachable` + `RuntimeObserved`).
|
||||
3. Sets confidence to the midpoint of the target state's confidence range.
|
||||
4. Returns a **warning** when overriding from `ConfirmedReachable` or `ConfirmedUnreachable`, since these are high-certainty states.
|
||||
|
||||
### DI Registration
|
||||
|
||||
`AddReachabilityCore()` registers `ILatticeTriageService → LatticeTriageService` (singleton, via `TryAddSingleton`).
|
||||
|
||||
### Observability (OTel Metrics)
|
||||
|
||||
Meter: `StellaOps.Reachability.Core.Triage`
|
||||
|
||||
| Metric | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `reachability.triage.entries_created` | Counter | Entries created |
|
||||
| `reachability.triage.evidence_applied` | Counter | Evidence applications |
|
||||
| `reachability.triage.overrides_applied` | Counter | Manual overrides |
|
||||
| `reachability.triage.resets` | Counter | Lattice resets |
|
||||
| `reachability.triage.contested` | Counter | Contested state transitions |
|
||||
|
||||
### Test Coverage
|
||||
|
||||
22 tests in `StellaOps.Reachability.Core.Tests/LatticeTriageServiceTests.cs`:
|
||||
- Entry creation (new, idempotent, distinct keys)
|
||||
- Evidence application (static→reachable, confirmed paths, conflicting→contested, digest recording)
|
||||
- Override (target state, warnings on confirmed, HasOverride flag)
|
||||
- Listing with filters (state, review, PURL prefix)
|
||||
- History retrieval
|
||||
- Reset transitions
|
||||
- VEX mapping (theory test)
|
||||
- Edge-case validation (null PURL, empty reason)
|
||||
|
||||
---
|
||||
|
||||
_Last updated: 2026-02-08_
|
||||
|
||||
@@ -166,6 +166,57 @@ public interface ICvssKevSources
|
||||
|
||||
---
|
||||
|
||||
## 4.4 Exploit Maturity Service
|
||||
|
||||
The **ExploitMaturityService** consolidates multiple exploitation signals into a unified maturity level for risk prioritization.
|
||||
|
||||
### Maturity Taxonomy
|
||||
|
||||
| Level | Description | Evidence |
|
||||
|-------|-------------|----------|
|
||||
| `Unknown` | No exploitation intelligence available | No signals or below thresholds |
|
||||
| `Theoretical` | Exploit theoretically possible | Low EPSS (<10%) |
|
||||
| `ProofOfConcept` | PoC exploit exists | Moderate EPSS (10-40%) |
|
||||
| `Active` | Active exploitation observed | High EPSS (40-80%), in-the-wild reports |
|
||||
| `Weaponized` | Weaponized exploit in campaigns | Very high EPSS (>80%), KEV listing |
|
||||
|
||||
### Signal Sources
|
||||
|
||||
```csharp
|
||||
public interface IExploitMaturityService
|
||||
{
|
||||
Task<ExploitMaturityResult> AssessMaturityAsync(string cveId, CancellationToken ct);
|
||||
Task<ExploitMaturityLevel?> GetMaturityLevelAsync(string cveId, CancellationToken ct);
|
||||
Task<IReadOnlyList<MaturityHistoryEntry>> GetMaturityHistoryAsync(string cveId, CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
**Signal aggregation:**
|
||||
1. **EPSS** - Maps probability score to maturity level via thresholds
|
||||
2. **KEV** - CISA Known Exploited Vulnerabilities → `Weaponized`
|
||||
3. **InTheWild** - Threat intel feeds → `Active`
|
||||
|
||||
### EPSS Threshold Mapping
|
||||
|
||||
| EPSS Score | Maturity Level |
|
||||
|------------|----------------|
|
||||
| ≥ 0.80 | Weaponized |
|
||||
| ≥ 0.40 | Active |
|
||||
| ≥ 0.10 | ProofOfConcept |
|
||||
| ≥ 0.01 | Theoretical |
|
||||
| < 0.01 | Unknown |
|
||||
|
||||
### Exploit Maturity API Endpoints
|
||||
|
||||
```
|
||||
GET /exploit-maturity/{cveId} → ExploitMaturityResult
|
||||
GET /exploit-maturity/{cveId}/level → { level: "Active" }
|
||||
GET /exploit-maturity/{cveId}/history → { entries: [...] }
|
||||
POST /exploit-maturity/batch { cveIds: [...] } → { results: [...] }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5) REST API (RiskEngine.WebService)
|
||||
|
||||
All under `/api/v1/risk`. Auth: **OpTok**.
|
||||
|
||||
@@ -419,6 +419,87 @@ Exports evidence pack for artifact(s).
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/v1/lineage/stream
|
||||
|
||||
Subscribe to real-time lineage updates via Server-Sent Events (SSE).
|
||||
|
||||
**Query Parameters:**
|
||||
- `watchDigests` (optional): Comma-separated list of digests to filter updates.
|
||||
|
||||
**Response:** SSE stream with events:
|
||||
```
|
||||
event: lineage-update
|
||||
data: {"id":"uuid","type":"SbomAdded","digest":"sha256:...","timestamp":"...","data":{...}}
|
||||
```
|
||||
|
||||
**Event Types:**
|
||||
- `SbomAdded` - New SBOM version created
|
||||
- `VexChanged` - VEX status changed for a component
|
||||
- `ReachabilityUpdated` - Reachability analysis completed
|
||||
- `EdgeChanged` - Lineage edge added/removed
|
||||
- `Heartbeat` - Keep-alive ping
|
||||
|
||||
### GET /api/v1/lineage/{artifactDigest}/optimized
|
||||
|
||||
Get paginated and depth-pruned lineage graph for performance.
|
||||
|
||||
**Query Parameters:**
|
||||
- `maxDepth` (optional, default: 3): Maximum traversal depth
|
||||
- `pageSize` (optional, default: 50): Nodes per page
|
||||
- `pageNumber` (optional, default: 0): Zero-indexed page number
|
||||
- `searchTerm` (optional): Filter nodes by name
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"centerDigest": "sha256:abc123...",
|
||||
"nodes": [...],
|
||||
"edges": [...],
|
||||
"boundaryNodes": [
|
||||
{"digest": "sha256:...", "hiddenChildrenCount": 5, "hiddenParentsCount": 0}
|
||||
],
|
||||
"totalNodes": 150,
|
||||
"hasMorePages": true,
|
||||
"pageNumber": 0,
|
||||
"pageSize": 50
|
||||
}
|
||||
```
|
||||
|
||||
### GET /api/v1/lineage/{artifactDigest}/levels
|
||||
|
||||
Progressive level-by-level graph traversal via SSE.
|
||||
|
||||
**Query Parameters:**
|
||||
- `direction` (optional, default: "Children"): Traversal direction (Children/Parents/Center)
|
||||
- `maxDepth` (optional, default: 5): Maximum depth to traverse
|
||||
|
||||
**Response:** SSE stream with level events:
|
||||
```
|
||||
event: level
|
||||
data: {"depth":0,"nodes":[...],"isComplete":true}
|
||||
|
||||
event: level
|
||||
data: {"depth":1,"nodes":[...],"isComplete":true}
|
||||
|
||||
event: complete
|
||||
data: {"status":"done"}
|
||||
```
|
||||
|
||||
### GET /api/v1/lineage/{artifactDigest}/metadata
|
||||
|
||||
Get cached metadata about a lineage graph.
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"centerDigest": "sha256:abc123...",
|
||||
"totalNodes": 150,
|
||||
"totalEdges": 175,
|
||||
"maxDepth": 8,
|
||||
"computedAt": "2025-12-28T10:30:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
## Caching Strategy
|
||||
|
||||
### Hover Card Cache (Valkey)
|
||||
@@ -434,6 +515,86 @@ Exports evidence pack for artifact(s).
|
||||
- **TTL:** 10 minutes
|
||||
- **Invalidation:** On new VEX data for either artifact
|
||||
|
||||
### Graph Metadata Cache (Valkey)
|
||||
|
||||
- **Key:** `lineage:metadata:{tenantId}:{centerDigest}`
|
||||
- **TTL:** 10 minutes
|
||||
- **Contents:** Node count, edge count, max depth
|
||||
- **Invalidation:** On lineage edge changes via `DELETE /api/v1/lineage/{digest}/cache`
|
||||
|
||||
### Optimization Cache
|
||||
|
||||
The `LineageGraphOptimizer` uses IDistributedCache to store:
|
||||
- Pre-computed metadata for large graphs
|
||||
- BFS distance calculations
|
||||
- Pagination state for consistent paging
|
||||
|
||||
## Real-Time Streaming Architecture
|
||||
|
||||
### LineageStreamService
|
||||
|
||||
Provides Server-Sent Events (SSE) for real-time lineage updates:
|
||||
|
||||
```csharp
|
||||
public interface ILineageStreamService
|
||||
{
|
||||
IAsyncEnumerable<LineageUpdateEvent> SubscribeAsync(
|
||||
Guid tenantId,
|
||||
IReadOnlyList<string>? watchDigests = null,
|
||||
CancellationToken ct = default);
|
||||
|
||||
Task NotifySbomAddedAsync(Guid tenantId, string artifactDigest,
|
||||
string? parentDigest, SbomVersionSummary summary, CancellationToken ct);
|
||||
|
||||
Task NotifyVexChangedAsync(Guid tenantId, string artifactDigest,
|
||||
VexChangeData change, CancellationToken ct);
|
||||
|
||||
Task NotifyReachabilityUpdatedAsync(Guid tenantId, string artifactDigest,
|
||||
ReachabilityUpdateData update, CancellationToken ct);
|
||||
|
||||
Task NotifyEdgeChangedAsync(Guid tenantId, string fromDigest,
|
||||
string toDigest, LineageEdgeChangeType changeType, CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
- Uses `Channel<T>` for tenant-scoped subscription management
|
||||
- Bounded channels with `DropOldest` policy to handle slow consumers
|
||||
- Optional digest filtering for targeted subscriptions
|
||||
- Automatic subscription cleanup on disconnect
|
||||
|
||||
### LineageGraphOptimizer
|
||||
|
||||
Optimizes large graphs for UI performance:
|
||||
|
||||
```csharp
|
||||
public interface ILineageGraphOptimizer
|
||||
{
|
||||
OptimizedLineageGraph Optimize(LineageOptimizationRequest request);
|
||||
|
||||
IAsyncEnumerable<LineageLevel> TraverseLevelsAsync(
|
||||
string centerDigest,
|
||||
ImmutableArray<LineageNode> nodes,
|
||||
ImmutableArray<LineageEdge> edges,
|
||||
TraversalDirection direction,
|
||||
int maxDepth = 10,
|
||||
CancellationToken ct = default);
|
||||
|
||||
Task<LineageGraphMetadata> GetOrComputeMetadataAsync(
|
||||
Guid tenantId, string centerDigest,
|
||||
ImmutableArray<LineageNode> nodes,
|
||||
ImmutableArray<LineageEdge> edges,
|
||||
CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
**Optimization Features:**
|
||||
1. **Depth Pruning:** BFS-based distance computation from center node
|
||||
2. **Search Filtering:** Case-insensitive node name matching
|
||||
3. **Pagination:** Stable ordering with configurable page size
|
||||
4. **Boundary Detection:** Identifies nodes with hidden children for expand-on-demand UI
|
||||
5. **Level Traversal:** Progressive rendering via async enumerable
|
||||
|
||||
## Determinism Guarantees
|
||||
|
||||
1. **Node Ordering:** Sorted by `sequenceNumber DESC`, then `createdAt DESC`
|
||||
|
||||
@@ -402,14 +402,33 @@ When configured, the worker runs the `reachability-analysis` stage to infer depe
|
||||
|
||||
Configuration lives in `src/Scanner/docs/sbom-reachability-filtering.md`, including policy schema, metadata keys, and report outputs.
|
||||
|
||||
### 5.5.6 VEX decision filter with reachability (Sprint 20260208_062)
|
||||
|
||||
Scanner now exposes a deterministic VEX+reachability matrix filter for triage pre-processing:
|
||||
|
||||
- Gate matrix component: `StellaOps.Scanner.Gate/VexReachabilityDecisionFilter` evaluates `(vendorStatus, reachabilityTier)` and returns one of `suppress`, `elevate`, `pass_through`, or `flag_for_review`.
|
||||
- Matrix examples: `(not_affected, unreachable) -> suppress`, `(affected, confirmed|likely) -> elevate`, `(not_affected, confirmed|likely) -> flag_for_review`.
|
||||
- API surface: `POST /api/v1/scans/vex-reachability/filter` accepts finding batches and returns annotated decisions plus action summary counts.
|
||||
- Determinism: batch order is preserved, rule IDs are explicit, and no network lookups are required for matrix evaluation.
|
||||
|
||||
### 5.5.7 Vulnerability-first triage clustering APIs (Sprint 20260208_063)
|
||||
|
||||
Scanner triage now includes deterministic exploit-path clustering primitives for vulnerability-first triage workflows:
|
||||
|
||||
- Core clustering service: `StellaOps.Scanner.Triage/Services/ExploitPathGroupingService` groups findings using common call-chain prefix similarity with configurable thresholds.
|
||||
- Inbox enhancements: `GET /api/v1/triage/inbox` supports `similarityThreshold`, `sortBy`, and `descending` for deterministic cluster filtering/sorting.
|
||||
- Cluster statistics: `GET /api/v1/triage/inbox/clusters/stats` returns per-cluster severity counts, reachability distribution, and priority scores.
|
||||
- Batch triage actions: `POST /api/v1/triage/inbox/clusters/{pathId}/actions` applies one action to all findings in the cluster and emits deterministic action records.
|
||||
- Offline/determinism posture: no network calls, stable ordering by IDs/path IDs, deterministic path-ID hashing, and replayable batch payload digests.
|
||||
|
||||
### 5.6 DSSE attestation (via Signer/Attestor)
|
||||
|
||||
* WebService constructs **predicate** with `image_digest`, `stellaops_version`, `license_id`, `policy_digest?` (when emitting **final reports**), timestamps.
|
||||
* Calls **Signer** (requires **OpTok + PoE**); Signer verifies **entitlement + scanner image integrity** and returns **DSSE bundle**.
|
||||
* **Attestor** logs to **Rekor v2**; returns `{uuid,index,proof}` → stored in `artifacts.rekor`.
|
||||
* **Verdict OCI idempotency**: `push-verdict` computes `sha256` idempotency from DSSE envelope bytes, writes `org.stellaops.idempotency.key`, uses stable `verdict-<hash>` manifest tags, retries transient push failures (429/5xx/timeouts), and treats conflict/already-submitted responses as success.
|
||||
* **Hybrid reachability attestations**: graph-level DSSE (mandatory) plus optional edge-bundle DSSEs for runtime/init/contested edges. See [`docs/modules/reach-graph/guides/hybrid-attestation.md`](../reach-graph/guides/hybrid-attestation.md) for verification runbooks and Rekor guidance.
|
||||
* Operator enablement runbooks (toggles, env-var map, rollout guidance) live in [`operations/dsse-rekor-operator-guide.md`](operations/dsse-rekor-operator-guide.md) per SCANNER-ENG-0015.
|
||||
* WebService constructs **predicate** with `image_digest`, `stellaops_version`, `license_id`, `policy_digest?` (when emitting **final reports**), timestamps.
|
||||
* Calls **Signer** (requires **OpTok + PoE**); Signer verifies **entitlement + scanner image integrity** and returns **DSSE bundle**.
|
||||
* **Attestor** logs to **Rekor v2**; returns `{uuid,index,proof}` → stored in `artifacts.rekor`.
|
||||
* **Verdict OCI idempotency**: `push-verdict` computes `sha256` idempotency from DSSE envelope bytes, writes `org.stellaops.idempotency.key`, uses stable `verdict-<hash>` manifest tags, retries transient push failures (429/5xx/timeouts), and treats conflict/already-submitted responses as success.
|
||||
* **Hybrid reachability attestations**: graph-level DSSE (mandatory) plus optional edge-bundle DSSEs for runtime/init/contested edges. See [`docs/modules/reach-graph/guides/hybrid-attestation.md`](../reach-graph/guides/hybrid-attestation.md) for verification runbooks and Rekor guidance.
|
||||
* Operator enablement runbooks (toggles, env-var map, rollout guidance) live in [`operations/dsse-rekor-operator-guide.md`](operations/dsse-rekor-operator-guide.md) per SCANNER-ENG-0015.
|
||||
|
||||
---
|
||||
|
||||
@@ -641,7 +660,7 @@ Diagnostics drive two metrics published by `EntryTraceMetrics`:
|
||||
|
||||
Structured logs include `entrytrace.path`, `entrytrace.command`, `entrytrace.reason`, and `entrytrace.depth`, all correlated with scan/job IDs. Timestamps are normalized to UTC (microsecond precision) to keep DSSE attestations and UI traces explainable.
|
||||
|
||||
### Appendix B — BOM‑Index sidecar
|
||||
### Appendix B — BOM‑Index sidecar
|
||||
|
||||
```
|
||||
struct Header { magic, version, imageDigest, createdAt }
|
||||
@@ -649,3 +668,43 @@ vector<string> purls
|
||||
map<purlIndex, roaring_bitmap> components
|
||||
optional map<purlIndex, roaring_bitmap> usedByEntrypoint
|
||||
```
|
||||
|
||||
### Appendix C — Stack-Trace Exploit Path View (Sprint 061)
|
||||
|
||||
The Triage library provides a stack-trace–style visualization layer on top of `ExploitPath`
|
||||
clusters, designed for UI rendering as collapsible call-chain frames.
|
||||
|
||||
#### Models (`StellaOps.Scanner.Triage.Models`)
|
||||
|
||||
| Type | Purpose |
|
||||
|------|---------|
|
||||
| `StackTraceExploitPathView` | Root view: ordered frames, CVE IDs, severity label, collapsed state |
|
||||
| `StackTraceFrame` | Single frame: symbol, role, source location, snippet, gate label |
|
||||
| `SourceSnippet` | Syntax-highlighted source extract at a frame location |
|
||||
| `FrameRole` | Entrypoint / Intermediate / Sink / GatedIntermediate |
|
||||
| `StackTraceViewRequest` | Build request with source mappings and gate labels |
|
||||
|
||||
#### Frame Role Assignment
|
||||
|
||||
| Position | Has Gate | Role |
|
||||
|----------|----------|------|
|
||||
| First | — | Entrypoint |
|
||||
| Last | — | Sink |
|
||||
| Middle | No | Intermediate |
|
||||
| Middle | Yes | GatedIntermediate |
|
||||
|
||||
#### Collapse Heuristic
|
||||
|
||||
Paths with > 3 frames are collapsed by default in the UI (showing only entrypoint + sink).
|
||||
The user can expand to see the full call chain.
|
||||
|
||||
#### Service (`IStackTraceExploitPathViewService`)
|
||||
|
||||
- `BuildView(StackTraceViewRequest)` — transforms a single `ExploitPath` into a `StackTraceExploitPathView`
|
||||
- `BuildViews(IReadOnlyList<StackTraceViewRequest>)` — batch transformation, ordered by priority score descending then path ID for determinism
|
||||
|
||||
#### Source Snippet Integration
|
||||
|
||||
When source mappings are provided (keyed by `file:line`), the service attaches
|
||||
`SourceSnippet` records to matching frames. This enables syntax-highlighted code
|
||||
display in the UI without requiring the scanner to store full source files.
|
||||
|
||||
41
docs/modules/scanner/reachability-ground-truth-corpus.md
Normal file
41
docs/modules/scanner/reachability-ground-truth-corpus.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Scanner Reachability Ground-Truth Corpus
|
||||
|
||||
This document defines the deterministic toy-service corpus used to validate
|
||||
reachability tier classification quality in Scanner tests.
|
||||
|
||||
## Location
|
||||
- `src/Scanner/__Tests/__Datasets/toys/`
|
||||
|
||||
## Service Set
|
||||
- `svc-01-log4shell-java`
|
||||
- `svc-02-prototype-pollution-node`
|
||||
- `svc-03-pickle-deserialization-python`
|
||||
- `svc-04-text-template-go`
|
||||
- `svc-05-xmlserializer-dotnet`
|
||||
- `svc-06-erb-injection-ruby`
|
||||
|
||||
Each service contains:
|
||||
- Minimal source code with a known vulnerability pattern.
|
||||
- `labels.yaml` with tier ground truth for one or more CVEs.
|
||||
|
||||
## labels.yaml Contract (v1)
|
||||
- Required top-level fields: `schema_version`, `service`, `language`, `entrypoint`, `cves`.
|
||||
- Each CVE entry requires: `id`, `package`, `tier`, `rationale`.
|
||||
- Allowed tier values:
|
||||
- `R0`: unreachable
|
||||
- `R1`: present in dependency only
|
||||
- `R2`: imported but not called
|
||||
- `R3`: called but not reachable from entrypoint
|
||||
- `R4`: reachable from entrypoint
|
||||
|
||||
## Deterministic Validation Harness
|
||||
- Test suite: `src/Scanner/__Tests/StellaOps.Scanner.Reachability.Tests/Benchmarks/ReachabilityTierCorpusTests.cs`
|
||||
- Harness capabilities:
|
||||
- Validates corpus structure and required schema fields.
|
||||
- Verifies `R0..R4` coverage across the toy corpus.
|
||||
- Maps `R0..R4` into Scanner confidence tiers for compatibility checks.
|
||||
- Computes precision, recall, and F1 per tier using deterministic ordering.
|
||||
|
||||
## Offline Posture
|
||||
- No external network access is required for corpus loading or metric computation.
|
||||
- Dataset files are copied into test output for stable local/CI execution.
|
||||
@@ -18,9 +18,9 @@
|
||||
## 3) Pipelines & Guardrails
|
||||
|
||||
- **Redaction.** Attribute processors strip PII/secrets based on policy-managed allowed keys. Redaction profiles mirrored in Offline Kit.
|
||||
- **Sampling.** Tail sampling by service/error; incident mode (triggered by Orchestrator) promotes services to 100 % sampling, extends retention, and toggles Notify alerts.
|
||||
- **Alerting.** Prometheus rules/Dashboards packaged with Export Center: service SLOs, queue depth, policy run latency, ingestion AOC violations.
|
||||
- **Sealed-mode guard.** `StellaOps.Telemetry.Core` enforces `IEgressPolicy` on OTLP exporters; when air-gap mode is sealed any non-loopback collector endpoints are automatically disabled and a structured warning with remediation is emitted.
|
||||
- **Sampling.** Tail sampling by service/error; incident mode (triggered by Orchestrator) promotes services to 100 % sampling, extends retention, and toggles Notify alerts.
|
||||
- **Alerting.** Prometheus rules/Dashboards packaged with Export Center: service SLOs, queue depth, policy run latency, ingestion AOC violations.
|
||||
- **Sealed-mode guard.** `StellaOps.Telemetry.Core` enforces `IEgressPolicy` on OTLP exporters; when air-gap mode is sealed any non-loopback collector endpoints are automatically disabled and a structured warning with remediation is emitted.
|
||||
|
||||
## 4) APIs & integration
|
||||
|
||||
@@ -39,4 +39,66 @@
|
||||
- Meta-metrics: `collector_export_failures_total`, `telemetry_bundle_generation_seconds`, `telemetry_incident_mode{state}`.
|
||||
- Health endpoints for collectors and storage clusters, plus dashboards for ingestion rate, retention, rule evaluations.
|
||||
|
||||
Refer to the module README and implementation plan for immediate context, and update this document once component boundaries and data flows are finalised.
|
||||
## 7) DORA Metrics
|
||||
|
||||
Stella Ops tracks the four key DORA (DevOps Research and Assessment) metrics for software delivery performance:
|
||||
|
||||
### 7.1) Metrics Tracked
|
||||
|
||||
- **Deployment Frequency** (`dora_deployments_total`, `dora_deployment_frequency_per_day`) — How often deployments occur per day/week.
|
||||
- **Lead Time for Changes** (`dora_lead_time_hours`) — Time from commit to deployment in production.
|
||||
- **Change Failure Rate** (`dora_deployment_failure_total`, `dora_change_failure_rate_percent`) — Percentage of deployments requiring rollback, hotfix, or failing.
|
||||
- **Mean Time to Recovery (MTTR)** (`dora_time_to_recovery_hours`) — Average time to recover from incidents.
|
||||
|
||||
### 7.2) Performance Classification
|
||||
|
||||
The system classifies teams into DORA performance levels:
|
||||
- **Elite**: On-demand deployments, <24h lead time, <15% CFR, <1h MTTR
|
||||
- **High**: Weekly deployments, <1 week lead time, <30% CFR, <1 day MTTR
|
||||
- **Medium**: Monthly deployments, <6 months lead time, <45% CFR, <1 week MTTR
|
||||
- **Low**: Quarterly or less frequent deployments with higher failure rates
|
||||
|
||||
### 7.3) Integration Points
|
||||
|
||||
- `IDoraMetricsService` — Service interface for recording deployments and incidents
|
||||
- `DoraMetrics` — OpenTelemetry-style metrics class with SLO breach tracking
|
||||
- DI registration: `services.AddDoraMetrics(options => { ... })`
|
||||
- Events are recorded when Release Orchestrator completes promotions or rollbacks
|
||||
|
||||
### 7.4) SLO Tracking
|
||||
|
||||
Configurable SLO targets via `DoraMetricsOptions`:
|
||||
- `LeadTimeSloHours` (default: 24)
|
||||
- `DeploymentFrequencySloPerDay` (default: 1)
|
||||
- `ChangeFailureRateSloPercent` (default: 15)
|
||||
- `MttrSloHours` (default: 1)
|
||||
|
||||
SLO breaches are recorded as `dora_slo_breach_total` with `metric` label.
|
||||
|
||||
### 7.5) Outcome Analytics and Attribution (Sprint 20260208_065)
|
||||
|
||||
Telemetry now includes deterministic executive outcome attribution built on top of the existing DORA event stream:
|
||||
|
||||
- `IOutcomeAnalyticsService` (`src/Telemetry/StellaOps.Telemetry.Core/StellaOps.Telemetry.Core/IOutcomeAnalyticsService.cs`)
|
||||
- `DoraOutcomeAnalyticsService` (`src/Telemetry/StellaOps.Telemetry.Core/StellaOps.Telemetry.Core/DoraOutcomeAnalyticsService.cs`)
|
||||
- Outcome report models (`src/Telemetry/StellaOps.Telemetry.Core/StellaOps.Telemetry.Core/OutcomeAnalyticsModels.cs`)
|
||||
|
||||
Outcome attribution behavior:
|
||||
|
||||
- Produces `OutcomeExecutiveReport` for a fixed tenant/environment/time window with deterministic ordering.
|
||||
- Adds MTTA support via `DoraIncidentEvent.AcknowledgedAt` and `TimeToAcknowledge`.
|
||||
- Groups deployment outcomes by normalized pipeline (`pipeline-a`, `pipeline-b`, `unknown`) with per-pipeline change failure rate and median lead time.
|
||||
- Groups incidents by severity with resolved/acknowledged counts plus MTTA/MTTR aggregates.
|
||||
- Produces daily cohort slices across the requested date range for executive trend views.
|
||||
|
||||
Dependency injection integration:
|
||||
|
||||
- `TelemetryServiceCollectionExtensions.AddDoraMetrics(...)` now also registers `IOutcomeAnalyticsService`, so existing telemetry entry points automatically expose attribution reporting without additional module wiring.
|
||||
|
||||
Verification coverage:
|
||||
|
||||
- `src/Telemetry/StellaOps.Telemetry.Core/StellaOps.Telemetry.Core.Tests/OutcomeAnalyticsServiceTests.cs`
|
||||
- `src/Telemetry/StellaOps.Telemetry.Core/StellaOps.Telemetry.Core.Tests/DoraMetricsServiceTests.cs`
|
||||
- Full telemetry core test suite pass (`262` tests) remains green after integration.
|
||||
|
||||
Refer to the module README and implementation plan for immediate context, and update this document once component boundaries and data flows are finalised.
|
||||
|
||||
@@ -74,6 +74,161 @@ src/Web/StellaOps.Web/
|
||||
| Policies | `/policies` | Policy configuration |
|
||||
| Settings | `/settings` | User and system settings |
|
||||
|
||||
### 3.1 VEX Gate Inline Actions (Sprint 20260208_073)
|
||||
|
||||
Quiet-triage promote actions now support inline VEX gating with deterministic evidence tiers:
|
||||
|
||||
- `src/Web/StellaOps.Web/src/app/features/vex_gate/vex-gate-button.directive.ts`
|
||||
- Morphs action buttons into tier-aware gate states:
|
||||
- Tier 1 -> green (`allow`)
|
||||
- Tier 2 -> amber (`review`)
|
||||
- Tier 3 -> red (`block`)
|
||||
- Emits a `gateBlocked` event on tier-3 actions.
|
||||
- `src/Web/StellaOps.Web/src/app/features/vex_gate/vex-evidence-sheet.component.ts`
|
||||
- Renders inline evidence details with tier/verdict metadata and optional DSSE verification hints.
|
||||
- Integrated in quiet-triage lane promote actions:
|
||||
- `src/Web/StellaOps.Web/src/app/features/triage/components/quiet-lane/quiet-lane-container.component.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/triage/components/quiet-lane/parked-item-card.component.ts`
|
||||
|
||||
The UI behavior remains offline-first and deterministic:
|
||||
- Tier mapping is derived from local finding attributes only.
|
||||
- No additional network dependency is required to render gate/evidence states.
|
||||
|
||||
### 3.2 Signals Runtime Dashboard (Sprint 20260208_072)
|
||||
|
||||
Signals runtime operations now include a dedicated dashboard route:
|
||||
|
||||
- Route: `ops/signals`
|
||||
- Route registration:
|
||||
- `src/Web/StellaOps.Web/src/app/app.routes.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/signals/signals.routes.ts`
|
||||
- Feature implementation:
|
||||
- `src/Web/StellaOps.Web/src/app/features/signals/signals-runtime-dashboard.component.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/signals/services/signals-runtime-dashboard.service.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/signals/models/signals-runtime-dashboard.models.ts`
|
||||
|
||||
Dashboard behavior:
|
||||
|
||||
- Aggregates signal runtime metrics (`signals/sec`, error rate, average latency) using `SignalsClient` and falls back to gateway request summaries when available.
|
||||
- Computes deterministic per-host probe health snapshots (eBPF/ETW/dyld/unknown) from signal payload telemetry.
|
||||
- Presents provider/status distribution summaries and probe status tables without introducing network-only dependencies beyond existing local API clients.
|
||||
|
||||
Verification coverage:
|
||||
|
||||
- `src/Web/StellaOps.Web/src/tests/signals_runtime_dashboard/signals-runtime-dashboard.service.spec.ts`
|
||||
- `src/Web/StellaOps.Web/src/tests/signals_runtime_dashboard/signals-runtime-dashboard.component.spec.ts`
|
||||
|
||||
### 3.3 Audit Trail Reason Capsule (Sprint 20260208_067)
|
||||
|
||||
Findings and triage views now expose a per-row "Why am I seeing this?" reason capsule:
|
||||
|
||||
- Audit reasons client contract:
|
||||
- `src/Web/StellaOps.Web/src/app/core/api/audit-reasons.client.ts`
|
||||
- Uses `/api/audit/reasons/:verdictId` with deterministic fallback records for offline/unavailable API conditions.
|
||||
- Reusable capsule component:
|
||||
- `src/Web/StellaOps.Web/src/app/features/triage/components/reason-capsule/reason-capsule.component.ts`
|
||||
- Displays policy name, rule ID, graph revision ID, and inputs digest.
|
||||
- UI integration points:
|
||||
- `src/Web/StellaOps.Web/src/app/features/findings/findings-list.component.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/findings/findings-list.component.html`
|
||||
- `src/Web/StellaOps.Web/src/app/features/triage/components/triage-list/triage-list.component.ts`
|
||||
|
||||
Verification coverage:
|
||||
|
||||
- `src/Web/StellaOps.Web/src/tests/audit_reason_capsule/audit-reasons.client.spec.ts`
|
||||
- `src/Web/StellaOps.Web/src/tests/audit_reason_capsule/reason-capsule.component.spec.ts`
|
||||
- `src/Web/StellaOps.Web/src/tests/audit_reason_capsule/findings-list.reason-capsule.spec.ts`
|
||||
|
||||
### 3.4 Pack Registry Browser (Sprint 20260208_068)
|
||||
|
||||
TaskRunner pack operations now include a dedicated registry browser route:
|
||||
|
||||
- Route: `ops/packs`
|
||||
- Route registration:
|
||||
- `src/Web/StellaOps.Web/src/app/app.routes.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/pack-registry/pack-registry.routes.ts`
|
||||
- Feature implementation:
|
||||
- `src/Web/StellaOps.Web/src/app/features/pack-registry/pack-registry-browser.component.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/pack-registry/services/pack-registry-browser.service.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/pack-registry/models/pack-registry-browser.models.ts`
|
||||
|
||||
Browser behavior:
|
||||
|
||||
- Lists available and installed packs using `PackRegistryClient`, with deterministic ordering and capability filters.
|
||||
- Displays DSSE signature status per pack and per version history entry (`verified`, `present`, `unsigned`) and signer metadata when available.
|
||||
- Executes install/upgrade actions only after compatibility evaluation; incompatible packs are blocked with explicit operator feedback.
|
||||
- Supports version-history drill-down per pack without introducing additional external dependencies.
|
||||
|
||||
Verification coverage:
|
||||
|
||||
- `src/Web/StellaOps.Web/src/tests/pack_registry_browser/pack-registry-browser.service.spec.ts`
|
||||
- `src/Web/StellaOps.Web/src/tests/pack_registry_browser/pack-registry-browser.component.spec.ts`
|
||||
|
||||
### 3.5 Pipeline Run-Centric View (Sprint 20260208_069)
|
||||
|
||||
Release Orchestrator now provides a unified pipeline run-centric surface that links release status, approvals, deployment progress, evidence state, and first-signal telemetry:
|
||||
|
||||
- Route registration:
|
||||
- `src/Web/StellaOps.Web/src/app/features/release-orchestrator/dashboard/dashboard.routes.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/release-orchestrator/runs/runs.routes.ts`
|
||||
- Feature implementation:
|
||||
- `src/Web/StellaOps.Web/src/app/features/release-orchestrator/runs/models/pipeline-runs.models.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/release-orchestrator/runs/services/pipeline-runs.service.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/release-orchestrator/runs/pipeline-runs-list.component.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/release-orchestrator/runs/pipeline-run-detail.component.ts`
|
||||
- Dashboard integration entry point:
|
||||
- `src/Web/StellaOps.Web/src/app/features/release-orchestrator/dashboard/dashboard.component.html`
|
||||
- `src/Web/StellaOps.Web/src/app/features/release-orchestrator/dashboard/dashboard.component.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/release-orchestrator/dashboard/dashboard.component.scss`
|
||||
|
||||
Run-centric behavior:
|
||||
|
||||
- Normalizes recent releases into deterministic `pipeline-<releaseId>` run IDs.
|
||||
- Correlates approvals and active deployments to each run for one-table operator triage.
|
||||
- Provides per-run stage progression (scan, gates, approval, evidence, deployment) with explicit status details.
|
||||
- Integrates `FirstSignalCardComponent` on run detail pages for first-signal evidence visibility.
|
||||
|
||||
Verification coverage:
|
||||
|
||||
- `src/Web/StellaOps.Web/src/tests/pipeline_run_centric/pipeline-runs.service.spec.ts`
|
||||
- `src/Web/StellaOps.Web/src/tests/pipeline_run_centric/pipeline-runs-list.component.spec.ts`
|
||||
|
||||
### 3.6 Reachability Center Coverage Summary (Sprint 20260208_070)
|
||||
|
||||
Reachability Center now includes explicit asset/sensor coverage summaries and missing-sensor indicators:
|
||||
|
||||
- Feature implementation:
|
||||
- `src/Web/StellaOps.Web/src/app/features/reachability/reachability-center.component.ts`
|
||||
- Verification coverage:
|
||||
- `src/Web/StellaOps.Web/src/app/features/reachability/reachability-center.component.spec.ts`
|
||||
|
||||
Coverage behavior:
|
||||
|
||||
- Computes deterministic fleet asset coverage percent from fixture rows.
|
||||
- Computes deterministic runtime sensor coverage percent from online vs expected sensors.
|
||||
- Surfaces a missing-sensor indicator section listing impacted assets and supports one-click filtering to `missing`.
|
||||
- Shows per-row sensor gap labels (`all sensors online`, `missing N sensors`) to make observation gaps explicit.
|
||||
|
||||
### 3.7 SBOM Graph Reachability Overlay with Time Slider (Sprint 20260208_071)
|
||||
|
||||
Graph explorer overlay behavior now supports deterministic lattice-state reachability halos with temporal snapshot exploration:
|
||||
|
||||
- Feature implementation:
|
||||
- `src/Web/StellaOps.Web/src/app/features/graph/graph-overlays.component.ts`
|
||||
- `src/Web/StellaOps.Web/src/app/features/graph/graph-canvas.component.ts`
|
||||
|
||||
Behavior details:
|
||||
|
||||
- Reachability legend in overlay controls maps lattice states `SR/SU/RO/RU/CR/CU/X` to explicit halo colors.
|
||||
- Time slider now binds to deterministic snapshot checkpoints (`current`, `1d`, `7d`, `30d`) and renders timeline event text for each selection.
|
||||
- Reachability mock data generation is deterministic per `(nodeId, snapshot)` so repeated runs produce stable lattice status, confidence, and observation timestamps.
|
||||
- Canvas halo stroke colors are derived from lattice state (not generic status), and halo titles include lattice state plus observed timestamp for operator audit context.
|
||||
|
||||
Verification coverage:
|
||||
|
||||
- `src/Web/StellaOps.Web/src/tests/graph_reachability_overlay/graph-overlays.component.spec.ts`
|
||||
- `src/Web/StellaOps.Web/src/tests/graph_reachability_overlay/graph-canvas.component.spec.ts`
|
||||
|
||||
---
|
||||
|
||||
## 4) Authentication
|
||||
|
||||
Reference in New Issue
Block a user