stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 3 Releases Wiki Activity

docs consoliation work

Browse Source
This commit is contained in:
StellaOps Bot
2025-12-24 14:19:46 +02:00
parent 40362de568
commit 5540ce9430
58 changed files with 2671 additions and 1751 deletions
Download Patch File Download Diff File Expand all files Collapse all files

0
docs/cli/archived/cli-consolidation-migration.md → docs/cli/migration-guide.md
View File

4
docs/db/RULES.md
View File

@@ -778,11 +778,13 @@ CREATE INDEX idx_large_table_col ON schema.large_table(column);
"Persistence": {
"Authority": "Postgres",
"Scheduler": "Postgres",
"Concelier": "Mongo"
"Concelier": "Postgres"
}
}
```
> **Note:** MongoDB storage was deprecated in Sprint 4400. All modules now use PostgreSQL. MongoDB-related patterns in this document are retained for historical reference only.
### 13.2 Connection String Security
**RULE:** Connection strings MUST NOT be logged or included in exception messages.

33
docs/dev/authority-rate-limit-tuning-outline.md
View File

@@ -1,33 +0,0 @@
# Authority Rate Limit Tuning Outline (2025-10-11)
## Purpose
- Drive the remaining work on SEC3.B (Security Guild) and PLG6.DOC (Docs Guild) by capturing the agreed baseline for Authority rate limits and related documentation deliverables.
- Provide a single reference for lockout + rate limit interplay so Docs can lift accurate copy into `docs/security/rate-limits.md` and `docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`.
## Baseline Configuration
- `/token`: fixed window, permitLimit 30, window 60s, queueLimit 0. Reduce to 10/60s for untrusted IP ranges; raise to 60/60s only with compensating controls (WAF + active monitoring).
- `/authorize`: permitLimit 60, window 60s, queueLimit 10. Intended for interactive browser flows; lowering below 30 requires UX review.
- `/internal/*`: disabled by default; recommended 5/60s with queueLimit 0 when bootstrap API exposed.
- Configuration path: `authority.security.rateLimiting.<endpoint>` (e.g., `token.permitLimit`). YAML/ENV bindings follow the standard options hierarchy.
- Retry metadata: middleware stamps `Retry-After` along with tags `authority.client_id`, `authority.remote_ip`, `authority.endpoint`. Docs should highlight these for operator dashboards.
## Parameter Matrix
| Scenario | permitLimit | window | queueLimit | Notes |
|----------|-------------|--------|------------|-------|
| Default production | 30 | 60s | 0 | Works with anonymous quota (33 scans/day). |
| High-trust clustered IPs | 60 | 60s | 5 | Requires `authorize_rate_limit_hits` alert ≤ 1% sustained. |
| Air-gapped lab | 10 | 120s | 0 | Emphasise reduced concurrency + manual queue draining. |
| Incident lockdown | 5 | 300s | 0 | Pair with lockout lowering to 3 attempts. |
## Lockout Interplay
- Ensure Docs explain difference between rate limit (per IP/client) vs lockout (per subject). Provide table mapping retry-after headers to recommended support scripts.
- Security Guild to define alert thresholds: trigger SOC ticket when 429 rate > 25% for 5 minutes or when limiter emits >100 events/hour per client.
## Observability
- Surface metrics: `aspnetcore_rate_limiting_rejections_total{limiter="authority-token"}` and custom log tags from `AuthorityRateLimiterMetadataMiddleware`.
- Recommend dashboard sections: request volume vs. rejections, top offending clientIds, per-endpoint heatmap.
## Action Items
1. Security Guild (SEC3.B): incorporate matrix + alert rules into `docs/security/rate-limits.md`, add YAML examples for override blocks, and cross-link lockout policy doc.
2. Docs Guild (PLG6.DOC): update developer guide section 9 with the middleware sequence and reference this outline for retry metadata + tuning guidance.
3. Authority Core: validate appsettings sample includes the `security.rateLimiting` block with comments and link back to published doc once ready.

6
docs/dev/merge_semver_playbook.md
View File

@@ -43,7 +43,9 @@ If multiple connectors emit identical constraints, the merge layer should:
2. Preserve a single normalized rule instance (thanks to `NormalizedVersionRuleEqualityComparer.Instance`).
3. Attach `decisionReason="precedence"` if one source overrides another.
## 4. Example Mongo pipeline
## 4. Example Query (Historical MongoDB Reference)
> **Note:** This section shows historical MongoDB syntax. The current system uses PostgreSQL.
Use the following aggregation to locate advisories that affect a specific SemVer:
@@ -102,7 +104,7 @@ For current database schema and query patterns, see `docs/db/SPECIFICATION.md`.
## 8. Storage projection reference
`NormalizedVersionDocumentFactory` copies each normalized rule into MongoDB using the shape below. Use this as a contract when reviewing connector fixtures or diagnosing merge/storage diffs:
`NormalizedVersionDocumentFactory` copies each normalized rule into PostgreSQL using the shape below. Use this as a contract when reviewing connector fixtures or diagnosing merge/storage diffs:
```json
{

2
docs/modules/attestor/README.md
View File

@@ -42,7 +42,7 @@ All predicates capture subjects, issuer metadata, policy context, materials, opt
- **Policy Studio:** Verification policies author required predicate types, issuers, witness requirements, and freshness windows; simulations show enforcement impact.
## Storage, offline & air-gap posture
- MongoDB stores entry metadata, dedupe keys, and audit events; object storage optionally archives DSSE bundles.
- PostgreSQL stores entry metadata, dedupe keys, and audit events; object storage optionally archives DSSE bundles.
- Export Center packages attestation bundles (`stella export attestation-bundle`) for Offline Kit delivery.
- Transparency logs can be mirrored; offline mode records gaps and provides compensating controls.

2
docs/modules/attestor/architecture.md
View File

@@ -482,7 +482,7 @@ The worker honours `bulkVerification.itemDelayMilliseconds` for throttling and r
* **Proof acquisition**:
* In synchronous mode, poll the log for inclusion up to `proofTimeoutMs`.
* In asynchronous mode, return `pending` and schedule a **proof fetcher** job (Mongo job doc + backoff).
* In asynchronous mode, return `pending` and schedule a **proof fetcher** job (PostgreSQL job record + backoff).
* **Mirrors/dual logs**:
* When `logPreference="both"`, submit to primary and mirror; store **both** UUIDs (primary canonical).

365
docs/modules/attestor/reg-evidence-architecture.md Normal file
View File

@@ -0,0 +1,365 @@
# Resolved Execution Graph (REG) Evidence Architecture
> **Status:** Proposed
> **Sprint Series:** 8100.0012.*
> **Last Updated:** 2025-12-24
## Overview
This document describes the architectural enhancements to StellaOps' evidence and attestation subsystems based on the **Merkle-Hash REG** (Resolved Execution Graph) pattern. The core insight: when every node in an execution graph is identified by a **Merkle hash of its normalized content**, evidence can be attached to *content itself* rather than brittle positional indices.
## Motivation
### Problem Statement
StellaOps currently has robust foundations for content-addressed identifiers, Merkle trees, and attestations. However, three gaps limit the system's verifiability:
1. **Canonicalizer versioning:** Hash collisions possible if canonicalization logic changes
2. **Fragmented evidence models:** Different modules use different evidence structures
3. **Implicit graph roots:** Merkle roots are computed but not independently attested
### Target Benefits
| Benefit | Description |
|---------|-------------|
| **Reproducible proofs** | Verifiers re-hash node content and check against stored hashes—no fragile indices |
| **Dedup & reuse** | Identical content across pipelines collapses to one ID; one piece of evidence justifies many occurrences |
| **Audits + time travel** | Snapshot the graph's Merkle root; any replay matching the root guarantees identical nodes & evidence |
| **Offline verification** | Attestations are self-contained; no network required to verify |
| **Exception stability** | Exceptions bind to content hashes, not "row 42"; stable across rebuilds |
## Architecture
### Component Diagram
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ StellaOps REG Architecture │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ StellaOps. │ │ StellaOps. │ │ StellaOps. │ │
│ │ Canonical.Json │────▶│ Evidence.Core │◀────│ Attestor. │ │
│ │ │ │ │ │ GraphRoot │ │
│ │ • CanonVersion │ │ • IEvidence │ │ │ │
│ │ • CanonJson │ │ • EvidenceRecord│ │ • IGraphRoot- │ │
│ │ • Versioned │ │ • IEvidenceStore│ │ Attestor │ │
│ │ Hashing │ │ • Adapters │ │ • Verification │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │
│ │ │ │ │
│ └────────────────────────┼────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ Content-Addressed Store │ │
│ │ │ │
│ │ Evidence by subject_node_id │ Graph roots by root_hash │ │
│ │ ──────────────────────────── │ ────────────────────────── │ │
│ │ sha256:abc... → [evidence] │ sha256:xyz... → attestation │ │
│ │ sha256:def... → [evidence] │ sha256:uvw... → attestation │ │
│ └──────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Scanner │ │ Policy │ │ Excititor │ │
│ │ ────────────── │ │ ────────────── │ │ ────────────── │ │
│ │ • EvidenceBundle│────▶│ • Exception │────▶│ • VexObservation│ │
│ │ • ProofSpine │ │ Application │ │ • VexLinkset │ │
│ │ • RichGraph │ │ • PolicyVerdict │ │ │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │
│ │ │ │ │
│ └────────────────────────┼────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ Unified IEvidence│ │
│ │ via Adapters │ │
│ └──────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Data Flow
```
1. Content Creation
─────────────────
Component/Node → Canonicalize(content, version) → SHA-256 → node_id
2. Evidence Attachment
────────────────────
Analysis Result → IEvidence { subject_node_id, type, payload, provenance }
→ EvidenceId = hash(subject + type + payload + provenance)
→ Store(evidence)
3. Graph Construction
───────────────────
Nodes + Edges → Sort(node_ids) + Sort(edge_ids) → Merkle Tree → root_hash
4. Graph Attestation
──────────────────
GraphRootAttestationRequest → GraphRootAttestor
→ In-Toto Statement { subject: [root, artifact] }
→ DSSE Sign
→ Optional: Rekor Publish
5. Verification
─────────────
Download attestation → Verify signature
→ Fetch nodes/edges by ID
→ Recompute Merkle root
→ Compare with attested root
```
## Implementation Sprints
| Sprint | Title | Dependency | Key Deliverables |
|--------|-------|------------|------------------|
| 8100.0012.0001 | Canonicalizer Versioning | None | `CanonVersion`, `CanonicalizeVersioned()`, backward compatibility |
| 8100.0012.0002 | Unified Evidence Model | 0001 | `IEvidence`, `EvidenceRecord`, `IEvidenceStore`, adapters |
| 8100.0012.0003 | Graph Root Attestation | 0001, 0002 | `IGraphRootAttestor`, in-toto statements, Rekor integration |
### Sprint Sequence Diagram
```
Week 1-2 Week 3-4 Week 5-6 Week 7-8
──────── ──────── ──────── ────────
│ │ │ │
│ 0001: Canon │ │ │
│ Versioning │ │ │
│ ┌─────────┐ │ │ │
│ │Wave 0-1 │────┼─▶ 0002: Unified│ │
│ │Wave 2-3 │ │ Evidence │ │
│ │Wave 4 │ │ ┌─────────┐ │ │
│ └─────────┘ │ │Wave 0-1 │──┼─▶ 0003: Graph │
│ │ │Wave 2-3 │ │ Root Attest │
│ │ │Wave 4 │ │ ┌─────────┐ │
│ │ └─────────┘ │ │Wave 0-1 │ │
│ │ │ │Wave 2-4 │ │
│ │ │ │Wave 5 │ │
│ │ │ └─────────┘ │
▼ ▼ ▼ ▼
```
## Technical Specifications
### Canonicalization Version Marker
```json
{
"_canonVersion": "stella:canon:v1",
"evidenceSource": "stellaops/scanner/reachability",
"sbomEntryId": "sha256:abc123...:pkg:npm/lodash@4.17.21",
...
}
```
The `_canonVersion` field:
- Uses underscore prefix to sort first lexicographically
- Identifies the exact canonicalization algorithm
- Enables graceful version migration
- Is included in all content-addressed hash computations
### Unified Evidence Record
```typescript
interface IEvidence {
// Content-addressed binding
subjectNodeId: string; // "sha256:{hex}" - what this evidence is about
evidenceId: string; // "sha256:{hex}" - ID of this evidence record
// Type and payload
evidenceType: EvidenceType; // reachability, scan, policy, vex, etc.
payload: Uint8Array; // Canonical JSON bytes
payloadSchemaVersion: string;
// Attestation
signatures: EvidenceSignature[];
// Provenance
provenance: {
generatorId: string;
generatorVersion: string;
generatedAt: DateTimeOffset;
inputsDigest?: string;
correlationId?: string;
};
// External storage
externalPayloadCid?: string;
}
```
### Graph Root Attestation (In-Toto)
```json
{
"_type": "https://in-toto.io/Statement/v1",
"subject": [
{
"name": "sha256:abc123...",
"digest": { "sha256": "abc123..." }
},
{
"name": "sha256:def456...",
"digest": { "sha256": "def456..." }
}
],
"predicateType": "https://stella-ops.org/attestation/graph-root/v1",
"predicate": {
"graphType": "ResolvedExecutionGraph",
"rootHash": "sha256:abc123...",
"nodeCount": 1247,
"edgeCount": 3891,
"nodeIds": ["sha256:...", ...],
"edgeIds": ["sha256:...", ...],
"inputs": {
"policyDigest": "sha256:...",
"feedsDigest": "sha256:...",
"toolchainDigest": "sha256:...",
"paramsDigest": "sha256:..."
},
"evidenceIds": ["sha256:...", ...],
"canonVersion": "stella:canon:v1",
"computedAt": "2025-12-24T10:30:00Z",
"computedBy": "stellaops/attestor/graph-root",
"computedByVersion": "1.0.0"
}
}
```
## Verification Guarantees
### What Can Be Verified
| Claim | Verification Method |
|-------|---------------------|
| "This graph contains exactly these nodes" | Recompute Merkle root from node IDs |
| "This evidence applies to node X" | Check evidence.subjectNodeId == node.id |
| "This attestation was signed by key K" | Verify DSSE envelope signature |
| "This graph was published to transparency log" | Check Rekor inclusion proof |
| "These inputs produced this graph" | Check inputs.* digests match expectations |
### What Cannot Be Verified (Without Additional Trust)
| Claim | Reason | Mitigation |
|-------|--------|------------|
| "The analysis was performed correctly" | Semantic, not structural | Trust the generator; audit toolchain |
| "No evidence was omitted" | Attester controls content | Require known evidence types; policy enforcement |
| "The inputs are authentic" | External dependency | Chain attestations; verify feed signatures |
## Migration Path
### Phase 1: Parallel Operation (Sprints 0001-0003)
- New versioned hashing alongside legacy
- New evidence model with adapters for existing types
- Graph root attestations optional
### Phase 2: Gradual Adoption (Post-Sprints)
- Emit migration warnings for legacy hashes
- Prefer IEvidence in new code
- Enable graph root attestations by default
### Phase 3: Deprecation (Future)
- Remove legacy hash acceptance
- Require IEvidence for all evidence storage
- Require graph root attestations for verification
## Compatibility Considerations
### Existing Attestations
Attestations generated before canonicalizer versioning remain valid:
- Verification detects legacy format (no `_canonVersion` field)
- Falls back to legacy canonicalization
- Logs warning for migration tracking
### Existing Evidence
Existing evidence types (`EvidenceBundle`, `EvidenceStatement`, etc.) continue working:
- Adapters convert to `IEvidence` on demand
- Original types remain in storage
- Gradual migration via write-through
### API Stability
Public APIs remain backward compatible:
- New methods added, not changed
- Optional parameters for new features
- Explicit opt-in for graph root attestations
## Performance Considerations
| Operation | Impact | Mitigation |
|-----------|--------|------------|
| Version field injection | ~100 bytes per hash | Negligible |
| Merkle tree computation | O(n log n) for n nodes | Existing algorithm; no change |
| Graph root attestation | +1 DSSE sign per graph | Batching; async |
| Evidence store queries | Index on subject_node_id | Composite index; pagination |
| Rekor publishing | Network latency | Optional; async; batching |
## Security Considerations
### Threat Model
| Threat | Mitigation |
|--------|------------|
| Hash collision attacks | SHA-256 with 256-bit security; version namespacing |
| Signature forgery | DSSE with ECDSA/EdDSA; key rotation |
| Evidence tampering | Content-addressed storage; Merkle verification |
| Replay attacks | Timestamp in provenance; Rekor log index |
| Canonicalization attacks | RFC 8785 compliance; explicit versioning |
### Key Management
Graph root attestations use the existing Signer module:
- Keys managed via Authority plugin
- Rotation policy applies
- Certificate chains optional for external verification
## References
- [RFC 8785 - JSON Canonicalization Scheme (JCS)](https://datatracker.ietf.org/doc/html/rfc8785)
- [in-toto Attestation Framework](https://github.com/in-toto/attestation)
- [DSSE - Dead Simple Signing Envelope](https://github.com/secure-systems-lab/dsse)
- [Sigstore Rekor](https://docs.sigstore.dev/rekor/overview/)
- [Merkle Tree - Wikipedia](https://en.wikipedia.org/wiki/Merkle_tree)
## Appendix A: File Locations
| Component | Path |
|-----------|------|
| CanonVersion | `src/__Libraries/StellaOps.Canonical.Json/CanonVersion.cs` |
| CanonJson (versioned) | `src/__Libraries/StellaOps.Canonical.Json/CanonJson.cs` |
| IEvidence | `src/__Libraries/StellaOps.Evidence.Core/IEvidence.cs` |
| EvidenceRecord | `src/__Libraries/StellaOps.Evidence.Core/EvidenceRecord.cs` |
| IEvidenceStore | `src/__Libraries/StellaOps.Evidence.Core/IEvidenceStore.cs` |
| Adapters | `src/__Libraries/StellaOps.Evidence.Core/Adapters/` |
| IGraphRootAttestor | `src/Attestor/__Libraries/StellaOps.Attestor.GraphRoot/IGraphRootAttestor.cs` |
| GraphRootAttestation | `src/Attestor/__Libraries/StellaOps.Attestor.GraphRoot/Models/` |
## Appendix B: Example Evidence Queries
```csharp
// Get all reachability evidence for a package
var evidence = await evidenceStore.GetBySubjectAsync(
subjectNodeId: "sha256:abc123...pkg:npm/lodash@4.17.21",
typeFilter: EvidenceType.Reachability);
// Verify a graph root attestation
var result = await graphRootAttestor.VerifyAsync(
envelope: downloadedEnvelope,
nodes: fetchedNodes,
edges: fetchedEdges);
if (!result.IsValid)
throw new VerificationException(result.FailureReason);
// Check if evidence exists before creating duplicate
if (!await evidenceStore.ExistsAsync(subjectNodeId, EvidenceType.Scan))
{
await evidenceStore.StoreAsync(newEvidence);
}
```

2
docs/modules/attestor/ttl-validation.md
View File

@@ -1,5 +1,7 @@
# Attestor TTL Validation Runbook
> **DEPRECATED:** This runbook tests MongoDB TTL indexes, which are no longer used. Attestor now uses PostgreSQL for persistence (Sprint 4400). See `docs/db/SPECIFICATION.md` for current database schema.
> **Purpose:** confirm MongoDB TTL indexes and Redis expirations for the attestation dedupe store behave as expected on a production-like stack.
## Prerequisites

2
docs/modules/authority/README.md
View File

@@ -25,7 +25,7 @@ Authority is the platform OIDC/OAuth2 control plane that mints short-lived, send
- Scheduler/Scanner for machine-to-machine scope enforcement.
## Operational notes
- MongoDB for tenant, client, and token state.
- PostgreSQL (schema `authority`) for tenant, client, and token state.
- Key material in KMS/HSM with rotation runbooks (`operations/key-rotation.md`).
- Monitoring runbook (`operations/monitoring.md`) and offline-import Grafana JSON (`operations/grafana-dashboard.json`).

2
docs/modules/concelier/README.md
View File

@@ -18,7 +18,7 @@ Concelier ingests signed advisories from dozens of sources and converts them int
- **2025-11-07:** Paragraph-anchored `/advisories/{advisoryKey}/chunks` endpoint shipped for Advisory AI paragraph retrieval. Details and rollout notes live in [`../../updates/2025-11-07-concelier-advisory-chunks.md`](../../updates/2025-11-07-concelier-advisory-chunks.md).
## Integrations & dependencies
- MongoDB for canonical observations and schedules.
- PostgreSQL (schema `vuln`) for canonical observations and schedules.
- Policy Engine / Export Center / CLI for evidence consumption.
- Notify and UI for advisory deltas.

2
docs/modules/concelier/operations/conflict-resolution.md
View File

@@ -131,7 +131,7 @@ Expect all logs at `Information`. Ensure OTEL exporters include the scope `Stell
- Canonical conflict rules: `src/DEDUP_CONFLICTS_RESOLUTION_ALGO.md`.
- Merge engine internals: `src/Concelier/__Libraries/StellaOps.Concelier.Merge/Services/AdvisoryPrecedenceMerger.cs`.
- Metrics definitions: `src/Concelier/__Libraries/StellaOps.Concelier.Merge/Services/AdvisoryMergeService.cs` (identity conflicts) and `AdvisoryPrecedenceMerger`.
- Storage audit trail: `src/Concelier/__Libraries/StellaOps.Concelier.Merge/Services/MergeEventWriter.cs`, `src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/MergeEvents`.
- Storage audit trail: `src/Concelier/__Libraries/StellaOps.Concelier.Merge/Services/MergeEventWriter.cs`, `src/Concelier/__Libraries/StellaOps.Concelier.Storage/MergeEvents`.
Keep this runbook synchronized with future sprint notes and update alert thresholds as baseline volumes change.

8
docs/modules/concelier/operations/connectors/apple.md
View File

@@ -42,10 +42,10 @@ concelier:
- `Apple software index fetch … processed=X newDocuments=Y`
- `Apple advisory parse complete … aliases=… affected=…`
- `Mapped Apple advisory … pendingMappings=0`
6. Confirm MongoDB state:
- `raw_documents` store contains the HT article HTML with metadata (`apple.articleId`, `apple.postingDate`).
- `dtos` store has `schemaVersion="apple.security.update.v1"`.
- `advisories` collection includes keys `HTxxxxxx` with normalized SemVer rules.
6. Confirm PostgreSQL state (schema `vuln`):
- `raw_documents` table contains the HT article HTML with metadata (`apple.articleId`, `apple.postingDate`).
- `dtos` table has `schemaVersion="apple.security.update.v1"`.
- `advisories` table includes keys `HTxxxxxx` with normalized SemVer rules.
- `source_states` entry for `apple` shows a recent `cursor.lastPosted`.
## 3. Production Monitoring

2
docs/modules/concelier/operations/connectors/certbund.md
View File

@@ -142,5 +142,5 @@ operating offline.
1. Observe `certbund.feed.fetch.success` and `certbund.detail.fetch.success` increments after runs; `certbund.feed.coverage.days` should hover near the observed RSS window.
2. Ensure summary logs report `truncated=false` in steady state—`true` indicates the fetch cap was hit.
3. During backfills, watch `certbund.feed.enqueued.count` trend to zero.
4. Spot-check stored advisories in Mongo to confirm `language="de"` and reference URLs match the portal detail endpoint.
4. Spot-check stored advisories in PostgreSQL (schema `vuln`) to confirm `language="de"` and reference URLs match the portal detail endpoint.
5. For Offline Kit exports, validate SHA256 hashes before distribution.

4
docs/modules/concelier/operations/connectors/cve-kev.md
View File

@@ -42,7 +42,7 @@ concelier:
- `cve.map.success`
4. Verify Prometheus shows matching `concelier.source.http.requests_total{concelier_source="cve"}` deltas (list vs detail phases) while `concelier.source.http.failures_total{concelier_source="cve"}` stays flat.
5. Confirm the info-level summary log `CVEs fetch window … pages=X detailDocuments=Y detailFailures=Z` appears once per fetch run and shows `detailFailures=0`.
6. Verify the MongoDB advisory store contains fresh CVE advisories (`advisoryKey` prefix `cve/`) and that the source cursor (`source_states` collection) advanced.
6. Verify the PostgreSQL advisory store (schema `vuln`) contains fresh CVE advisories (`advisoryKey` prefix `cve/`) and that the source cursor (`source_states` table) advanced.
### 1.3 Production Monitoring
@@ -115,7 +115,7 @@ Treat repeated schema failures or growing anomaly counts as an upstream regressi
- `kev.map.advisories` (tag `catalogVersion`)
4. Confirm `concelier.source.http.requests_total{concelier_source="kev"}` increments once per fetch and that the paired `concelier.source.http.failures_total` stays flat (zero increase).
5. Inspect the info logs `Fetched KEV catalog document … pendingDocuments=…` and `Parsed KEV catalog document … entries=…`—they should appear exactly once per run and `Mapped X/Y… skipped=0` should match the `kev.map.advisories` delta.
6. Confirm MongoDB documents exist for the catalog JSON (`raw_documents` & `dtos`) and that advisories with prefix `kev/` are written.
6. Confirm PostgreSQL records exist for the catalog JSON (`raw_documents` and `dtos` tables in schema `vuln`) and that advisories with prefix `kev/` are written.
### 2.4 Production Monitoring

8
docs/modules/concelier/operations/connectors/kisa.md
View File

@@ -38,10 +38,10 @@ concelier:
- `KISA fetched detail for {Idx} … category={Category}`
- `KISA mapped advisory {AdvisoryId} (severity={Severity})`
- Absence of warnings such as `document missing GridFS payload`.
5. Validate MongoDB state:
- `raw_documents.metadata` has `kisa.idx`, `kisa.category`, `kisa.title`.
- DTO store contains `schemaVersion="kisa.detail.v1"`.
- Advisories include aliases (`IDX`, CVE) and `language="ko"`.
5. Validate PostgreSQL state (schema `vuln`):
- `raw_documents` table metadata has `kisa.idx`, `kisa.category`, `kisa.title`.
- `dtos` table contains `schemaVersion="kisa.detail.v1"`.
- `advisories` table includes aliases (`IDX`, CVE) and `language="ko"`.
- `source_states` entry for `kisa` shows recent `cursor.lastFetchAt`.
## 3. Production Monitoring

4
docs/modules/concelier/operations/connectors/msrc.md
View File

@@ -72,8 +72,8 @@ Run the helper:
```bash
dotnet run --project src/Tools/SourceStateSeeder -- \
--connection-string "mongodb://localhost:27017" \
--database concelier \
--connection-string "Host=localhost;Database=stellaops;Username=stella;Password=..." \
--schema vuln \
--input seeds/msrc-backfill.json
```

10
docs/modules/concelier/operations/observation-events.md
View File

@@ -1,19 +1,19 @@
# Observation Event Transport (advisory.observation.updated@1)
Purpose: document how to emit `advisory.observation.updated@1` events via Mongo outbox with optional NATS JetStream transport.
Purpose: document how to emit `advisory.observation.updated@1` events via PostgreSQL outbox with optional NATS JetStream transport.
## Configuration (appsettings.yaml / config)
```yaml
advisoryObservationEvents:
enabled: false # set true to publish beyond Mongo outbox
transport: "mongo" # "mongo" (no-op publisher) or "nats"
enabled: false # set true to publish beyond PostgreSQL outbox
transport: "postgres" # "postgres" (no-op publisher) or "nats"
natsUrl: "nats://127.0.0.1:4222"
subject: "concelier.advisory.observation.updated.v1"
deadLetterSubject: "concelier.advisory.observation.updated.dead.v1"
stream: "CONCELIER_OBS"
```
Defaults: disabled, transport `mongo`; subject/stream as above.
Defaults: disabled, transport `postgres`; subject/stream as above.
## Flow
1) Observation sink writes event to `advisory_observation_events` (idempotent on `observationHash`).
@@ -24,7 +24,7 @@ Defaults: disabled, transport `mongo`; subject/stream as above.
- Ensure NATS JetStream is reachable before enabling `transport: nats` to avoid retry noise.
- Stream is auto-created if missing with current subject; size capped at 512 KiB per message.
- Dead-letter subject reserved; not yet wired—keep for future schema validation failures.
- Backlog monitoring: count documents in `advisory_observation_events` with `publishedAt: null`.
- Backlog monitoring: count rows in `advisory_observation_events` table with `published_at IS NULL`.
## Testing
- Without NATS: leave `enabled=false`; app continues writing outbox only.

24
docs/modules/devops/architecture.md
View File

@@ -2,7 +2,7 @@
> Draws from the AOC guardrails, Orchestrator, Export Center, and Observability module plans to describe how Stella Ops is built, signed, distributed, and operated.
> **Scope.** Implementation‑ready blueprint for **how Stella Ops is built, versioned, signed, distributed, upgraded, licensed (PoE)**, and operated in customer environments (online and air‑gapped). Covers reproducible builds, supply‑chain attestations, registries, offline kits, migration/rollback, artifact lifecycle (RustFS default + Mongo, S3 fallback), monitoring SLOs, and customer activation.
> **Scope.** Implementation‑ready blueprint for **how Stella Ops is built, versioned, signed, distributed, upgraded, licensed (PoE)**, and operated in customer environments (online and air‑gapped). Covers reproducible builds, supply‑chain attestations, registries, offline kits, migration/rollback, artifact lifecycle (RustFS default + PostgreSQL, S3 fallback), monitoring SLOs, and customer activation.
---
@@ -180,9 +180,9 @@ helm install stella stellaops/platform \
--set global.channel=stable \
--set authority.issuer=https://authority.stella.local \
--set scanner.minio.endpoint=http://minio.stella.local:9000 \
--set scanner.mongo.uri=mongodb://mongo/scanner \
--set concelier.mongo.uri=mongodb://mongo/concelier \
--set excititor.mongo.uri=mongodb://mongo/excititor
--set scanner.mongo.uri=postgresb://mongo/scanner \
--set concelier.mongo.uri=postgresb://mongo/concelier \
--set excititor.mongo.uri=postgresb://mongo/excititor
```
* Post‑install job registers **Authority clients** (Scanner, Signer, Attestor, UI) and prints **bootstrap** URLs and client credentials (sealed secrets).
@@ -210,7 +210,7 @@ helm install stella stellaops/platform \
* Images referenced by **digest**; keep previous release manifest `K` versions back.
* `helm rollback` or compose `docker compose -f release-K.yml up -d`.
* Mongo migrations are additive; **no destructive changes** within a single minor.
* PostgreSQL migrations are additive; **no destructive changes** within a single minor.
---
@@ -268,7 +268,7 @@ Signer validates **scanner** image’s cosign identity + calendar tag for **rele
---
## 7) Artifact lifecycle & storage (RustFS/Mongo)
## 7) Artifact lifecycle & storage (RustFS/PostgreSQL)
### 7.1 Buckets & prefixes (RustFS)
@@ -306,16 +306,16 @@ rustfs://stellaops/
> **Migration note.** Follow `docs/modules/scanner/operations/rustfs-migration.md` when transitioning existing
> MinIO buckets to RustFS. The provided migrator is idempotent and safe to rerun per prefix.
### 7.4 Mongo retention
### 7.4 PostgreSQL retention
* **Scanner**: `runtime.events` use TTL (e.g., 30–90 days); **catalog** permanent.
* **Concelier/Excititor**: raw docs keep **last N windows**; canonical stores permanent.
* **Attestor**: `entries` permanent; `dedupe` TTL 24–48h.
### 7.5 Mongo server baseline
### 7.5 PostgreSQL server baseline
* **Minimum supported server:** MongoDB **4.2+**. Driver 3.5.0 removes compatibility shims for 4.0; upstream has already announced 4.0 support will be dropped in upcoming C# driver releases. citeturn1open1
* **Deploy images:** Compose/Helm defaults stay on `mongo:7.x`. For air-gapped installs, refresh Offline Kit bundles so the packaged `mongod` matches ≥4.2.
* **Deploy images:** Compose/Helm defaults stay on `postgres:16`. For air-gapped installs, refresh Offline Kit bundles so the packaged `postgres` matches ≥4.2.
* **Upgrade guard:** During rollout, verify replica sets reach FCV `4.2` or above before swapping binaries; automation should hard-stop if FCV is <4.2.
---
@@ -327,7 +327,7 @@ rustfs://stellaops/
* **Golden signals**:
* **Latency**: token issuance, sign→attest round‑trip, scan enqueue→emit, export build.
* **Saturation**: queue depth, Mongo write IOPS, RustFS throughput / queue depth (or S3 metrics when in fallback mode).
* **Saturation**: queue depth, PostgreSQL write IOPS, RustFS throughput / queue depth (or S3 metrics when in fallback mode).
* **Traffic**: scans/min, attestations/min, webhook admits/min.
* **Errors**: 5xx rates, cosign verification failures, Rekor timeouts.
@@ -355,7 +355,7 @@ Prometheus + OTLP; Grafana dashboards ship in the charts.
* **Backups/DR**:
* Mongo nightly snapshots; MinIO versioning + replication (if configured).
* PostgreSQL nightly snapshots; MinIO versioning + replication (if configured).
* Restore runbooks tested quarterly with synthetic data.
---
@@ -474,7 +474,7 @@ services:
* `attestor.submit_latency_seconds{quantile=0.95}` < 0.3.
* `scanner.scan_latency_seconds{quantile=0.95}` < target per image size.
* `concelier.export.duration_seconds` stable; `excititor.consensus.conflicts_total` not exploding after policy changes.
* RustFS request error rate near zero (or `s3_requests_errors_total` when operating against S3); Mongo `opcounters` hit expected baseline.
* RustFS request error rate near zero (or `s3_requests_errors_total` when operating against S3); PostgreSQL `pg_stat_bgwriter` counters hit expected baseline.
### Appendix B — Upgrade safety checklist

2
docs/modules/excititor/README.md
View File

@@ -35,7 +35,7 @@ Excititor converts heterogeneous VEX feeds into raw observations and linksets th
- Notify for VEX-driven alerts.
## Operational notes
- MongoDB for observation storage and job metadata.
- PostgreSQL (schema `vex`) for observation storage and job metadata.
- Offline kit packaging aligned with Concelier merges.
- Connector-specific runbooks (see `docs/modules/concelier/operations/connectors`).
- Ubuntu CSAF provenance knobs: [`operations/ubuntu-csaf.md`](operations/ubuntu-csaf.md) captures TrustWeight/Tier, cosign, and fingerprint configuration for the sprint 120 enrichment.

6
docs/modules/excititor/operations/vex-raw-validator.md
View File

@@ -1,10 +1,12 @@
# Excititor · VEX Raw Collection Validator (AOC-19-001/002)
> **DEPRECATED:** This document describes MongoDB validators which are no longer used. Excititor now uses PostgreSQL for persistence (Sprint 4400). Schema validation is now performed via PostgreSQL constraints and check constraints. See `docs/db/SPECIFICATION.md` for current database schema.
- **Date:** 2025-11-25
- **Scope:** EXCITITOR-STORE-AOC-19-001 / 19-002
- **Working directory:** `src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo`
- **Working directory:** ~~`src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo`~~ (deprecated)
## What shipped
## What shipped (historical)
- `$jsonSchema` validator applied to `vex_raw` (migration `20251125-vex-raw-json-schema`) with `validationAction=warn`, `validationLevel=moderate` to surface contract violations without impacting ingestion.
- Schema lives at `docs/modules/excititor/schemas/vex_raw.schema.json` (mirrors Mongo validator fields: digest/id, providerId, format, sourceUri, retrievedAt, optional content/GridFS object id, metadata strings).
- Migration is auto-registered in DI; hosted migration runner applies it on service start. New collections created with the validator if missing.

34
docs/modules/excitor/AGENTS.md
View File

@@ -1,34 +0,0 @@
# Excitor agent guide
## Mission
Excitor computes deterministic consensus across VEX claims, preserving conflicts and producing attestable evidence for policy suppression.
## Key docs
- [Module README](./README.md)
- [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md)
## How to get started
1. Open sprint file `/docs/implplan/SPRINT_*.md` and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change.
## Required Reading
- `docs/modules/excitor/README.md`
- `docs/modules/excitor/architecture.md`
- `docs/modules/excitor/implementation_plan.md`
- `docs/modules/platform/architecture-overview.md`
## Working Agreement
- 1. Update task status to `DOING`/`DONE` in both correspoding sprint file `/docs/implplan/SPRINT_*.md` and the local `TASKS.md` when you start or finish work.
- 2. Review this charter and the Required Reading documents before coding; confirm prerequisites are met.
- 3. Keep changes deterministic (stable ordering, timestamps, hashes) and align with offline/air-gap expectations.
- 4. Coordinate doc updates, tests, and cross-guild communication whenever contracts or workflows change.
- 5. Revert to `TODO` if you pause the task without shipping changes; leave notes in commit/PR descriptions for context.

39
docs/modules/excitor/README.md
View File

@@ -1,39 +0,0 @@
# StellaOps Excitor
Excitor computes deterministic consensus across VEX claims, preserving conflicts and producing attestable evidence for policy suppression.
## Latest updates (2025-11-05)
- Consensus API beta documented with canonical JSON samples and DSSE packaging guidance (`docs/updates/2025-11-05-excitor-consensus-beta.md`).
- README links to Link-Not-Merge consensus milestone and preview endpoints for downstream integration.
## Responsibilities
- Ingest Excititor observations and compute per-product consensus snapshots.
- Provide APIs for querying canonical VEX positions and conflict sets.
- Publish exports and DSSE-ready digests for downstream consumption.
- Keep provenance weights and disagreement metadata.
## Key components
- Consensus engine and API host in `StellaOps.Excitor.*` (to-be-implemented).
- Storage schema for consensus graphs.
- Integration hooks for Policy Engine suppression logic.
## Integrations & dependencies
- Excititor for raw observations.
- Policy Engine and UI for suppression stories.
- CLI for evidence inspection.
## Operational notes
- Deterministic consensus algorithms (see architecture).
- Planned telemetry for disagreement counts and freshness.
- Offline exports aligning with Concelier/Excititor timelines.
## Related resources
- ./scoring.md
- ../../vex/consensus-json.md (beta consensus payload sample)
## Backlog references
- DOCS-EXCITOR backlog referenced in architecture doc.
- CLI parity tracked in ../../TASKS.md (CLI-GRAPH/VEX stories).
## Epic alignment
- **Epic 7 – VEX Consensus Lens:** deliver trust-weighted consensus snapshots, disagreement metadata, and explain APIs.

465
docs/modules/excitor/architecture.md
View File

@@ -1,465 +0,0 @@
# component_architecture_excitor.md — **Stella Ops Excitor** (2025Q4)
> Built to satisfy Epic 7 – VEX Consensus Lens requirements.
> **Scope.** This document specifies the **Excitor** service: its purpose, trust model, data structures, APIs, plug‑in contracts, storage schema, normalization/consensus algorithms, performance budgets, testing matrix, and how it integrates with Scanner, Policy, Conselier, and the attestation chain. It is implementation‑ready.
---
## 0) Mission & role in the platform
**Mission.** Convert heterogeneous **VEX** statements (OpenVEX, CSAF VEX, CycloneDX VEX; vendor/distro/platform sources) into **canonical, queryable claims**; compute **deterministic consensus** per *(vuln, product)*; preserve **conflicts with provenance**; publish **stable, attestable exports** that the backend uses to suppress non‑exploitable findings, prioritize remaining risk, and explain decisions.
**Boundaries.**
* Excitor **does not** decide PASS/FAIL. It supplies **evidence** (statuses + justifications + provenance weights).
* Excitor preserves **conflicting claims** unchanged; consensus encodes how we would pick, but the raw set is always exportable.
* VEX consumption is **backend‑only**: Scanner never applies VEX. The backend’s **Policy Engine** asks Excitor for status evidence and then decides what to show.
---
## 1) Inputs, outputs & canonical domain
### 1.1 Accepted input formats (ingest)
* **OpenVEX** JSON documents (attested or raw).
* **CSAF VEX** 2.x (vendor PSIRTs and distros commonly publish CSAF).
* **CycloneDX VEX** 1.4+ (standalone VEX or embedded VEX blocks).
* **OCI‑attached attestations** (VEX statements shipped as OCI referrers) — optional connectors.
All connectors register **source metadata**: provider identity, trust tier, signature expectations (PGP/cosign/PKI), fetch windows, rate limits, and time anchors.
### 1.2 Canonical model (normalized)
Every incoming statement becomes a set of **VexClaim** records:
```
VexClaim
- providerId // 'redhat', 'suse', 'ubuntu', 'github', 'vendorX'
- vulnId // 'CVE-2025-12345', 'GHSA-xxxx', canonicalized
- productKey // canonical product identity (see §2.2)
- status // affected | not_affected | fixed | under_investigation
- justification? // for 'not_affected'/'affected' where provided
- introducedVersion? // semantics per provider (range or exact)
- fixedVersion? // where provided (range or exact)
- lastObserved // timestamp from source or fetch time
- provenance // doc digest, signature status, fetch URI, line/offset anchors
- evidence[] // raw source snippets for explainability
- supersedes? // optional cross-doc chain (docDigest → docDigest)
```
### 1.3 Exports (consumption)
* **VexConsensus** per `(vulnId, productKey)` with:
* `rollupStatus` (after policy weights/justification gates),
* `sources[]` (winning + losing claims with weights & reasons),
* `policyRevisionId` (identifier of the Excitor policy used),
* `consensusDigest` (stable SHA‑256 over canonical JSON).
* **Raw claims** export for auditing (unchanged, with provenance).
* **Provider snapshots** (per source, last N days) for operator debugging.
* **Index** optimized for backend joins: `(productKey, vulnId) → (status, confidence, sourceSet)`.
All exports are **deterministic**, and (optionally) **attested** via DSSE and logged to Rekor v2.
---
## 2) Identity model — products & joins
### 2.1 Vuln identity
* Accepts **CVE**, **GHSA**, vendor IDs (MSRC, RHSA…), distro IDs (DSA/USN/RHSA…) — normalized to `vulnId` with alias sets.
* **Alias graph** maintained (from Conselier) to map vendor/distro IDs → CVE (primary) and to **GHSA** where applicable.
### 2.2 Product identity (`productKey`)
* **Primary:** `purl` (Package URL).
* **Secondary links:** `cpe`, **OS package NVRA/EVR**, NuGet/Maven/Golang identity, and **OS package name** when purl unavailable.
* **Fallback:** `oci:<registry>/<repo>@<digest>` for image‑level VEX.
* **Special cases:** kernel modules, firmware, platforms → provider‑specific mapping helpers (connector captures provider’s product taxonomy → canonical `productKey`).
> Excitor does not invent identities. If a provider cannot be mapped to purl/CPE/NVRA deterministically, we keep the native **product string** and mark the claim as **non‑joinable**; the backend will ignore it unless a policy explicitly whitelists that provider mapping.
---
## 3) Storage schema (MongoDB)
Database: `excitor`
### 3.1 Collections
**`vex.providers`**
```
_id: providerId
name, homepage, contact
trustTier: enum {vendor, distro, platform, hub, attestation}
signaturePolicy: { type: pgp|cosign|x509|none, keys[], certs[], cosignKeylessRoots[] }
fetch: { baseUrl, kind: http|oci|file, rateLimit, etagSupport, windowDays }
enabled: bool
createdAt, modifiedAt
```
**`vex.raw`** (immutable raw documents)
```
_id: sha256(doc bytes)
providerId
uri
ingestedAt
contentType
sig: { verified: bool, method: pgp|cosign|x509|none, keyId|certSubject, bundle? }
payload: GridFS pointer (if large)
disposition: kept|replaced|superseded
correlation: { replaces?: sha256, replacedBy?: sha256 }
```
**`vex.claims`** (normalized rows; dedupe on providerId+vulnId+productKey+docDigest)
```
_id
providerId
vulnId
productKey
status
justification?
introducedVersion?
fixedVersion?
lastObserved
docDigest
provenance { uri, line?, pointer?, signatureState }
evidence[] { key, value, locator }
indices:
- {vulnId:1, productKey:1}
- {providerId:1, lastObserved:-1}
- {status:1}
- text index (optional) on evidence.value for debugging
```
**`vex.consensus`** (rollups)
```
_id: sha256(canonical(vulnId, productKey, policyRevision))
vulnId
productKey
rollupStatus
sources[]: [
{ providerId, status, justification?, weight, lastObserved, accepted:bool, reason }
]
policyRevisionId
evaluatedAt
consensusDigest // same as _id
indices:
- {vulnId:1, productKey:1}
- {policyRevisionId:1, evaluatedAt:-1}
```
**`vex.exports`** (manifest of emitted artifacts)
```
_id
querySignature
format: raw|consensus|index
artifactSha256
rekor { uuid, index, url }?
createdAt
policyRevisionId
cacheable: bool
```
**`vex.cache`**
```
querySignature -> exportId (for fast reuse)
ttl, hits
```
**`vex.migrations`**
* ordered migrations applied at bootstrap to ensure indexes.
### 3.2 Indexing strategy
* Hot path queries use exact `(vulnId, productKey)` and time‑bounded windows; compound indexes cover both.
* Providers list view by `lastObserved` for monitoring staleness.
* `vex.consensus` keyed by `(vulnId, productKey, policyRevision)` for deterministic reuse.
---
## 4) Ingestion pipeline
### 4.1 Connector contract
```csharp
public interface IVexConnector
{
string ProviderId { get; }
Task FetchAsync(VexConnectorContext ctx, CancellationToken ct); // raw docs
Task NormalizeAsync(VexConnectorContext ctx, CancellationToken ct); // raw -> VexClaim[]
}
```
* **Fetch** must implement: window scheduling, conditional GET (ETag/If‑Modified‑Since), rate limiting, retry/backoff.
* **Normalize** parses the format, validates schema, maps product identities deterministically, emits `VexClaim` records with **provenance**.
### 4.2 Signature verification (per provider)
* **cosign (keyless or keyful)** for OCI referrers or HTTP‑served JSON with Sigstore bundles.
* **PGP** (provider keyrings) for distro/vendor feeds that sign docs.
* **x509** (mutual TLS / provider‑pinned certs) where applicable.
* Signature state is stored on **vex.raw.sig** and copied into **provenance.signatureState** on claims.
> Claims from sources failing signature policy are marked `"signatureState.verified=false"` and **policy** can down‑weight or ignore them.
### 4.3 Time discipline
* For each doc, prefer **provider’s document timestamp**; if absent, use fetch time.
* Claims carry `lastObserved` which drives **tie‑breaking** within equal weight tiers.
---
## 5) Normalization: product & status semantics
### 5.1 Product mapping
* **purl** first; **cpe** second; OS package NVRA/EVR mapping helpers (distro connectors) produce purls via canonical tables (e.g., rpm→purl:rpm, deb→purl:deb).
* Where a provider publishes **platform‑level** VEX (e.g., “RHEL 9 not affected”), connectors expand to known product inventory rules (e.g., map to sets of packages/components shipped in the platform). Expansion tables are versioned and kept per provider; every expansion emits **evidence** indicating the rule applied.
* If expansion would be speculative, the claim remains **platform‑scoped** with `productKey="platform:redhat:rhel:9"` and is flagged **non‑joinable**; backend can decide to use platform VEX only when Scanner proves the platform runtime.
### 5.2 Status + justification mapping
* Canonical **status**: `affected | not_affected | fixed | under_investigation`.
* **Justifications** normalized to a controlled vocabulary (CISA‑aligned), e.g.:
* `component_not_present`
* `vulnerable_code_not_in_execute_path`
* `vulnerable_configuration_unused`
* `inline_mitigation_applied`
* `fix_available` (with `fixedVersion`)
* `under_investigation`
* Providers with free‑text justifications are mapped by deterministic tables; raw text preserved as `evidence`.
---
## 6) Consensus algorithm
**Goal:** produce a **stable**, explainable `rollupStatus` per `(vulnId, productKey)` given possibly conflicting claims.
### 6.1 Inputs
* Set **S** of `VexClaim` for the key.
* **Excitor policy snapshot**:
* **weights** per provider tier and per provider overrides.
* **justification gates** (e.g., require justification for `not_affected` to be acceptable).
* **minEvidence** rules (e.g., `not_affected` must come from ≥1 vendor or 2 distros).
* **signature requirements** (e.g., require verified signature for ‘fixed’ to be considered).
### 6.2 Steps
1. **Filter invalid** claims by signature policy & justification gates → set `S'`.
2. **Score** each claim:
`score = weight(provider) * freshnessFactor(lastObserved)` where freshnessFactor ∈ [0.8, 1.0] for staleness decay (configurable; small effect).
3. **Aggregate** scores per status: `W(status) = Σ score(claims with that status)`.
4. **Pick** `rollupStatus = argmax_status W(status)`.
5. **Tie‑breakers** (in order):
* Higher **max single** provider score wins (vendor > distro > platform > hub).
* More **recent** lastObserved wins.
* Deterministic lexicographic order of status (`fixed` > `not_affected` > `under_investigation` > `affected`) as final tiebreaker.
6. **Explain**: mark accepted sources (`accepted=true; reason="weight"`/`"freshness"`), mark rejected sources with explicit `reason` (`"insufficient_justification"`, `"signature_unverified"`, `"lower_weight"`).
> The algorithm is **pure** given S and policy snapshot; result is reproducible and hashed into `consensusDigest`.
---
## 7) Query & export APIs
All endpoints are versioned under `/api/v1/vex`.
### 7.1 Query (online)
```
POST /claims/search
body: { vulnIds?: string[], productKeys?: string[], providers?: string[], since?: timestamp, limit?: int, pageToken?: string }
→ { claims[], nextPageToken? }
POST /consensus/search
body: { vulnIds?: string[], productKeys?: string[], policyRevisionId?: string, since?: timestamp, limit?: int, pageToken?: string }
→ { entries[], nextPageToken? }
POST /excititor/resolve (scope: vex.read)
body: { productKeys?: string[], purls?: string[], vulnerabilityIds: string[], policyRevisionId?: string }
→ { policy, resolvedAt, results: [ { vulnerabilityId, productKey, status, sources[], conflicts[], decisions[], signals?, summary?, envelope: { artifact, contentSignature?, attestation?, attestationEnvelope?, attestationSignature? } } ] }
```
### 7.2 Exports (cacheable snapshots)
```
POST /exports
body: { signature: { vulnFilter?, productFilter?, providers?, since? }, format: raw|consensus|index, policyRevisionId?: string, force?: bool }
→ { exportId, artifactSha256, rekor? }
GET /exports/{exportId} → bytes (application/json or binary index)
GET /exports/{exportId}/meta → { signature, policyRevisionId, createdAt, artifactSha256, rekor? }
```
### 7.3 Provider operations
```
GET /providers → provider list & signature policy
POST /providers/{id}/refresh → trigger fetch/normalize window
GET /providers/{id}/status → last fetch, doc counts, signature stats
```
**Auth:** service‑to‑service via Authority tokens; operator operations via UI/CLI with RBAC.
---
## 8) Attestation integration
* Exports can be **DSSE‑signed** via **Signer** and logged to **Rekor v2** via **Attestor** (optional but recommended for regulated pipelines).
* `vex.exports.rekor` stores `{uuid, index, url}` when present.
* **Predicate type**: `https://stella-ops.org/attestations/vex-export/1` with fields:
* `querySignature`, `policyRevisionId`, `artifactSha256`, `createdAt`.
---
## 9) Configuration (YAML)
```yaml
excitor:
mongo: { uri: "mongodb://mongo/excitor" }
s3:
endpoint: http://minio:9000
bucket: stellaops
policy:
weights:
vendor: 1.0
distro: 0.9
platform: 0.7
hub: 0.5
attestation: 0.6
providerOverrides:
redhat: 1.0
suse: 0.95
requireJustificationForNotAffected: true
signatureRequiredForFixed: true
minEvidence:
not_affected:
vendorOrTwoDistros: true
connectors:
- providerId: redhat
kind: csaf
baseUrl: https://access.redhat.com/security/data/csaf/v2/
signaturePolicy: { type: pgp, keys: [ "…redhat-pgp-key…" ] }
windowDays: 7
- providerId: suse
kind: csaf
baseUrl: https://ftp.suse.com/pub/projects/security/csaf/
signaturePolicy: { type: pgp, keys: [ "…suse-pgp-key…" ] }
- providerId: ubuntu
kind: openvex
baseUrl: https://…/vex/
signaturePolicy: { type: none }
- providerId: vendorX
kind: cyclonedx-vex
ociRef: ghcr.io/vendorx/vex@sha256:…
signaturePolicy: { type: cosign, cosignKeylessRoots: [ "sigstore-root" ] }
```
---
## 10) Security model
* **Input signature verification** enforced per provider policy (PGP, cosign, x509).
* **Connector allowlists**: outbound fetch constrained to configured domains.
* **Tenant isolation**: per‑tenant DB prefixes or separate DBs; per‑tenant S3 prefixes; per‑tenant policies.
* **AuthN/Z**: Authority‑issued OpToks; RBAC roles (`vex.read`, `vex.admin`, `vex.export`).
* **No secrets in logs**; deterministic logging contexts include providerId, docDigest, claim keys.
---
## 11) Performance & scale
* **Targets:**
* Normalize 10k VEX claims/minute/core.
* Consensus compute ≤ 50 ms for 1k unique `(vuln, product)` pairs in hot cache.
* Export (consensus) 1M rows in ≤ 60 s on 8 cores with streaming writer.
* **Scaling:**
* WebService handles control APIs; **Worker** background services (same image) execute fetch/normalize in parallel with rate‑limits; Mongo writes batched; upserts by natural keys.
* Exports stream straight to S3 (MinIO) with rolling buffers.
* **Caching:**
* `vex.cache` maps query signatures → export; TTL to avoid stampedes; optimistic reuse unless `force`.
---
## 12) Observability
* **Metrics:**
* `vex.ingest.docs_total{provider}`
* `vex.normalize.claims_total{provider}`
* `vex.signature.failures_total{provider,method}`
* `vex.consensus.conflicts_total{vulnId}`
* `vex.exports.bytes{format}` / `vex.exports.latency_seconds`
* **Tracing:** spans for fetch, verify, parse, map, consensus, export.
* **Dashboards:** provider staleness, top conflicting vulns/components, signature posture, export cache hit‑rate.
---
## 13) Testing matrix
* **Connectors:** golden raw docs → deterministic claims (fixtures per provider/format).
* **Signature policies:** valid/invalid PGP/cosign/x509 samples; ensure rejects are recorded but not accepted.
* **Normalization edge cases:** platform‑only claims, free‑text justifications, non‑purl products.
* **Consensus:** conflict scenarios across tiers; check tie‑breakers; justification gates.
* **Performance:** 1M‑row export timing; memory ceilings; stream correctness.
* **Determinism:** same inputs + policy → identical `consensusDigest` and export bytes.
* **API contract tests:** pagination, filters, RBAC, rate limits.
---
## 14) Integration points
* **Backend Policy Engine** (in Scanner.WebService): calls `POST /excititor/resolve` (scope `vex.read`) with batched `(purl, vulnId)` pairs to fetch `rollupStatus + sources`.
* **Conselier**: provides alias graph (CVE↔vendor IDs) and may supply VEX‑adjacent metadata (e.g., KEV flag) for policy escalation.
* **UI**: VEX explorer screens use `/claims/search` and `/consensus/search`; show conflicts & provenance.
* **CLI**: `stellaops vex export --consensus --since 7d --out vex.json` for audits.
---
## 15) Failure modes & fallback
* **Provider unreachable:** stale thresholds trigger warnings; policy can down‑weight stale providers automatically (freshness factor).
* **Signature outage:** continue to ingest but mark `signatureState.verified=false`; consensus will likely exclude or down‑weight per policy.
* **Schema drift:** unknown fields are preserved as `evidence`; normalization rejects only on **invalid identity** or **status**.
---
## 16) Rollout plan (incremental)
1. **MVP**: OpenVEX + CSAF connectors for 3 major providers (e.g., Red Hat/SUSE/Ubuntu), normalization + consensus + `/excititor/resolve`.
2. **Signature policies**: PGP for distros; cosign for OCI.
3. **Exports + optional attestation**.
4. **CycloneDX VEX** connectors; platform claim expansion tables; UI explorer.
5. **Scale hardening**: export indexes; conflict analytics.
---
## 17) Appendix — canonical JSON (stable ordering)
All exports and consensus entries are serialized via `VexCanonicalJsonSerializer`:
* UTF‑8 without BOM;
* keys sorted (ASCII);
* arrays sorted by `(providerId, vulnId, productKey, lastObserved)` unless semantic order mandated;
* timestamps in `YYYY‑MM‑DDThh:mm:ssZ`;
* no insignificant whitespace.

65
docs/modules/excitor/implementation_plan.md
View File

@@ -1,65 +0,0 @@
# Implementation plan — Excitor
## Delivery phases
- **Phase 1 – Connectors & normalization**
Build connectors for OpenVEX, CSAF VEX, CycloneDX VEX, OCI attestations; capture provenance, signatures, and source metadata; normalise into `VexClaim`.
- **Phase 2 – Mapping & trust registry**
Implement product mapping (CPE → purl/version), issuer registry (trust tiers, signatures), scope scoring, and justification taxonomy.
- **Phase 3 – Consensus & projections**
Deliver consensus computation, conflict preservation, projections (`vex_consensus`, history, provider snapshots), and DSSE events.
- **Phase 4 – APIs & integrations**
Expose REST/CLI endpoints for claims, consensus, conflicts, exports; integrate Policy Engine, Vuln Explorer, Advisory AI, Export Center.
- **Phase 5 – Observability & offline**
Ship metrics, logs, traces, dashboards, incident runbooks, Offline Kit bundles, and performance tuning (10M claims/tenant).
## Work breakdown
- **Connectors**
- Fetchers for vendor feeds, CSAF repositories, OpenVEX docs, OCI referrers.
- Signature verification (PGP, cosign, PKI) per source; schema validation; rate limiting.
- Source configuration (trust tier, fetch cadence, blackout windows) stored in metadata registry.
- **Normalization**
- Canonical `VexClaim` schema with deterministic IDs, provenance, supersedes chains.
- Product tree parsing, mapping to canonical product keys and environments.
- Justification and scope scoring derived from source semantics.
- **Consensus & projections**
- Lattice join with precedence rules, conflict tracking, confidence scores, recency decay.
- Append-only history, conflict queue, DSSE events (`vex.consensus.updated`).
- Export-ready JSONL & DSSE bundles for Offline Kit and Export Center.
- **APIs & UX**
- REST endpoints (`/claims`, `/consensus`, `/conflicts`, `/providers`) with tenant RBAC.
- CLI commands `stella vex claims|consensus|conflicts|export`.
- Console modules (list/detail, conflict diagnostics, provider health, simulation hooks).
- **Integrations**
- Policy Engine trust knobs, Vuln Explorer consensus badges, Advisory AI narrative generation, Notify alerts for conflicts.
- Orchestrator jobs for recompute/backfill triggered by Excitor deltas.
- **Observability & Ops**
- Metrics (ingest latency, signature failure rate, conflict rate, consensus latency).
- Logs/traces with tenant/issuer/provenance context.
- Runbooks for mapping failures, signature errors, recompute storms, quota exhaustion.
## Acceptance criteria
- Connectors ingest validated VEX statements with signed provenance, deterministic mapping, and tenant isolation.
- Consensus outputs reproducible, include conflicts, and integrate with Policy Engine/Vuln Explorer/Export Center.
- CLI/Console provide evidence inspection, conflict analysis, and exports; Offline Kit bundles replay verification offline.
- Observability dashboards/alerts capture ingest health, trust anomalies, conflict spikes, and performance budgets.
- Recompute pipeline handles policy changes and new evidence without dropping deterministic outcomes.
## Risks & mitigations
- **Mapping ambiguity:** maintain scope scores, manual overrides, highlight warnings.
- **Signature trust gaps:** issuer registry with auditing, fallback trust policies, tenant overrides.
- **Evidence surges:** orchestrator backpressure, prioritised queues, shardable workers.
- **Performance regressions:** indexing, caching, load tests, budget enforcement.
- **Tenant leakage:** strict RBAC/filters, fuzz tests, compliance reviews.
## Test strategy
- **Unit:** connector parsers, normalization, mapping conversions, lattice operations.
- **Property:** randomised evidence ensuring commutative consensus and deterministic digests.
- **Integration:** end-to-end pipeline from Excitor to consensus export, policy simulation, conflict handling.
- **Performance:** large feed ingestion, recompute stress, CLI export throughput.
- **Security:** signature tampering, issuer revocation, RBAC.
- **Offline:** export/import verification, DSSE bundle validation.
## Definition of done
- Connectors, normalization, consensus, APIs, and integrations deployed with telemetry, runbooks, and Offline Kit parity.
- Documentation (overview, architecture, algorithm, issuer registry, API/CLI, runbooks) updated with imposed rule compliance.
- ./TASKS.md and ../../TASKS.md reflect active status and dependencies.

2
docs/modules/graph/architecture.md
View File

@@ -18,7 +18,7 @@
## 2) Pipelines
1. **Ingestion:** Cartographer/SBOM Service emit SBOM snapshots (`sbom_snapshot` events) captured by the Graph Indexer. Ledger lineage references become `SBOM_VERSION_OF` + `SBOM_LINEAGE_*` edges. Advisories/VEX from Concelier/Excititor generate edge updates, policy runs attach overlay metadata.
2. **ETL:** Normalises nodes/edges into canonical IDs, deduplicates, enforces tenant partitions, and writes to the graph store (planned: Neo4j-compatible or document + adjacency lists in Mongo).
2. **ETL:** Normalises nodes/edges into canonical IDs, deduplicates, enforces tenant partitions, and writes to the graph store (planned: Neo4j-compatible or PostgreSQL adjacency lists).
3. **Overlay computation:** Batch workers build materialised views for frequently used queries (impact lists, saved queries, policy overlays) and store as immutable blobs for Offline Kit exports.
4. **Diffing:** `graph_diff` jobs compare two snapshots (e.g., pre/post deploy) and generate signed diff manifests for UI/CLI consumption.
5. **Analytics (Runtime & Signals 140.A):** background workers run Louvain-style clustering + degree/betweenness approximations on ingested graphs, emitting overlays per tenant/snapshot and writing cluster ids back to nodes when enabled.

2
docs/modules/issuer-directory/operations/deployment.md
View File

@@ -57,7 +57,7 @@
5. **Upgrade & rollback**
- Update Compose images to the desired release manifest (`deploy/releases/*.yaml`), re-run `docker compose config`, then `docker compose up -d`.
- Rollbacks follow the same steps with the previous manifest. Mongo collections are backwards compatible within `2025.10.x`.
- Rollbacks follow the same steps with the previous manifest. PostgreSQL schemas are backwards compatible within `2025.10.x`.
## 3 · Deploy with Helm
1. **Create or update the secret**

4
docs/modules/notify/README.md
View File

@@ -18,7 +18,7 @@ Notify (Notifications Studio) converts platform events into tenant-scoped alerts
- **Worker pipeline:** `StellaOps.Notify.Worker` ingests bus events, evaluates match predicates, applies per-tenant throttles, and dispatches deliveries.
- **Connector plug-ins:** Restart-time plug-ins under `StellaOps.Notify.Connectors.*` (Slack, Teams, Email, generic webhook) with health checks and retry policy hints declared in `notify-plugin.json`.
- **Template engine:** Deterministic rendering with safe helpers, locale bundles, and redaction defaults that keep Offline Kit parity.
- **Delivery ledger:** Mongo-backed ledger storing hashed payloads, attempts, throttled/digested markers, and provenance links for audit + exports.
- **Delivery ledger:** PostgreSQL-backed ledger storing hashed payloads, attempts, throttled/digested markers, and provenance links for audit + exports.
## In progress / upcoming (Sprint 39 focus)
- `NOTIFY-SVC-39-001` correlation engine with token-bucket throttles, incident lifecycle, and quiet-hours evaluator.
@@ -36,7 +36,7 @@ Status for these items is tracked in `src/Notifier/StellaOps.Notifier/TASKS.md`
- [`docs/updates/2025-10-29-notify-docs.md`](../../updates/2025-10-29-notify-docs.md) — latest release note; follow-ups remain to validate connector metadata, quiet-hours semantics, and simulation payloads once Sprint 39 drops land.
## Integrations & dependencies
- **Storage:** MongoDB (`rules`, `channels`, `deliveries`, `digests`, `throttles`) with change streams for worker snapshots.
- **Storage:** PostgreSQL (schema `notify`) for rules, channels, deliveries, digests, and throttles; Valkey for worker coordination.
- **Queues:** Redis Streams or NATS JetStream for ingestion, throttling, and DLQs (`notify.dlq`).
- **Authority:** OpTok-protected APIs, DPoP-backed CLI/UI scopes (`notify.viewer`, `notify.operator`, `notify.admin`), and secret references for channel credentials.
- **Observability:** Prometheus metrics (`notify.sent_total`, `notify.failed_total`, `notify.digest_coalesced_total`, etc.), OTEL traces, and dashboards documented in `docs/notifications/architecture.md#12-observability-prometheus--otel`.

2
docs/modules/notify/bootstrap-pack.md
View File

@@ -23,7 +23,7 @@ copied directly onto the hosts that run `StellaOps.Notifier.WebService`.
`chmod 600`).
2. **Drop configuration** – place `notify.yaml` in the location expected by
the runtime (`/app/etc/notify.yaml` for the containers we ship). The file
assumes MongoDB is reachable at `mongodb://stellaops:airgap-password@mongo:27017`
assumes PostgreSQL is reachable at `Host=postgres;Database=stellaops;Username=stellaops;Password=airgap-password`
and Authority at `https://authority.airgap.local` – adjust if your
deployment uses different hostnames.
3. **Import rule/template** – with the Notify CLI or REST API, import

4
docs/modules/orchestrator/architecture.md
View File

@@ -5,8 +5,8 @@
## 1) Topology
- **Orchestrator API (`StellaOps.Orchestrator`).** Minimal API providing job state, throttling controls, replay endpoints, and dashboard data. Authenticated via Authority scopes (`orchestrator:*`).
- **Job ledger (Mongo).** Collections `jobs`, `job_history`, `sources`, `quotas`, `throttles`, `incidents`. Append-only history ensures auditability.
- **Queue abstraction.** Supports Mongo queue, Redis Streams, or NATS JetStream (pluggable). Each job carries lease metadata and retry policy.
- **Job ledger (PostgreSQL).** Tables `jobs`, `job_history`, `sources`, `quotas`, `throttles`, `incidents` (schema `orchestrator`). Append-only history ensures auditability.
- **Queue abstraction.** Supports Valkey Streams or NATS JetStream (pluggable). Each job carries lease metadata and retry policy.
- **Dashboard feeds.** SSE/GraphQL endpoints supply Console UI with job timelines, throughput, error distributions, and rate-limit status.
## 2) Job lifecycle

2
docs/modules/policy/README.md
View File

@@ -13,7 +13,7 @@ Policy Engine compiles and evaluates Stella DSL policies deterministically, prod
- Shared libraries under `StellaOps.Policy.*` for evaluation, storage, DSL tooling.
## Integrations & dependencies
- MongoDB findings collections, RustFS explain bundles.
- PostgreSQL (schema `policy`) for findings, RustFS explain bundles.
- Scheduler for incremental re-evaluation triggers.
- CLI/UI for policy authoring and runs.

382
docs/modules/provcache/README.md Normal file
View File

@@ -0,0 +1,382 @@
# Provcache Module
> Provenance Cache — Maximizing Trust Evidence Density
## Overview
Provcache is a caching layer that maximizes "provenance density" — the amount of trustworthy evidence retained per byte — enabling faster security decisions, offline replays, and smaller air-gap bundles.
### Key Benefits
- **Trust Latency**: Warm cache lookups return in single-digit milliseconds
- **Bandwidth Efficiency**: Avoid re-fetching bulky SBOMs/attestations
- **Offline Operation**: Decisions usable without full SBOM/VEX payloads
- **Audit Transparency**: Full evidence chain verifiable via Merkle proofs
## Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ Policy Evaluator │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ VeriKey │───▶│ Provcache │───▶│ TrustLatticeEngine │ │
│ │ Builder │ │ Service │ │ (if cache miss) │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ Provcache Store │
│ ┌─────────────┐ ┌────────────────┐ │
│ │ Valkey │◀──▶│ Postgres │ │
│ │ (read-thru) │ │ (write-behind) │ │
│ └─────────────┘ └────────────────┘ │
└─────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────┐
│ Evidence Chunk Store │
│ ┌─────────────────────────────────────┐│
│ │ prov_evidence_chunks (Postgres) ││
│ │ - Chunked SBOM/VEX/CallGraph ││
│ │ - Merkle tree verification ││
│ └─────────────────────────────────────┘│
└─────────────────────────────────────────┘
```
## Core Concepts
### VeriKey (Provenance Identity Key)
A composite hash that uniquely identifies a provenance decision context:
```
VeriKey = SHA256(
source_hash || // Image/artifact digest
sbom_hash || // Canonical SBOM hash
vex_hash_set_hash || // Sorted VEX statement hashes
merge_policy_hash || // PolicyBundle hash
signer_set_hash || // Signer certificate hashes
time_window // Epoch bucket
)
```
**Why each component?**
| Component | Purpose |
|-----------|---------|
| `source_hash` | Different artifacts → different keys |
| `sbom_hash` | SBOM changes (new packages) → new key |
| `vex_hash_set` | VEX updates → new key |
| `policy_hash` | Policy changes → new key |
| `signer_set_hash` | Key rotation → new key (security) |
| `time_window` | Temporal bucketing → controlled expiry |
### DecisionDigest
Canonicalized representation of an evaluation result:
```json
{
"digestVersion": "v1",
"veriKey": "sha256:abc123...",
"verdictHash": "sha256:def456...",
"proofRoot": "sha256:789abc...",
"replaySeed": {
"feedIds": ["cve-2024", "ghsa-2024"],
"ruleIds": ["default-policy-v2"]
},
"trustScore": 85,
"createdAt": "2025-12-24T12:00:00Z",
"expiresAt": "2025-12-25T12:00:00Z"
}
```
### Trust Score
A composite score (0-100) indicating decision confidence:
| Component | Weight | Calculation |
|-----------|--------|-------------|
| Reachability | 25% | Call graph coverage, entry points analyzed |
| SBOM Completeness | 20% | Package count, license data presence |
| VEX Coverage | 20% | Vendor statements, justifications |
| Policy Freshness | 15% | Time since last policy update |
| Signer Trust | 20% | Key age, reputation, chain validity |
### Evidence Chunks
Large evidence (SBOM, VEX, call graphs) is stored in fixed-size chunks:
- **Default size**: 64 KB per chunk
- **Merkle verification**: Each chunk is a Merkle leaf
- **Lazy fetch**: Only fetch chunks needed for audit
- **LRU eviction**: Old chunks evicted under storage pressure
## API Reference
### Endpoints
| Method | Path | Description |
|--------|------|-------------|
| GET | `/v1/provcache/{veriKey}` | Lookup cached decision |
| POST | `/v1/provcache` | Store decision (idempotent) |
| POST | `/v1/provcache/invalidate` | Invalidate by pattern |
| GET | `/v1/proofs/{proofRoot}` | List evidence chunks |
| GET | `/v1/proofs/{proofRoot}/chunks/{index}` | Download chunk |
### Cache Lookup Flow
```mermaid
sequenceDiagram
participant Client
participant PolicyEngine
participant Provcache
participant Valkey
participant Postgres
participant TrustLattice
Client->>PolicyEngine: Evaluate(artifact)
PolicyEngine->>Provcache: Get(VeriKey)
Provcache->>Valkey: GET verikey
alt Cache Hit
Valkey-->>Provcache: DecisionDigest
Provcache-->>PolicyEngine: CacheResult(hit)
PolicyEngine-->>Client: Decision (cached)
else Cache Miss
Valkey-->>Provcache: null
Provcache->>Postgres: SELECT * FROM provcache_items
alt DB Hit
Postgres-->>Provcache: ProvcacheEntry
Provcache->>Valkey: SET (backfill)
Provcache-->>PolicyEngine: CacheResult(hit, source=postgres)
else DB Miss
Postgres-->>Provcache: null
Provcache-->>PolicyEngine: CacheResult(miss)
PolicyEngine->>TrustLattice: Evaluate
TrustLattice-->>PolicyEngine: EvaluationResult
PolicyEngine->>Provcache: Set(VeriKey, DecisionDigest)
Provcache->>Valkey: SET
Provcache->>Postgres: INSERT (async)
PolicyEngine-->>Client: Decision (computed)
end
end
```
## Invalidation
### Automatic Invalidation Triggers
| Trigger | Event | Scope |
|---------|-------|-------|
| Signer Revocation | `SignerRevokedEvent` | All entries with matching `signer_set_hash` |
| Feed Epoch Advance | `FeedEpochAdvancedEvent` | Entries with older `feed_epoch` |
| Policy Update | `PolicyUpdatedEvent` | Entries with matching `policy_hash` |
| TTL Expiry | Background job | Entries past `expires_at` |
### Manual Invalidation
```bash
# Invalidate by signer
POST /v1/provcache/invalidate
{
"by": "signer_set_hash",
"value": "sha256:revoked-signer...",
"reason": "key-compromise"
}
# Invalidate by policy
POST /v1/provcache/invalidate
{
"by": "policy_hash",
"value": "sha256:old-policy...",
"reason": "policy-update"
}
```
## Air-Gap Integration
### Export Workflow
```bash
# Export minimal proof (digest only)
stella prov export --verikey sha256:abc123 --density lite
# Export with evidence chunks
stella prov export --verikey sha256:abc123 --density standard
# Export full evidence
stella prov export --verikey sha256:abc123 --density strict --sign
```
### Import Workflow
```bash
# Import and verify Merkle root
stella prov import --input proof.bundle
# Import with lazy chunk fetch
stella prov import --input proof-lite.json --lazy-fetch --backend https://api.stellaops.com
```
### Density Levels
| Level | Contents | Size | Use Case |
|-------|----------|------|----------|
| `lite` | DecisionDigest + ProofRoot | ~2 KB | Quick verification |
| `standard` | + First N chunks | ~200 KB | Normal audit |
| `strict` | + All chunks | Variable | Full compliance |
## Configuration
```yaml
provcache:
# TTL configuration
defaultTtl: 24h
maxTtl: 168h # 7 days
timeWindowBucket: 1h
# Storage
valkeyKeyPrefix: "stellaops:prov:"
enableWriteBehind: true
writeBehindFlushInterval: 5s
writeBehindMaxBatchSize: 100
# Evidence chunking
chunkSize: 65536 # 64 KB
maxChunksPerEntry: 1000
# Behavior
allowCacheBypass: true
digestVersion: "v1"
```
## Observability
### Metrics
| Metric | Type | Description |
|--------|------|-------------|
| `provcache_requests_total` | Counter | Total cache requests |
| `provcache_hits_total` | Counter | Cache hits |
| `provcache_misses_total` | Counter | Cache misses |
| `provcache_latency_seconds` | Histogram | Operation latency |
| `provcache_items_count` | Gauge | Current item count |
| `provcache_invalidations_total` | Counter | Invalidation count |
### Alerts
```yaml
# Low cache hit rate
- alert: ProvcacheLowHitRate
expr: rate(provcache_hits_total[5m]) / rate(provcache_requests_total[5m]) < 0.5
for: 10m
labels:
severity: warning
annotations:
summary: "Provcache hit rate below 50%"
# High invalidation rate
- alert: ProvcacheHighInvalidationRate
expr: rate(provcache_invalidations_total[5m]) > 100
for: 5m
labels:
severity: warning
annotations:
summary: "High cache invalidation rate"
```
## Security Considerations
### Signer-Aware Caching
The `signer_set_hash` is part of the VeriKey, ensuring:
- Key rotation → new cache entries
- Key revocation → immediate invalidation
- No stale decisions from compromised signers
### Merkle Verification
All evidence chunks are Merkle-verified:
- `ProofRoot` = Merkle root of all chunks
- Individual chunks verifiable without full tree
- Tamper detection on import
### Audit Trail
All invalidations are logged to `prov_revocations` table:
```sql
SELECT * FROM provcache.prov_revocations
WHERE created_at > NOW() - INTERVAL '24 hours'
ORDER BY created_at DESC;
```
## Database Schema
### provcache_items
```sql
CREATE TABLE provcache.provcache_items (
verikey TEXT PRIMARY KEY,
digest_version TEXT NOT NULL,
verdict_hash TEXT NOT NULL,
proof_root TEXT NOT NULL,
replay_seed JSONB NOT NULL,
policy_hash TEXT NOT NULL,
signer_set_hash TEXT NOT NULL,
feed_epoch TEXT NOT NULL,
trust_score INTEGER NOT NULL,
hit_count BIGINT DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL,
expires_at TIMESTAMPTZ NOT NULL,
updated_at TIMESTAMPTZ NOT NULL
);
```
### prov_evidence_chunks
```sql
CREATE TABLE provcache.prov_evidence_chunks (
chunk_id UUID PRIMARY KEY,
proof_root TEXT NOT NULL REFERENCES provcache_items(proof_root),
chunk_index INTEGER NOT NULL,
chunk_hash TEXT NOT NULL,
blob BYTEA NOT NULL,
blob_size INTEGER NOT NULL,
content_type TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
```
### prov_revocations
```sql
CREATE TABLE provcache.prov_revocations (
revocation_id UUID PRIMARY KEY,
revocation_type TEXT NOT NULL,
target_hash TEXT NOT NULL,
reason TEXT,
actor TEXT,
entries_affected BIGINT NOT NULL,
created_at TIMESTAMPTZ NOT NULL
);
```
## Implementation Sprints
| Sprint | Focus | Key Deliverables |
|--------|-------|------------------|
| [8200.0001.0001](../../implplan/SPRINT_8200_0001_0001_provcache_core_backend.md) | Core Backend | VeriKey, DecisionDigest, Valkey+Postgres, API |
| [8200.0001.0002](../../implplan/SPRINT_8200_0001_0002_provcache_invalidation_airgap.md) | Invalidation & Air-Gap | Signer revocation, feed epochs, CLI export/import |
| [8200.0001.0003](../../implplan/SPRINT_8200_0001_0003_provcache_ux_observability.md) | UX & Observability | UI badges, proof tree, Grafana, OCI attestation |
## Related Documentation
- [Policy Engine Architecture](../policy/README.md)
- [TrustLattice Engine](../policy/design/policy-deterministic-evaluator.md)
- [Offline Kit Documentation](../../24_OFFLINE_KIT.md)
- [Air-Gap Controller](../airgap/README.md)
- [Authority Key Rotation](../authority/README.md)

86
docs/modules/router/messaging-valkey-transport.md
View File

@@ -1,8 +1,9 @@
# Router · Messaging Transport over Valkey (Draft v0.1)
# Router · Messaging Transport over Valkey
## Status
- Draft; intended for implementation via a dedicated sprint.
- Last updated: 2025-12-23 (UTC).
- **Implemented** in Sprint 8100.0011.0003.
- Core components: Gateway DI wiring, GatewayHostedService integration, GatewayTransportClient dispatch.
- Last updated: 2025-12-24 (UTC).
## Purpose
Enable Gateway ↔ microservice Router traffic over an offline-friendly, Redis-compatible transport (Valkey) by using the existing **Messaging** transport layer:
@@ -11,15 +12,18 @@ Enable Gateway ↔ microservice Router traffic over an offline-friendly, Redis-c
This supports environments where direct TCP/TLS microservice connections are undesirable, and where an internal message bus is the preferred control plane.
## What Exists Today (Repository Reality)
- Messaging transport server/client:
- `src/__Libraries/StellaOps.Router.Transport.Messaging/MessagingTransportServer.cs`
- `src/__Libraries/StellaOps.Router.Transport.Messaging/MessagingTransportClient.cs`
- Valkey-backed message queue factory:
- `src/__Libraries/StellaOps.Messaging.Transport.Valkey/ValkeyMessageQueueFactory.cs`
- Gateway WebService currently starts only TCP/TLS transport servers:
- `src/Gateway/StellaOps.Gateway.WebService/Program.cs`
- `src/Gateway/StellaOps.Gateway.WebService/Services/GatewayHostedService.cs`
## Implementation Summary
### Libraries
- `StellaOps.Router.Transport.Messaging` — Router transport layer over messaging
- `StellaOps.Messaging.Transport.Valkey` — Valkey/Redis backend for messaging
- `StellaOps.Messaging` — Core messaging abstractions and DI
### Gateway Integration
- **Program.cs** — Conditional registration of `ValkeyTransportPlugin` and `AddMessagingTransportServer()`
- **GatewayOptions.cs** — `GatewayMessagingTransportOptions` for Valkey connection and queue configuration
- **GatewayHostedService.cs** — Start/stop `MessagingTransportServer`, subscribe to events
- **GatewayTransportClient.cs** — Dispatch to `TransportType.Messaging` connections
## High-Level Flow
1) Microservice connects via messaging transport:
@@ -39,14 +43,35 @@ The Messaging transport uses a small set of queues (names are configurable):
- **Per-service request queues**: gateway publishes REQUEST frames targeted to a service
- **Dead letter queues** (optional): for messages that exceed retries/leases
## Configuration (Draft)
### Gateway
- Register Valkey messaging services (`StellaOps.Messaging.Transport.Valkey`)
- Add messaging transport server (`AddMessagingTransportServer`)
- Add Gateway config section for messaging transport options:
- Valkey connection info (host/port/auth)
- queue naming prefix
- consumer group / lease duration / dead-letter suffix
## Configuration
### Gateway YAML Configuration
```yaml
Gateway:
Transports:
Messaging:
Enabled: true
ConnectionString: "valkey:6379"
Database: 0
RequestQueueTemplate: "router:requests:{service}"
ResponseQueueName: "router:responses"
ConsumerGroup: "router-gateway"
RequestTimeout: "30s"
LeaseDuration: "5m"
BatchSize: 10
HeartbeatInterval: "10s"
```
### Gateway DI Registration
```csharp
// In Program.cs (already implemented)
if (bootstrapOptions.Transports.Messaging.Enabled)
{
builder.Services.AddMessagingTransport<ValkeyTransportPlugin>(
builder.Configuration, "Gateway:Transports:Messaging");
builder.Services.AddMessagingTransportServer();
}
```
### Microservice
- Register Valkey messaging services (`StellaOps.Messaging.Transport.Valkey`)
@@ -63,13 +88,20 @@ The Messaging transport uses a small set of queues (names are configurable):
- The Gateway must not trust client-supplied identity headers; it must overwrite reserved headers before dispatch.
- See `docs/modules/gateway/identity-header-policy.md`.
## Gaps / Implementation Work
1) Wire Messaging transport into Gateway:
- start/stop `MessagingTransportServer`
- subscribe to HELLO/HEARTBEAT/RESPONSE events and reuse existing HELLO validation + routing state updates
2) Extend Gateway transport client to support `TransportType.Messaging` for dispatch.
3) Add config mapping and deployment examples (compose/helm) for Valkey transport.
4) Add integration tests covering:
## Implementation Status
### Completed (Sprint 8100.0011.0003)
1. ✅ Wire Messaging transport into Gateway:
- start/stop `MessagingTransportServer` in `GatewayHostedService`
- subscribe to `OnHelloReceived`, `OnHeartbeatReceived`, `OnResponseReceived`, `OnConnectionClosed` events
- reuse routing state updates and claims store updates
2. ✅ Extend Gateway transport client to support `TransportType.Messaging` for dispatch.
3. ✅ Add config options (`GatewayMessagingTransportOptions`) and DI mappings.
### Remaining Work
1. Add deployment examples (compose/helm) for Valkey transport.
2. Add integration tests using ValkeyFixture:
- microservice HELLO registration via messaging
- request dispatch + response return
3. Validate streaming support (or document as out-of-scope).

2
docs/modules/scanner/design/ruby-analyzer.md
View File

@@ -93,7 +93,7 @@
|----------|-------|----------|
| `ruby_packages.json` | `RubyPackageInventory { scanId, imageDigest, generatedAt, packages[] }` where each package mirrors `{id, name, version, source, provenance, groups[], platform, runtime.*}` | SBOM Composer, Policy Engine |
`ruby_packages.json` records are persisted in Mongo’s `ruby.packages` collection via the `RubyPackageInventoryStore`. Scanner.WebService exposes the same payload through `GET /api/scans/{scanId}/ruby-packages` so Policy, CLI, and Offline Kit consumers can reuse the canonical inventory without re-running the analyzer. Each document is keyed by `scanId` and includes the resolved `imageDigest` plus the UTC timestamp recorded by the Worker.
`ruby_packages.json` records are persisted in PostgreSQL's `scanner.ruby_packages` table via the `RubyPackageInventoryStore`. Scanner.WebService exposes the same payload through `GET /api/scans/{scanId}/ruby-packages` so Policy, CLI, and Offline Kit consumers can reuse the canonical inventory without re-running the analyzer. Each record is keyed by `scanId` and includes the resolved `imageDigest` plus the UTC timestamp recorded by the Worker.
| `ruby_runtime_edges.json` | Edges `{from, to, reason, confidence}` | EntryTrace overlay, Policy explain traces |
| `ruby_capabilities.json` | Capability `{kind, location, evidenceHash, params}` | Policy Engine (capability predicates) |
| `ruby_observation.json` | Summary document (packages, runtime edges, capability flags) | Surface manifest, Policy explain traces |

2
docs/modules/scanner/operations/dsse-rekor-operator-guide.md
View File

@@ -37,7 +37,7 @@
- `scanner.attestation.attestorEndpoint` → Attestor base URL.
- `attestor.rekor.api` & `attestor.rekor.pubkey` set for the target log.
3. **Storage**
- Mongo collections `attestations` & `rekorProofs` sized for retention (7–30 days recommended).
- PostgreSQL tables `attestations` & `rekor_proofs` sized for retention (7–30 days recommended).
- Object store tier with at-rest encryption for DSSE payloads.
4. **Observability**
- Metrics: `attestor_rekor_success_total`, `attestor_rekor_retry_total`, `rekor_inclusion_latency`.

2
docs/modules/scanner/operations/secret-leak-detection.md
View File

@@ -74,7 +74,7 @@ Restart the worker so the analyzer reloads the updated bundle. Bundles are immut
- Scanner.Analysis.SecretFindingsTtl
```
The migration adds `secretFindings` documents to `ScanAnalysisStore` with the standard TTL (default 90 days). Adjust Mongo TTL via the deployment overlay if longer retention is required.
The migration adds `secretFindings` records to `ScanAnalysisStore` with the standard TTL (default 90 days). Adjust PostgreSQL retention policy via the deployment overlay if longer retention is required.
3. **Activate policy ingestion** (WebService):

2
docs/modules/scanner/readiness-checkpoints.md
View File

@@ -13,7 +13,7 @@
| Phase V · Sprint 0134 (PHP fixtures/runtime/package) | Green | PHP analyzer fixtures, runtime evidence, and packaging shipped; docs updated. | Keep fixture hashes stable; rerun benchmarks when dependencies change. |
| Phase VI · Sprint 0135 (Python container + Ruby VFS/edges) | Green | Python container/zipapp adapters shipped; Ruby VFS/dependency edges/observations/runtime capture packaged; EntryTrace 18-502/503 delivered. | Maintain determinism; re-run EntryTrace suite in CI. |
| Phase VII · Sprint 0136 (EntryTrace surface/CLI) | Green | EntryTrace phase VII tasks 18-504/505/506 completed; CLI/WebService surfaces show best-terminal metadata and confidence. | Keep NDJSON schema stable; rerun worker payload tests in CI. |
| Sprint 0138 (Ruby parity & future analyzers) | Amber/Red | Ruby parity shipped; Mongo package inventory live. PHP pipeline SCANNER-ENG-0010 blocked on composer/autoload design + restore stability (design at `docs/modules/scanner/design/php-autoload-design.md`); Deno scope drafted (`docs/modules/scanner/design/deno-analyzer-scope.md`); Dart/Swift scope drafted (`docs/modules/scanner/design/dart-swift-analyzer-scope.md`); Kubernetes/VM roadmap pending. | Implement PHP autoload parser/fixtures per design; add Deno fixtures and validation evidence; align with Zastava/Runtime and update readiness once fixtures land. |
| Sprint 0138 (Ruby parity & future analyzers) | Amber/Red | Ruby parity shipped; PostgreSQL package inventory live. PHP pipeline SCANNER-ENG-0010 blocked on composer/autoload design + restore stability (design at `docs/modules/scanner/design/php-autoload-design.md`); Deno scope drafted (`docs/modules/scanner/design/deno-analyzer-scope.md`); Dart/Swift scope drafted (`docs/modules/scanner/design/dart-swift-analyzer-scope.md`); Kubernetes/VM roadmap pending. | Implement PHP autoload parser/fixtures per design; add Deno fixtures and validation evidence; align with Zastava/Runtime and update readiness once fixtures land. |
## Overall
- Green areas: native analyzers, PHP fixtures/runtime packaging, Ruby analyzer, Python container adapters, EntryTrace phases VI–VII.

2
docs/modules/scheduler/README.md
View File

@@ -14,7 +14,7 @@ Scheduler detects advisory/VEX deltas, computes impact windows, and orchestrates
- Shared libraries under `StellaOps.Scheduler.*`.
## Integrations & dependencies
- MongoDB for impact models.
- PostgreSQL (schema `scheduler`) for impact models.
- Redis/NATS for queueing.
- Policy Engine, Scanner, Notify.

215
docs/modules/signals/architecture.md Normal file
View File

@@ -0,0 +1,215 @@
# Signals Module Architecture
## Overview
The **Signals** module provides a unified evidence-weighted scoring system for vulnerability findings. It aggregates evidence from multiple sources (reachability analysis, runtime observations, backport detection, exploit intelligence, source trust, and mitigations) into a single 0-100 score that enables rapid triage.
## Module Purpose
- **Unify Scoring:** Combine disparate evidence signals into one actionable score
- **Enable Triage:** Support sorting, filtering, and prioritization by evidence strength
- **Maintain Determinism:** Same inputs + policy = same score, always
- **Provide Transparency:** Decomposable scores with explanations
## Location
```
src/Signals/
├── StellaOps.Signals/
│ └── EvidenceWeightedScore/
│ ├── EvidenceWeightedScoreCalculator.cs
│ ├── EvidenceWeightedScoreInput.cs
│ ├── EvidenceWeightedScoreResult.cs
│ ├── Normalizers/
│ │ ├── BackportEvidenceNormalizer.cs
│ │ ├── ExploitLikelihoodNormalizer.cs
│ │ ├── MitigationNormalizer.cs
│ │ ├── ReachabilityNormalizer.cs
│ │ ├── RuntimeSignalNormalizer.cs
│ │ └── SourceTrustNormalizer.cs
│ ├── Guardrails/
│ │ └── ScoreGuardrails.cs
│ └── Policy/
│ ├── EvidenceWeightPolicy.cs
│ └── EvidenceWeightPolicyProvider.cs
└── __Tests/
└── StellaOps.Signals.Tests/
└── EvidenceWeightedScore/
```
## Core Concepts
### Evidence Dimensions
| Dimension | Symbol | Description | Source Module |
|-----------|--------|-------------|---------------|
| Reachability | RCH | Code path reachability to vulnerable sink | Policy |
| Runtime | RTS | Live observation strength (eBPF/dyld/ETW) | Policy |
| Backport | BKP | Patch evidence from distro/changelog/binary | Concelier |
| Exploit | XPL | Exploit probability (EPSS + KEV) | Scanner, Concelier |
| Source Trust | SRC | VEX source trustworthiness | Excititor |
| Mitigations | MIT | Active protective controls | Policy |
### Scoring Formula
```
Score = clamp01(W_rch*RCH + W_rts*RTS + W_bkp*BKP + W_xpl*XPL + W_src*SRC - W_mit*MIT) * 100
```
Note: MIT is **subtractive** — mitigations reduce risk.
### Guardrails
Hard caps and floors based on evidence conditions:
1. **Not-Affected Cap:** If vendor says not-affected (BKP=1) and no runtime contradiction (RTS<0.6), cap at 15
2. **Runtime Floor:** If strong live signal (RTS>=0.8), floor at 60
3. **Speculative Cap:** If no reachability and no runtime (RCH=0, RTS=0), cap at 45
### Score Buckets
| Bucket | Range | Meaning |
|--------|-------|---------|
| ActNow | 90-100 | Strong evidence of exploitable risk; immediate action |
| ScheduleNext | 70-89 | Likely real; schedule for next sprint |
| Investigate | 40-69 | Moderate evidence; investigate when touching component |
| Watchlist | 0-39 | Low/insufficient evidence; monitor |
## Architecture
### Component Diagram
```
┌─────────────────────────────────────────────────────────────────┐
│ Signals Module │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ ┌──────────────────────────────────┐ │
│ │ NormalizerAggr. │───▶│ EvidenceWeightedScoreCalculator │ │
│ └────────┬────────┘ └──────────────┬───────────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────┐ ┌──────────────────────────────────┐ │
│ │ Normalizers │ │ ScoreGuardrails │ │
│ │ ┌─────┐ ┌─────┐ │ └──────────────────────────────────┘ │
│ │ │ BKP │ │ XPL │ │ │
│ │ └─────┘ └─────┘ │ ┌──────────────────────────────────┐ │
│ │ ┌─────┐ ┌─────┐ │ │ EvidenceWeightPolicyProvider │ │
│ │ │ MIT │ │ RCH │ │ └──────────────────────────────────┘ │
│ │ └─────┘ └─────┘ │ │
│ │ ┌─────┐ ┌─────┐ │ │
│ │ │ RTS │ │ SRC │ │ │
│ │ └─────┘ └─────┘ │ │
│ └─────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
│ │
▼ ▼
┌─────────────────────┐ ┌───────────────────────────────────────┐
│ External Modules │ │ Consumers │
│ │ │ │
│ • Policy (RCH, RTS) │ │ • Policy Engine (verdict enrichment) │
│ • Concelier (BKP) │ │ • Findings API (score endpoints) │
│ • Scanner (XPL) │ │ • Web UI (score components) │
│ • Excititor (SRC) │ │ │
└─────────────────────┘ └───────────────────────────────────────┘
```
### Data Flow
1. **Finding arrives** for scoring
2. **NormalizerAggregator** collects evidence from source modules
3. **Individual normalizers** convert raw evidence to 0-1 values
4. **EvidenceWeightedScoreCalculator** applies formula
5. **ScoreGuardrails** apply caps/floors
6. **Result** returned with score, bucket, breakdown, explanations
### Policy Configuration
Weight policies are loaded from YAML and can vary by:
- **Tenant:** Different organizations may prioritize differently
- **Environment:** Production may weight runtime higher than dev
- **Policy version:** Changes are versioned for reproducibility
```yaml
version: "ews.v1"
profile: production
weights:
rch: 0.30
rts: 0.25
bkp: 0.15
xpl: 0.15
src: 0.10
mit: 0.10
guardrails:
not_affected_cap:
enabled: true
max_score: 15
runtime_floor:
enabled: true
min_score: 60
speculative_cap:
enabled: true
max_score: 45
```
## Integration Points
### Inbound (Evidence Sources)
| Source | Data | Interface |
|--------|------|-----------|
| Policy/ConfidenceCalculator | ReachabilityEvidence, RuntimeEvidence | Direct type reference |
| Concelier/BackportProofService | ProofBlob | Direct type reference |
| Scanner/EpssPriorityCalculator | EPSS score, percentile | Direct type reference |
| Concelier/VendorRiskSignalExtractor | KEV status | Direct type reference |
| Excititor/TrustVector | Trust vector components | Direct type reference |
| Policy/GateMultipliers | Active mitigations | Direct type reference |
### Outbound (Consumers)
| Consumer | Usage |
|----------|-------|
| Policy Engine | Verdict enrichment, score-based rules |
| Findings API | Score endpoints (calculate, get, history) |
| Web UI | Score display components |
| Webhooks | Score change notifications |
| Attestor | Scoring proofs in attestations |
## Determinism
The Signals module maintains strict determinism:
1. **Same inputs + same policy = same score:** No randomness, no time-dependent factors in formula
2. **Policy versioning:** Every policy has a digest; scores include policy digest
3. **Reproducible proofs:** Scoring proofs can be verified by recalculation
4. **Stable ordering:** Input order doesn't affect result
## Performance
| Metric | Target |
|--------|--------|
| Single score calculation | <10ms |
| Batch (100 findings) | <1s |
| Policy load | <100ms |
| Cache hit ratio | >80% |
## Testing Strategy
| Level | Focus | Location |
|-------|-------|----------|
| L0 Unit | Calculator, normalizers, guardrails | `*.Tests` |
| L0 Property | Monotonicity, bounds, determinism | `*.Tests/Properties` |
| S1 Integration | Cross-module evidence flow | `*.Tests/Integration` |
| S1 Snapshot | Result JSON structure | `*.Tests/Snapshots` |
## Related Documentation
- Product Advisory: `docs/product-advisories/24-Dec-2025 - Evidence-Weighted Score Model.md`
- Sprint Plans: `docs/implplan/SPRINT_8200_0012_*.md`
- Policy Confidence (deprecated): `docs/modules/policy/confidence-scoring.md`
- Backport Detection: `docs/modules/concelier/backport-detection.md`
- EPSS Enrichment: `docs/modules/scanner/epss-enrichment.md`
- Trust Vector: `docs/modules/excititor/trust-vector.md`

2
docs/modules/signer/architecture.md
View File

@@ -400,7 +400,7 @@ signer:
* Run ≥ 2 replicas; front with L7 LB; **sticky** not required.
* Redis for replay/quota caches (HA).
* Audit sink (Mongo/Postgres) in primary region; asynchronous write with local fallback buffer.
* Audit sink (PostgreSQL) in primary region; asynchronous write with local fallback buffer.
* Fulcio/KMS clients configured with retries/backoff; circuit breakers.
---

99
docs/modules/taskrunner/migrations/pack-run-collections.md
View File

@@ -1,99 +0,0 @@
# Task Runner Collections — Initial Migration
Last updated: 2025-11-06
This migration seeds the MongoDB collections that back the Task Runner service. It is implemented as `20251106-task-runner-baseline.mongosh` under the platform migration runner and must be applied **before** enabling the TaskRunner service in any environment.
## Collections
### `pack_runs`
| Field | Type | Notes |
|------------------|-----------------|-----------------------------------------------------------|
| `_id` | `string` | Run identifier (same as `runId`). |
| `planHash` | `string` | Deterministic hash produced by the planner. |
| `plan` | `object` | Full `TaskPackPlan` payload used to execute the run. |
| `failurePolicy` | `object` | Retry/backoff directives resolved at plan time. |
| `requestedAt` | `date` | Timestamp when the client requested the run. |
| `createdAt` | `date` | Timestamp when the run was persisted. |
| `updatedAt` | `date` | Timestamp of the last mutation. |
| `steps` | `array<object>` | Flattened step records (`stepId`, `status`, attempts…). |
| `tenantId` | `string` | Optional multi-tenant scope (reserved for future phases). |
**Indexes**
1. `{ _id: 1 }` — implicit primary key / uniqueness guarantee.
2. `{ updatedAt: -1 }` — serves `GET /runs` listings and staleness checks.
3. `{ tenantId: 1, updatedAt: -1 }` — activated once tenancy is enforced; remains sparse until then.
### `pack_run_logs`
| Field | Type | Notes |
|---------------|-----------------|--------------------------------------------------------|
| `_id` | `ObjectId` | Generated per log entry. |
| `runId` | `string` | Foreign key to `pack_runs._id`. |
| `sequence` | `long` | Monotonic counter assigned by the writer. |
| `timestamp` | `date` | UTC timestamp of the log event. |
| `level` | `string` | `trace`, `debug`, `info`, `warn`, `error`. |
| `eventType` | `string` | Machine-friendly event identifier (e.g. `step.started`). |
| `message` | `string` | Human-readable summary. |
| `stepId` | `string` | Optional step identifier. |
| `metadata` | `object` | Deterministic key/value payload (string-only values). |
**Indexes**
1. `{ runId: 1, sequence: 1 }` (unique) — guarantees ordered retrieval and enforces idempotence.
2. `{ runId: 1, timestamp: 1 }` — accelerates replay and time-window queries.
3. `{ timestamp: 1 }` — optional TTL (disabled by default) for retention policies.
### `pack_artifacts`
| Field | Type | Notes |
|--------------|------------|-------------------------------------------------------------|
| `_id` | `ObjectId` | Generated per artifact record. |
| `runId` | `string` | Foreign key to `pack_runs._id`. |
| `name` | `string` | Output name from the Task Pack manifest. |
| `type` | `string` | `file`, `object`, or other future evidence categories. |
| `sourcePath` | `string` | Local path captured during execution (nullable). |
| `storedPath` | `string` | Object store path or bundle-relative URI (nullable). |
| `status` | `string` | `pending`, `copied`, `materialized`, `skipped`. |
| `notes` | `string` | Free-form notes (deterministic messages only). |
| `capturedAt` | `date` | UTC timestamp recorded by the worker. |
**Indexes**
1. `{ runId: 1, name: 1 }` (unique) — ensures a run emits at most one record per output.
2. `{ runId: 1 }` — supports artifact listing alongside run inspection.
## Execution Order
1. Create collections with `validator` envelopes mirroring the field expectations above (if MongoDB schema validation is enabled in the environment).
2. Apply the indexes in the order listed — unique indexes first to surface data issues early.
3. Backfill existing filesystem-backed runs by importing the serialized state/log/artifact manifests into the new collections. A dedicated importer script (`tools/taskrunner/import-filesystem-state.ps1`) accompanies the migration.
4. Switch the Task Runner service configuration to point at the Mongo-backed stores (`TaskRunner:Storage:Mode = "Mongo"`), then redeploy workers and web service.
## Rollback
To revert, switch the Task Runner configuration back to the filesystem provider and stop the Mongo migration runner. Collections can remain in place; they are append-only and harmless when unused.
## Configuration Reference
Enable the Mongo-backed stores by updating the worker and web service configuration (Compose/Helm values or `appsettings*.json`):
```json
"TaskRunner": {
"Storage": {
"Mode": "mongo",
"Mongo": {
"ConnectionString": "mongodb://127.0.0.1:27017/taskrunner",
"Database": "taskrunner",
"RunsCollection": "pack_runs",
"LogsCollection": "pack_run_logs",
"ArtifactsCollection": "pack_artifacts",
"ApprovalsCollection": "pack_run_approvals"
}
}
}
```
The worker uses the mirrored structure under the `Worker` section. Omit the `Database` property to fall back to the name embedded in the connection string.

2
docs/modules/ui/console-architecture.md
View File

@@ -18,7 +18,7 @@ This dossier describes the end-to-end architecture of the StellaOps Console as d
- Stream live status (ingestion deltas, scheduler progress, telemetry) via SSE with graceful degradation to polling when offline or throttled.
- Maintain CLI parity by embedding `stella` commands alongside interactive actions.
Non-goals: authoring ingestion logic, mutating Policy overlays, exposing internal Mongo collections, or performing cryptographic signing in-browser.
Non-goals: authoring ingestion logic, mutating Policy overlays, exposing internal database tables, or performing cryptographic signing in-browser.
---

6
docs/modules/vex-lens/README.md
View File

@@ -9,7 +9,7 @@ VEX Lens produces a deterministic, provenance-rich consensus view of VEX stateme
- Provide simulation/export APIs and Offline Kit bundles so tenants can rehearse policy changes and mirror consensus data in air-gapped environments.
## Architecture snapshot (Sprint 30 groundwork)
- **StellaOps.VexLens service & workers** — orchestrate normalisation, trust weighting, lattice join, and persistence into `vex_consensus`, `vex_consensus_history`, and `vex_conflict_queue` collections.
- **StellaOps.VexLens service & workers** — orchestrate normalisation, trust weighting, lattice join, and persistence into `vex_consensus`, `vex_consensus_history`, and `vex_conflict_queue` tables (PostgreSQL).
- **Issuer Directory integration** — maintains publisher metadata, keys, and trust tiers that feed weighting engines and revocation workflows.
- **Consensus APIs** — `/v1/vex/consensus`, `/v1/vex/conflicts`, `/v1/vex/trust/weights`, and export streams with DSSE manifests for Offline Kit + Export Center.
- **Explainability traces** — capture derived-from chains, conflicting issuers, and trust deltas to power UI drilldowns and CLI audits.
@@ -39,7 +39,7 @@ VEX Lens produces a deterministic, provenance-rich consensus view of VEX stateme
- **Notify / Task Runner** receive conflict and override events for operator actions once notification bridges ship.
## Data & observability
- Collections: `vex_consensus`, `vex_consensus_history`, `vex_conflict_queue`, plus issuer registry tables managed with tenant isolation and deterministic indexes.
- PostgreSQL tables: `vex_consensus`, `vex_consensus_history`, `vex_conflict_queue`, plus issuer registry tables managed with tenant isolation and deterministic indexes.
- Metrics: `vex_consensus_conflicts_total`, `vex_consensus_latency_seconds`, `vex_consensus_recompute_seconds{reason}`, signature failure counters.
- Traces/logs: `consensus.group`, `consensus.join`, `consensus.persist` spans with correlation IDs and issuer details; structured logs capture trust adjustments and reconciliation outcomes.
- Offline bundles include `consensus.jsonl`, `conflicts.jsonl`, manifest + DSSE signatures, enabling mirror deployments and replay validation.
@@ -47,7 +47,9 @@ VEX Lens produces a deterministic, provenance-rich consensus view of VEX stateme
## Key docs & references
- [`architecture.md`](architecture.md) — implementation-ready blueprint covering inputs, algorithm, APIs, storage, observability, and exports.
- [`implementation_plan.md`](implementation_plan.md) — phased delivery roadmap and acceptance criteria.
- [`scoring.md`](scoring.md) — future risk scoring model and formula reference.
- [`../../vex/aggregation.md`](../../vex/aggregation.md) — Aggregation-Only Contract boundaries for VEX ingestion and downstream consumers.
- **Operations:** [`operations/deployment.md`](operations/deployment.md), [`operations/offline-kit.md`](operations/offline-kit.md) — deployment guides and offline bundle preparation.
- Sprint tracking in `docs/implplan/SPRINT_0332_0001_0001_docs_modules_vex_lens.md`; module engineering tasks in `src/VexLens/StellaOps.VexLens/TASKS.md`; doc TASKS mirror in `docs/modules/vex-lens/TASKS.md`.
## Epic alignment

0
docs/modules/vexlens/operations/deployment.md → docs/modules/vex-lens/operations/deployment.md
View File

6
docs/modules/vexlens/operations/offline-kit.md → docs/modules/vex-lens/operations/offline-kit.md
View File

@@ -192,10 +192,10 @@ vexlens:
storage:
# Local storage only
mongodb:
connectionString: mongodb://localhost:27017
postgresql:
connectionString: Host=localhost;Database=stellaops;Username=stellaops;Password=password
# No external cache
redis:
valkey:
enabled: false
time:

0
docs/modules/excitor/scoring.md → docs/modules/vex-lens/scoring.md
View File

319
docs/modules/vexlens/architecture.md
View File

@@ -1,319 +0,0 @@
# component_architecture_vexlens.md — **Stella Ops VexLens** (2025Q4)
> Supports deliverables from Epic 30 – VEX Consensus Engine and Epic 31 – Advisory AI Integration.
> **Scope.** Implementation-ready architecture for **VexLens**: the consensus engine for computing authoritative VEX (Vulnerability Exploitability eXchange) status from multiple overlapping statements. It supports trust-weighted voting, lattice-based conflict resolution, and provides policy integration for vulnerability decisioning.
---
## 0) Mission & Boundaries
**Mission.** Compute deterministic VEX consensus status from multiple sources with full audit trail, enabling automated vulnerability triage based on exploitability data.
**Boundaries.**
* **VexLens does not fetch VEX documents** — it receives normalized statements from Excititor or direct API input.
* **VexLens does not store raw VEX documents** — it stores computed projections and consensus results.
* **VexLens does not make policy decisions** — it provides VEX status to Policy Engine for final determination.
---
## 1) Responsibilities (contract)
1. **Normalize** VEX documents from OpenVEX, CSAF VEX, CycloneDX VEX, and SPDX VEX formats.
2. **Map products** using PURL and CPE identifiers with configurable matching strictness.
3. **Verify signatures** on VEX documents (DSSE, JWS, PGP, PKCS#7).
4. **Compute trust weights** based on issuer authority, signature status, freshness, and other factors.
5. **Compute consensus** using configurable modes:
- **HighestWeight**: Single highest-weighted statement wins
- **WeightedVote**: Weighted voting among all statements
- **Lattice**: Most conservative status wins (affected > under_investigation > not_affected > fixed)
- **AuthoritativeFirst**: Authoritative sources override others
- **MostRecent**: Most recent statement wins
6. **Store projections** for historical tracking and audit.
7. **Emit events** on consensus computation, status changes, and conflict detection.
8. **Integrate** with Policy Engine for vulnerability suppression and severity adjustment.
---
## 2) External Dependencies
* **Excititor**: Provides normalized VEX statements from connectors.
* **Policy Engine**: Consumes VEX consensus for vulnerability decisioning.
* **Vuln Explorer**: Enriches vulnerability data with VEX status.
* **Orchestrator**: Schedules consensus compute jobs for batch processing.
* **Authority**: Validates issuer trust and key fingerprints.
* **Config stores**: PostgreSQL (projections, issuer directory), Redis (caches).
---
## 3) API Surface
Base path: `/api/v1/vexlens`. Full OpenAPI spec at `docs/api/vexlens-openapi.yaml`.
### 3.1 Consensus Operations
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/consensus` | POST | Compute consensus for a vulnerability-product pair |
| `/consensus/batch` | POST | Compute consensus for multiple pairs in batch |
### 3.2 Projection Queries
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/projections` | GET | Query consensus projections with filtering |
| `/projections/{projectionId}` | GET | Get a projection by ID |
| `/projections/latest` | GET | Get latest projection for a vuln-product pair |
| `/projections/history` | GET | Get projection history |
### 3.3 Issuer Directory
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/issuers` | GET | List registered issuers |
| `/issuers` | POST | Register a new issuer |
| `/issuers/{issuerId}` | GET | Get issuer details |
| `/issuers/{issuerId}` | DELETE | Revoke an issuer |
| `/issuers/{issuerId}/keys` | POST | Add a key to an issuer |
| `/issuers/{issuerId}/keys/{fingerprint}` | DELETE | Revoke a key |
### 3.4 Statistics
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/statistics` | GET | Get consensus statistics |
---
## 4) Data Flow
```
┌─────────────┐ ┌──────────────┐ ┌─────────────────┐
│ Excititor │────▶│ Normalizer │────▶│ Trust Weighting │
│ (VEX Docs) │ │ (OpenVEX, │ │ (9 factors) │
└─────────────┘ │ CSAF, CDX) │ └────────┬────────┘
└──────────────┘ │
▼
┌─────────────┐ ┌──────────────┐ ┌─────────────────┐
│ Policy │◀────│ Projection │◀────│ Consensus │
│ Engine │ │ Store │ │ Engine │
└─────────────┘ └──────────────┘ └─────────────────┘
│
▼
┌──────────────┐
│ Events │
│ (Computed, │
│ StatusChange,│
│ Conflict) │
└──────────────┘
```
---
## 5) VEX Status Lattice
VexLens uses a status lattice for conservative conflict resolution:
```
affected (most restrictive)
│
▼
under_investigation
│
▼
not_affected
│
▼
fixed (least restrictive)
```
In lattice mode, the most restrictive status always wins. This ensures that when sources disagree, the system errs on the side of caution.
---
## 6) Trust Weight Factors
| Factor | Weight | Description |
|--------|--------|-------------|
| IssuerBase | 25% | Base trust from issuer directory |
| SignatureStatus | 15% | Valid/invalid/unsigned signature |
| Freshness | 15% | Document age with exponential decay |
| IssuerCategory | 10% | Vendor > Distributor > Aggregator |
| IssuerTier | 10% | Authoritative > Trusted > Untrusted |
| StatusQuality | 10% | Has justification, specific status |
| TransparencyLog | 5% | Sigstore Rekor entry |
| SourceMatch | 5% | Source URI pattern match |
| ProductAuthority | 5% | Issuer is authoritative for product |
---
## 7) Configuration
```yaml
vexlens:
consensus:
defaultMode: WeightedVote # HighestWeight, WeightedVote, Lattice, AuthoritativeFirst, MostRecent
minimumConfidence: 0.1
conflictThreshold: 0.3
requireJustificationForNotAffected: false
trust:
freshnessHalfLifeDays: 90
minimumFreshness: 0.3
allowUnsigned: true
unsignedPenalty: 0.3
allowUnknownIssuers: true
unknownIssuerPenalty: 0.5
storage:
projectionRetentionDays: 365
eventRetentionDays: 90
issuerDirectory:
source: postgresql # postgresql, file, api
refreshIntervalMinutes: 60
```
---
## 8) Storage Schema
### 8.1 Consensus Projection
```json
{
"projectionId": "proj-abc123",
"vulnerabilityId": "CVE-2024-1234",
"productKey": "pkg:npm/lodash@4.17.21",
"tenantId": "tenant-001",
"status": "not_affected",
"justification": "vulnerable_code_not_present",
"confidenceScore": 0.95,
"outcome": "Unanimous",
"statementCount": 3,
"conflictCount": 0,
"rationaleSummary": "Unanimous consensus from 3 authoritative sources",
"computedAt": "2025-12-06T12:00:00Z",
"storedAt": "2025-12-06T12:00:01Z",
"previousProjectionId": null,
"statusChanged": true
}
```
### 8.2 Issuer Record
```json
{
"issuerId": "npm-security",
"name": "npm Security Team",
"category": "Vendor",
"trustTier": "Authoritative",
"status": "Active",
"keyFingerprints": [
{
"fingerprint": "ABCD1234EFGH5678",
"keyType": "Pgp",
"algorithm": "EdDSA",
"status": "Active",
"registeredAt": "2025-01-01T00:00:00Z",
"expiresAt": null
}
],
"metadata": {
"description": "Official npm security advisories",
"uri": "https://www.npmjs.com/advisories",
"email": "security@npmjs.com"
},
"registeredAt": "2025-01-01T00:00:00Z"
}
```
---
## 9) Events
### 9.1 ConsensusComputedEvent
Emitted after every consensus computation.
```json
{
"eventId": "evt-abc123",
"projectionId": "proj-abc123",
"vulnerabilityId": "CVE-2024-1234",
"productKey": "pkg:npm/lodash@4.17.21",
"status": "not_affected",
"confidenceScore": 0.95,
"outcome": "Unanimous",
"statementCount": 3,
"computedAt": "2025-12-06T12:00:00Z",
"emittedAt": "2025-12-06T12:00:01Z"
}
```
### 9.2 ConsensusStatusChangedEvent
Emitted when consensus status changes from previous projection.
### 9.3 ConsensusConflictDetectedEvent
Emitted when conflicts are detected during consensus computation.
---
## 10) Observability
### 10.1 Metrics (OpenTelemetry)
| Metric | Type | Description |
|--------|------|-------------|
| `vexlens.consensus.computed_total` | Counter | Total consensus computations |
| `vexlens.consensus.conflicts_total` | Counter | Total conflicts detected |
| `vexlens.consensus.confidence` | Histogram | Confidence score distribution |
| `vexlens.consensus.duration_seconds` | Histogram | Computation duration |
| `vexlens.consensus.status_changes_total` | Counter | Status changes detected |
| `vexlens.normalization.documents_total` | Counter | Documents normalized |
| `vexlens.trust.weight_value` | Histogram | Trust weight distribution |
| `vexlens.issuer.registered_total` | Counter | Issuers registered |
### 10.2 Traces
Activity source: `StellaOps.VexLens`
| Activity | Description |
|----------|-------------|
| `vexlens.normalize` | VEX document normalization |
| `vexlens.compute_trust_weight` | Trust weight computation |
| `vexlens.compute_consensus` | Consensus computation |
| `vexlens.store_projection` | Projection storage |
| `vexlens.query_projections` | Projection query |
### 10.3 Logging
Structured logging with event IDs in `VexLensLogEvents`:
- 1xxx: Normalization events
- 2xxx: Product mapping events
- 3xxx: Signature verification events
- 4xxx: Trust weight events
- 5xxx: Consensus events
- 6xxx: Projection events
- 7xxx: Issuer directory events
---
## 11) Security Considerations
1. **Issuer Trust**: All issuers must be registered with verified key fingerprints.
2. **Signature Verification**: Documents should be cryptographically signed for production use.
3. **Tenant Isolation**: Projections are scoped to tenants; no cross-tenant data access.
4. **Audit Trail**: All consensus computations are logged with full rationale.
5. **Determinism**: All computations are deterministic for reproducibility.
---
## 12) Test Matrix
| Test Category | Coverage | Notes |
|---------------|----------|-------|
| Unit tests | Normalizer, Parser, Trust, Consensus | 89+ tests |
| Determinism harness | Normalization, Trust, Consensus | Verify reproducibility |
| Integration tests | API service, Storage, Events | End-to-end flows |
| Property-based tests | Lattice semantics, Weight computation | Invariant verification |

2
docs/modules/vuln-explorer/architecture.md
View File

@@ -60,7 +60,7 @@ CLI mirrors these endpoints (`stella findings list|view|update|export`). Console
## 6) Identity & access integration
- **Scopes** – `vuln:view`, `vuln:investigate`, `vuln:operate`, `vuln:audit` map to read-only, triage, workflow, and audit experiences respectively. The deprecated `vuln:read` scope is still honoured for legacy tokens but is no longer advertised.
- **Attribute filters (ABAC)** – Authority enforces per-service-account filters via the client-credential parameters `vuln_env`, `vuln_owner`, and `vuln_business_tier`. Service accounts define the allowed values in `authority.yaml` (`attributes` block). Tokens include the resolved filters as claims (`stellaops:vuln_env`, `stellaops:vuln_owner`, `stellaops:vuln_business_tier`), and tokens persisted to Mongo retain the same values for audit and revocation.
- **Attribute filters (ABAC)** – Authority enforces per-service-account filters via the client-credential parameters `vuln_env`, `vuln_owner`, and `vuln_business_tier`. Service accounts define the allowed values in `authority.yaml` (`attributes` block). Tokens include the resolved filters as claims (`stellaops:vuln_env`, `stellaops:vuln_owner`, `stellaops:vuln_business_tier`), and tokens persisted to PostgreSQL retain the same values for audit and revocation.
- **Audit trail** – Every token issuance emits `authority.vuln_attr.*` audit properties that mirror the resolved filter set, along with `delegation.service_account` and ordered `delegation.actor[n]` entries so Vuln Explorer can correlate access decisions.
- **Permalinks** – Signed permalinks inherit the caller’s ABAC filters; consuming services must enforce the embedded claims in addition to scope checks when resolving permalinks.
- **Attachment tokens** – Authority mints short-lived tokens (`POST /vuln/attachments/tokens/issue`) to gate evidence downloads. Verification (`POST /vuln/attachments/tokens/verify`) enforces tenant, scope, and ABAC metadata, and emits `vuln.attachment.token.*` audit records.

2
docs/modules/vuln-explorer/runbooks/observability.md
View File

@@ -20,7 +20,7 @@
- Trace exemplar anchors: `traceparent` headers are copied into logs; exporters stay disabled by default for air-gap. Enable by setting `Telemetry:ExportEnabled=true` and pointing to on-prem Tempo/Jaeger.
## Health/diagnostics
- `/health/liveness` and `/health/readiness` (HTTP 200 expected; readiness checks Mongo + cache reachability).
- `/health/liveness` and `/health/readiness` (HTTP 200 expected; readiness checks PostgreSQL + cache reachability).
- `/status` returns build version, git commit, and enabled features; safe for anonymous fetch in sealed environments.
- Ledger replay check: `GET /v1/findings?projectionMode=verify` emits `X-Vuln-Projection-Head` for quick consistency probes.

6
docs/notifications/digests.md
View File

@@ -17,11 +17,11 @@ Digests coalesce multiple matching events into a single notification when rules
## 2. Storage model
Digest state lives in Mongo (`digests` collection) and mirrors the schema described in [modules/notify/architecture.md](../modules/notify/architecture.md#7-data-model-mongo):
Digest state lives in PostgreSQL (`notify.digests` table) and mirrors the schema described in [modules/notify/architecture.md](../modules/notify/architecture.md#7-data-model):
```json
{
"_id": "tenant-dev:act-email-compliance:1h",
"id": "tenant-dev:act-email-compliance:1h",
"tenantId": "tenant-dev",
"actionKey": "act-email-compliance",
"window": "1h",
@@ -76,7 +76,7 @@ All routes honour the tenant header and reuse the standard Notify rate limits.
- **Throttles.** Digest generation respects action throttles; setting an aggressive throttle together with a digest window may result in deliberate skips (logged as `Throttled` in the delivery ledger).
- **Quiet hours.** Future sprint work (`NOTIFY-SVC-39-004`) integrates quiet-hour calendars. When enabled, flush timers pause during quiet windows and resume afterwards.
- **Back-pressure.** When the window reaches the configured item cap before the timer, the worker flushes early and starts a new window immediately.
- **Crash resilience.** Workers rebuild in-flight windows from Mongo on startup; partially flushed windows remain closed after success or reopened if the flush fails.
- **Crash resilience.** Workers rebuild in-flight windows from PostgreSQL on startup; partially flushed windows remain closed after success or reopened if the flush fails.
---

6
docs/notifications/overview.md
View File

@@ -23,7 +23,7 @@ Notifications Studio turns raw platform events into concise, tenant-scoped alert
| Channel catalog | Slack, Teams, Email, Webhook connectors loaded via restart-time plug-ins; metadata stored without secrets. | [`notifications/architecture.md`](architecture.md) |
| Templates | Locale-aware, deterministic rendering via safe helpers; channel defaults plus tenant-specific overrides, including the attestation lifecycle suite (`tmpl-attest-*`). | [`notifications/templates.md`](templates.md#7-attestation--signing-lifecycle-templates-notify-attest-74-001) |
| Digests | Coalesce bursts into periodic summaries with deterministic IDs and audit trails. | [`notifications/digests.md`](digests.md) |
| Delivery ledger | Tracks rendered payload hashes, attempts, throttles, and outcomes for every action. | [`modules/notify/architecture.md`](../modules/notify/architecture.md#7-data-model-mongo) |
| Delivery ledger | Tracks rendered payload hashes, attempts, throttles, and outcomes for every action. | [`modules/notify/architecture.md`](../modules/notify/architecture.md#7-data-model) |
| Ack tokens | DSSE-signed acknowledgement tokens with webhook allowlists and escalation guardrails enforced by Authority. | [`modules/notify/architecture.md`](../modules/notify/architecture.md#81-ack-tokens--escalation-workflows) |
---
@@ -45,7 +45,7 @@ The Notify WebService fronts worker state with REST APIs used by the UI and CLI.
|------|----------|
| **Tenancy** | Each rule, channel, template, and delivery belongs to exactly one tenant. Cross-tenant sharing is intentionally unsupported. |
| **Determinism** | Configuration persistence normalises strings and sorts collections. Template rendering produces identical `bodyHash` values when inputs match; attestation events always reference the canonical `tmpl-attest-*` keys documented in the template guide. |
| **Scaling** | Workers scale horizontally; per-tenant rule snapshots are cached and refreshed from Mongo change streams. Redis (or equivalent) guards throttles and locks. |
| **Scaling** | Workers scale horizontally; per-tenant rule snapshots are cached and refreshed from PostgreSQL change notifications. Valkey (or Redis-compatible) guards throttles and locks. |
| **Offline** | Offline Kits include plug-ins, default templates, and seed rules. Operators can edit YAML/JSON manifests before air-gapped deployment. |
| **Security** | Channel secrets use indirection (`secretRef`), Authority-protected OAuth clients secure API access, and delivery payloads are redacted before storage where required. |
| **Module boundaries** | 2025-11-02 decision: keep `src/Notify/` as the shared notification toolkit and `src/Notifier/` as the Notifications Studio runtime host until a packaging RFC covers the implications of merging. |
@@ -56,7 +56,7 @@ The Notify WebService fronts worker state with REST APIs used by the UI and CLI.
| Step | Goal | Reference |
|------|------|-----------|
| 1 | Deploy Notify WebService + Worker with Mongo and Redis | [`modules/notify/architecture.md`](../modules/notify/architecture.md#1-runtime-shape--projects) |
| 1 | Deploy Notify WebService + Worker with PostgreSQL and Valkey | [`modules/notify/architecture.md`](../modules/notify/architecture.md#1-runtime-shape--projects) |
| 2 | Register OAuth clients/scopes in Authority | [`etc/authority.yaml.sample`](../../etc/authority.yaml.sample) |
| 3 | Install channel plug-ins and capture secret references | [`plugins/notify`](../../plugins) |
| 4 | Create a tenant rule and test preview | [`POST /channels/{id}/test`](../modules/notify/architecture.md#8-external-apis-webservice) |

413
docs/product-advisories/24-Dec-2025 - Deterministic Resolver Architecture.md Normal file
View File

@@ -0,0 +1,413 @@
# Product Advisory: Deterministic Resolver Architecture
**Date:** 2025-12-24
**Author:** Architecture Guild
**Status:** Approved for Implementation
**Sprint Epoch:** 9100
**Priority:** P0 (Critical Path for Audit Compliance)
---
## Executive Summary
This advisory defines the architecture for a **unified deterministic resolver** that evaluates vulnerability graphs using only declared evidence edges and produces cryptographically verifiable, reproducible verdicts. The resolver guarantees: **same inputs → same traversal order → same per-node inputs → same verdicts → same output digest**.
This is a foundational capability for:
- Auditor-friendly verification ("same inputs → same verdicts")
- Offline replay by vendors/distributors
- Single-digest comparison across runs
- CI/CD gate assertions
---
## Problem Statement
Current implementation has strong building blocks but lacks unified orchestration:
| Component | Location | Gap |
|-----------|----------|-----|
| Topological Sort | `DeterministicGraphOrderer` | Traversal only; no verdict output |
| Lattice Evaluation | `TrustLatticeEngine`, `PolicyEvaluator` | No traversal sequence in output |
| Content Addressing | `ContentAddressedIdGenerator` | Node IDs only; no edge IDs |
| Purity Analysis | `ProhibitedPatternAnalyzer` | Static only; no runtime enforcement |
| Digest Computation | Various | No unified "run digest" |
**Result:** Cannot produce a single `ResolutionResult` that captures:
1. The exact traversal sequence used
2. Per-node verdicts with individual digests
3. A composite `FinalDigest` for verification
---
## Solution Architecture
### Core Data Model
```csharp
// New: src/__Libraries/StellaOps.Resolver/
/// <summary>
/// Content-addressed node identifier.
/// Format: sha256(kind || ":" || normalized-key)
/// </summary>
public sealed record NodeId(string Value) : IComparable<NodeId>
{
public int CompareTo(NodeId? other) =>
string.Compare(Value, other?.Value, StringComparison.Ordinal);
public static NodeId From(string kind, string normalizedKey) =>
new(HexSha256($"{kind}:{normalizedKey}"));
}
/// <summary>
/// Content-addressed edge identifier.
/// Format: sha256(srcID || "->" || edgeKind || "->" || dstID)
/// </summary>
public sealed record EdgeId(string Value) : IComparable<EdgeId>
{
public static EdgeId From(NodeId src, string kind, NodeId dst) =>
new(HexSha256($"{src.Value}->{kind}->{dst.Value}"));
}
/// <summary>
/// Graph edge with content-addressed identity and optional cycle-cut marker.
/// </summary>
public sealed record Edge(
EdgeId Id,
NodeId Src,
string Kind,
NodeId Dst,
JsonElement Attrs,
bool IsCycleCut = false);
/// <summary>
/// Graph node with content-addressed identity.
/// </summary>
public sealed record Node(
NodeId Id,
string Kind,
JsonElement Attrs);
/// <summary>
/// Immutable policy with content-addressed identity.
/// </summary>
public sealed record Policy(
string Version,
JsonElement Rules,
string ConstantsDigest);
/// <summary>
/// Individual verdict with its own content-addressed digest.
/// </summary>
public sealed record Verdict(
NodeId Node,
string Status,
JsonElement Evidence,
string VerdictDigest);
/// <summary>
/// Complete resolution result with all digests for verification.
/// </summary>
public sealed record ResolutionResult(
ImmutableArray<NodeId> TraversalSequence,
ImmutableArray<Verdict> Verdicts,
string GraphDigest,
string PolicyDigest,
string FinalDigest);
```
### Resolver Algorithm
```text
normalize(graph); // IDs, fields, ordering
validate(graph); // No implicit data, cycles declared
assert no-ambient-inputs(); // Runtime purity check
order = topoSortWithLexTieBreak(graph); // Uses only evidence edges
verdicts = []
for node in order:
inbound = gatherInboundEvidence(node) // Pure, no IO
verdict = Evaluate(node, inbound, policy) // Pure, no IO
verdicts.append(canonicalize(verdict) with VerdictDigest)
result = {
traversal: order,
verdicts: verdicts,
graphDigest: digest(canonical(graph)),
policyDigest: digest(canonical(policy))
}
result.finalDigest = digest(canonical(result))
return result
```
### Key Invariants
1. **Canonical IDs**: `NodeId = sha256(kind:normalized-key)`, `EdgeId = sha256(src->kind->dst)`
2. **Canonical Serialization**: Alphabetical keys, sorted arrays, fixed numerics, ISO-8601 Z timestamps
3. **Fixed Traversal**: Kahn's algorithm with lexicographic tie-breaker (`SortedSet<NodeId>`)
4. **Explicit Cycles**: Cycles require `IsCycleCut = true` edge; unmarked cycles → invalid graph
5. **Evidence-Only Evaluation**: Node verdict = f(node, inbound_edges, policy) — no ambient IO
6. **Stable Outputs**: `(TraversalSequence, Verdicts[], GraphDigest, PolicyDigest, FinalDigest)`
---
## Gap Analysis vs Current Implementation
### Already Implemented (No Work Needed)
| Requirement | Implementation | Location |
|-------------|---------------|----------|
| Canonical JSON | RFC 8785 compliant | `CanonicalJsonSerializer`, `Rfc8785JsonCanonicalizer` |
| Alphabetical key sorting | `StableDictionaryConverter` | `StellaOps.Canonicalization` |
| ISO-8601 UTC timestamps | `Iso8601DateTimeConverter` | `StellaOps.Canonicalization` |
| Topological sort with lex tie-break | `SortedSet<string>` | `DeterministicGraphOrderer:134` |
| K4 Four-Valued Lattice | Complete | `K4Lattice.cs` |
| VEX Lattice Merging | With traces | `OpenVexStatementMerger` |
| Merkle Tree Proofs | SHA256-based | `DeterministicMerkleTreeBuilder` |
| Static Purity Analysis | 16+ patterns | `ProhibitedPatternAnalyzer` |
| Injected Timestamp | Context-based | `PolicyEvaluationContext.Now` |
| Graph Content Hash | SHA256 | `DeterministicGraphOrderer.ComputeCanonicalHash()` |
### Gaps to Implement
| Gap | Priority | Sprint | Description |
|-----|----------|--------|-------------|
| Unified Resolver | P0 | 9100.0001.0001 | Single entry point producing `ResolutionResult` |
| Cycle-Cut Edges | P1 | 9100.0001.0002 | `IsCycleCut` edge property; validation |
| EdgeId | P2 | 9100.0001.0003 | Content-addressed edge identifiers |
| FinalDigest | P1 | 9100.0002.0001 | Composite run-level digest |
| Per-Node VerdictDigest | P2 | 9100.0002.0002 | Individual verdict digests |
| Runtime Purity | P1 | 9100.0003.0001 | Runtime enforcement beyond static analysis |
| Graph Validation + NFC | P3 | 9100.0003.0002 | Pre-traversal validation; NFC normalization |
---
## Implementation Phases
### Phase 1: Core Resolver Package (Sprint 9100.0001.*)
**Goal:** Create `StellaOps.Resolver` library with unified resolver pattern.
1. **Sprint 9100.0001.0001** — Core Resolver
- `DeterministicResolver` class
- `ResolutionResult` record
- Integration with existing `DeterministicGraphOrderer`
- Integration with existing `TrustLatticeEngine`
2. **Sprint 9100.0001.0002** — Cycle-Cut Edges
- `IsCycleCut` property on edges
- Cycle validation (unmarked cycles → error)
- Graph validation before traversal
3. **Sprint 9100.0001.0003** — Content-Addressed EdgeId
- `EdgeId` record
- Edge content addressing
- Merkle tree inclusion of edges
### Phase 2: Digest Infrastructure (Sprint 9100.0002.*)
**Goal:** Implement comprehensive digest chain for verification.
1. **Sprint 9100.0002.0001** — FinalDigest
- Composite digest computation
- `sha256(canonical({graphDigest, policyDigest, verdicts[]}))`
- Integration with attestation system
2. **Sprint 9100.0002.0002** — Per-Node VerdictDigest
- Verdict-level content addressing
- Drill-down debugging ("which node changed?")
- Delta detection between runs
### Phase 3: Purity & Validation (Sprint 9100.0003.*)
**Goal:** Harden determinism guarantees with runtime enforcement.
1. **Sprint 9100.0003.0001** — Runtime Purity Enforcement
- Runtime guards beyond static analysis
- Dependency injection shims for ambient services
- Test harness for purity verification
2. **Sprint 9100.0003.0002** — Graph Validation & NFC
- Pre-traversal validation ("no implicit data")
- Unicode NFC normalization for string fields
- Evidence completeness assertions
---
## Testing Requirements
### Mandatory Test Types
1. **Replay Test**: Same input twice → identical `FinalDigest`
2. **Permutation Test**: Shuffle input nodes/edges → identical outputs
3. **Cycle Test**: Introduce cycle → fail unless `IsCycleCut` edge present
4. **Ambience Test**: Forbid calls to time/env/network during evaluation
5. **Serialization Test**: Canonicalization changes → predictable digest changes
### Property-Based Tests
```csharp
// Idempotency
[Property]
public void Resolver_SameInput_SameOutput(Graph graph)
{
var result1 = resolver.Run(graph);
var result2 = resolver.Run(graph);
Assert.Equal(result1.FinalDigest, result2.FinalDigest);
}
// Order Independence
[Property]
public void Resolver_ShuffledInputs_SameOutput(Graph graph, int seed)
{
var shuffled = ShuffleNodesAndEdges(graph, seed);
var result1 = resolver.Run(graph);
var result2 = resolver.Run(shuffled);
Assert.Equal(result1.FinalDigest, result2.FinalDigest);
}
// Cycle Detection
[Property]
public void Resolver_UnmarkedCycle_Throws(Graph graphWithCycle)
{
Assume.That(!graphWithCycle.Edges.Any(e => e.IsCycleCut));
Assert.Throws<InvalidGraphException>(() => resolver.Run(graphWithCycle));
}
```
---
## Integration Points
### Existing Components to Integrate
| Component | Integration |
|-----------|-------------|
| `DeterministicGraphOrderer` | Use for traversal order computation |
| `TrustLatticeEngine` | Use for K4 lattice evaluation |
| `CanonicalJsonSerializer` | Use for all serialization |
| `ContentAddressedIdGenerator` | Extend for EdgeId and VerdictId |
| `ProhibitedPatternAnalyzer` | Combine with runtime guards |
| `PolicyEvaluationContext` | Use for injected inputs |
### New Modules
| Module | Purpose |
|--------|---------|
| `StellaOps.Resolver` | Core resolver library |
| `StellaOps.Resolver.Testing` | Test fixtures and harnesses |
| `StellaOps.Resolver.Attestation` | Integration with proof chain |
---
## Success Criteria
1. **Unified API**: Single `resolver.Run(graph)` produces complete `ResolutionResult`
2. **Deterministic**: 100% pass rate on replay/permutation/cycle tests
3. **Auditable**: `FinalDigest` enables single-value verification
4. **Performant**: No more than 10% overhead vs current fragmented approach
5. **Documented**: Full API documentation and integration guide
---
## Appendix: C# Reference Implementation
```csharp
public sealed class DeterministicResolver
{
private readonly Policy _policy;
private readonly string _policyDigest;
private readonly DeterministicGraphOrderer _orderer;
private readonly TrustLatticeEngine _lattice;
private readonly CanonicalJsonSerializer _serializer;
public DeterministicResolver(
Policy policy,
DeterministicGraphOrderer orderer,
TrustLatticeEngine lattice,
CanonicalJsonSerializer serializer)
{
_policy = Canon(policy);
_policyDigest = HexSha256(serializer.SerializeToBytes(_policy));
_orderer = orderer;
_lattice = lattice;
_serializer = serializer;
}
public ResolutionResult Run(EvidenceGraph graph)
{
// 1. Canonicalize and validate
var canonical = _orderer.Canonicalize(graph);
ValidateCycles(canonical);
EnsureNoAmbientInputs();
// 2. Traverse in deterministic order
var traversal = canonical.Nodes
.Select(n => NodeId.Parse(n.Id))
.ToImmutableArray();
// 3. Evaluate each node purely
var verdicts = new List<Verdict>(traversal.Length);
foreach (var nodeId in traversal)
{
var node = GetNode(canonical, nodeId);
var inbound = GatherInboundEvidence(canonical, nodeId);
var verdict = EvaluatePure(node, inbound, _policy);
var verdictBytes = _serializer.SerializeToBytes(verdict);
verdicts.Add(verdict with { VerdictDigest = HexSha256(verdictBytes) });
}
// 4. Compute digests
var verdictsArray = verdicts.ToImmutableArray();
var resultWithoutFinal = new ResolutionResult(
traversal,
verdictsArray,
canonical.ContentHash,
_policyDigest,
"");
var finalBytes = _serializer.SerializeToBytes(resultWithoutFinal);
return resultWithoutFinal with { FinalDigest = HexSha256(finalBytes) };
}
private void ValidateCycles(CanonicalGraph graph)
{
// Detect cycles; require IsCycleCut for each
var cycles = DetectCycles(graph);
var unmarked = cycles.Where(c => !c.CutEdge.IsCycleCut).ToList();
if (unmarked.Any())
{
throw new InvalidGraphException(
$"Graph contains {unmarked.Count} unmarked cycle(s). " +
"Add IsCycleCut=true to break cycles.");
}
}
private Verdict EvaluatePure(Node node, InboundEvidence evidence, Policy policy)
{
// Pure evaluation: no IO, no ambient state
return _lattice.Evaluate(node, evidence, policy);
}
}
```
---
## Related Documents
- `docs/modules/attestor/proof-chain-specification.md`
- `docs/modules/policy/architecture.md`
- `docs/testing/testing-strategy-models.md`
- `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/K4Lattice.cs`
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Ordering/DeterministicGraphOrderer.cs`
---
## Approval
| Role | Name | Date | Decision |
|------|------|------|----------|
| Architecture Lead | — | 2025-12-24 | Approved |
| Policy Guild Lead | — | 2025-12-24 | Approved |
| Scanner Guild Lead | — | 2025-12-24 | Approved |

271
docs/product-advisories/24-Dec-2025 - Evidence-Weighted Score Model.md Normal file
View File

@@ -0,0 +1,271 @@
# Product Advisory: Evidence-Weighted Score Model
**Date:** 24-Dec-2025
**Status:** Approved for Implementation
**Sprint Batch:** 8200.0012
---
## Executive Summary
This advisory introduces a **unified evidence-weighted scoring model** that combines six evidence dimensions into a single 0-100 score per vulnerability finding. The score enables rapid triage by surfacing the most "real" risks with transparent, decomposable evidence.
### Key Benefits
- **Prioritize quickly:** Sort by score to fix the most "real" risks first
- **Reduce noise:** Thin, speculative findings sink to the bottom
- **Stay explainable:** Score decomposes into evidence "slices" on hover
---
## Problem Statement
The current platform has **four independent scoring systems** that don't interoperate:
1. `ConfidenceCalculator` (Policy module) — 5 factors, outputs 0-1
2. `ScorePolicy` (Policy module) — 4 factors in basis points
3. `EvidenceDensityScorer` (Scanner module) — 8 factors, outputs 0-1
4. `TrustVector` (Excititor module) — 3 factors, outputs 0-1
Users cannot sort findings by a single "risk strength" metric. They must mentally combine multiple signals, leading to:
- Inconsistent prioritization across teams
- Missed high-risk findings buried in noise
- Difficulty explaining triage decisions to stakeholders
---
## Proposed Solution
### Scoring Model
```
Score = clamp01(
W_rch*RCH + W_rts*RTS + W_bkp*BKP + W_xpl*XPL + W_src*SRC - W_mit*MIT
) * 100
```
### Evidence Dimensions
| Symbol | Name | Description | Weight |
|--------|------|-------------|--------|
| **RCH** | Reachability | Static/dynamic reachability confidence | 0.30 |
| **RTS** | Runtime | Runtime signal strength (eBPF, hits, recency) | 0.25 |
| **BKP** | Backport | Backport/patch evidence strength | 0.15 |
| **XPL** | Exploit | Exploit likelihood (EPSS + KEV) | 0.15 |
| **SRC** | Source Trust | Source trust (vendor > distro > community) | 0.10 |
| **MIT** | Mitigations | Active mitigations (subtractive) | 0.10 |
### Guardrails
| Condition | Action | Rationale |
|-----------|--------|-----------|
| BKP=1 + not_affected + RTS<0.6 | Cap at 15 | Vendor confirms not affected |
| RTS >= 0.8 | Floor at 60 | Strong live signal |
| RCH=0 + RTS=0 | Cap at 45 | Speculative finding |
### Score Buckets
| Bucket | Range | Action |
|--------|-------|--------|
| **ActNow** | 90-100 | Immediate action required |
| **ScheduleNext** | 70-89 | Schedule for next sprint |
| **Investigate** | 40-69 | Investigate when touching component |
| **Watchlist** | 0-39 | Low priority, monitor |
---
## Gap Analysis
### Existing Infrastructure (Reusable)
| Component | Location | Reuse |
|-----------|----------|-------|
| Reachability states | `Policy/ConfidenceCalculator.cs` | Direct mapping to RCH |
| Runtime evidence | `Policy/RuntimeEvidence` | Direct mapping to RTS |
| Backport detection | `Concelier/BackportProofService.cs` | Needs normalization |
| EPSS bands | `Scanner/EpssPriorityCalculator.cs` | Needs normalization |
| KEV status | `Concelier/VendorRiskSignalExtractor.cs` | Direct flag |
| Trust vector | `Excititor/TrustVector.cs` | Direct mapping to SRC |
| Gate multipliers | `Policy/GateMultipliersBps` | Needs inversion to MIT |
### New Components Required
1. **Unified Score Calculator** — Core formula implementation
2. **Normalizers** — Convert raw evidence to 0-1 inputs
3. **Guardrails Engine** — Apply caps/floors
4. **API Endpoints** — Expose scores via REST
5. **UI Components** — Score pill, breakdown popover, badges
---
## Implementation Sprints
| Sprint | Focus | Key Deliverables |
|--------|-------|------------------|
| **8200.0012.0001** | Core Library | Calculator, input models, weight policy, guardrails |
| **8200.0012.0002** | Normalizers | BKP, XPL, MIT, RCH, RTS, SRC normalizers |
| **8200.0012.0003** | Policy Integration | Score-based rules, verdict enrichment, attestation |
| **8200.0012.0004** | API Endpoints | Single/batch score, history, webhooks |
| **8200.0012.0005** | Frontend UI | Score pill, breakdown, badges, history chart |
**Total Tasks:** 268 across 5 sprints
**Estimated Duration:** 6 months
---
## API Shape
### Score Response
```json
{
"findingId": "CVE-2024-1234@pkg:deb/debian/curl@7.64.0-4",
"score": 78,
"bucket": "ScheduleNext",
"inputs": {
"rch": 0.85, "rts": 0.40, "bkp": 0.00,
"xpl": 0.70, "src": 0.80, "mit": 0.10
},
"weights": {
"rch": 0.30, "rts": 0.25, "bkp": 0.15,
"xpl": 0.15, "src": 0.10, "mit": 0.10
},
"flags": ["live-signal", "proven-path"],
"explanations": [
"Static path to vulnerable sink (confidence: 85%)",
"EPSS: 0.8% probability (High band)",
"Distro VEX signed (trust: 80%)"
],
"caps": {
"speculativeCap": false,
"notAffectedCap": false,
"runtimeFloor": false
},
"policyDigest": "sha256:abc123...",
"calculatedAt": "2026-01-15T14:30:00Z"
}
```
---
## UI Design
### Score Pill
```
┌───────┐
│ 78 │ ← White text on amber background (ScheduleNext bucket)
└───────┘
```
### Breakdown Popover
```
┌─────────────────────────────────────────┐
│ Evidence Score: 78/100 │
│ Bucket: Schedule Next Sprint │
├─────────────────────────────────────────┤
│ Reachability ████████▒▒ 0.85 │
│ Runtime ████▒▒▒▒▒▒ 0.40 │
│ Backport ▒▒▒▒▒▒▒▒▒▒ 0.00 │
│ Exploit ███████▒▒▒ 0.70 │
│ Source Trust ████████▒▒ 0.80 │
│ Mitigations -█▒▒▒▒▒▒▒▒▒ 0.10 │
├─────────────────────────────────────────┤
│ 🟢 Live signal detected │
│ ✓ Proven reachability path │
└─────────────────────────────────────────┘
```
---
## Governance & Tuning
1. **Policy-Driven Weights:** Different environments (prod/staging/dev) can have different weight profiles
2. **Versioned Policies:** Policy changes are versioned and signed; same inputs + policy = same score
3. **Audit Trail:** All score calculations logged with inputs, policy digest, and result
4. **Threshold Tuning:** Bucket thresholds configurable per tenant
---
## Migration Strategy
### Phase 1: Dual Emit (3 months)
- Both Confidence and EWS emitted in verdicts
- Feature flag controls which is primary
- Telemetry compares rankings
### Phase 2: EWS Primary (3 months)
- EWS is default; Confidence available for compatibility
- Deprecation warnings for Confidence consumers
### Phase 3: Confidence Removal (v3.0)
- Confidence scoring removed
- EWS is sole scoring system
---
## Risks & Mitigations
| Risk | Impact | Mitigation |
|------|--------|------------|
| Weight tuning requires iteration | Suboptimal prioritization | Conservative defaults; telemetry |
| Migration confusion | User errors | Clear docs; gradual rollout |
| Performance regression | Slower evaluation | Caching; benchmarks (<50ms) |
| Attestation size increase | Storage cost | Compact proof format |
---
## Success Criteria
1. **Adoption:** 80% of findings viewed with EWS score within 6 months
2. **Triage Time:** 30% reduction in mean-time-to-triage
3. **False Positive Rate:** <5% of "ActNow" findings downgraded after review
4. **Determinism:** 100% of scores reproducible given same inputs + policy
---
## Appendix: Normalization Details
### BKP (Backport) Normalization
| Evidence Tier | BKP Range |
|---------------|-----------|
| No evidence | 0.00 |
| Distro advisory text | 0.40-0.55 |
| Changelog mention | 0.45-0.60 |
| Patch header match | 0.70-0.85 |
| HunkSig match | 0.80-0.92 |
| Binary fingerprint | 0.90-1.00 |
### XPL (Exploit) Normalization
| Signal | XPL Value |
|--------|-----------|
| KEV present | floor 0.40 |
| EPSS top 1% | 0.90-1.00 |
| EPSS top 5% | 0.70-0.89 |
| EPSS top 25% | 0.40-0.69 |
| EPSS below 25% | 0.20-0.39 |
| No EPSS data | 0.30 |
### MIT (Mitigation) Normalization
| Mitigation | MIT Value |
|------------|-----------|
| Feature flag disabled | 0.20-0.40 |
| Auth required | 0.10-0.20 |
| Admin only | 0.15-0.25 |
| Seccomp profile | 0.10-0.25 |
| Network isolation | 0.05-0.15 |
---
## References
- Sprint Files: `docs/implplan/SPRINT_8200_0012_*.md`
- Module Docs: `docs/modules/signals/architecture.md` (to be created)
- Existing Confidence: `docs/modules/policy/confidence-scoring.md`
- EPSS Enrichment: `docs/modules/scanner/epss-enrichment.md`
- Backport Detection: `docs/modules/concelier/backport-detection.md`

2
docs/reachability/DELIVERY_GUIDE.md
View File

@@ -37,7 +37,7 @@ This guide translates the deterministic reachability blueprint into concrete wor
| Stream | Owner Guild(s) | Key deliverables |
|--------|----------------|------------------|
| **Native symbols & callgraphs** | Scanner Worker · Symbols Guild | Ship `Scanner.Symbols.Native` + `Scanner.CallGraph.Native`, integrate Symbol Manifest v1, demangle Itanium/MSVC names, emit `FuncNode`/`CallEdge` CAS bundles (task `SCANNER-NATIVE-401-015`). |
| **Reachability store** | Signals · BE-Base Platform | Provision shared Mongo collections (`func_nodes`, `call_edges`, `cve_func_hits`), indexes, and repositories plus REST hooks for reuse (task `SIG-STORE-401-016`). |
| **Reachability store** | Signals · BE-Base Platform | Provision shared PostgreSQL tables (`func_nodes`, `call_edges`, `cve_func_hits`), indexes, and repositories plus REST hooks for reuse (task `SIG-STORE-401-016`). |
| **Language lifters** | Scanner Worker | CLI/hosted lifters for DotNet, Go, Node/Deno, JVM, Rust, Swift, Binary, Shell with CAS uploads and richgraph output |
| **Signals ingestion & scoring** | Signals | `/callgraphs`, `/runtime-facts` (JSON + NDJSON/gzip), `/graphs/{id}`, `/reachability/recompute` GA; CAS-backed storage, runtime dedupe, BFS+predicates scoring |
| **Runtime capture** | Zastava + Runtime Guild | EntryTrace/eBPF samplers, NDJSON batches (symbol IDs + timestamps + counts) |

4
docs/reachability/runtime-facts.md
View File

@@ -21,8 +21,8 @@
- Empty runtime stream is rejected.
## Storage and cache
- Stored alongside reachability facts in Mongo collection `reachability_facts`.
- Runtime hits cached in Redis via `reachability_cache:*` entries; invalidated on ingest.
- Stored alongside reachability facts in PostgreSQL table `reachability_facts`.
- Runtime hits cached in Valkey via `reachability_cache:*` entries; invalidated on ingest.
## Interaction with scoring
- Ingest triggers recompute: runtime hits added to prior facts’ hits, targets set to symbols observed, entryPoints taken from callgraph.

286
docs/reproducibility.md Normal file
View File

@@ -0,0 +1,286 @@
# StellaOps Reproducibility Specification
This document defines the reproducibility guarantees, verdict identity computation, and replay procedures for StellaOps artifacts.
## Overview
StellaOps provides **deterministic, reproducible outputs** for all security artifacts:
- SBOM generation (CycloneDX 1.6, SPDX 3.0.1)
- VEX statements (OpenVEX)
- Policy verdicts
- Delta computations
- DSSE attestations and Sigstore bundles
**Core Guarantee:** Given identical inputs (image digest, advisory feeds, policies, tool versions), StellaOps produces byte-for-byte identical outputs with matching content-addressed identifiers.
## Verdict Identity Formula
### Content-Addressed Verdict ID
All verdicts use content-addressed identifiers computed as:
```
VerdictId = SHA256(Canonicalize(VerdictPayload))
```
Where `VerdictPayload` includes:
- **Delta ID**: Content hash of the security delta
- **Blocking Drivers**: Sorted list of risk-increasing factors
- **Warning Drivers**: Sorted list of advisory factors
- **Applied Exceptions**: Sorted list of exception IDs covering findings
- **Gate Level**: Recommended gate (G0-G4)
- **Input Stamps**: Hashes of all inputs (see below)
### Input Stamps
Every artifact includes `InputStamps` capturing the provenance of all inputs:
```json
{
"feedSnapshotHash": "sha256:abc123...",
"policyManifestHash": "sha256:def456...",
"sourceCodeHash": "sha256:789ghi...",
"baseImageDigest": "sha256:jkl012...",
"vexDocumentHashes": ["sha256:mno345..."],
"toolchainVersion": "1.0.0",
"custom": {}
}
```
### Determinism Manifest
The `DeterminismManifest` (schema v1.0) tracks artifact reproducibility:
```json
{
"schemaVersion": "1.0",
"artifact": {
"type": "verdict",
"name": "scan-verdict",
"version": "2025-12-24T12:00:00Z",
"format": "StellaOps.DeltaVerdict@1"
},
"canonicalHash": {
"algorithm": "SHA-256",
"value": "abc123def456...",
"encoding": "hex"
},
"inputs": {
"feedSnapshotHash": "sha256:...",
"policyManifestHash": "sha256:...",
"baseImageDigest": "sha256:..."
},
"toolchain": {
"platform": ".NET 10.0",
"components": [
{"name": "StellaOps.Scanner", "version": "1.0.0"},
{"name": "StellaOps.Policy", "version": "1.0.0"}
]
},
"reproducibility": {
"clockFixed": true,
"orderingGuarantee": "stable-sort",
"normalizationRules": ["UTF-8", "LF", "canonical-json"]
},
"generatedAt": "2025-12-24T12:00:00Z"
}
```
## Canonical JSON Serialization
All JSON outputs follow RFC 8785 (JSON Canonicalization Scheme):
1. Keys sorted lexicographically
2. No whitespace between tokens
3. Unicode escaping for non-ASCII
4. Numbers without leading zeros
5. UTF-8 encoding
## DSSE Attestation Format
### Envelope Structure
```json
{
"payloadType": "application/vnd.in-toto+json",
"payload": "<base64url-encoded statement>",
"signatures": [
{
"keyid": "sha256:...",
"sig": "<base64url-encoded signature>"
}
]
}
```
### In-toto Statement
```json
{
"_type": "https://in-toto.io/Statement/v1",
"subject": [
{
"name": "registry.example.com/image:tag",
"digest": {"sha256": "abc123..."}
}
],
"predicateType": "https://stellaops.io/attestation/verdict/v1",
"predicate": {
"verdictId": "sha256:...",
"status": "pass",
"gate": "G2",
"inputs": {...},
"evidence": [...]
}
}
```
## Sigstore Bundle Format
StellaOps produces Sigstore bundles (v0.3) for offline verification:
```json
{
"$mediaType": "application/vnd.dev.sigstore.bundle.v0.3+json",
"verificationMaterial": {
"certificate": {...},
"tlogEntries": [{
"logIndex": "12345",
"logId": {...},
"inclusionProof": {...}
}]
},
"dsseEnvelope": {...}
}
```
## Replay Procedure
### Prerequisites
1. Offline bundle containing:
- Advisory feed snapshot
- Policy pack
- VEX documents
- Tool binaries (pinned versions)
### Steps
```bash
# 1. Extract offline bundle
stella offline extract --bundle offline-kit.tar.gz --output ./replay
# 2. Set deterministic environment
export STELLAOPS_DETERMINISTIC_SEED=42
export STELLAOPS_CLOCK_FIXED=2025-12-24T12:00:00Z
# 3. Run scan with pinned inputs
stella scan \
--image registry.example.com/image@sha256:abc123 \
--feeds ./replay/feeds \
--policies ./replay/policies \
--output ./replay/output
# 4. Verify hash matches original
stella verify \
--manifest ./replay/output/manifest.json \
--expected-hash sha256:def456...
# 5. Verify DSSE attestation
stella attest verify \
--bundle ./replay/output/bundle.sigstore \
--policy verification-policy.yaml
```
### Verification Policy
```yaml
apiVersion: stellaops.io/v1
kind: VerificationPolicy
metadata:
name: audit-verification
spec:
requiredPredicateTypes:
- https://stellaops.io/attestation/verdict/v1
trustedIssuers:
- https://accounts.stellaops.io
maxAge: 90d
requireRekorEntry: true
unknownBudget:
maxTotal: 5
action: fail
```
## Unknown Budget Attestation
Policy thresholds are attested in verdict bundles:
```json
{
"predicateType": "https://stellaops.io/attestation/budget-check/v1",
"predicate": {
"environment": "production",
"budgetConfig": {
"maxUnknownCount": 5,
"maxCumulativeUncertainty": 2.0,
"reasonLimits": {
"Reachability": 0,
"Identity": 2,
"Provenance": 2
}
},
"actualCounts": {
"total": 3,
"byReason": {"Identity": 2, "Provenance": 1}
},
"result": "pass",
"configHash": "sha256:..."
}
}
```
## Schema Versions
| Format | Version | Schema Location |
|--------|---------|-----------------|
| CycloneDX | 1.6 | `docs/schemas/cyclonedx-bom-1.6.schema.json` |
| SPDX | 3.0.1 | `docs/schemas/spdx-3.0.1.schema.json` |
| OpenVEX | 0.2.0 | `docs/schemas/openvex-0.2.0.schema.json` |
| Sigstore Bundle | 0.3 | `docs/schemas/sigstore-bundle-0.3.schema.json` |
| DeterminismManifest | 1.0 | `docs/schemas/determinism-manifest-1.0.schema.json` |
## CI Integration
### Schema Validation
```yaml
# .gitea/workflows/schema-validation.yml
- name: Validate CycloneDX
run: |
sbom-utility validate \
--input-file ${{ matrix.fixture }} \
--schema docs/schemas/cyclonedx-bom-1.6.schema.json
```
### Determinism Gate
```yaml
# .gitea/workflows/determinism-gate.yml
- name: Verify Verdict Hash
run: |
HASH1=$(stella scan --image test:latest --output /tmp/run1 | jq -r '.verdictId')
HASH2=$(stella scan --image test:latest --output /tmp/run2 | jq -r '.verdictId')
[ "$HASH1" = "$HASH2" ] || exit 1
```
## Related Documentation
- [Testing Strategy](testing/testing-strategy-models.md)
- [Determinism Verification](testing/determinism-verification.md)
- [DSSE Attestation Guide](modules/attestor/README.md)
- [Offline Operation](24_OFFLINE_KIT.md)
- [Proof Bundle Spec](modules/triage/proof-bundle-spec.md)
## Changelog
| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2025-12-24 | Initial specification based on product advisory gap analysis |

2
docs/security/audit-events.md
View File

@@ -44,7 +44,7 @@ Event names follow dotted notation:
## Persistence
The Authority host converts audit records into `AuthorityLoginAttemptDocument` rows for MongoDB persistence. Documents must:
The Authority host converts audit records into `AuthorityLoginAttemptDocument` rows for PostgreSQL persistence (schema `authority`). Documents must:
- Preserve `CorrelationId`, `SubjectId`, `ClientId`, `Plugin`, `Outcome`, `Reason`, and `OccurredAt`.
- Store remote address in `remoteAddress` only after classification as PII.

2
docs/signals/unknowns-registry.md
View File

@@ -64,7 +64,7 @@ See [Unknowns Ranking Algorithm](./unknowns-ranking.md) for the complete formula
## 6. Storage & CAS
- Primary store: append-only KV/graph in Mongo (collections `unknowns`, `unknown_metrics`).
- Primary store: append-only KV/graph in PostgreSQL (tables `unknowns`, `unknown_metrics`).
- Evidence blobs: CAS under `cas://unknowns/{sha256}` for large payloads (runtime traces, partial SBOMs).
- Include analyzer fingerprint + schema version in each record for replay.

4
docs/task-packs/registry.md
View File

@@ -10,7 +10,7 @@ The Packs Registry stores, verifies, and serves Task Pack bundles across environ
- **Service name:** `StellaOps.PacksRegistry`
- **Interfaces:** REST/GraphQL API, OCI-compatible registry endpoints, event streams for mirroring.
- **Data stores:** MongoDB (`packs`, `pack_versions`, `pack_provenance`), object storage (bundle blobs, signatures), timeline events.
- **Data stores:** PostgreSQL (`packs`, `pack_versions`, `pack_provenance` tables), object storage (bundle blobs, signatures), timeline events.
- **Dependencies:** Authority scopes (`packs.*`), Export Center (manifests), DevOps signing service, Notifications (optional).
---
@@ -145,7 +145,7 @@ Extensions must be deterministic and derived from signed bundle data.
## 9 · Operations
- **Backups:** Daily snapshots of Mongo collections + object storage, retained for 30 days.
- **Backups:** Daily snapshots of PostgreSQL tables + object storage, retained for 30 days.
- **Retention:** Old versions retained indefinitely; mark as `deprecated` instead of deleting.
- **Maintenance:**
- Run `registry vacuum` weekly to prune orphaned blobs.