docs: Add comprehensive component architecture documentation
Created detailed architectural documentation showing component interactions, communication patterns, and data flows across all StellaOps services. ## New Documentation **docs/ARCHITECTURE_DETAILED.md** - Comprehensive architecture guide: - Component topology diagram (all 36+ services) - Infrastructure layer details (PostgreSQL, Valkey, RustFS, NATS) - Service-by-service catalog with responsibilities - Communication patterns with WHY (business purpose) - 5 detailed data flow diagrams: 1. Scan Request Flow (CLI → Scanner → Worker → Policy → Signer → Attestor → Notify) 2. Advisory Update Flow (Concelier → Scheduler → Scanner re-evaluation) 3. VEX Update Flow (Excititor → IssuerDirectory → Scheduler → Policy) 4. Notification Delivery Flow (Scanner → Valkey → Notify → Slack/Teams/Email) 5. Policy Evaluation Flow (Scanner → Policy.Gateway → OPA → PostgreSQL replication) - Database schema isolation details per service - Security boundaries and authentication flows ## Updated Documentation **docs/DEVELOPER_ONBOARDING.md**: - Added link to detailed architecture - Simplified overview with component categories - Quick reference topology tree **docs/07_HIGH_LEVEL_ARCHITECTURE.md**: - Updated infrastructure requirements section - Clarified PostgreSQL as ONLY database - Emphasized Valkey as REQUIRED (not optional) - Marked NATS as optional (Valkey is default transport) **docs/README.md**: - Added link to detailed architecture in navigation ## Key Architectural Insights Documented **Communication Patterns:** - 11 communication steps in scan flow (Gateway → Scanner → Valkey → Worker → Concelier → Policy → Signer → Attestor → Valkey → Notify → Slack) - PostgreSQL logical replication (advisory_raw_stream, vex_raw_stream → Policy Engine) - Valkey Streams for async job queuing (XADD/XREADGROUP pattern) - HTTP webhooks for delta events (Concelier/Excititor → Scheduler) **Security Boundaries:** - Authority issues OpToks with DPoP binding (RFC 9449) - Signer enforces PoE validation + scanner digest verification - All services validate JWT + DPoP on every request - Tenant isolation via tenant_id in all PostgreSQL queries **Database Patterns:** - 8 dedicated PostgreSQL schemas (authority, scanner, vuln, vex, scheduler, notify, policy, orchestrator) - Append-only advisory/VEX storage (AOC - Aggregation-Only Contract) - BOM-Index for impact selection (CVE → PURL → image mapping) This documentation provides complete visibility into who calls who, why they communicate, what data flows through the system, and how security is enforced at every layer. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,248 @@
|
||||
# SPRINT_4300_0002_0001: Unknowns Budget Policy Integration
|
||||
|
||||
## Topic & Scope
|
||||
- Add unknown budget policy rules and enforcement gates tied to Unknowns state.
|
||||
- Provide configuration and reporting for environment-scoped unknown thresholds.
|
||||
- Surface budget status in scan reports and notifications.
|
||||
- **Working directory:** `src/Policy/`, `src/Signals/`, `src/Scanner/`.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
- **Upstream:** UncertaintyTier (exists), UnknownStateLedger (exists).
|
||||
- **Downstream:** Policy decisions, notification workflows, UI reporting.
|
||||
- **Safe to parallelize with:** Other SPRINT_4300_0002_* sprints.
|
||||
|
||||
## Documentation Prerequisites
|
||||
- `docs/modules/policy/architecture.md`
|
||||
- `docs/modules/signals/unknowns/2025-12-01-unknowns-registry.md`
|
||||
- `docs/modules/scanner/architecture.md`
|
||||
|
||||
## Sprint Metadata
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Sprint ID** | 4300_0002_0001 |
|
||||
| **Title** | Unknowns Budget Policy Integration |
|
||||
| **Priority** | P1 (High) |
|
||||
| **Moat Strength** | 4 (Strong moat) |
|
||||
| **Working Directory** | `src/Policy/`, `src/Signals/`, `src/Scanner/` |
|
||||
| **Estimated Effort** | 2 weeks |
|
||||
| **Dependencies** | UncertaintyTier (exists), UnknownStateLedger (exists) |
|
||||
|
||||
---
|
||||
|
||||
## Objective
|
||||
|
||||
Implement policy-level enforcement of unknown budgets, enabling rules like "fail if unknowns > N in production" or "warn if uncertainty tier is T1 for critical components."
|
||||
|
||||
**Moat thesis**: "We quantify uncertainty and gate on it."
|
||||
|
||||
---
|
||||
|
||||
## Background
|
||||
|
||||
The advisory identifies "Unknowns as first-class state" as a **Moat 4** feature. Current implementation has:
|
||||
- `UncertaintyTier` (T1-T4) with entropy classification
|
||||
- `UnknownStateLedger` tracking marker kinds
|
||||
- Risk modifiers from uncertainty
|
||||
|
||||
**Gap**: No policy integration to enforce unknown budgets.
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
### D1: Unknown Budget Rule DSL
|
||||
- Define policy rules for unknown thresholds
|
||||
- Support tier-based, count-based, and entropy-based rules
|
||||
- Environment scoping (dev/staging/prod)
|
||||
|
||||
### D2: Policy Engine Integration
|
||||
- Extend `PolicyGateEvaluator` with unknown budget gates
|
||||
- Add unknown state to evaluation context
|
||||
- Emit violation on budget exceeded
|
||||
|
||||
### D3: Unknown Budget Configuration
|
||||
- Admin UI for setting budgets per environment
|
||||
- API endpoints for budget CRUD
|
||||
- Default budgets per tier
|
||||
|
||||
### D4: Reporting & Alerts
|
||||
- Include unknown budget status in scan reports
|
||||
- Notify on budget threshold crossings
|
||||
- Dashboard widget for unknown trends
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
### Phase 1: Policy Rule DSL
|
||||
|
||||
| ID | Task | Status | Assignee |
|
||||
|----|------|--------|----------|
|
||||
| BUDGET-001 | Define `UnknownBudgetRule` schema | DONE | Agent |
|
||||
| BUDGET-002 | Add budget rules to policy bundle format | DONE | Agent |
|
||||
| BUDGET-003 | Create `UnknownBudgetRuleParser` | DONE | Agent |
|
||||
| BUDGET-004 | Support expressions: `unknowns.count > 10`, `unknowns.tier == T1` | DONE | Agent |
|
||||
| BUDGET-005 | Add environment scope filter | DONE | Agent |
|
||||
|
||||
### Phase 2: Policy Engine Integration
|
||||
|
||||
| ID | Task | Status | Assignee |
|
||||
|----|------|--------|----------|
|
||||
| BUDGET-006 | Extend `PolicyEvaluationContext` with unknown state | DONE | Agent |
|
||||
| BUDGET-007 | Add `UnknownBudgetGate` to `PolicyGateEvaluator` | DONE | Agent |
|
||||
| BUDGET-008 | Implement tier-based gate: block on T1, warn on T2 | DONE | Agent |
|
||||
| BUDGET-009 | Implement count-based gate: fail if count > threshold | DONE | Agent |
|
||||
| BUDGET-010 | Implement entropy-based gate: fail if mean entropy > threshold | DONE | Agent |
|
||||
| BUDGET-011 | Emit `BudgetExceededViolation` with details | DONE | Agent |
|
||||
| BUDGET-012 | Unit tests for all gate types | DONE | Agent |
|
||||
|
||||
### Phase 3: Configuration
|
||||
|
||||
| ID | Task | Status | Assignee |
|
||||
|----|------|--------|----------|
|
||||
| BUDGET-013 | Add `UnknownBudgetOptions` configuration | DONE | Agent |
|
||||
| BUDGET-014 | Create budget management API endpoints | DONE | Agent |
|
||||
| BUDGET-015 | Implement default budgets (prod: T2 max, staging: T1 warn) | DONE | Agent |
|
||||
| BUDGET-016 | Add budget configuration to policy YAML | DONE | Agent |
|
||||
|
||||
### Phase 4: Reporting
|
||||
|
||||
| ID | Task | Status | Assignee |
|
||||
|----|------|--------|----------|
|
||||
| BUDGET-017 | Add unknown budget section to scan report | DONE | Agent |
|
||||
| BUDGET-018 | Create `UnknownBudgetExceeded` notification event | DONE | Agent |
|
||||
| BUDGET-019 | Integrate with Notify module for alerts | DONE | Agent |
|
||||
| BUDGET-020 | Add budget status to policy evaluation response | DONE | Agent |
|
||||
|
||||
---
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Dependency | Owners | Task Definition |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | BUDGET-001 | DONE | — | Agent | Define `UnknownBudgetRule` schema |
|
||||
| 2 | BUDGET-002 | DONE | — | Agent | Add budget rules to policy bundle format |
|
||||
| 3 | BUDGET-003 | DONE | — | Agent | Create `UnknownBudgetRuleParser` |
|
||||
| 4 | BUDGET-004 | DONE | — | Agent | Support expressions: `unknowns.count > 10`, `unknowns.tier == T1` |
|
||||
| 5 | BUDGET-005 | DONE | — | Agent | Add environment scope filter |
|
||||
| 6 | BUDGET-006 | DONE | — | Agent | Extend `PolicyEvaluationContext` with unknown state |
|
||||
| 7 | BUDGET-007 | DONE | — | Agent | Add `UnknownBudgetGate` to `PolicyGateEvaluator` |
|
||||
| 8 | BUDGET-008 | DONE | — | Agent | Implement tier-based gate: block on T1, warn on T2 |
|
||||
| 9 | BUDGET-009 | DONE | — | Agent | Implement count-based gate: fail if count > threshold |
|
||||
| 10 | BUDGET-010 | DONE | — | Agent | Implement entropy-based gate: fail if mean entropy > threshold |
|
||||
| 11 | BUDGET-011 | DONE | — | Agent | Emit `BudgetExceededViolation` with details |
|
||||
| 12 | BUDGET-012 | DONE | — | Agent | Unit tests for all gate types |
|
||||
| 13 | BUDGET-013 | DONE | — | Agent | Add `UnknownBudgetOptions` configuration |
|
||||
| 14 | BUDGET-014 | DONE | — | Agent | Create budget management API endpoints |
|
||||
| 15 | BUDGET-015 | DONE | — | Agent | Implement default budgets (prod: T2 max, staging: T1 warn) |
|
||||
| 16 | BUDGET-016 | DONE | — | Agent | Add budget configuration to policy YAML |
|
||||
| 17 | BUDGET-017 | DONE | — | Agent | Add unknown budget section to scan report |
|
||||
| 18 | BUDGET-018 | DONE | — | Agent | Create `UnknownBudgetExceeded` notification event |
|
||||
| 19 | BUDGET-019 | DONE | — | Agent | Integrate with Notify module for alerts |
|
||||
| 20 | BUDGET-020 | DONE | — | Agent | Add budget status to policy evaluation response |
|
||||
|
||||
---
|
||||
|
||||
## Wave Coordination
|
||||
|
||||
- Single wave for unknown budget policy integration.
|
||||
|
||||
## Wave Detail Snapshots
|
||||
|
||||
- N/A (single wave).
|
||||
|
||||
## Interlocks
|
||||
|
||||
- Admin UI and Notify integration depend on cross-module coordination.
|
||||
|
||||
## Upcoming Checkpoints
|
||||
|
||||
| Date (UTC) | Checkpoint | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-12-22 | Sprint template normalization complete. | Agent |
|
||||
|
||||
## Action Tracker
|
||||
|
||||
| Date (UTC) | Action | Owner | Status |
|
||||
| --- | --- | --- | --- |
|
||||
| 2025-12-22 | Normalize sprint file to standard template. | Agent | DONE |
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-12-22 | Sprint created from moat hardening advisory (19-Dec-2025). | Agent |
|
||||
| 2025-12-22 | Normalized sprint file to standard template; no semantic changes. | Agent |
|
||||
| 2025-12-22 | Status review: UnknownBudgetOptions, UnknownBudgetService, UnknownsBudgetGate, UncertaintyTier system all pre-existing. Phase 1-2 and BUDGET-013 marked DONE. | Agent |
|
||||
| 2025-12-22 | Completed remaining tasks: BUDGET-002 (PolicyBundle.UnknownBudgets), BUDGET-014 (BudgetEndpoints.cs), BUDGET-015 (DefaultBudgets.cs), BUDGET-016 (policy-engine.yaml.sample), BUDGET-017 (UnknownBudgetSectionDto), BUDGET-018-020 (BudgetExceededEventFactory, NotifyEventKinds). Sprint complete. | Agent |
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
1. **AC1**: Policy can define `unknowns.count <= 5` threshold
|
||||
2. **AC2**: Policy can define `unknowns.tier != T1` requirement
|
||||
3. **AC3**: Budget violations appear in scan results
|
||||
4. **AC4**: Notifications fire on budget exceeded
|
||||
5. **AC5**: Environment-specific budgets work correctly
|
||||
|
||||
---
|
||||
|
||||
## Technical Notes
|
||||
|
||||
### Policy Rule Examples
|
||||
```yaml
|
||||
unknown_budgets:
|
||||
- name: "production-strict"
|
||||
environment: "production"
|
||||
rules:
|
||||
- tier_max: T2 # Block if any T1 unknowns
|
||||
- count_max: 5 # Block if > 5 unknowns total
|
||||
- entropy_max: 0.4 # Block if mean entropy > 0.4
|
||||
action: block
|
||||
|
||||
- name: "staging-warn"
|
||||
environment: "staging"
|
||||
rules:
|
||||
- tier_max: T1 # Warn on T1, allow T2-T4
|
||||
- count_max: 20
|
||||
action: warn
|
||||
```
|
||||
|
||||
### Gate Evaluation
|
||||
```csharp
|
||||
public sealed class UnknownBudgetGate : IPolicyGate
|
||||
{
|
||||
public GateResult Evaluate(UnknownBudgetRule rule, UnknownState state)
|
||||
{
|
||||
if (rule.TierMax.HasValue && state.MaxTier < rule.TierMax.Value)
|
||||
return GateResult.Fail($"Tier {state.MaxTier} exceeds budget {rule.TierMax}");
|
||||
|
||||
if (rule.CountMax.HasValue && state.Count > rule.CountMax.Value)
|
||||
return GateResult.Fail($"Count {state.Count} exceeds budget {rule.CountMax}");
|
||||
|
||||
return GateResult.Pass();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
| Item | Type | Owner | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| Default budgets | Decision | Policy Team | Align with advisory defaults (prod strict, staging warn) |
|
||||
| Budget actions | Decision | Policy Team | `block` and `warn` actions supported in v1 |
|
||||
|
||||
| Risk | Impact | Mitigation |
|
||||
|------|--------|------------|
|
||||
| Too strict budgets block all deployments | Adoption friction | Provide sensible defaults, gradual rollout |
|
||||
| Unknown counting varies by scan | Inconsistent gates | Normalize counting methodology |
|
||||
|
||||
---
|
||||
|
||||
## Documentation Updates
|
||||
|
||||
- [ ] Update `docs/modules/policy/architecture.md`
|
||||
- [ ] Add `docs/operations/unknown-budgets-guide.md`
|
||||
- [ ] Update policy DSL reference
|
||||
@@ -0,0 +1,185 @@
|
||||
# SPRINT_4300_0002_0002: Unknowns Attestation Predicates
|
||||
|
||||
## Topic & Scope
|
||||
- Define in-toto predicate types for unknown state and unknown budget evaluations.
|
||||
- Emit unknown attestations in the proof chain and extend verification to cover them.
|
||||
- Publish schemas for the new predicates.
|
||||
- **Working directory:** `src/Attestor/`, `src/Signals/`, `src/Unknowns/`.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
- **Upstream:** SPRINT_4300_0002_0001, UncertaintyTier (exists).
|
||||
- **Downstream:** Verdict verification and audit replay workflows.
|
||||
- **Safe to parallelize with:** Other SPRINT_4300_0002_* sprints.
|
||||
|
||||
## Documentation Prerequisites
|
||||
- `docs/modules/attestor/architecture.md`
|
||||
- `docs/modules/signals/unknowns/2025-12-01-unknowns-registry.md`
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
|
||||
## Sprint Metadata
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Sprint ID** | 4300_0002_0002 |
|
||||
| **Title** | Unknowns Attestation Predicates |
|
||||
| **Priority** | P1 (High) |
|
||||
| **Moat Strength** | 4 (Strong moat) |
|
||||
| **Working Directory** | `src/Attestor/`, `src/Signals/`, `src/Unknowns/` |
|
||||
| **Estimated Effort** | 1 week |
|
||||
| **Dependencies** | SPRINT_4300_0002_0001, UncertaintyTier (exists) |
|
||||
|
||||
---
|
||||
|
||||
## Objective
|
||||
|
||||
Create in-toto attestation predicates for unknown states, making uncertainty auditable, portable, and verifiable as part of the proof chain.
|
||||
|
||||
**Moat thesis**: "We quantify uncertainty and gate on it." — Extended to: uncertainty is attestable.
|
||||
|
||||
---
|
||||
|
||||
## Background
|
||||
|
||||
Unknowns need to be:
|
||||
1. Recorded in attestations for audit trails
|
||||
2. Portable with verdicts for external verification
|
||||
3. Queryable by admission controllers
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
### D1: Unknown State Attestation Predicate
|
||||
- Define `uncertainty.stella/v1` predicate type
|
||||
- Include: tier, entropy, marker kinds, evidence
|
||||
|
||||
### D2: Unknown Budget Attestation Predicate
|
||||
- Define `uncertainty-budget.stella/v1` predicate type
|
||||
- Include: budget definition, evaluation result, violations
|
||||
|
||||
### D3: Integration with Proof Chain
|
||||
- Emit unknown attestations as part of `ProofSpineAssembler`
|
||||
- Link to verdict attestation
|
||||
|
||||
### D4: Verification Support
|
||||
- Extend `stella verdict verify` to check unknown attestations
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
| ID | Task | Status | Assignee |
|
||||
|----|------|--------|----------|
|
||||
| UATT-001 | Define `UncertaintyStatement` in-toto predicate | DONE | Agent |
|
||||
| UATT-002 | Define `UncertaintyBudgetStatement` predicate | DONE | Agent |
|
||||
| UATT-003 | Create statement builders in `StellaOps.Attestor.ProofChain` | DONE | Agent |
|
||||
| UATT-004 | Integrate into `ProofSpineAssembler` | DONE | Agent |
|
||||
| UATT-005 | Add unknown attestation to verdict bundle | DONE | Agent |
|
||||
| UATT-006 | Extend verification CLI for unknown predicates | DONE | Agent |
|
||||
| UATT-007 | Add JSON schema for predicates | DONE | Agent |
|
||||
| UATT-008 | Write attestation round-trip tests | DONE | Agent |
|
||||
|
||||
---
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Dependency | Owners | Task Definition |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | UATT-001 | DONE | — | Agent | Define `UncertaintyStatement` in-toto predicate |
|
||||
| 2 | UATT-002 | DONE | — | Agent | Define `UncertaintyBudgetStatement` predicate |
|
||||
| 3 | UATT-003 | DONE | — | Agent | Create statement builders in `StellaOps.Attestor.ProofChain` |
|
||||
| 4 | UATT-004 | DONE | — | Agent | Integrate into `ProofSpineAssembler` |
|
||||
| 5 | UATT-005 | DONE | — | Agent | Add unknown attestation to verdict bundle |
|
||||
| 6 | UATT-006 | DONE | — | Agent | Extend verification CLI for unknown predicates |
|
||||
| 7 | UATT-007 | DONE | — | Agent | Add JSON schema for predicates |
|
||||
| 8 | UATT-008 | DONE | — | Agent | Write attestation round-trip tests |
|
||||
|
||||
---
|
||||
|
||||
## Wave Coordination
|
||||
|
||||
- Single wave for unknown attestation predicate delivery.
|
||||
|
||||
## Wave Detail Snapshots
|
||||
|
||||
- N/A (single wave).
|
||||
|
||||
## Interlocks
|
||||
|
||||
- Verification CLI depends on predicate schema and proof chain emission.
|
||||
|
||||
## Upcoming Checkpoints
|
||||
|
||||
| Date (UTC) | Checkpoint | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-12-22 | Sprint template normalization complete. | Agent |
|
||||
|
||||
## Action Tracker
|
||||
|
||||
| Date (UTC) | Action | Owner | Status |
|
||||
| --- | --- | --- | --- |
|
||||
| 2025-12-22 | Normalize sprint file to standard template. | Agent | DONE |
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-12-22 | Sprint created from moat hardening advisory (19-Dec-2025). | Agent |
|
||||
| 2025-12-22 | Normalized sprint file to standard template; no semantic changes. | Agent |
|
||||
| 2025-12-22 | UATT-001,002,003: Created UncertaintyStatement, UncertaintyBudgetStatement predicates and builders. | Agent |
|
||||
| 2025-12-22 | UATT-008: Wrote 7 unit tests for attestation predicates (all passing). | Agent |
|
||||
| 2025-12-22 | UATT-004: Extended ProofSpinePayload and ProofSpineRequest with uncertainty statement IDs. | Agent |
|
||||
| 2025-12-22 | UATT-005: Extended VerdictOutputs and VerdictOciPublisher with uncertainty attestation references. | Agent |
|
||||
| 2025-12-22 | UATT-006: Extended VerdictCommandGroup with --verify-uncertainty, --max-tier, --max-unknowns, --max-entropy options. | Agent |
|
||||
| 2025-12-22 | UATT-007: Created uncertainty-statement.v1.schema.json and uncertainty-budget-statement.v1.schema.json in Attestor.Types/schemas. Sprint complete. | Agent |
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
1. **AC1**: Unknown state is captured in attestation
|
||||
2. **AC2**: Budget evaluation result is attestable
|
||||
3. **AC3**: Attestations are signed and verifiable
|
||||
4. **AC4**: Proof chain links unknown to verdict
|
||||
|
||||
---
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
| Item | Type | Owner | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| Predicate types | Decision | Attestor Team | `uncertainty.stella/v1`, `uncertainty-budget.stella/v1` |
|
||||
|
||||
| Risk | Impact | Mitigation |
|
||||
| --- | --- | --- |
|
||||
| Predicate schema drift | Verification failures | Version and publish schemas alongside code |
|
||||
| Missing unknown state data | Incomplete attestations | Validate upstream Unknowns/Signals inputs |
|
||||
|
||||
---
|
||||
|
||||
## Technical Notes
|
||||
|
||||
### Uncertainty Statement
|
||||
```json
|
||||
{
|
||||
"_type": "https://in-toto.io/Statement/v1",
|
||||
"subject": [{"digest": {"sha256": "<sbom-digest>"}}],
|
||||
"predicateType": "uncertainty.stella/v1",
|
||||
"predicate": {
|
||||
"graphRevisionId": "...",
|
||||
"aggregateTier": "T2",
|
||||
"meanEntropy": 0.35,
|
||||
"unknownCount": 7,
|
||||
"markers": [
|
||||
{"kind": "U1", "count": 3, "entropy": 0.45},
|
||||
{"kind": "U2", "count": 4, "entropy": 0.28}
|
||||
],
|
||||
"evaluatedAt": "2025-12-22T00:00:00Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Documentation Updates
|
||||
|
||||
- [ ] Update attestation type catalog
|
||||
- [ ] Add uncertainty predicate specification
|
||||
@@ -0,0 +1,252 @@
|
||||
# SPRINT_4300_0003_0001: Sealed Knowledge Snapshot Export/Import
|
||||
|
||||
## Topic & Scope
|
||||
- Implement sealed knowledge snapshot export/import for air-gapped environments.
|
||||
- Package advisories, VEX, and policy bundles with time anchors and trust roots.
|
||||
- Add diff and staleness controls for snapshot lifecycle.
|
||||
- **Working directory:** `src/AirGap/`, `src/Concelier/`, `src/Excititor/`, `src/Cli/`.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
- **Upstream:** AirGap.Importer (exists), ReplayManifest (exists).
|
||||
- **Downstream:** Offline scans, advisory synchronization workflows.
|
||||
- **Safe to parallelize with:** Other SPRINT_4300_0003_* sprints.
|
||||
|
||||
## Documentation Prerequisites
|
||||
- `docs/modules/airgap/` (air-gap workflow docs)
|
||||
- `docs/modules/concelier/architecture.md`
|
||||
- `docs/modules/excititor/architecture.md`
|
||||
- `docs/modules/cli/architecture.md`
|
||||
|
||||
## Sprint Metadata
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Sprint ID** | 4300_0003_0001 |
|
||||
| **Title** | Sealed Knowledge Snapshot Export/Import |
|
||||
| **Priority** | P1 (High) |
|
||||
| **Moat Strength** | 4 (Strong moat) |
|
||||
| **Working Directory** | `src/AirGap/`, `src/Concelier/`, `src/Excititor/`, `src/Cli/` |
|
||||
| **Estimated Effort** | 2 weeks |
|
||||
| **Dependencies** | AirGap.Importer (exists), ReplayManifest (exists) |
|
||||
|
||||
---
|
||||
|
||||
## Objective
|
||||
|
||||
Implement a "sealed knowledge snapshot" workflow for air-gapped environments, packaging all advisory feeds, VEX statements, and policies into a cryptographically verifiable bundle that can be transferred offline and validated on import.
|
||||
|
||||
**Moat thesis**: Air-gapped "runtime" is common; air-gapped **reproducibility** is not.
|
||||
|
||||
---
|
||||
|
||||
## Background
|
||||
|
||||
The advisory identifies air-gapped epistemic mode as **Moat 4**. Current implementation has:
|
||||
- `AirGap.Controller` with state management
|
||||
- `ReplayVerifier` with depth levels
|
||||
- `TrustStore` for offline validation
|
||||
|
||||
**Gap**: No unified export/import workflow for knowledge snapshots.
|
||||
|
||||
---
|
||||
|
||||
## Deliverables
|
||||
|
||||
### D1: Knowledge Snapshot Format
|
||||
- Define snapshot bundle structure
|
||||
- Include: advisories, VEX, policies, time anchor, trust roots
|
||||
- Merkle tree for content integrity
|
||||
|
||||
### D2: Snapshot Export CLI
|
||||
- `stella airgap export --output=./knowledge-2025-12-22.tar.gz`
|
||||
- Point-in-time feed extraction
|
||||
- Sign snapshot with designated key
|
||||
|
||||
### D3: Snapshot Import CLI
|
||||
- `stella airgap import --bundle=./knowledge-2025-12-22.tar.gz`
|
||||
- Verify signature and merkle root
|
||||
- Validate time anchor freshness
|
||||
- Apply to local database
|
||||
|
||||
### D4: Snapshot Diff
|
||||
- Compare two snapshots
|
||||
- Report: new advisories, updated VEX, policy changes
|
||||
|
||||
### D5: Staleness Policy
|
||||
- Configurable max age for snapshots
|
||||
- Warn/block on stale knowledge
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
### Phase 1: Snapshot Format
|
||||
|
||||
| ID | Task | Status | Assignee |
|
||||
|----|------|--------|----------|
|
||||
| SEAL-001 | Define `KnowledgeSnapshotManifest` schema | DONE | Agent |
|
||||
| SEAL-002 | Implement merkle tree builder for bundle contents | DONE | Agent |
|
||||
| SEAL-003 | Create `SnapshotBundleWriter` | DONE | Agent |
|
||||
| SEAL-004 | Add DSSE signing for manifest | DONE | Agent |
|
||||
|
||||
### Phase 2: Export
|
||||
|
||||
| ID | Task | Status | Assignee |
|
||||
|----|------|--------|----------|
|
||||
| SEAL-005 | Add `stella airgap export` command | DONE | Agent |
|
||||
| SEAL-006 | Implement advisory snapshot extractor | DONE | Agent |
|
||||
| SEAL-007 | Implement VEX snapshot extractor | DONE | Agent |
|
||||
| SEAL-008 | Implement policy bundle extractor | DONE | Agent |
|
||||
| SEAL-009 | Add time anchor token generation | DONE | Agent |
|
||||
| SEAL-010 | Package into signed bundle | DONE | Agent |
|
||||
|
||||
### Phase 3: Import
|
||||
|
||||
| ID | Task | Status | Assignee |
|
||||
|----|------|--------|----------|
|
||||
| SEAL-011 | Add `stella airgap import` command | DONE | Agent |
|
||||
| SEAL-012 | Implement signature verification | DONE | Agent |
|
||||
| SEAL-013 | Implement merkle root validation | DONE | Agent |
|
||||
| SEAL-014 | Validate time anchor against staleness policy | DONE | Agent |
|
||||
| SEAL-015 | Apply advisories to Concelier database | DONE | Agent |
|
||||
| SEAL-016 | Apply VEX to Excititor database | DONE | Agent |
|
||||
| SEAL-017 | Apply policies to Policy registry | DONE | Agent |
|
||||
|
||||
### Phase 4: Diff & Staleness
|
||||
|
||||
| ID | Task | Status | Assignee |
|
||||
|----|------|--------|----------|
|
||||
| SEAL-018 | Implement `stella airgap diff` command | DONE | Agent |
|
||||
| SEAL-019 | Add staleness policy configuration | DONE | Agent |
|
||||
| SEAL-020 | Emit warnings on stale imports | DONE | Agent |
|
||||
|
||||
---
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Dependency | Owners | Task Definition |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | SEAL-001 | DONE | — | Agent | Define `KnowledgeSnapshotManifest` schema |
|
||||
| 2 | SEAL-002 | DONE | — | Agent | Implement merkle tree builder for bundle contents |
|
||||
| 3 | SEAL-003 | DONE | — | Agent | Create `SnapshotBundleWriter` |
|
||||
| 4 | SEAL-004 | DONE | — | Agent | Add DSSE signing for manifest |
|
||||
| 5 | SEAL-005 | DONE | — | Agent | Add `stella airgap export` command |
|
||||
| 6 | SEAL-006 | DONE | — | Agent | Implement advisory snapshot extractor |
|
||||
| 7 | SEAL-007 | DONE | — | Agent | Implement VEX snapshot extractor |
|
||||
| 8 | SEAL-008 | DONE | — | Agent | Implement policy bundle extractor |
|
||||
| 9 | SEAL-009 | DONE | — | Agent | Add time anchor token generation |
|
||||
| 10 | SEAL-010 | DONE | — | Agent | Package into signed bundle |
|
||||
| 11 | SEAL-011 | DONE | — | Agent | Add `stella airgap import` command |
|
||||
| 12 | SEAL-012 | DONE | — | Agent | Implement signature verification |
|
||||
| 13 | SEAL-013 | DONE | — | Agent | Implement merkle root validation |
|
||||
| 14 | SEAL-014 | DONE | — | Agent | Validate time anchor against staleness policy |
|
||||
| 15 | SEAL-015 | DONE | — | Agent | Apply advisories to Concelier database |
|
||||
| 16 | SEAL-016 | DONE | — | Agent | Apply VEX to Excititor database |
|
||||
| 17 | SEAL-017 | DONE | — | Agent | Apply policies to Policy registry |
|
||||
| 18 | SEAL-018 | DONE | — | Agent | Implement `stella airgap diff` command |
|
||||
| 19 | SEAL-019 | DONE | — | Agent | Add staleness policy configuration |
|
||||
| 20 | SEAL-020 | DONE | — | Agent | Emit warnings on stale imports |
|
||||
|
||||
---
|
||||
|
||||
## Wave Coordination
|
||||
|
||||
- Single wave for sealed knowledge snapshot delivery.
|
||||
|
||||
## Wave Detail Snapshots
|
||||
|
||||
- N/A (single wave).
|
||||
|
||||
## Interlocks
|
||||
|
||||
- Snapshot import depends on data model compatibility in Concelier and Excititor.
|
||||
|
||||
## Upcoming Checkpoints
|
||||
|
||||
| Date (UTC) | Checkpoint | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-12-22 | Sprint template normalization complete. | Agent |
|
||||
|
||||
## Action Tracker
|
||||
|
||||
| Date (UTC) | Action | Owner | Status |
|
||||
| --- | --- | --- | --- |
|
||||
| 2025-12-22 | Normalize sprint file to standard template. | Agent | DONE |
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-12-22 | Sprint created from moat hardening advisory (19-Dec-2025). | Agent |
|
||||
| 2025-12-22 | Normalized sprint file to standard template; no semantic changes. | Agent |
|
||||
| 2025-12-22 | Completed SEAL-005, SEAL-011, SEAL-018: Created AirGapCommandGroup with export/import/diff/status commands. | Agent |
|
||||
| 2025-12-22 | Completed SEAL-019, SEAL-020: Created etc/airgap.yaml.sample with staleness policy and warning configuration. | Agent |
|
||||
| 2025-12-22 | Completed SEAL-002, SEAL-003, SEAL-004: Created SnapshotBundleWriter with merkle tree and DSSE signing. | Agent |
|
||||
| 2025-12-22 | Completed SEAL-006, SEAL-007, SEAL-008: Created Advisory, VEX, and Policy snapshot extractors in AirGap.Bundle. | Agent |
|
||||
| 2025-12-22 | Completed SEAL-009, SEAL-010: Created TimeAnchorService for time anchor generation. | Agent |
|
||||
| 2025-12-22 | Completed SEAL-012, SEAL-013, SEAL-014: Created SnapshotBundleReader with signature/merkle/time anchor verification. | Agent |
|
||||
| 2025-12-23 | Completed SEAL-015, SEAL-016, SEAL-017: Created KnowledgeSnapshotImporter.cs with IAdvisoryImportTarget, IVexImportTarget, IPolicyImportTarget interfaces. Created module-specific adapters: ConcelierAdvisoryImportTarget, ExcititorVexImportTarget, PolicyRegistryImportTarget in AirGap.Bundle. Sprint now 20/20 complete (100%). | Agent |
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
1. **AC1**: Export produces self-contained knowledge bundle
|
||||
2. **AC2**: Import validates signature and merkle root
|
||||
3. **AC3**: Stale snapshots are rejected (configurable age)
|
||||
4. **AC4**: Diff shows changes between snapshots
|
||||
5. **AC5**: Imported knowledge enables offline scans
|
||||
|
||||
---
|
||||
|
||||
## Technical Notes
|
||||
|
||||
### Bundle Structure
|
||||
```
|
||||
knowledge-2025-12-22.tar.gz
|
||||
├── manifest.json # Snapshot metadata + merkle root
|
||||
├── manifest.sig # DSSE signature
|
||||
├── time-anchor.json # RFC 3161 or Roughtime token
|
||||
├── advisories/
|
||||
│ ├── nvd/ # NVD advisories
|
||||
│ ├── ghsa/ # GitHub advisories
|
||||
│ └── ... # Other feeds
|
||||
├── vex/
|
||||
│ ├── cisco/
|
||||
│ ├── redhat/
|
||||
│ └── ...
|
||||
├── policies/
|
||||
│ └── policy-bundle.tar # OPA bundle
|
||||
└── trust/
|
||||
└── trust-roots.pem # Signing key roots
|
||||
```
|
||||
|
||||
### Staleness Budget
|
||||
```yaml
|
||||
airgap:
|
||||
staleness:
|
||||
max_age_hours: 168 # 7 days default
|
||||
warn_age_hours: 72 # Warn after 3 days
|
||||
require_time_anchor: true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
| Item | Type | Owner | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| Snapshot bundle format | Decision | AirGap Team | `knowledge-YYYY-MM-DD.tar.gz` with manifest + merkle root |
|
||||
| Staleness policy | Decision | AirGap Team | Default max age 7 days, warn after 3 days |
|
||||
|
||||
| Risk | Impact | Mitigation |
|
||||
|------|--------|------------|
|
||||
| Large bundle size | Transfer challenges | Incremental updates, compression |
|
||||
| Key compromise | Trust broken | Support key rotation, revocation lists |
|
||||
| Time anchor unavailable | Cannot validate freshness | Fallback to operator attestation |
|
||||
|
||||
---
|
||||
|
||||
## Documentation Updates
|
||||
|
||||
- [ ] Add `docs/operations/airgap-knowledge-sync.md`
|
||||
- [ ] Update air-gap architecture documentation
|
||||
- [ ] Add staleness policy guide
|
||||
Reference in New Issue
Block a user