docs consolidation work
This commit is contained in:
StellaOps Bot
77
docs/_archive/implementation-plans/README.md
Normal file
77
docs/_archive/implementation-plans/README.md
Normal file
@@ -0,0 +1,77 @@
|
||||
# Implementation Plans Archive
|
||||
|
||||
This directory contains archived implementation_plan.md files from module documentation.
|
||||
|
||||
## Archive Date: 2025-12-25
|
||||
|
||||
Implementation plans have been consolidated into the respective module README.md files under an "Implementation Status" section. The original implementation_plan.md files have been archived here for reference.
|
||||
|
||||
## Archived Files
|
||||
|
||||
### Modules Consolidated (2025-12-25 - Final Batch)
|
||||
- orchestrator-implementation-plan.md
|
||||
- platform-implementation-plan.md
|
||||
- policy-implementation-plan.md
|
||||
- registry-implementation-plan.md
|
||||
- scanner-implementation-plan.md
|
||||
- scheduler-implementation-plan.md
|
||||
- signer-implementation-plan.md
|
||||
- telemetry-implementation-plan.md
|
||||
- ui-implementation-plan.md
|
||||
- vex-lens-implementation-plan.md
|
||||
- vuln-explorer-implementation-plan.md
|
||||
- zastava-implementation-plan.md
|
||||
|
||||
### Previously Archived Modules
|
||||
- advisory-ai-implementation-plan.md
|
||||
- attestor-implementation-plan.md
|
||||
- authority-implementation-plan.md
|
||||
- ci-implementation-plan.md
|
||||
- cli-implementation-plan.md
|
||||
- concelier-implementation-plan.md
|
||||
- devops-implementation-plan.md
|
||||
- excititor-implementation-plan.md
|
||||
- export-center-implementation-plan.md
|
||||
- findings-ledger-implementation-plan.md
|
||||
- graph-implementation-plan.md
|
||||
- notify-implementation-plan.md
|
||||
|
||||
## What Was Consolidated
|
||||
|
||||
For each module, the following content was extracted from implementation_plan.md and added to README.md:
|
||||
|
||||
### Included in Implementation Status:
|
||||
- High-level delivery phases with status (Complete/In Progress/Planned/Not Started)
|
||||
- Key acceptance criteria (5-10 bullets max)
|
||||
- Technical decisions and risk mitigations (5-10 bullets max)
|
||||
- Epic alignments and milestones
|
||||
- Current workstreams and blocking items (where applicable)
|
||||
- Recent updates and operational assets
|
||||
|
||||
### Excluded from README.md:
|
||||
- Detailed task IDs and sprint tables
|
||||
- TODO/DOING/DONE status columns
|
||||
- Execution logs and sprint trackers
|
||||
- Granular work breakdown structures
|
||||
- Detailed test strategies (unit/integration/performance specifics)
|
||||
- Sprint readiness tracker tables
|
||||
|
||||
## Finding Current Status
|
||||
|
||||
For current implementation status, see:
|
||||
- Module-specific README.md: `docs/modules/{module}/README.md`
|
||||
- Sprint files: `docs/implplan/SPRINT_*.md`
|
||||
- Module task trackers: `docs/modules/{module}/TASKS.md` or `src/{Module}/**/TASKS.md`
|
||||
|
||||
## Rationale
|
||||
|
||||
Implementation plans contained extensive sprint-level detail that:
|
||||
1. Duplicated information in sprint trackers and TASKS.md files
|
||||
2. Created maintenance burden across multiple files
|
||||
3. Obscured high-level status for newcomers and operators
|
||||
|
||||
The consolidation provides:
|
||||
1. Single source of truth for high-level status in README.md
|
||||
2. Detailed sprint/task tracking remains in dedicated TASKS.md and sprint files
|
||||
3. Improved discoverability of module status and epic alignment
|
||||
4. Reduced documentation drift across files
|
||||
@@ -0,0 +1,19 @@
|
||||
# Implementation plan — Advisory AI
|
||||
|
||||
## Current objectives
|
||||
- Deliver Epic milestones summarised below while maintaining determinism and offline parity.
|
||||
- Keep documentation, telemetry, and runbooks aligned with sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Roadmap: reconcile open stories in ../../TASKS.md with module backlog.
|
||||
- Delivery: ship features outlined in the epic while preserving AOC guardrails.
|
||||
- Validation: extend tests/fixtures to guarantee reproducibility and provenance.
|
||||
|
||||
## Epic milestones
|
||||
- Epic 8: Advisory AI Assistant.
|
||||
- DOCS-AI stories to be tracked in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up work.
|
||||
- Sync with owners listed in sprint file `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
@@ -0,0 +1,167 @@
|
||||
# Implementation plan — Attestor
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Foundations**
|
||||
Build the Attestor service skeleton, DSSE bundle ingestion, mTLS/OpTok enforcement, Rekor v2 client, and cache the `{uuid,index,proof}` tuple. Publish base API (`POST /rekor/entries`, `GET /entries/{uuid}`) and Mongo schemas.
|
||||
- **Phase 2 – Policies & UI**
|
||||
Deliver verification policy authoring (Policy Studio integration), console views (evidence browser, verification reports, issuer management), and CLI verbs (`stella attest sign|verify|list|fetch`).
|
||||
- **Phase 3 – Scan & VEX support**
|
||||
Accept SBOM, ScanResults, VEX, and PolicyEvaluation predicates; integrate with Scanner, Export Center, Excititor, and Policy Engine pipelines. Ensure AOC invariants on ingestion.
|
||||
- **Phase 4 – Transparency & keys**
|
||||
Add multi-log submission (primary + mirror), witness endorsements, KMS/HSM/FIDO2 drivers, key rotation/revocation workflows, and audit trails.
|
||||
- **Phase 5 – Bulk & air gap**
|
||||
Implement batch submission/verification, DSSE archival to CAS/object storage, export/import bundles for Offline Kit, and mirror transparency log snapshots.
|
||||
- **Phase 6 – Performance & hardening**
|
||||
Optimise cache usage, parallel verification (target ≥1 k envelopes/minute per worker), extend observability (metrics/logs/traces), fuzz parsers, and finalise incident playbooks.
|
||||
|
||||
## Work breakdown
|
||||
- **Attestor service & libraries**
|
||||
- DSSE validation pipeline (payload whitelist, signature verification, trust roots).
|
||||
- Rekor client with inclusion-proof acquisition, retry/backoff, mirroring controls.
|
||||
- Mongo repositories for entries, dedupe, audit; CAS storage for DSSE envelopes.
|
||||
- Batch submission/verification APIs, verification cache, deterministic serialization.
|
||||
- Observability hooks: metrics (`attestor_submission_total`, `attestor_verify_seconds`), structured logs, OpenTelemetry traces.
|
||||
- **Signer & Authority integration**
|
||||
- Enforce mTLS peer validation, Authority scope mapping (`attestor.write`, `attestor.verify`), and DPoP binding.
|
||||
- Provide signer identity attestation metadata consumed by Attestor.
|
||||
- **Policy & Console**
|
||||
- Extend Policy Studio with `VerificationPolicy` authoring, approvals, and simulated results.
|
||||
- Console workflows: Evidence browser, verification reports, chain-of-custody graph, key management UI, bulk verification screens.
|
||||
- **CLI & SDK**
|
||||
- `stella attest` command group (sign/verify/list/fetch/key management) with DSSE canonicalisation and cosign interoperability.
|
||||
- SDK helpers for DSSE envelope creation, verification, and proof inspection.
|
||||
- **Export Center & Offline Kit**
|
||||
- Export Center adapters for attestation bundles; CLI/Console flows to export & import evidence in air-gapped environments.
|
||||
- Offline Kit scripts for replaying verification, mirroring transparency logs, and reporting gaps.
|
||||
- **Security & key management**
|
||||
- KMS/HSM/FIDO2 driver abstraction, key rotation and revocation runbooks, witness endorsements, and revocation telemetry.
|
||||
- **Docs & training**
|
||||
- Update module dossier (overview, architecture, implementation plan), key management guides, transparency reference, CLI/Console documentation, and air-gap runbooks.
|
||||
|
||||
## Cross-module dependencies
|
||||
- **Policy Studio / Policy Engine:** verification policy artefacts, explain integration, remediation hints.
|
||||
- **Export Center:** attestation bundle export/import, provenance linking.
|
||||
- **Authority & Tenancy:** scopes, identity attestations, tenant-aware issuer catalogues.
|
||||
- **Notifications:** attestation success/failure events, key rotation alerts.
|
||||
- **Observability:** dashboards and alerting for signing/verification pipelines.
|
||||
|
||||
## Acceptance criteria
|
||||
- Service ingests DSSE envelopes for all supported predicate types, logs them to configured transparency logs, and returns proofs with deterministic hashes.
|
||||
- Verification APIs/CLI/UI validate signatures, inclusion proofs, and policy compliance; cached verification accelerates repeated checks.
|
||||
- Verification policies gate attestation usage, enforcing issuer, freshness, signature count, and witness requirements.
|
||||
- Export Center and Offline Kit workflows bundle attestations and replay verification offline.
|
||||
- Observability coverage includes metrics, traces, logs, audit events, and alert triggers for key compromise, log outages, and verification failure spikes.
|
||||
- Performance target met (≥1 k envelopes/minute per worker) with horizontal scaling.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Key compromise or leakage:** enforce hardware-backed keys, rotation procedures, revocation checks, and incident runbooks.
|
||||
- **Parser bugs / malformed DSSE:** fuzz DSSE and predicate schemas, strict schema validation, fail closed.
|
||||
- **Transparency outage:** mirror logs, support witness endorsements, queue submissions for retry with exponential backoff.
|
||||
- **Policy complexity:** ship curated starter policies, provide simulation tooling, and document common scenarios.
|
||||
- **Offline gaps:** archive bundles and proof material, surface gaps to operators, and document compensating controls.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** DSSE validation, Rekor client, dedupe logic, key drivers, policy enforcement.
|
||||
- **Integration:** submit/verify flows across predicate types, multi-log publishing, batch operations, CLI/UI end-to-end exercises.
|
||||
- **Security:** tenant isolation, scope enforcement, key rotation regression, tamper detection.
|
||||
- **Performance:** throughput benchmarks, cache hit-rate monitoring, large batch verification.
|
||||
- **Chaos:** inject Rekor outages, network failures, corrupt bundles; ensure graceful degradation and auditable alerts.
|
||||
|
||||
## Definition of done
|
||||
- Phased milestones delivered with telemetry, documentation, and runbooks in place.
|
||||
- CLI/Console parity verified; Offline Kit procedures validated in sealed environment.
|
||||
- Cross-module dependencies acknowledged in ./TASKS.md and ../../TASKS.md.
|
||||
- Documentation set refreshed (overview, architecture, key management, transparency, CLI/UI) with imposed rule statement.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs sprint: `docs/implplan/SPRINT_0313_0001_0001_docs_modules_attestor.md`; statuses mirrored in `docs/modules/attestor/TASKS.md`.
|
||||
- Observability evidence stub lives in `operations/observability.md` with Grafana placeholder under `operations/dashboards/`; finalize after next demo outputs.
|
||||
- ATTESTOR-OPS-0001 remains BLOCKED until next demo provides observability data; update sprint/TASKS when available.
|
||||
|
||||
---
|
||||
|
||||
## Sprint readiness tracker
|
||||
|
||||
> Last updated: 2025-11-27 (ATTESTOR-ENG-0001)
|
||||
|
||||
This section maps delivery phases to implementation sprints and tracks readiness checkpoints.
|
||||
|
||||
### Phase 1 — Foundations
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| ATTEST-73-001 | ✅ DONE (2025-11-25) | SPRINT_110_ingestion_evidence | Attestation claims builder verified; TRX archived. |
|
||||
| ATTEST-73-002 | ✅ DONE (2025-11-25) | SPRINT_110_ingestion_evidence | Internal verify endpoint validated; TRX archived. |
|
||||
| ATTEST-PLAN-2001 | ✅ DONE (2025-11-24) | SPRINT_0200_0001_0001_attestation_coord | Coordination plan published at `docs/modules/attestor/prep/2025-11-24-attest-plan-2001.md`. |
|
||||
| ELOCKER-CONTRACT-2001 | ✅ DONE (2025-11-24) | SPRINT_0200_0001_0001_attestation_coord | Evidence Locker contract published. |
|
||||
| KMSI-73-001/002 | ✅ DONE (2025-11-03) | SPRINT_100_identity_signing | KMS key management and FIDO2 profile. |
|
||||
|
||||
**Checkpoint:** Foundations complete — service skeleton, DSSE ingestion, Rekor client, and cache layer operational.
|
||||
|
||||
### Phase 2 — Policies & UI
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| POLICY-ATTEST-73-001 | ⏳ BLOCKED | SPRINT_0123_0001_0001_policy_reasoning | VerificationPolicy schema/persistence; awaiting prep artefact finalization. |
|
||||
| POLICY-ATTEST-73-002 | ⏳ BLOCKED | SPRINT_0123_0001_0001_policy_reasoning | Editor DTOs/validation; depends on 73-001. |
|
||||
| POLICY-ATTEST-74-001 | ⏳ BLOCKED | SPRINT_0123_0001_0001_policy_reasoning | Surface attestation reports; depends on 73-002. |
|
||||
| POLICY-ATTEST-74-002 | ⏳ BLOCKED | SPRINT_0123_0001_0001_policy_reasoning | Console report integration; depends on 74-001. |
|
||||
| CLI-ATTEST-73-001 | ⏳ BLOCKED | SPRINT_0201_0001_0001_cli_i | `stella attest sign` command; blocked by scanner analyzer issues. |
|
||||
| CLI-ATTEST-73-002 | ⏳ BLOCKED | SPRINT_0201_0001_0001_cli_i | `stella attest verify` command; depends on 73-001. |
|
||||
| CLI-ATTEST-74-001 | ⏳ BLOCKED | SPRINT_0201_0001_0001_cli_i | `stella attest list` command; depends on 73-002. |
|
||||
| CLI-ATTEST-74-002 | ⏳ BLOCKED | SPRINT_0201_0001_0001_cli_i | `stella attest fetch` command; depends on 74-001. |
|
||||
|
||||
**Checkpoint:** Policy Studio integration and Console verification views blocked on upstream schema/API deliverables.
|
||||
|
||||
### Phase 3 — Scan & VEX support
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| ATTEST-01-003 | ✅ DONE (2025-11-23) | SPRINT_110_ingestion_evidence | Excititor attestation payloads shipped on frozen bundle v1. |
|
||||
| CONCELIER-ATTEST-73-001 | ✅ DONE (2025-11-25) | SPRINT_110_ingestion_evidence | Core/WebService attestation suites executed. |
|
||||
| CONCELIER-ATTEST-73-002 | ✅ DONE (2025-11-25) | SPRINT_110_ingestion_evidence | Attestation verify endpoint validated. |
|
||||
|
||||
**Checkpoint:** Scan/VEX attestation payloads integrated; ingestion flows verified.
|
||||
|
||||
### Phase 4 — Transparency & keys
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-ATTEST-74-001 | ✅ DONE (2025-11-16) | SPRINT_0171_0001_0001_notifier_i | Notification templates for verification/key events created. |
|
||||
| NOTIFY-ATTEST-74-002 | 📝 TODO | SPRINT_0171_0001_0001_notifier_i | Wire notifications to key rotation/revocation; blocked on payload localization freeze. |
|
||||
| ATTEST-REPLAY-187-003 | 📝 TODO | SPRINT_187_evidence_locker_cli_integration | Wire Attestor/Rekor anchoring for replay manifests. |
|
||||
|
||||
**Checkpoint:** Key event notifications partially complete; witness endorsements and rotation workflows pending.
|
||||
|
||||
### Phase 5 — Bulk & air gap
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| EXPORT-ATTEST-74-001 | ⏳ BLOCKED | SPRINT_0162_0001_0001_exportcenter_i | Export job producing attestation bundles; needs EvidenceLocker DSSE layout. |
|
||||
| EXPORT-ATTEST-74-002 | ⏳ BLOCKED | SPRINT_0162_0001_0001_exportcenter_i | CI/offline kit integration; depends on 74-001. |
|
||||
| EXPORT-ATTEST-75-001 | ⏳ BLOCKED | SPRINT_0162_0001_0001_exportcenter_i | CLI `stella attest bundle verify/import`; depends on 74-002. |
|
||||
| EXPORT-ATTEST-75-002 | ⏳ BLOCKED | SPRINT_0162_0001_0001_exportcenter_i | Offline kit integration; depends on 75-001. |
|
||||
|
||||
**Checkpoint:** Bulk/air-gap workflows blocked awaiting Export Center contracts.
|
||||
|
||||
### Phase 6 — Performance & hardening
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| ATTEST-73-003 | 📝 TODO | SPRINT_302_docs_tasks_md_ii | Evidence documentation; waiting on ATEL0102 evidence. |
|
||||
| ATTEST-73-004 | 📝 TODO | SPRINT_302_docs_tasks_md_ii | Extended documentation; depends on 73-003. |
|
||||
|
||||
**Checkpoint:** Performance benchmarks and incident playbooks pending; observability coverage to be validated.
|
||||
|
||||
---
|
||||
|
||||
### Overall readiness summary
|
||||
|
||||
| Phase | Status | Blocking items |
|
||||
|-------|--------|----------------|
|
||||
| **1 – Foundations** | ✅ Complete | — |
|
||||
| **2 – Policies & UI** | ⏳ Blocked | POLICY-ATTEST-73-001 prep; CLI build issues |
|
||||
| **3 – Scan & VEX** | ✅ Complete | — |
|
||||
| **4 – Transparency & keys** | 🔄 In progress | NOTIFY-ATTEST-74-002 payload freeze |
|
||||
| **5 – Bulk & air gap** | ⏳ Blocked | EXPORT-ATTEST-74-001 contract |
|
||||
| **6 – Performance** | 📝 Not started | Upstream phase completion |
|
||||
|
||||
### Next actions
|
||||
1. Track POLICY-ATTEST-73-001 prep artefact publication (Sprint 0123).
|
||||
2. Resolve CLI build blockers to unblock CLI-ATTEST-73-001 (Sprint 0201).
|
||||
3. Complete NOTIFY-ATTEST-74-002 wiring once payload localization freezes (Sprint 0171).
|
||||
4. Monitor Export Center contract finalization for Phase 5 tasks (Sprint 0162).
|
||||
@@ -0,0 +1,101 @@
|
||||
# Implementation plan — Authority
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** deliver OpTok scopes, guardrails, and AOC verifier hooks for ingestion services.
|
||||
- **Epic 2 – Policy Engine & Editor:** support policy evaluator flows (device-code, client credentials, scope sandboxing).
|
||||
- **Epic 4 – Policy Studio:** provide registry/promotion signing, approvals, and fresh-auth prompts.
|
||||
- **Epic 14 – Identity & Tenancy:** implement tenant isolation, RBAC hierarchies, audit trails, and PoE integration.
|
||||
- Track additional work (DOCS-SEC-62-001, AUTH-POLICY-20-001/002) in ../../TASKS.md and src/Authority/**/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0314_0001_0001_docs_modules_authority.md`; statuses mirrored in `docs/modules/authority/TASKS.md`.
|
||||
- Observability assets remain in `operations/monitoring.md` with Grafana JSON `operations/grafana-dashboard.json` (offline import).
|
||||
- Authority readiness tracker (AUTHORITY-ENG-0001) delivered 2025-11-27; future updates should adjust both sprint and TASKS.
|
||||
|
||||
---
|
||||
|
||||
## Sprint readiness tracker
|
||||
|
||||
> Last updated: 2025-11-27 (AUTHORITY-ENG-0001)
|
||||
|
||||
This section maps epic milestones to implementation sprints and tracks readiness checkpoints.
|
||||
|
||||
### Epic 1 — AOC enforcement
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-SIG-26-001 | ✅ DONE (2025-10-29) | SPRINT_0143_0001_0001_signals | Signals scopes + AOC role templates; propagation validation complete. |
|
||||
| AUTH-AIRGAP-57-001 | ✅ DONE (2025-11-08) | SPRINT_100_identity_signing | Sealed-mode CI gating; refuses tokens when sealed install lacks confirmation. |
|
||||
|
||||
**Checkpoint:** AOC enforcement operational with guardrails and scope policies in place.
|
||||
|
||||
### Epic 2 — Policy Engine & Editor
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-DPOP-11-001 | ✅ DONE (2025-11-08) | SPRINT_100_identity_signing | DPoP validation on `/token` grants; interactive tokens inherit `cnf.jkt`. |
|
||||
| AUTH-MTLS-11-002 | ✅ DONE (2025-11-08) | SPRINT_100_identity_signing | Refresh grants enforce original client cert; `x5t#S256` metadata persisted. |
|
||||
|
||||
**Checkpoint:** DPoP and mTLS sender-constraint flows operational.
|
||||
|
||||
### Epic 4 — Policy Studio
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-PACKS-43-001 | ✅ DONE (2025-11-09) | SPRINT_100_identity_signing | Pack signing policies, approval RBAC, CLI CI token scopes, audit logging. |
|
||||
|
||||
**Checkpoint:** Pack signing and approval flows with fresh-auth prompts complete.
|
||||
|
||||
### Epic 14 — Identity & Tenancy
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-TEN-47-001 | ✅ Contract published | SPRINT_0115_0001_0004_concelier_iv | Tenant-scope contract at `docs/modules/authority/tenant-scope-47-001.md`. |
|
||||
| AUTH-CRYPTO-90-001 | 🔄 DOING | SPRINT_0514_0001_0001_sovereign_crypto | Sovereign signing provider; key-loading path migration in progress. |
|
||||
|
||||
**Checkpoint:** Tenancy contract published; sovereign crypto provider integration in progress.
|
||||
|
||||
### Future tasks
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-REACH-401-005 | 📝 TODO | SPRINT_0401_0001_0001_reachability_evidence_chain | DSSE predicate types for SBOM/Graph/VEX/Replay; blocked on predicate definitions. |
|
||||
| AUTH-VERIFY-186-007 | 📝 TODO | SPRINT_186_record_deterministic_execution | Verification helper for DSSE signatures and Rekor proofs; awaits provenance harness. |
|
||||
|
||||
**Checkpoint:** Attestation predicate support and verification helpers pending upstream dependencies.
|
||||
|
||||
---
|
||||
|
||||
### Overall readiness summary
|
||||
|
||||
| Epic | Status | Blocking items |
|
||||
|------|--------|----------------|
|
||||
| **1 – AOC enforcement** | ✅ Complete | — |
|
||||
| **2 – Policy Engine & Editor** | ✅ Complete | — |
|
||||
| **4 – Policy Studio** | ✅ Complete | — |
|
||||
| **14 – Identity & Tenancy** | 🔄 In progress | AUTH-CRYPTO-90-001 provider contract |
|
||||
| **Future (Attestation)** | 📝 Not started | DSSE predicate schema; provenance harness |
|
||||
|
||||
### Cross-module dependencies
|
||||
|
||||
| Dependency | Required by | Status |
|
||||
|------------|-------------|--------|
|
||||
| Signals scope propagation | AUTH-SIG-26-001 | ✅ Validated |
|
||||
| Sealed-mode CI evidence | AUTH-AIRGAP-57-001 | ✅ Implemented |
|
||||
| DSSE predicate definitions | AUTH-REACH-401-005 | Schema draft pending |
|
||||
| Provenance harness (PROB0101) | AUTH-VERIFY-186-007 | In progress |
|
||||
| Sovereign crypto keystore plan | AUTH-CRYPTO-90-001 | ✅ Prep published |
|
||||
|
||||
### Next actions
|
||||
1. Complete AUTH-CRYPTO-90-001 provider registry wiring (Sprint 0514).
|
||||
2. Coordinate DSSE predicate schema with Signer guild for AUTH-REACH-401-005 (Sprint 0401).
|
||||
3. Monitor PROB0101 provenance harness for AUTH-VERIFY-186-007 (Sprint 186).
|
||||
22
docs/_archive/implementation-plans/ci-implementation-plan.md
Normal file
22
docs/_archive/implementation-plans/ci-implementation-plan.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# Implementation plan — CI Recipes
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** ensure pipelines enforce schemas, provenance, and verifier jobs.
|
||||
- **Epic 10 – Export Center:** add export/signing/Offline Kit automation templates.
|
||||
- **Epic 11 – Notifications Studio:** document CI hooks for notification previews/tests.
|
||||
- Track DOCS-CI stories in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Mirror task status changes in `./TASKS.md` and the owning sprint file.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change; record deterministic/offline considerations with each recipe addition.
|
||||
@@ -0,0 +1,24 @@
|
||||
# Implementation plan — CLI
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
- Documentation sync: keep module docs aligned with active sprint `docs/implplan/SPRINT_0316_0001_0001_docs_modules_cli.md`.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver deterministic policy verbs, simulation, and explain outputs.
|
||||
- **Epic 4 – Policy Studio:** add registry/promotion workflows, lint tooling, and approvals UX.
|
||||
- **Epic 6 – Vulnerability Explorer:** integrate ledger/triage operations.
|
||||
- **Epic 10 – Export Center:** automate export verification and Offline Kit flows.
|
||||
- **Epic 11 – Notifications Studio:** manage rule/channel authoring and previews via CLI.
|
||||
- Track CLI-specific work (e.g., CLI-CORE-41-001, DOCS-CLI-OBS-52-001) in ../../TASKS.md and src/Cli/**/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
@@ -0,0 +1,73 @@
|
||||
# Implementation plan — Concelier
|
||||
|
||||
## Delivery timeline
|
||||
- **Phase 1 — Guardrails & schema**
|
||||
Stand up PostgreSQL JSON schema validators for `advisory_raw` and `vex_raw`, wire the `AOCWriteGuard` repository interceptor, and seed deterministic linkset builders. Freeze legacy normalisation paths and migrate callers to the new raw schema.
|
||||
- **Phase 2 — API & observability**
|
||||
Publish ingestion and verification endpoints (`POST /ingest/*`, `GET /advisories.raw`, `POST /aoc/verify`) with Authority scopes, expose telemetry (`aoc_violation_total`, guard spans, structured logs), and ensure Offline Kit packaging captures validator deployment steps.
|
||||
- **Phase 3 — Experience polish**
|
||||
Ship CLI/Console affordances (`stella sources ingest --dry-run`, dashboard tiles, violation drill-downs), finish Export Center hand-off metadata, and close out CI enforcement (`stella aoc verify` preflight, AST lint, seeded fixtures).
|
||||
|
||||
## Work breakdown by component
|
||||
- **Concelier WebService & worker**
|
||||
- Add PostgreSQL validators and unique indexes over `(tenant, source.vendor, upstream.upstream_id, upstream.content_hash)`.
|
||||
- Implement write interceptors rejecting forbidden fields, missing provenance, or merge attempts.
|
||||
- Deterministically compute linksets and persist canonical JSON payloads.
|
||||
- Introduce `/ingest/advisory`, `/advisories/raw*`, and `/aoc/verify` surfaces guarded by `advisory:*` and `aoc:verify` scopes.
|
||||
- Emit guard metrics/traces and surface supersedes/violation audit logs.
|
||||
- **Excititor (shared ingestion contract)**
|
||||
- Mirror Concelier guard and schema changes for `vex_raw`.
|
||||
- Maintain restart-time plug-in determinism and linkset extraction parity.
|
||||
- **Shared libraries**
|
||||
- Publish `StellaOps.Ingestion.AOC` (forbidden key catalog, guard middleware, provenance helpers, signature verification).
|
||||
- Share error codes (`ERR_AOC_00x`) and deterministic hashing utilities.
|
||||
- **Policy Engine integration**
|
||||
- Enforce `effective_finding_*` write exclusivity.
|
||||
- Consume only raw documents + linksets, removing any implicit normalisation.
|
||||
- **Authority scopes**
|
||||
- Provision `advisory:ingest|read`, `vex:ingest|read`, `aoc:verify`; propagate tenant claims to ingestion services.
|
||||
- **CLI & Console**
|
||||
- Implement `stella sources ingest --dry-run` and `stella aoc verify` (with exit codes mapped to `ERR_AOC_00x`).
|
||||
- Surface AOC dashboards, violation drill-down, and verification shortcuts in the Console.
|
||||
- **CI/CD**
|
||||
- Add Roslyn analyzer / AST lint to block forbidden writes.
|
||||
- Seed fixtures and run `stella aoc verify` against snapshots in pipeline gating.
|
||||
|
||||
## Documentation deliverables
|
||||
- Update `docs/aoc/aggregation-only-contract.md` with guard invariants, schemas, error codes, and migration guidance.
|
||||
- Refresh `docs/modules/concelier/operations/*.md` (mirror, conflict-resolution, authority audit) with validator rollouts and observability dashboards.
|
||||
- Cross-link Authority scope definitions, CLI reference, Console sources guide, and observability runbooks to the AOC guard changes.
|
||||
- Ensure Offline Kit documentation captures validator bootstrap and verify workflows.
|
||||
|
||||
## Acceptance criteria
|
||||
- PostgreSQL validators and runtime guards reject forbidden fields and missing provenance with the documented `ERR_AOC_00x` codes.
|
||||
- Linksets and supersedes chains are deterministic; rerunning ingestion over identical payloads yields byte-identical documents.
|
||||
- CLI `stella aoc verify` exits non-zero on seeded violations and zero on clean datasets; Console dashboards show real-time guard status.
|
||||
- Export Center consumes advisory datasets without relying on legacy normalised fields.
|
||||
- CI fails if lint rules detect forbidden writes or if seeded guard tests regress.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Collector drift introduces new forbidden keys.** Mitigated by guard middleware + CI lint + schema validation; RFC required for linkset changes.
|
||||
- **Migration complexity from legacy normalisation.** Staged cutover with `_backup_*` copies and temporary views to keep Policy Engine parity.
|
||||
- **Performance overhead during ingest.** Guard remains O(number of keys); index review ensures insert latency stays within warm (<5 s) / cold (<30 s) targets.
|
||||
- **Tenancy leakage.** `tenant` required in schema, Authority-supplied claims enforced per request, observability alerts fire on missing tenant identifiers.
|
||||
|
||||
## Test strategy
|
||||
- **Unit**: guard rejection paths, provenance enforcement, idempotent insertions, linkset determinism.
|
||||
- **Property**: fuzz upstream payloads to guarantee no forbidden fields emerge.
|
||||
- **Integration**: batch ingest (50k advisories, mixed VEX fixtures), verifying zero guard violations and consistent supersedes.
|
||||
- **Contract**: Policy Engine consumers verify raw-only reads; Export Center consumes canonical datasets.
|
||||
- **End-to-end**: ingest/verify flow with CLI + Console actions to confirm observability and guard reporting.
|
||||
|
||||
## Definition of done
|
||||
- Validators deployed and verified in staging/offline environments.
|
||||
- Runtime guards, CLI/Console workflows, and CI linting all active.
|
||||
- Observability dashboards and runbooks updated; metrics visible.
|
||||
- Documentation updates merged; Offline Kit instructions published.
|
||||
- ./TASKS.md reflects status transitions; cross-module dependencies acknowledged in ../../TASKS.md.
|
||||
|
||||
## Readiness checkpoints (2025-11-25)
|
||||
- Sprint 110 attestation chain validated: `/internal/attestations/verify` endpoint and evidence bundle tests green (`TestResults/concelier-attestation/web.trx`, `core.trx`).
|
||||
- Link-Not-Merge cache + console consumption docs frozen (see `operations/lnm-cache-plan.md`, `operations/console-lnm-consumption.md`); cache headers remain deterministic.
|
||||
- Observation events transport reviewed; backlog guardrails and NATS/air-gap guidance updated in `operations/observation-events.md`.
|
||||
- Next gating dependency: TaskRunner contract drop (sprint 0157 blockers) before wiring approvals/pack ingest flows into Concelier.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Implementation plan — DevOps
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** ensure CI/CD guardrails, schema validation, and verifier pipelines are enforced.
|
||||
- **Epic 9 – Orchestrator Dashboard:** deliver dashboards, recovery runbooks, and rate-limit governance.
|
||||
- **Epic 10 – Export Center:** manage signing/promotions and Offline Kit bundle publishing.
|
||||
- **Epic 15 – Observability & Forensics:** coordinate telemetry deployments, evidence retention, and forensic automation.
|
||||
- Track module runbooks (DEVOPS-LAUNCH-18-001/900) and telemetry automation via ../../TASKS.md and ops/devops/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Implementation plan — Excititor
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** enforce immutable VEX observation schema, provenance capture, and guardrails.
|
||||
- **Epic 7 – VEX Consensus Lens:** provide lens-ready metadata (issuer trust, temporal scoping) and consensus APIs.
|
||||
- **Epic 8 – Advisory AI:** guarantee citation-ready payloads and normalized context for AI summaries/explainers.
|
||||
- Track DOCS-LNM-22-006/007 and CLI-EXC-25-001..002 in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
|
||||
| Sprint task | State (SPRINT_0333_0001_0001_docs_modules_excititor) | Notes |
|
||||
| --- | --- | --- |
|
||||
| EXCITOR-DOCS-0001 | DONE | README release alignment + consensus beta references refreshed (DSSE/export guidance). |
|
||||
| EXCITOR-ENG-0001 | DONE | Implementation plan mirrored to sprint status; TASKS board created. |
|
||||
| EXCITOR-OPS-0001 | DONE | Runbook/observability checklist added to `mirrors.md`. |
|
||||
| EXCITITOR-DOCS-0001 | BLOCKED | Waiting on chunk API CI validation + OpenAPI freeze. |
|
||||
| EXCITITOR-ENG-0001 | TODO | Will update engineering notes once DOCS unblock. |
|
||||
| EXCITITOR-OPS-0001 | TODO | Sync observability/runbook updates after OpenAPI freeze. |
|
||||
|
||||
See `/docs/implplan/SPRINT_0333_0001_0001_docs_modules_excititor.md` for the canonical status table.
|
||||
@@ -0,0 +1,71 @@
|
||||
# Implementation plan — Export Center
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – JSON & mirror foundations**
|
||||
Stand up the Export Center service + worker, deliver canonical JSON (`json:raw`, `json:policy`) and `mirror:full` profiles as download-only bundles, seed schema migrations, and publish manifest/provenance formats.
|
||||
- **Phase 2 – Trivy adapters & distribution**
|
||||
Implement Trivy DB / Java DB adapters, wire OCI/object storage distribution paths, and expose policy snapshot embedding + verification tooling.
|
||||
- **Phase 3 – Delta, encryption, scheduling**
|
||||
Release mirror deltas, bundle encryption, advanced scheduling/automation, resumable downloads, and CLI/Console verification workflows.
|
||||
|
||||
## Component work breakdown
|
||||
- **Service & worker**
|
||||
- Define migrations for `export_profiles`, `export_runs`, `export_inputs`, `export_distributions`.
|
||||
- Implement planner, adapter host, signing/attestation layer, distribution engines, and deterministic manifests.
|
||||
- Enforce tenant quotas, concurrency controls, and audit logging for create/cancel/distribute events.
|
||||
- **Adapters**
|
||||
- JSON adapters: canonical JSONL writers, redaction guardrails, compression (zstd).
|
||||
- Trivy adapters: field mapping, schema compatibility gating, validation suite.
|
||||
- Mirror adapters: filesystem/OCI layout, delta computation, optional encryption with manifest updates.
|
||||
- **Integrations**
|
||||
- Findings Ledger streaming APIs for advisories, VEX, SBOMs, findings.
|
||||
- Policy Engine deterministic snapshot endpoint; VEX Lens consensus snapshot.
|
||||
- Export Center telemetry surfaced through Observability stack.
|
||||
- **Surfaces**
|
||||
- Console: profiles CRUD, run wizard, run detail + verification panel, distribution dashboards.
|
||||
- CLI: `stella export profile|run|download|verify` with resumable downloads and signature verification.
|
||||
- **Security / RBAC**
|
||||
- Scope enforcement per tenant, role matrix coverage, encryption key rotation tests, redaction filters.
|
||||
- **Docs & ops**
|
||||
- Author module dossier (overview, architecture, profiles, API, CLI, mirror bundles, Trivy adapter, provenance & signing).
|
||||
- Produce runbooks (`docs/operations/export-runbook.md`) and hardening guidance (`docs/security/export-hardening.md`).
|
||||
|
||||
## Documentation deliverables
|
||||
- `docs/modules/export-center/overview.md` — responsibilities, profiles, surfaces.
|
||||
- `docs/modules/export-center/architecture.md` — service topology, adapters, manifests, distribution flow.
|
||||
- `docs/modules/export-center/profiles.md`, `trivy-adapter.md`, `mirror-bundles.md`, `provenance-and-signing.md`, `api.md`, `cli.md` — keep aligned with shipped features.
|
||||
- Cross-link Orchestrator, Policy, VEX Lens, CLI, and Offline Kit docs whenever exports become dependencies.
|
||||
|
||||
## Acceptance criteria
|
||||
- Operators can create, monitor, and download an export; `cosign verify` (and CLI verify) succeeds against manifest + provenance, mapping back to source artifacts.
|
||||
- Trivy bundles import cleanly into Trivy across supported versions; mirror bundles run in Offline Kit reference environment (full + delta).
|
||||
- Policy snapshot runs reproduce deterministic decisions and include embedded `policyVersion` + `inputsHash`.
|
||||
- Tenant scoping and RBAC block unauthorized actions; encryption-enabled bundles lock data to recipient keys.
|
||||
- Metrics (`exporter_run_duration_seconds`, `exporter_bundle_bytes_total`, `exporter_run_failures_total`) and dashboards reflect live runs; alerts trigger on sustained failure rates.
|
||||
- Retried runs remain idempotent: manifests, hashes, and distribution artefacts match across identical inputs.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Schema drift (Trivy / policy):** versioned adapters with compatibility gates, CI integration tests, fail-fast with actionable errors.
|
||||
- **Bundle bloat:** zstd compression, sharding, delta exports, OCI dedupe.
|
||||
- **Data leakage:** strict schema allowlists, tenancy filters, redaction enforcement, encryption options.
|
||||
- **Non-determinism:** embed policy snapshots, enforce deterministic ordering, include content hashes in manifest.
|
||||
- **Operational slowness:** streaming downloads with range support, resumable CLI, concurrency limits, retry policies for workers.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** adapter mapping, manifest hashing, signing/attestation, delta computation, encryption round-trips.
|
||||
- **Integration:** end-to-end runs for every profile, verification workflows, OCI push/pull, resume/abort scenarios.
|
||||
- **Compatibility:** matrix tests for Trivy versions, mirror bundle import in Offline Kit sample environment.
|
||||
- **Security:** tenant fuzzing, RBAC coverage, redaction/PII filters, key rotation.
|
||||
- **Performance & chaos:** stress exports with large datasets, simulate worker/API failures mid-run, confirm deterministic recovery.
|
||||
|
||||
## Definition of done
|
||||
- Service, worker, and adapters deployed with telemetry & alerting.
|
||||
- CLI & Console workflows published, Offline Kit instructions updated.
|
||||
- Documentation set listed above refreshed; imposed rule statements appended where required.
|
||||
- CI pipelines include schema validation, profile verification, and determinism checks.
|
||||
- ./TASKS.md + ../../TASKS.md reflect current status for in-flight stories.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs sprint: `docs/implplan/SPRINT_0320_0001_0001_docs_modules_export_center.md`; statuses mirrored in `docs/modules/export-center/TASKS.md`.
|
||||
- Observability evidence stub lives in `operations/observability.md` with Grafana placeholder under `operations/dashboards/`.
|
||||
- Bundle/profile/offline manifest guidance maintained in `devportal-offline*.md`, `mirror-bundles.md`, and `provenance-and-signing.md`; update sprint/TASKS if these change.
|
||||
@@ -0,0 +1,45 @@
|
||||
# Implementation Plan — Findings Ledger (Sprint 120)
|
||||
|
||||
## Phase 1 – Observability baselines (LEDGER-29-007)
|
||||
- Instrument writer/projector with metrics listed in `observability.md` (`ledger_write_latency_seconds`, `ledger_events_total`, `ledger_projection_lag_seconds`, etc.).
|
||||
- Emit structured logs (Serilog JSON) including chain/sequence/hash metadata.
|
||||
- Wire OTLP exporters, ensure `/metrics` endpoint exposes histogram buckets with exemplars.
|
||||
- Publish Grafana dashboards + alert rules (Policy SLO pack).
|
||||
- Deliver doc updates + sample Grafana JSON in repo (`docs/observability/dashboards/findings-ledger/`).
|
||||
|
||||
## Phase 2 – Determinism harness (LEDGER-29-008)
|
||||
- Finalize NDJSON fixtures for ≥5 M findings/tenant (per tenant/test scenario).
|
||||
- Implement `tools/LedgerReplayHarness` CLI as specified in `replay-harness.md`.
|
||||
- Add GitHub/Gitea pipeline job(s) running nightly (1 M) + weekly (5 M) harness plus DSSE signing.
|
||||
- Capture CPU/memory/latency metrics and commit signed reports for validation.
|
||||
- Provide runbook for QA + Ops to rerun harness in their environments.
|
||||
|
||||
## Phase 3 – Deployment & backup collateral (LEDGER-29-009)
|
||||
- Integrate ledger service into Compose (`docker-compose.prod.yaml`) and Helm values.
|
||||
- Automate PostgreSQL migrations (DatabaseMigrator invocation pre-start).
|
||||
- Document backup cadence (pg_dump + WAL archiving) and projection rebuild process (call harness).
|
||||
- Ensure Offline Kit packaging pulls binaries, migrations, harness, and default dashboards.
|
||||
|
||||
## Phase 4 – Provenance & air-gap extensions
|
||||
- LEDGER-34-101: ingest orchestrator run export metadata, index by artifact hash, expose audit endpoint.
|
||||
- LEDGER-AIRGAP-56/57/58: extend ledger events to capture bundle provenance, staleness metrics, timeline events.
|
||||
- LEDGER-ATTEST-73-001: store attestation pointers (DSSE IDs, Rekor metadata) for explainability.
|
||||
- For each extension, update schema doc + workflow inference doc to describe newly recorded fields and tenant-safe defaults.
|
||||
|
||||
## Dependencies & sequencing
|
||||
1. AdvisoryAI Sprint 110.A completion (raw findings parity).
|
||||
2. Observability schema approval (Nov 15) to unblock Phase 1 instrumentation.
|
||||
3. QA lab capacity for 5 M replay (Nov 18 checkpoint).
|
||||
4. DevOps review of Compose/Helm overlays (Nov 20).
|
||||
5. Orchestrator export schema freeze (Nov 25) for provenance linkage.
|
||||
|
||||
## Deliverables checklist
|
||||
- [ ] Metrics/logging/tracing implementation merged, dashboards exported.
|
||||
- [ ] Harness CLI + fixtures + signed reports committed.
|
||||
- [ ] Compose/Helm overlays + backup/restore runbooks validated.
|
||||
- [ ] Air-gap provenance fields documented + implemented.
|
||||
- [ ] Sprint tracker and release notes updated after each phase.
|
||||
|
||||
---
|
||||
|
||||
*Draft: 2025-11-13. Update when sequencing or dependencies change.*
|
||||
@@ -0,0 +1,65 @@
|
||||
# Implementation plan — Graph
|
||||
|
||||
## Delivery phases
|
||||
> Current active execution sprint: `docs/implplan/SPRINT_0141_0001_0001_graph_indexer.md` (Runtime & Signals 140.A).
|
||||
- **Phase 1 – Graph Indexer foundations**
|
||||
Stand up Graph Indexer service, node/edge schemas, ingestion from SBOM/Concelier/Excititor events, identity stability, and snapshot materialisation.
|
||||
- **Phase 2 – Graph API service**
|
||||
Expose search, query, path, impact, diff, and overlay endpoints with RBAC, cost controls, and streaming responses.
|
||||
- **Phase 3 – Console & CLI experiences**
|
||||
Ship Graph Explorer UI (WebGL canvas, filters, diff mode, overlays) and CLI (`stella sbom graph ...`) for automation pipelines.
|
||||
- **Phase 4 – Advanced analytics**
|
||||
Implement clustering, centrality, saved queries, overlay caching, and Policy Engine explain integration.
|
||||
- **Phase 5 – Exports & offline**
|
||||
Deliver GraphML/CSV/NDJSON exports, Offline Kit bundles (`nodes.jsonl`, `edges.jsonl`, overlays), and deterministic manifests.
|
||||
- **Phase 6 – Observability & hardening**
|
||||
Complete dashboards, alerts, runbooks, load/perf testing, and a11y/accessibility review.
|
||||
|
||||
## Work breakdown
|
||||
- **Services**
|
||||
- Graph Indexer: event consumers, node/edge builders, snapshot/version handling, aggregate metrics.
|
||||
- Graph API: validation, planner/cost guard, streaming tile engine, diff/overlay builder, exports.
|
||||
- Worker jobs: clustering, diff, overlay materialisation with backpressure awareness.
|
||||
- **Data model & storage**
|
||||
- Collections/tables (`graph_nodes`, `graph_edges`, `graph_snapshots`, `graph_saved_queries`, `graph_overlays_cache`), indexes, tenant partitioning, append-only change logs.
|
||||
- Evaluate document + adjacency vs graph DB abstraction; ensure deterministic serialization for exports.
|
||||
- **Console**
|
||||
- Feature module `graph-explorer` with routes, canvas renderer, panels, diff UI, saved queries, export workflows, a11y pass.
|
||||
- Telemetry instrumentation for user interactions and query budgets.
|
||||
- **CLI & SDK**
|
||||
- `stella sbom graph query|diff|impact|export`, with JSON schema and piping support.
|
||||
- SDK utilities for automation and CI pipelines.
|
||||
- **Policy & VEX integration**
|
||||
- Fetch explain traces for policy overlays, integrate VEX suppressions, align with Policy Engine & VEX Lens data models.
|
||||
- **Observability & Ops**
|
||||
- Metrics (ingest lag, query latency, cache hit rate), log/traces, dashboards, alerting for runaway queries and OOM.
|
||||
- Runbooks for incident classes (query denial, cache poisoning, degraded render).
|
||||
- **Documentation**
|
||||
- Maintain overview, API, query language, console guide, CLI reference, policy/VEX integration docs with compliance checklists.
|
||||
|
||||
## Acceptance criteria
|
||||
- Graph Indexer ingests SBOM/advisory/VEX events deterministically with tenant isolation and append-only provenance.
|
||||
- Graph API serves search/query/path/diff/overlay endpoints within budgeted latency and enforces cost limits + RBAC.
|
||||
- Console explorer visualises topology, overlays, diffs, saved queries; CLI commands mirror functionality for automation.
|
||||
- Exports (GraphML/CSV/NDJSON) and Offline Kit bundles reproduce snapshots and overlays with signed manifests.
|
||||
- Observability dashboards/alerts detect ingest lag, query failures, cache churn, and memory pressure; runbooks guide remediation.
|
||||
- Policy/VEX overlays align with Policy Engine explain traces and VEX suppressions.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Graph scale/complexity:** adopt adjacency compression, cached overlays, streaming pagination, enforced query budgets.
|
||||
- **Tenant bleed:** strict tenant filters, fuzz tests, data masking, compliance reviews.
|
||||
- **Runaway queries/visualization:** cost planner, query timeout, UI hints, safe mode renders.
|
||||
- **Cache poisoning:** input validation, schema versioning, eviction policies.
|
||||
- **Offline parity gaps:** deterministic export pipeline, integration tests for Offline Kit import.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** node/edge builders, identifier stability, overlay computations, query planner, diff engine.
|
||||
- **Integration:** end-to-end ingest + query flows across SBOM/advisory/VEX, saved query execution, diff exports.
|
||||
- **Performance:** large SBOM datasets, concurrency, memory profiling, WebGL rendering.
|
||||
- **Security:** tenant isolation tests, RBAC, query cost abuse.
|
||||
- **Offline:** export/import verification, manifest hashing, CLI replay.
|
||||
|
||||
## Definition of done
|
||||
- All phases delivered with telemetry, documentation, runbooks, and Offline Kit parity.
|
||||
- Console/CLI parity validated; a11y review complete.
|
||||
- ./TASKS.md and ../../TASKS.md updated; README/architecture/plan kept current with imposed rule references.
|
||||
160
docs/_archive/implementation-plans/notify-implementation-plan.md
Normal file
160
docs/_archive/implementation-plans/notify-implementation-plan.md
Normal file
@@ -0,0 +1,160 @@
|
||||
# Implementation plan — Notify
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Core rules engine & delivery ledger**
|
||||
Implement rules/channels schema, event ingestion, rule evaluation, idempotent deliveries, and audit logging.
|
||||
- **Phase 2 – Connectors & rendering**
|
||||
Ship Slack/Teams/Email/Webhook connectors, template rendering, localization, throttling, retries, and secret referencing.
|
||||
- **Phase 3 – Console & CLI authoring**
|
||||
Provide UI/CLI for rule authoring, previews, channel health, delivery browsing, digests, and test sends.
|
||||
- **Phase 4 – Governance & observability**
|
||||
Add approvals, RBAC, tenant quotas, Notify metrics/logs/traces, dashboards, Notify-specific alerts, and Notify runbooks.
|
||||
- **Phase 5 – Offline & compliance**
|
||||
Produce Offline Kit bundles (rules/channels/deploy scripts), signed exports, retention policies, and auditing for regulated environments.
|
||||
|
||||
## Work breakdown
|
||||
- **Service & worker**
|
||||
- REST API for rules/channels/delivery history, idempotency middleware, digest scheduler.
|
||||
- Worker pipelines for event intake, rule matching, template rendering, delivery execution, retries, and throttling.
|
||||
- Delivery ledger capturing payload metadata, response, retry state, DSSE signatures.
|
||||
- **Connectors**
|
||||
- Slack/Teams/Email/Webhook plug-ins with configuration validation, rate limiting, error classification.
|
||||
- Secrets referenced via Authority/Secret store; no plaintext storage.
|
||||
- **Console & CLI**
|
||||
- Console module for rules builder, condition editor, preview, test send, delivery insights, digests and schedule configuration.
|
||||
- CLI (`stella notify rule|channel|delivery`) for automation, export/import.
|
||||
- **Integrations**
|
||||
- Event sources: Concelier, Excititor, Policy Engine, Vuln Explorer, Export Center, Attestor, Zastava, Scheduler.
|
||||
- Notify events to Notify (meta) for failure escalations, accepted-risk expiration reminders.
|
||||
- **Observability & ops**
|
||||
- Metrics: delivery success/failure, retry counts, throttle hits, digest generation, channel health.
|
||||
- Logs/traces with tenant, rule ID, channel, correlation ID; dashboards and alerts.
|
||||
- Runbooks for misconfigured channels, throttling, event backlog, incident digest.
|
||||
- **Docs & compliance**
|
||||
- Update Notifications Studio guides, channel runbooks, security/RBAC docs, Offline Kit instructions.
|
||||
- Provide compliance checklist (audit logging, retention, opt-out).
|
||||
|
||||
## Acceptance criteria
|
||||
- Rules evaluate deterministically per event; deliveries idempotent with audit trail and DSSE signatures.
|
||||
- Channel connectors support retries, rate limits, health checks, previews; secrets referenced securely.
|
||||
- Console/CLI support rule creation, testing, digests, delivery browsing, and export/import workflows.
|
||||
- Observability dashboards track delivery health; alerts fire for sustained failures or backlog; runbooks cover remediation.
|
||||
- Offline Kit bundle contains configs, rules, digests, and deployment scripts for air-gapped installs.
|
||||
- Notify respects tenancy and RBAC; governance (approvals, change log) enforced for high-impact rules.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Notification storms:** throttling, digests, dedupe windows, preview/test gating.
|
||||
- **Secret compromise:** secret references only, rotation workflows, audit logging.
|
||||
- **Connector API changes:** versioned adapter layer, nightly health checks, fallback channels.
|
||||
- **Noise vs signal:** simulation previews, metrics, rule scoring, recommended defaults.
|
||||
- **Offline parity:** export/import of rules, connectors, and digests with signed manifests.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** rule evaluation, template rendering, connector clients, throttling, digests.
|
||||
- **Integration:** end-to-end events from core services, multi-channel routing, retries, audit logging.
|
||||
- **Performance:** burst throttling, digest creation, large rule sets.
|
||||
- **Security:** RBAC tests, tenant isolation, secret reference validation, DSSE signature verification.
|
||||
- **Offline:** export/import round-trips, Offline Kit deployment, manual delivery replay.
|
||||
|
||||
## Definition of done
|
||||
- Notify service, workers, connectors, Console/CLI, observability, and Offline Kit assets shipped with documentation and runbooks.
|
||||
- Compliance checklist appended to docs; ./TASKS.md and ../../TASKS.md updated with progress.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs sprint: `docs/implplan/SPRINT_322_docs_modules_notify.md`; statuses mirrored in `docs/modules/notify/TASKS.md`.
|
||||
- Observability evidence stub: `operations/observability.md` and `operations/dashboards/notify-observability.json` (to be populated after next demo outputs).
|
||||
- NOTIFY-DOCS-0002 remains blocked pending NOTIFY-SVC-39-001..004 (correlation/digests/simulation/quiet hours); keep sprint/TASKS synced when those land.
|
||||
|
||||
---
|
||||
|
||||
## Sprint readiness tracker
|
||||
|
||||
> Last updated: 2025-11-27 (NOTIFY-ENG-0001)
|
||||
|
||||
This section maps delivery phases to implementation sprints and tracks readiness checkpoints.
|
||||
|
||||
### Phase 1 — Core rules engine & delivery ledger
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-SVC-37-001 | ✅ DONE (2025-11-24) | SPRINT_0172_0001_0002_notifier_ii | Pack approval contract published (OpenAPI schema, payloads). |
|
||||
| NOTIFY-SVC-37-002 | ✅ DONE (2025-11-24) | SPRINT_0172_0001_0002_notifier_ii | Ingestion endpoint with Mongo persistence, idempotent writes, audit trail. |
|
||||
| NOTIFY-SVC-37-003 | 🔄 DOING | SPRINT_0172_0001_0002_notifier_ii | Approval/policy templates, routing predicates; dispatch/rendering pending. |
|
||||
| NOTIFY-SVC-37-004 | ✅ DONE (2025-11-24) | SPRINT_0172_0001_0002_notifier_ii | Acknowledgement API, test harness, metrics. |
|
||||
| NOTIFY-OAS-61-001 | ✅ DONE (2025-11-17) | SPRINT_0171_0001_0001_notifier_i | OAS with rules/templates/incidents/quiet hours endpoints. |
|
||||
| NOTIFY-OAS-61-002 | ✅ DONE (2025-11-17) | SPRINT_0171_0001_0001_notifier_i | `/.well-known/openapi` discovery endpoint. |
|
||||
| NOTIFY-OAS-62-001 | ✅ DONE (2025-11-17) | SPRINT_0171_0001_0001_notifier_i | SDK examples for rule CRUD. |
|
||||
| NOTIFY-OAS-63-001 | ✅ DONE (2025-11-17) | SPRINT_0171_0001_0001_notifier_i | Deprecation headers and templates. |
|
||||
|
||||
**Checkpoint:** Core rules engine mostly complete; template dispatch/rendering in progress.
|
||||
|
||||
### Phase 2 — Connectors & rendering
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-SVC-38-002 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Channel adapters (email, chat webhook, generic webhook) with retry policies. |
|
||||
| NOTIFY-SVC-38-003 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Template service, renderer with redaction and localization. |
|
||||
| NOTIFY-SVC-38-004 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | REST + WS APIs for rules CRUD, templates preview, incidents. |
|
||||
| NOTIFY-DOC-70-001 | ✅ DONE (2025-11-02) | SPRINT_0171_0001_0001_notifier_i | Architecture docs for `src/Notify` vs `src/Notifier` split. |
|
||||
|
||||
**Checkpoint:** Connector and rendering work not yet started; depends on Phase 1 completion.
|
||||
|
||||
### Phase 3 — Console & CLI authoring
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-SVC-39-001 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Correlation engine with throttler, quiet hours, incident lifecycle. |
|
||||
| NOTIFY-SVC-39-002 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Digest generator with schedule runner. |
|
||||
| NOTIFY-SVC-39-003 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Simulation engine for dry-run rules against historical events. |
|
||||
| NOTIFY-SVC-39-004 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Quiet hour calendars with audit logging. |
|
||||
|
||||
**Checkpoint:** Console/CLI authoring work not started; depends on Phase 2 completion.
|
||||
|
||||
### Phase 4 — Governance & observability
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-SVC-40-001 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Escalations, on-call schedules, PagerDuty/OpsGenie adapters. |
|
||||
| NOTIFY-SVC-40-002 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Summary storm breaker, localization bundles. |
|
||||
| NOTIFY-SVC-40-003 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Security hardening (signed ack links, webhook HMAC). |
|
||||
| NOTIFY-SVC-40-004 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Observability metrics/traces, dead-letter handling, chaos tests. |
|
||||
| NOTIFY-OBS-51-001 | ✅ DONE (2025-11-22) | SPRINT_0171_0001_0001_notifier_i | SLO evaluator webhooks with templates/routing/suppression. |
|
||||
| NOTIFY-OBS-55-001 | ✅ DONE (2025-11-22) | SPRINT_0171_0001_0001_notifier_i | Incident mode templates with evidence/trace/retention context. |
|
||||
| NOTIFY-ATTEST-74-001 | ✅ DONE (2025-11-16) | SPRINT_0171_0001_0001_notifier_i | Templates for verification failures, key revocations, transparency. |
|
||||
| NOTIFY-ATTEST-74-002 | 📝 TODO | SPRINT_0171_0001_0001_notifier_i | Wire notifications to key rotation/revocation events. |
|
||||
| NOTIFY-RISK-66-001 | ⏳ BLOCKED | SPRINT_0171_0001_0001_notifier_i | Risk severity escalation triggers; needs POLICY-RISK-40-002. |
|
||||
| NOTIFY-RISK-67-001 | ⏳ BLOCKED | SPRINT_0171_0001_0001_notifier_i | Risk profile publish/deprecate notifications. |
|
||||
| NOTIFY-RISK-68-001 | ⏳ BLOCKED | SPRINT_0171_0001_0001_notifier_i | Per-profile routing, quiet hours, dedupe. |
|
||||
|
||||
**Checkpoint:** Core observability complete; governance and risk notifications blocked on upstream dependencies.
|
||||
|
||||
### Phase 5 — Offline & compliance
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-AIRGAP-56-002 | ✅ DONE | SPRINT_0171_0001_0001_notifier_i | Bootstrap Pack with deterministic secrets and offline validation. |
|
||||
| NOTIFY-TEN-48-001 | ⏳ BLOCKED | SPRINT_0173_0001_0003_notifier_iii | Tenant-scope rules/templates; needs Sprint 0172 tenancy model. |
|
||||
|
||||
**Checkpoint:** Offline basics complete; tenancy work blocked on upstream Sprint 0172.
|
||||
|
||||
---
|
||||
|
||||
### Overall readiness summary
|
||||
|
||||
| Phase | Status | Blocking items |
|
||||
|-------|--------|----------------|
|
||||
| **1 – Core rules engine** | 🔄 In progress | NOTIFY-SVC-37-003 dispatch/rendering |
|
||||
| **2 – Connectors & rendering** | 📝 Not started | Phase 1 completion |
|
||||
| **3 – Console & CLI** | 📝 Not started | Phase 2 completion |
|
||||
| **4 – Governance & observability** | 🔄 Partial | POLICY-RISK-40-002 for risk notifications |
|
||||
| **5 – Offline & compliance** | 🔄 Partial | Sprint 0172 tenancy model |
|
||||
|
||||
### Cross-module dependencies
|
||||
|
||||
| Dependency | Required by | Status |
|
||||
|------------|-------------|--------|
|
||||
| Attestor payload localization | NOTIFY-ATTEST-74-002 | Freeze pending |
|
||||
| POLICY-RISK-40-002 export | NOTIFY-RISK-66/67/68 | BLOCKED |
|
||||
| Sprint 0172 tenancy model | NOTIFY-TEN-48-001 | In progress |
|
||||
| Telemetry SLO webhook schema | NOTIFY-OBS-51-001 | ✅ Published (`docs/modules/notify/slo-webhook-schema.md`) |
|
||||
|
||||
### Next actions
|
||||
1. Complete NOTIFY-SVC-37-003 dispatch/rendering wiring (Sprint 0172).
|
||||
2. Start NOTIFY-SVC-38-002 channel adapters once Phase 1 closes.
|
||||
3. Track POLICY-RISK-40-002 to unblock risk notification tasks.
|
||||
4. Monitor Sprint 0172 tenancy model for NOTIFY-TEN-48-001.
|
||||
@@ -0,0 +1,62 @@
|
||||
# Implementation plan — Source & Job Orchestrator
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Core service & job ledger**
|
||||
Implement source registry, run/job tables, queue abstraction, lease management, token-bucket rate limiting, watchdogs, and API primitives (`/sources`, `/runs`, `/jobs`).
|
||||
- **Phase 2 – Worker SDK & artifact registry**
|
||||
Embed worker SDK in Conseiller, Excititor, SBOM, Policy Engine; capture artifact metadata + hashes, enforce idempotency, publish progress/metrics.
|
||||
- **Phase 3 – Observability & dashboard**
|
||||
Ship metrics, traces, incident logging, SSE/WebSocket feeds, and Console dashboard (DAG/timeline, heatmaps, error clustering, SLO burn rate).
|
||||
- **Phase 4 – Controls & resilience**
|
||||
Deliver pause/resume/throttle/retry/backfill tooling, dead-letter review, circuit breakers, blackouts, backpressure handling, and automation hooks.
|
||||
- **Phase 5 – Offline & compliance**
|
||||
Generate deterministic audit bundles (`jobs.jsonl`, `history.jsonl`, `throttles.jsonl`), provenance manifests, and offline replay scripts.
|
||||
|
||||
## Work breakdown
|
||||
- **Service & persistence**
|
||||
- Postgres schema (`sources`, `runs`, `jobs`, `artifacts`, `dag_edges`, `quotas`, `schedules`, `incidents`).
|
||||
- Lease manager with heartbeats, retries, and dead-letter queues.
|
||||
- Token-bucket rate limiter per `{tenant, source.host}`; adaptive refill on upstream throttles.
|
||||
- Watermark/backfill orchestration for event-time windows.
|
||||
- **Worker SDK**
|
||||
- Claim/heartbeat/report contract, deterministic artifact hashing, idempotency enforcement.
|
||||
- Library release for .NET workers plus language bindings for Rust/Go ingestion agents.
|
||||
- **Control plane APIs**
|
||||
- CRUD for sources, runs, jobs, quotas, schedules; control actions (retry, cancel, prioritize, pause/resume, backfill).
|
||||
- SSE/WebSocket stream for Console updates.
|
||||
- **Observability**
|
||||
- Metrics: queue depth, job latency, failure classes, rate-limit hits, burn rate.
|
||||
- Error clustering (HTTP 429/5xx, schema mismatch, parse errors), incident logging with reason codes.
|
||||
- Gantt timeline and DAG JSON for Console visualisation.
|
||||
- **Console & CLI**
|
||||
- Console app sections: overview, sources, runs, job detail, incidents, throttles.
|
||||
- CLI commands: `stella orchestrator sources|runs|jobs|throttle|backfill`.
|
||||
- **Compliance & offline**
|
||||
- Immutable audit bundles with signatures; exports via Export Center; Offline Kit instructions.
|
||||
- Tenant isolation validation and secret redaction for logs/UI.
|
||||
|
||||
## Acceptance criteria
|
||||
- Orchestrator schedules all advisory/VEX/SBOM/policy jobs with quotas, rate limits, and idempotency; retries and replay preserve provenance.
|
||||
- Console dashboard reflects real-time DAG status, queue depth, SLO burn rate, and allows pause/resume/throttle/backfill with audit trail.
|
||||
- Worker SDK integrated across producer services, emitting progress and artifact metadata.
|
||||
- Observability stack exposes metrics, logs, traces, incidents, and alerts for stuck jobs, throttling, and failure spikes.
|
||||
- Offline audit bundles reproduce job history deterministically and verify signatures.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Backpressure/queue overload:** adaptive token buckets, circuit breakers, dynamic concurrency; degrade gracefully.
|
||||
- **Upstream vendor throttles:** throttle management with user-visible state, automatic jitter and retry.
|
||||
- **Tenant leakage:** enforce tenant filters at API/queue/storage, fuzz tests, redaction.
|
||||
- **Complex DAG errors:** built-in diagnostics, error clustering, partial replay tooling.
|
||||
- **Operator error:** confirmation prompts, RBAC, runbook guidance, reason codes logged.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** scheduling, quota enforcement, lease renewals, token bucket, watermark arithmetic.
|
||||
- **Integration:** worker SDK with Conseiller/Excititor/SBOM pipelines, pause/resume/backfill flows, failure recovery.
|
||||
- **Performance:** high-volume job workloads, queue backpressure, concurrency caps, dashboard SSE load tests.
|
||||
- **Chaos:** simulate upstream outages, stuck workers, clock skew, Postgres failover.
|
||||
- **Compliance:** audit bundle generation, signature verification, offline replay.
|
||||
|
||||
## Definition of done
|
||||
- All phases delivered with telemetry, dashboards, and runbooks published.
|
||||
- Console + CLI parity validated; Offline Kit instructions complete.
|
||||
- ./TASKS.md and ../../TASKS.md updated with status; documentation (README/architecture/this plan) reflects latest behaviour.
|
||||
@@ -0,0 +1,27 @@
|
||||
# Implementation plan — Platform
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** maintain authoritative guardrail docs, schemas, and verifier checklists.
|
||||
- **Epics 2 & 4 – Policy Engine/Studio:** coordinate platform-wide governance, approvals, and tenancy scopes.
|
||||
- **Epic 5 – SBOM Graph Explorer:** define shared contracts for graph indexing and overlays.
|
||||
- **Epics 6–11:** ensure cross-cutting contracts (Explorer, Lens, AI, Orchestrator, Notifications) stay aligned.
|
||||
- Track additional platform updates in ../../TASKS.md and docs/implplan/SPRINTS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs sprint: `docs/implplan/SPRINT_0324_0001_0001_docs_modules_platform.md`; statuses mirrored in `docs/modules/platform/TASKS.md`.
|
||||
- Keep links to `architecture-overview.md` and `../../07_HIGH_LEVEL_ARCHITECTURE.md` current; update both sprint and TASKS if platform guardrails change.
|
||||
- Platform is docs-only; ensure Offline Kit and AOC references remain discoverable from README/architecture.
|
||||
@@ -0,0 +1,67 @@
|
||||
# Implementation plan — Policy Engine
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Deterministic evaluation core**
|
||||
Finalise DSL compiler, runtime guardrails, evaluation workers, change-stream integration (advisories, VEX, SBOM), and append-only effective findings.
|
||||
- **Phase 2 – Orchestration & incremental runs**
|
||||
Implement run scheduler, incremental deltas, change-stream replay, simulation hooks, and determinism hashing.
|
||||
- **Phase 3 – Policy Studio workflows**
|
||||
Deliver policy registry, versioning, approvals, explain trace API, client editor integration, and signed promotion pipelines.
|
||||
- **Phase 4 – Simulation & approvals**
|
||||
Provide diff/simulation APIs, approval queues, change management, and integration with CLI/Console.
|
||||
- **Phase 5 – Exports & offline parity**
|
||||
Produce policy bundles, explain archives, Offline Kit assets, and deterministic manifests; integrate with Export Center.
|
||||
- **Phase 6 – Observability & hardening**
|
||||
Ship metrics, logs, traces, incident response runbooks, guardrail analyzers, and compliance attestations.
|
||||
|
||||
## Work breakdown
|
||||
- **Evaluation engine**
|
||||
- DSL compiler with caching, static analysis, and guard rails (no wall-clock/random/network outside allowlist).
|
||||
- Batch evaluator with deterministic ordering, change-stream inputs, policy IR caching.
|
||||
- Explain trace generation, evidence linking, storage in object store.
|
||||
- **Run orchestration**
|
||||
- Scheduler for incremental runs, job leasing, fair-share per tenant/policy.
|
||||
- Determinism hash + replay verification, time-travel snapshots, resume cursors.
|
||||
- Simulation endpoints returning diff summaries, rationale breakdown, exit codes.
|
||||
- **Policy Studio**
|
||||
- Policy registry (draft→review→approved), signed promotion pipeline, approvals workflow (multi-step).
|
||||
- Console integration (editor, simulation, approvals, explain viewer) and CLI parity.
|
||||
- **Integrations**
|
||||
- Inputs: Concelier, Excititor, SBOM Service, VEX Lens, runtime signals.
|
||||
- Outputs: Findings ledger, Vuln Explorer, Notify (policy events), Export Center (policy bundles).
|
||||
- Authority scopes, tenancy enforcement, RBAC for policy author/reviewer/operator.
|
||||
- **Observability & compliance**
|
||||
- Metrics: run duration, evaluation verdict counts, simulation latency, guard violations.
|
||||
- Logs/traces with trace ID propagation, policy version references, tenant scoping.
|
||||
- Guard analyzers (static + runtime), unit/property tests, compliance reports.
|
||||
- **Docs & tooling**
|
||||
- Update DSL guide, policy lifecycle/runbooks, simulation manual, CLI reference, Offline Kit instructions.
|
||||
- Provide sample policies, fixtures, and analyzer rules.
|
||||
|
||||
## Acceptance criteria
|
||||
- Evaluation engine deterministic across runs; effective findings materialised only by Policy Engine; guardrails prevent forbidden IO.
|
||||
- Incremental runs handle advisory/VEX/SBOM deltas within ≤5 min SLA; determinism hash and replay verification succeed.
|
||||
- Policy Studio supports draft/review/approval, signed promotions, simulation diffing, and explain traces in UI/CLI.
|
||||
- Exports (policy bundles, explain archives) reproducible with signed manifests; Offline Kit packages deliver same tooling.
|
||||
- Observability dashboards show run metrics, guard violations, simulation usage; alerts trigger on determinism hash mismatch or backlog.
|
||||
- CLI/Console parity for policy management, simulation, approvals, and export workflows.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Non-determinism:** strict static analysis, runtime guard, determinism hash, replay tests.
|
||||
- **Policy drift vs reality:** simulation diff previews, approval workflow, history/audit trail.
|
||||
- **Scaling evaluations:** sharded workers, incremental deltas, caching, job queue fairness.
|
||||
- **Guard bypass:** analyzers integrated into CI, runtime guard rejects forbidden operations.
|
||||
- **Offline compliance:** deterministic exports, manifest verification, documentation for sealed-mode deployments.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** DSL parsing, guard analyzer, evaluation pipeline, simulation diff calculations.
|
||||
- **Property:** randomised policy inputs verifying determinism and guard enforcement.
|
||||
- **Integration:** Concelier/Excititor/SBOM feeds → Policy Engine → findings ledger, simulation, approvals.
|
||||
- **Performance:** evaluation throughput, change-stream backlog recovery, simulation under load.
|
||||
- **Security/compliance:** RBAC/tenancy, analyzer enforcement, audit logging, signed promotions.
|
||||
- **Offline:** export/import of policy bundles, explain archives, CLI verification.
|
||||
|
||||
## Definition of done
|
||||
- Policy Engine core, orchestration, Policy Studio workflows, exports, and observability delivered with runbooks and Offline Kit parity.
|
||||
- Documentation suite (overview, architecture, DSL, lifecycle, Studio, simulation, CLI) updated with imposed rule statements.
|
||||
- ./TASKS.md and ../../TASKS.md reflect status; analyzers integrated into CI; compliance evidence captured.
|
||||
@@ -0,0 +1,20 @@
|
||||
# Implementation plan — Registry Token Service
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 10 – Export Center:** deliver signed registry token bundles supporting mirror/Offline Kit workflows.
|
||||
- **Epic 14 – Identity & Tenancy:** integrate tenant-aware scope validation, revocation, and audit trails.
|
||||
- Track follow-ups in ../../TASKS.md and src/Registry/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
@@ -0,0 +1,64 @@
|
||||
# Implementation plan — Scanner
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Control plane & job queue**
|
||||
Finalise Scanner WebService, queue abstraction (Valkey/NATS), job leasing, CAS layer cache, artifact catalog, and API endpoints.
|
||||
- **Phase 2 – Analyzer parity & SBOM assembly**
|
||||
Implement OS/Lang/Native analyzers, inventory/usage SBOM views, entry trace resolution, deterministic component identity.
|
||||
- **Phase 3 – Diff & attestations**
|
||||
Deliver three-way diff engine, DSSE SBOM/report signing pipeline, attestation hand-off (Signer→Attestor), metadata for Export Center.
|
||||
- **Phase 4 – Integrations & exports**
|
||||
Integrate with Policy Engine, Vuln Explorer, Export Center, CLI/Console; provide buildx plugin, CLI commands, and offline scanning support.
|
||||
- **Phase 5 – Observability & resilience**
|
||||
Metrics/logs/traces, queue backpressure handling, cache eviction, runbooks, smoke tests, SLO dashboards.
|
||||
|
||||
## Work breakdown
|
||||
- **Control plane**
|
||||
- REST API for scan requests, diff, catalog listing, artifact retrieval.
|
||||
- Queue service with idempotency, retries, dead-letter handling; worker scaling.
|
||||
- CAS storage (RustFS + S3 fallback), GC, ILM policies, offline mode.
|
||||
- **Analyzers**
|
||||
- OS (apk/dpkg/rpm), language (Java/Node/Python/Go/DotNet/Rust), native (ELF/PE/MachO).
|
||||
- Deterministic metadata (purl, version, source location), heuristics optional under flags.
|
||||
- Entry trace/usage analysis, dependency resolution, license detection.
|
||||
- **SBOM & diff**
|
||||
- Inventory/usage SBOM assembly, CycloneDX/SPDX emitters, schema validation.
|
||||
- Three-way diff (base, target, runtime), evidence linking, JSON export.
|
||||
- **Attestation & export**
|
||||
- DSSE bundle signing, attestation metadata for Signer/Attestor, provenance summary.
|
||||
- Export Center integration (SBOM/diff artifacts, manifests), CLI builder plugin (buildx).
|
||||
- **CLI/Console**
|
||||
- CLI commands `stella scan`, `stella sbom diff`, `stella sbom export`, offline caching.
|
||||
- Console flows for scan requests, diff viewer, SBOM downloads, attestation status.
|
||||
- **Observability & ops**
|
||||
- Metrics (queue depth, scan latency, cache hit/miss, analyzer timing), logs/traces with job IDs.
|
||||
- Alerts for backlog, failed scans, attestation issues, storage pressure.
|
||||
- Runbooks for stuck jobs, cache corruption, analyzer regressions, offline mode.
|
||||
|
||||
## Acceptance criteria
|
||||
- Scans produce deterministic SBOM inventory/usage views with component identity stability and reproducible diffs.
|
||||
- Queue/worker pipeline handles retries, backpressure, offline kits, and exports DSSE attestations for Signer/Attestor.
|
||||
- Export Center consumes SBOM/diff artifacts; Vuln Explorer receives metadata and explain traces.
|
||||
- CLI/Console parity for scan submission, diffing, exports, attestation verification.
|
||||
- Observability dashboards cover queue health, analyzer success rates, performance; alerts fire on SLO breaches.
|
||||
- Offline scanning (air-gapped) supported with local caches and manifest verification.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Analyzer drift/determinism:** golden fixtures, hash-based regression tests, deterministic sorting, strict identity rules.
|
||||
- **Queue overload:** adaptive backpressure, scaling workers, dead-letter review, priority lanes.
|
||||
- **Storage growth:** CAS dedupe, ILM policies, offline bundle pruning.
|
||||
- **Attestation failures:** retry with backoff, attestation health checks, Notify integration.
|
||||
- **Offline divergence:** packaging of analyzers/configs, manifest signatures, parity tests.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** analyzer parsers, component identity, diff calculations, API validation.
|
||||
- **Integration:** end-to-end scan/diff/attestation flows, Export Center integration, CLI automation.
|
||||
- **Performance:** large images, concurrent scans, cache stress, queue throughput.
|
||||
- **Determinism:** repeated scans/diffs across systems, hash comparisons, property tests.
|
||||
- **Security:** RBAC, tenant isolation, attestation key handling, path sanitisation.
|
||||
- **Offline:** air-gap scanning, manifest verification, CLI offline mode.
|
||||
|
||||
## Definition of done
|
||||
- Scanner services, analyzers, diffing, attestation pipeline, exports, and observability delivered with runbooks and Offline Kit parity.
|
||||
- Documentation (architecture, analyzer guides, CLI, offline mode, operations) updated with imposed rule statements.
|
||||
- ./TASKS.md and ../../TASKS.md updated with progress; regression fixtures maintained in repo.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Implementation plan — Scheduler
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver incremental policy run orchestration, change streams, and explain trace propagation.
|
||||
- **Epic 6 – Vulnerability Explorer:** ensure findings updates and remediation triggers integrate with scheduler outputs.
|
||||
- **Epic 9 – Orchestrator Dashboard:** provide job telemetry and control surfaces consumed by the orchestrator UI/CLI.
|
||||
- Track additional work (SCHED-MODELS-20-001, observability follow-ups) in ../../TASKS.md and src/Scheduler/**/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
136
docs/_archive/implementation-plans/signer-implementation-plan.md
Normal file
136
docs/_archive/implementation-plans/signer-implementation-plan.md
Normal file
@@ -0,0 +1,136 @@
|
||||
# Implementation plan — Signer
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Core service & PoE**
|
||||
Harden OpTok validation, Proof-of-Entitlement (PoE) checks, quota enforcement, scanner release verification, and DSSE signing pipeline (keyless + keyful).
|
||||
- **Phase 2 – Export Center integration**
|
||||
Produce signing bundles with provenance manifests for Export Center, deliver cosign-compatible outputs, and document verification workflows for offline exports.
|
||||
- **Phase 3 – Attestor alignment**
|
||||
Emit DSSE envelopes, metadata, and signer identity information required by Attestor (key metadata, certificate chains, bundle hashes); expose audit APIs.
|
||||
- **Phase 4 – Observability & resilience**
|
||||
Complete telemetry, throttling dashboards, audit trails, fallback key rotation, and offline kit packaging.
|
||||
|
||||
## Work breakdown
|
||||
- **Authentication & entitlement**
|
||||
- Enforce Authority-issued DPoP/mTLS tokens with `signer.sign` scope and tenant checks.
|
||||
- Integrate PoE introspection (cloud licensing) and plan quotas.
|
||||
- Validate scanner release signature via OCI referrers before signing reports/SBOMs.
|
||||
- **Signing pipeline**
|
||||
- Implement DSSE canonicalisation, keyless (Fulcio) and keyful (KMS/HSM/FIDO2) signing.
|
||||
- Support multi-signature output, certificate chain embedding, and deterministic bundle hashing.
|
||||
- Provide policy metadata (policy digest, view set) for report predicates.
|
||||
- **Integrations**
|
||||
- Coordinate with Attestor on bundle schema, signer identity payload, and error codes.
|
||||
- Expose Export Center-ready signing API (`/sign/export`) that returns manifest + signature tuple.
|
||||
- Surface CLI flows (`stella sign sbom/report`, offline verification helpers).
|
||||
- **Security & key management**
|
||||
- Key rotation and revocation runbooks; separation of signing keys per tenancy/plan.
|
||||
- Hardware-backed key support (HSM/FIDO2) with audit logging and attested builds.
|
||||
- **Observability**
|
||||
- Metrics: signing latency, PoE failures, quota hits, key usage distribution.
|
||||
- Structured logs with trace/context IDs, subject digests, issuer mode, decision outcome.
|
||||
- Alerts for PoE outages, key exhaustion, quota breach, signing failure spikes.
|
||||
- **Documentation & runbooks**
|
||||
- Update README/architecture/implementation plan, operator runbooks, offline verification guidance, and CLI reference.
|
||||
|
||||
## Acceptance criteria
|
||||
- Signer only signs requests that satisfy OpTok, PoE, quota, and scanner provenance checks.
|
||||
- DSSE outputs (keyless + keyful) verify with standard cosign tooling; Attestor logs them without additional transformation.
|
||||
- Export Center receives signed bundles, provenance manifests, and signature metadata to package exports deterministically.
|
||||
- Audit logs capture every signing request with tenant, issuer, subject digest, PoE state, and key source.
|
||||
- Observability dashboards/alerts reflect latency, failure rate, PoE status, and quota usage.
|
||||
- CLI/Offline workflows verify signatures using Offline Kit trust roots.
|
||||
|
||||
## Risks & mitigations
|
||||
- **PoE/entitlement outages:** cache last-known entitlement within TTL, provide emergency bypass toggles with audit trail.
|
||||
- **Key compromise:** enforce hardware-backed keys, rotation cadence, immediate revocation workflow, incident runbook.
|
||||
- **Release verification failures:** maintain allowlist for trusted scanner digests, fallback to manual approval with audit.
|
||||
- **Determinism drift:** canonicalise JSON, lock timestamp sources, regression tests for DSSE hashing.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** OpTok/PoE validation, quota enforcement, scanner signature verification, DSSE canonicalisation, multi-sig bundling.
|
||||
- **Integration:** end-to-end signing for SBOM, report, export artifacts; Attestor ingestion; Export Center bundle signing.
|
||||
- **Security:** fuzz signing inputs, simulate PoE tampering, ensure unauthorized actors are rejected.
|
||||
- **Performance:** signing throughput benchmarks (keyless vs keyful), quota pressure, concurrency checks.
|
||||
- **Offline:** verify signatures using Offline Kit trust roots and cosign CLI without network access.
|
||||
|
||||
## Definition of done
|
||||
- Signing pipeline deployed with observability and incident runbooks.
|
||||
- Export Center + Attestor dependencies validated; CLI parity confirmed.
|
||||
- Documentation updated (README, architecture, runbooks, CLI guides) with imposed rule compliance.
|
||||
- ./TASKS.md and ../../TASKS.md reflect the latest status transitions.
|
||||
|
||||
---
|
||||
|
||||
## Sprint readiness tracker
|
||||
|
||||
> Last updated: 2025-11-27 (SIGNER-ENG-0001)
|
||||
|
||||
This section maps delivery phases to implementation sprints and tracks readiness checkpoints.
|
||||
|
||||
### Phase 1 — Core service & PoE
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| KMSI-73-001 | ✅ DONE (2025-11-03) | SPRINT_100_identity_signing | KMS key management foundations with staffing + DSSE contract. |
|
||||
| KMSI-73-002 | ✅ DONE (2025-11-03) | SPRINT_100_identity_signing | FIDO2 profile integration. |
|
||||
| PROV-OBS-53-001 | ✅ DONE (2025-11-17) | SPRINT_0513_0001_0001_provenance | DSSE/SLSA BuildDefinition + BuildMetadata models with canonical JSON serializer. |
|
||||
| PROV-OBS-53-002 | ✅ DONE (2025-11-23) | SPRINT_0513_0001_0001_provenance | Signer abstraction (cosign/KMS/offline) with key rotation hooks and audit logging. |
|
||||
| SEC-CRYPTO-90-020 | 🔄 IN PROGRESS | SPRINT_0514_0001_0001_sovereign_crypto | CryptoPro signer plugin; Windows CSP runner pending. |
|
||||
|
||||
**Checkpoint:** Core signing infrastructure operational — KMS drivers, signer abstractions, and DSSE models delivered.
|
||||
|
||||
### Phase 2 — Export Center integration
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| PROV-OBS-53-003 | ✅ DONE (2025-11-23) | SPRINT_0513_0001_0001_provenance | PromotionAttestationBuilder feeding canonicalised payloads to Signer. |
|
||||
| SIGN-REPLAY-186-003 | 📝 TODO | SPRINT_186_record_deterministic_execution | Extend Signer/Authority DSSE flows for replay manifest/bundle payloads. |
|
||||
| SIGN-CORE-186-004 | 📝 TODO | SPRINT_186_record_deterministic_execution | Replace HMAC demo with StellaOps.Cryptography providers (keyless + KMS). |
|
||||
| SIGN-CORE-186-005 | 📝 TODO | SPRINT_186_record_deterministic_execution | Refactor SignerStatementBuilder for StellaOps predicate types. |
|
||||
| SIGN-TEST-186-006 | 📝 TODO | SPRINT_186_record_deterministic_execution | Upgrade signer integration tests with real crypto + fixture predicates. |
|
||||
|
||||
**Checkpoint:** Export Center signing APIs partially complete; replay manifest support and crypto provider refactoring pending.
|
||||
|
||||
### Phase 3 — Attestor alignment
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-REACH-401-005 | 📝 TODO | SPRINT_0401_0001_0001_reachability_evidence_chain | DSSE predicate types for SBOM/Graph/VEX/Replay; blocked on predicate definitions. |
|
||||
| SIGN-VEX-401-018 | 📝 TODO | SPRINT_0401_0001_0001_reachability_evidence_chain | Extend predicate catalog with `stella.ops/vexDecision@v1`. |
|
||||
| PROV-OBS-54-001 | 📝 TODO | SPRINT_0513_0001_0001_provenance | Verification library for DSSE signatures, Merkle roots, timeline chain. |
|
||||
| PROV-OBS-54-002 | 📝 TODO | SPRINT_0513_0001_0001_provenance | .NET global tool for local verification + CLI `stella forensic verify`. |
|
||||
|
||||
**Checkpoint:** Attestor DSSE alignment pending; predicate catalog extension and verification library not started.
|
||||
|
||||
### Phase 4 — Observability & resilience
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| DOCS-PROMO-70-001 | 📝 TODO | SPRINT_0304_0001_0004_docs_tasks_md_iv | Promotion attestations doc (CLI commands, Signer/Attestor integration, offline verification). |
|
||||
| CLI-PROMO-70-002 | 📝 TODO | SPRINT_203_cli_iii | `stella promotion attest` / `promotion verify` commands. |
|
||||
| CLI-FORENSICS-54-002 | 📝 TODO | SPRINT_202_cli_ii | `stella forensic attest show <artifact>` listing signer details. |
|
||||
|
||||
**Checkpoint:** Observability and CLI integration pending; waiting on upstream signing pipeline completion.
|
||||
|
||||
---
|
||||
|
||||
### Overall readiness summary
|
||||
|
||||
| Phase | Status | Blocking items |
|
||||
|-------|--------|----------------|
|
||||
| **1 – Core service & PoE** | ✅ Complete | — |
|
||||
| **2 – Export Center integration** | 🔄 In progress | SIGN-CORE-186-004/005 crypto provider refactoring |
|
||||
| **3 – Attestor alignment** | 📝 Not started | AUTH-REACH-401-005 predicate definitions |
|
||||
| **4 – Observability & resilience** | 📝 Not started | Upstream phase completion |
|
||||
|
||||
### Cross-module dependencies
|
||||
|
||||
| Dependency | Required by | Status |
|
||||
|------------|-------------|--------|
|
||||
| Attestor DSSE bundle schema | SIGN-VEX-401-018 | Documented in `docs/modules/attestor/architecture.md` §1 |
|
||||
| Provenance library canonicalisation | SIGN-CORE-186-005 | Available via PROV-OBS-53-001/002 |
|
||||
| Export Center bundle manifest | SIGN-REPLAY-186-003 | Pending Sprint 162/163 deliverables |
|
||||
| Authority predicate definitions | AUTH-REACH-401-005 | Schema draft pending |
|
||||
|
||||
### Next actions
|
||||
1. Complete CryptoPro signer plugin Windows smoke test (SEC-CRYPTO-90-020, Sprint 0514).
|
||||
2. Start SIGN-CORE-186-004 once replay bundle schema finalises (Sprint 186).
|
||||
3. Track AUTH-REACH-401-005 predicate schema draft for Attestor alignment (Sprint 401).
|
||||
4. Monitor PROV-OBS-54-001/002 for verification library availability.
|
||||
@@ -0,0 +1,69 @@
|
||||
# Implementation plan — Telemetry
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Collector & pipeline profiles**
|
||||
Publish OpenTelemetry collector configs (`default`, `forensic`, `airgap`), establish ingest gateways, TLS/mTLS, and attribute redaction policies.
|
||||
- **Phase 2 – Storage backends & retention**
|
||||
Deploy Prometheus/Tempo/Loki (or equivalents) with retention tiers, bucket/object storage, deterministic manifest generation, and sealed-mode allowlists.
|
||||
- **Phase 3 – Incident mode & forensic capture**
|
||||
Implement incident toggles (CLI/API), tail sampling adjustments, forensic bundle generation (OTLP archives, manifest/signature), and Notify hooks.
|
||||
- **Phase 4 – Observability dashboards & automation**
|
||||
Deliver dashboards (service SLOs, queue depth, policy latency), alert rules, Grafana packages, and CLI automation for deployment and capture.
|
||||
- **Phase 5 – Offline & compliance**
|
||||
Ship Offline Kit artefacts (collectors, configs, dashboards, replay tooling), signed bundles, and documentation for air-gapped review workflows.
|
||||
- **Phase 6 – Hardening & SOC handoff**
|
||||
Complete RBAC integration, audit logging, incident response runbooks, performance tuning, and integration tests across services.
|
||||
|
||||
## Work breakdown
|
||||
- **Collector configs**
|
||||
- Maintain config templates per profile with processors (redaction, batching, resource detection) and exporters.
|
||||
- CLI automation (`stella telemetry deploy`, `stella telemetry profile diff`), validation tests, and config signing.
|
||||
- **Storage & retention**
|
||||
- Provision Prometheus/Tempo/Loki (or vendor equivalents) with retention tiers (default, forensic, airgap).
|
||||
- Ensure determinism (chunk manifests, content hashing), remote-write allowlists, sealed/offline modes.
|
||||
- Implement archivers for forensic bundles (metrics/traces/logs) with cosign signatures.
|
||||
- **Incident mode**
|
||||
- API/CLI to toggle incident sampling, retention escalation, Notify signals, and auto bundle capture.
|
||||
- Hook into Orchestrator to respond to incidents and revert after cooldown.
|
||||
- **Dashboards & alerts**
|
||||
- Dashboard packages for core services (ingestion, policy, export, attestation).
|
||||
- Alert rules for SLO burn, collector failure, exporter backlog, bundle generation errors.
|
||||
- Self-observability metrics (`collector_export_failures_total`, `telemetry_incident_mode{}`).
|
||||
- **Offline support**
|
||||
- Offline Kit assets: collector binaries/configs, import scripts, dashboards, replay instructions, compliance checklists.
|
||||
- File-based exporters and manual transfer workflows with signed manifests.
|
||||
- **Docs & runbooks**
|
||||
- Update observability overview, forensic capture guide, incident response checklist, sealed-mode instructions, RBAC matrix.
|
||||
- SOC handoff package with control objectives and audit evidence.
|
||||
|
||||
## Acceptance criteria
|
||||
- Collectors ingest metrics/logs/traces across deployments, applying redaction rules and tenant isolation; profiles validate via CI.
|
||||
- Storage backends retain data per default/forensic/airgap SLAs with deterministic chunk manifests and sealed-mode compliance.
|
||||
- Incident mode toggles sampling to 100 %, extends retention, triggers Notify, and captures forensic bundles signed with cosign.
|
||||
- Dashboards and alerts cover service SLOs, queue depth, policy latency, ingestion violations, and telemetry stack health.
|
||||
- CLI commands (`stella telemetry deploy/capture/status`) automate config rollout, forensic capture, and verification.
|
||||
- Offline bundles replay telemetry in sealed environments using provided scripts and manifests.
|
||||
|
||||
## Risks & mitigations
|
||||
- **PII leakage:** strict redaction processors, policy-managed allowlists, audit tests.
|
||||
- **Collector overload:** horizontal scaling, batching, circuit breakers, incident mode throttling.
|
||||
- **Storage cost:** tiered retention, compression, pruning policies, offline archiving.
|
||||
- **Air-gap drift:** offline kit refresh schedule, deterministic manifest verification.
|
||||
- **Alert fatigue:** burn-rate alerts, deduping, SOC runbooks.
|
||||
|
||||
## Test strategy
|
||||
- **Config lint/tests:** schema validation, unit tests for processors/exporters, golden configs.
|
||||
- **Integration:** simulate service traces/logs/metrics, verify pipelines, incident toggles, bundle generation.
|
||||
- **Performance:** load tests with peak ingestion, long retention windows, failover scenarios.
|
||||
- **Security:** redaction verification, RBAC/tenant scoping, sealed-mode tests, signed config verification.
|
||||
- **Offline:** capture bundles, transfer, replay, compliance attestation.
|
||||
|
||||
## Definition of done
|
||||
- Collector profiles, storage backends, incident mode, dashboards, CLI, and offline kit delivered with telemetry and documentation.
|
||||
- Runbooks and SOC handoff packages published; compliance checklists appended.
|
||||
- ./TASKS.md and ../../TASKS.md updated; imposed rule statements confirmed in documentation.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0330_0001_0001_docs_modules_telemetry.md`; statuses mirrored in `docs/modules/telemetry/TASKS.md`.
|
||||
- Observability evidence lives in `operations/observability.md` with Grafana JSON stub under `operations/dashboards/`.
|
||||
- Keep future doc/ops updates mirrored across sprint, TASKS, and module front doors to avoid drift.
|
||||
30
docs/_archive/implementation-plans/ui-implementation-plan.md
Normal file
30
docs/_archive/implementation-plans/ui-implementation-plan.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# Implementation plan — Console UI
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver policy editor simulation and explain UX.
|
||||
- **Epic 4 – Policy Studio:** build registry, approvals, and promotion experiences.
|
||||
- **Epic 5 – SBOM Graph Explorer:** implement graph navigation, overlays, and diff views.
|
||||
- **Epic 6 – Vulnerability Explorer:** surface triage dashboards, findings ledger, and audit exports.
|
||||
- **Epic 8 – Advisory AI:** integrate advisory summaries and remediation hints with strict provenance.
|
||||
- **Epic 9 – Orchestrator Dashboard:** expose job/source monitoring controls.
|
||||
- **Epic 11 – Notifications Studio:** provide notifications workspace with previews and audit trails.
|
||||
- Track supporting tasks (e.g., DOCS-CONSOLE-23-001, CONSOLE-OBS-52-001) in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Current doc refresh tracked in `docs/implplan/SPRINT_0331_0001_0001_docs_modules_ui.md`; statuses mirrored in `docs/modules/ui/TASKS.md`.
|
||||
- Observability evidence for latest demo lives in `operations/observability.md` with Grafana JSON stub under `operations/dashboards/`.
|
||||
- Keep future doc/ops changes mirrored across sprint, TASKS, and module front doors to prevent drift.
|
||||
@@ -0,0 +1,68 @@
|
||||
# Implementation plan — VEX Consensus Lens
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Core lens service**
|
||||
Build normalisation pipeline (CSAF/OpenVEX/CycloneDX), product mapping library, trust weighting functions, consensus algorithm, and persistence (`vex_consensus`, history, conflicts).
|
||||
- **Phase 2 – API & integrations**
|
||||
Expose `/vex/consensus` query/detail/simulate/export endpoints, integrate Policy Engine thresholds, Vuln Explorer UI chips, and VEX Lens change events.
|
||||
- **Phase 3 – Issuer Directory & signatures**
|
||||
Deliver issuer registry, key management, signature verification, RBAC, audit logs, and tenant overrides.
|
||||
- **Phase 4 – Console & CLI experiences**
|
||||
Ship Console module (lists, evidence table, quorum bar, conflicts, simulation drawer) and CLI commands (`stella vex consensus ...`) with export support.
|
||||
- **Phase 5 – Recompute & performance**
|
||||
Implement recompute scheduling (policy activation, Excitor deltas), caching, load tests (10M records/tenant), observability dashboards, and Offline Kit exports.
|
||||
|
||||
## Work breakdown
|
||||
- **VEX Lens service**
|
||||
- Normalise VEX payloads, maintain scope scores, compute consensus digest.
|
||||
- Trust weighting functions (issuer tier, freshness decay, scope quality) ingest the new `vex.provenance.*` contract emitted by Excititor connectors (provider weight/tier, cosign metadata, fingerprints) so connector-tuned trust flows all the way to consensus.
|
||||
- Idempotent workers for consensus projection and history tracking.
|
||||
- Conflict handling queue for manual review and notifications.
|
||||
- **Integrations**
|
||||
- Excitor: enrich VEX events with issuer hints, signatures, product trees, and now connector-supplied trust weights/tiers that Lens consumes directly.
|
||||
- Policy Engine: trust knobs, simulation endpoints, policy-driven recompute.
|
||||
- Vuln Explorer & Advisory AI: consensus badges, conflict surfacing.
|
||||
- **Issuer Directory**
|
||||
- CRUD for issuers/keys, audit logs, import CSAF publishers, tenant overrides.
|
||||
- Signature verification endpoints consumed by Lens.
|
||||
- **APIs & UX**
|
||||
- REST endpoints for query/detail/conflict export, trust weight updates.
|
||||
- Console module with filters, saved views, evidence table, simulation drawer.
|
||||
- CLI commands for list/show/simulate/export with JSON/CSV output.
|
||||
- **Observability & Ops**
|
||||
- Metrics (consensus latency, conflict rate, signature failures, cache hit rate), logs, traces.
|
||||
- Dashboards + runbooks for recompute storms, mapping failures, signature errors, quota breaches.
|
||||
- Offline exports for Export Center/Offline Kit.
|
||||
|
||||
## Acceptance criteria
|
||||
- Consensus results reproducible across supported VEX formats with deterministic digests and provenance.
|
||||
- Signature verification influences trust weights; unverifiable evidence is down-weighted without pipeline failure.
|
||||
- Policy simulations show quorum shifts without persisting state; Vuln Explorer consumes consensus signals.
|
||||
- Issuer Directory enforces RBAC, audit logs, and key rotation; CLI & Console parity achieved.
|
||||
- Recompute pipeline handles Excitor deltas and policy activations with backpressure and incident surfacing.
|
||||
- Observability dashboards/alerts cover ingestion lag, conflict spikes, signature failures, performance budgets (P95 < 500 ms for 100-row pages at 10M records/tenant).
|
||||
|
||||
## Risks & mitigations
|
||||
- **Product mapping ambiguity:** conservative scope scoring, manual overrides, surfaced warnings, policy review hooks.
|
||||
- **Issuer compromise:** signature verification, trust weighting, tenant overrides, revocation runbooks.
|
||||
- **Evidence storms:** batching, worker sharding, orchestrator rate limiting, priority queues.
|
||||
- **Performance degradation:** caching, indexing, load tests, quota enforcement.
|
||||
- **Offline gaps:** deterministic exports, manifest hashes, Offline Kit tests.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** normalisers, mapping, trust weights, consensus lattice, signature verification.
|
||||
- **Property:** randomised evidence sets verifying lattice commutativity and determinism.
|
||||
- **Integration:** Excitor → Lens → Policy/Vuln Explorer flow, issuer overrides, simulation.
|
||||
- **Performance:** large tenant datasets, cache behaviour, concurrency tests.
|
||||
- **Security:** RBAC, tenant scoping, signature tampering, issuer revocation.
|
||||
- **Offline:** export/import verification, CLI parity.
|
||||
|
||||
## Definition of done
|
||||
- Lens service, issuer directory, API/CLI/Console components deployed with telemetry and runbooks.
|
||||
- Documentation set (overview, algorithm, issuer directory, API, console, policy trust) updated with imposed rule statements.
|
||||
- ./TASKS.md and ../../TASKS.md reflect current status; Offline Kit parity confirmed.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0332_0001_0001_docs_modules_vex_lens.md`; statuses mirrored in `docs/modules/vex-lens/TASKS.md`.
|
||||
- Observability evidence lives in `runbooks/observability.md` with Grafana JSON stub under `runbooks/dashboards/`.
|
||||
- Keep future doc/ops updates mirrored across sprint, TASKS, and module front doors to avoid drift.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Implementation plan — Vulnerability Explorer
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Findings Ledger & resolver**
|
||||
Create append-only ledger, projector, ecosystem resolvers (npm/Maven/PyPI/Go/RPM/DEB), canonical advisory keys, and provenance hashing.
|
||||
- **Phase 2 – API & simulation**
|
||||
Ship Vuln Explorer API (list/detail/grouping/simulation), batch evaluation with Policy Engine rationales, and export orchestrator.
|
||||
- **Phase 3 – Console & CLI workflows**
|
||||
Deliver triage UI (assignments, comments, remediation plans, simulation bar), keyboard accessibility, and CLI commands (`stella vuln ...`) with JSON/CSV output.
|
||||
- **Phase 4 – Automation & integrations**
|
||||
Integrate Advisory AI hints, Zastava runtime exposure, Notify rules, Scheduler follow-up scans, and Graph Explorer deep links.
|
||||
- **Phase 5 – Exports & offline parity**
|
||||
Generate deterministic bundles (JSON, CSV, PDF, Offline Kit manifests), audit logs, and signed reports.
|
||||
- **Phase 6 – Observability & hardening**
|
||||
Complete dashboards (projection lag, MTTR, accepted-risk cadence), alerts, runbooks, performance tuning (5M findings/tenant), and security/RBAC validation.
|
||||
|
||||
## Work breakdown
|
||||
- **Findings Ledger**
|
||||
- Define event schema, Merkle root anchoring, append-only storage, history tables.
|
||||
- Projector to `finding_records` and `finding_history`, idempotent event processing, time travel snapshots.
|
||||
- Resolver pipelines referencing SBOM inventory deltas, policy outputs, VEX consensus, runtime signals.
|
||||
- **API & exports**
|
||||
- REST endpoints (`/v1/findings`, `/v1/findings/{id}`, `/actions`, `/reports`, `/exports`) with ABAC filters.
|
||||
- Simulation endpoint returning diffs, integration with Policy Engine batch evaluation.
|
||||
- Export jobs for JSON/CSV/PDF plus Offline Kit bundle assembly and signing.
|
||||
- **Console**
|
||||
- Feature module `vuln-explorer` with grid, filters, saved views, deep links, detail tabs (policy, evidence, paths, remediation).
|
||||
- Simulation drawer, delta chips, accepted-risk approvals, evidence bundle viewer.
|
||||
- Accessibility (keyboard navigation, ARIA), virtualization for large result sets.
|
||||
- **CLI**
|
||||
- Commands `stella vuln list|show|simulate|assign|accept-risk|verify-fix|export`.
|
||||
- Stable schemas for automation; piping support; tests for exit codes.
|
||||
- **Integrations**
|
||||
- Conseiller/Excitor: normalized advisory keys, linksets, evidence retrieval.
|
||||
- SBOM Service: inventory deltas with scope/runtime flags, safe version hints.
|
||||
- Notify: events for SLA breaches, accepted-risk expiries, remediation deadlines.
|
||||
- Scheduler: trigger rescans when remediation plan milestones complete.
|
||||
- **Observability & ops**
|
||||
- Metrics (open findings, MTTR, projection lag, export duration, SLA burn), logs/traces with correlation IDs.
|
||||
- Alerting on projector backlog, API 5xx spikes, export failures, accepted-risk nearing expiry.
|
||||
- Runbooks covering recompute storms, mapping errors, report issues.
|
||||
|
||||
## Acceptance criteria
|
||||
- Ledger/event sourcing reproduces historical states byte-for-byte; Merkle hashes verify integrity.
|
||||
- Resolver respects ecosystem semantics, scope, and runtime context; path evidence presented in UI/CLI.
|
||||
- Triage workflows (assignment, comments, accepted-risk) enforce justification and approval requirements with audit records.
|
||||
- Simulation returns policy diffs without mutating state; CLI/UI parity achieved for simulation and exports.
|
||||
- Exports and Offline Kit bundles reproducible with signed manifests and provenance; reports available in JSON/CSV/PDF.
|
||||
- Observability dashboards show green SLOs, alerts fire for projection lag or SLA burns, and runbooks documented.
|
||||
- RBAC/ABAC validated; attachments encrypted; tenant isolation guaranteed.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Advisory identity collisions:** strict canonicalization, linkset references, raw evidence access.
|
||||
- **Resolver inaccuracies:** property-based tests, path verification, manual override workflows.
|
||||
- **Projection lag/backlog:** autoscaling, queue backpressure, alerting, pause controls.
|
||||
- **Export size/performance:** streaming NDJSON, size estimators, chunked downloads.
|
||||
- **User confusion on suppression:** rationale tab, explicit badges, explain traces.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** resolver algorithms, state machine transitions, policy mapping, export builders.
|
||||
- **Integration:** ingestion → ledger → projector → API flow, simulation, Notify notifications.
|
||||
- **E2E:** Console triage scenarios, CLI flows, accessibility tests.
|
||||
- **Performance:** 5M findings/tenant, projection rebuild, export generation.
|
||||
- **Security:** RBAC/ABAC matrix, CSRF, attachment encryption, signed URL expiry.
|
||||
- **Determinism:** time-travel snapshots, export manifest hashing, Offline Kit replay.
|
||||
|
||||
## Definition of done
|
||||
- Services, UI/CLI, integrations, exports, and observability deployed with runbooks and Offline Kit parity.
|
||||
- Documentation suite (overview, using-console, API, CLI, findings ledger, policy mapping, VEX/SBOM integration, telemetry, security, runbooks, install) updated with imposed rule statement.
|
||||
- ./TASKS.md and ../../TASKS.md reflect active progress; compliance checklists appended where required.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0334_0001_0001_docs_modules_vuln_explorer.md` (Docs/OPS/ENG streams).
|
||||
- Observability runbook evidence lives under `runbooks/observability.md` with stub dashboard JSON for offline demos.
|
||||
- API/OpenAPI references: `api.md` and `openapi/vuln-explorer.v1.yaml` are the current contract drafts; keep Console/CLI generators synced to these.
|
||||
@@ -0,0 +1,24 @@
|
||||
# Implementation plan — Zastava
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Backlog references
|
||||
- ZASTAVA runtime tasks in ../../TASKS.md.
|
||||
- Webhook smoke tests tracked in src/Zastava/**/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0335_0001_0001_docs_modules_zastava.md`; statuses mirrored in `docs/modules/zastava/TASKS.md`.
|
||||
- Observability evidence lives in `operations/observability.md` with Grafana JSON stub under `operations/dashboards/`.
|
||||
- Surface.Env/Surface.Secrets contracts remain dependencies; align with platform docs before enabling sealed mode.
|
||||
Reference in New Issue
Block a user