docs consolidation work
This commit is contained in:
StellaOps Bot
@@ -49,6 +49,20 @@ Advisory AI is the retrieval-augmented assistant that synthesizes advisory and V
|
||||
- Renders plan metadata (cache key, prompt template, token budget), guardrail state, provenance hashes, signatures, and citations in a deterministic table view.
|
||||
- Honors `STELLAOPS_ADVISORYAI_URL` when set; otherwise the CLI reuses the backend URL and scopes requests via `X-StellaOps-Scopes`.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
**Current Phase:** Production-ready (Epic 8 - Advisory AI Assistant)
|
||||
|
||||
**Completed Milestones:**
|
||||
- RAG pipeline with Concelier/Excititor/VEX Lens integration
|
||||
- Guardrail enforcement and provenance tracking
|
||||
- Offline bundle packaging and air-gap support
|
||||
- CLI and API surfaces with deterministic outputs
|
||||
|
||||
**Active Workstreams:**
|
||||
- Ongoing: Documentation, telemetry, and runbook alignment with sprint outcomes
|
||||
- Epic 8 stories tracked in `../../TASKS.md`
|
||||
|
||||
## Epic alignment
|
||||
- Epic 8: Advisory AI Assistant.
|
||||
- DOCS-AI stories to be tracked in ../../TASKS.md.
|
||||
- Epic 8: Advisory AI Assistant
|
||||
- DOCS-AI stories tracked in `../../TASKS.md`
|
||||
|
||||
@@ -1,19 +0,0 @@
|
||||
# Implementation plan — Advisory AI
|
||||
|
||||
## Current objectives
|
||||
- Deliver Epic milestones summarised below while maintaining determinism and offline parity.
|
||||
- Keep documentation, telemetry, and runbooks aligned with sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Roadmap: reconcile open stories in ../../TASKS.md with module backlog.
|
||||
- Delivery: ship features outlined in the epic while preserving AOC guardrails.
|
||||
- Validation: extend tests/fixtures to guarantee reproducibility and provenance.
|
||||
|
||||
## Epic milestones
|
||||
- Epic 8: Advisory AI Assistant.
|
||||
- DOCS-AI stories to be tracked in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up work.
|
||||
- Sync with owners listed in sprint file `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
51
docs/modules/airgap/README.md
Normal file
51
docs/modules/airgap/README.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# AirGap
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/AirGap/`
|
||||
**Owner:** Platform Team
|
||||
|
||||
## Purpose
|
||||
|
||||
AirGap manages sealed knowledge snapshot export and import for offline/air-gapped deployments. Provides time-anchored snapshots with staleness policies, deterministic bundle creation, and secure import validation for complete offline operation.
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- `StellaOps.AirGap.Controller` - Snapshot orchestration and staleness enforcement
|
||||
- `StellaOps.AirGap.Importer` - Import validation and bundle ingestion
|
||||
|
||||
**Libraries:**
|
||||
- `StellaOps.AirGap.Policy` - Staleness policy evaluation
|
||||
- `StellaOps.AirGap.Time` - Time anchor validation and trust
|
||||
- `StellaOps.AirGap.Storage.Postgres` - PostgreSQL storage for snapshots
|
||||
- `StellaOps.AirGap.Storage.Postgres.Tests` - Storage integration tests
|
||||
|
||||
## Configuration
|
||||
|
||||
See `etc/airgap.yaml.sample` for configuration options.
|
||||
|
||||
Key settings:
|
||||
- Staleness policy (maxAgeHours, warnAgeHours, staleAction)
|
||||
- Time anchor requirements (requireTimeAnchor)
|
||||
- Per-content staleness budgets (advisories, VEX, packages, mitigations)
|
||||
- PostgreSQL connection (schema: `airgap`)
|
||||
- Export/import paths and validation rules
|
||||
|
||||
## Dependencies
|
||||
|
||||
- PostgreSQL (schema: `airgap`)
|
||||
- Authority (authentication)
|
||||
- ExportCenter (bundle creation)
|
||||
- Mirror (snapshot sources)
|
||||
- All data modules (Concelier, VexHub, SbomService, etc.)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Operations: `./operations/` (if exists)
|
||||
- Offline Kit: `../../24_OFFLINE_KIT.md`
|
||||
- Mirror: `../mirror/`
|
||||
- ExportCenter: `../export-center/`
|
||||
|
||||
## Current Status
|
||||
|
||||
Implemented with Controller for snapshot export and Importer for secure ingestion. Staleness policies enforce time-bound validity. Integrated with ExportCenter for bundle packaging and all data modules for content export/import.
|
||||
@@ -63,4 +63,28 @@ All predicates capture subjects, issuer metadata, policy context, materials, opt
|
||||
- **Epic 19 – Attestor Console:** console experience, verification APIs, issuer/key governance, transparency integration, and offline bundles.
|
||||
- **Epic 10 – Export Center:** provenance alignment so exports carry signed manifests and attestation bundles.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
**Delivery Phases:**
|
||||
- Phase 1 (Foundations) – Complete: service skeleton, DSSE ingestion, Rekor client, and cache layer operational
|
||||
- Phase 2 (Policies & UI) – Blocked: Policy Studio integration and CLI commands awaiting upstream dependencies
|
||||
- Phase 3 (Scan & VEX support) – Complete: SBOM, VEX, and scan result predicates integrated
|
||||
- Phase 4 (Transparency & keys) – In progress: key event notifications, witness endorsements, and rotation workflows
|
||||
- Phase 5 (Bulk & air gap) – Blocked: Export Center contract required for attestation bundle workflows
|
||||
- Phase 6 (Performance & hardening) – Not started: benchmarks and incident playbooks pending
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- Service ingests DSSE envelopes, logs to transparency, returns proofs with deterministic hashes
|
||||
- Verification APIs/CLI/UI validate signatures, inclusion proofs, policy compliance with caching
|
||||
- Performance target: ≥1k envelopes/minute per worker with horizontal scaling
|
||||
- Export Center and Offline Kit workflows bundle attestations for offline replay
|
||||
- Observability coverage: metrics, traces, logs, audit events, and alerts
|
||||
|
||||
**Key Risks & Mitigations:**
|
||||
- Key compromise: enforce hardware-backed keys, rotation procedures, revocation checks, incident runbooks
|
||||
- Parser bugs: fuzz DSSE/predicate schemas, strict validation, fail closed
|
||||
- Transparency outage: mirror logs, witness endorsements, queue submissions with exponential backoff
|
||||
- Policy complexity: ship starter policies, simulation tooling, documented scenarios
|
||||
- Offline gaps: archive bundles and proof material, surface gaps to operators, document compensating controls
|
||||
|
||||
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
|
||||
|
||||
@@ -1,167 +0,0 @@
|
||||
# Implementation plan — Attestor
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Foundations**
|
||||
Build the Attestor service skeleton, DSSE bundle ingestion, mTLS/OpTok enforcement, Rekor v2 client, and cache the `{uuid,index,proof}` tuple. Publish base API (`POST /rekor/entries`, `GET /entries/{uuid}`) and Mongo schemas.
|
||||
- **Phase 2 – Policies & UI**
|
||||
Deliver verification policy authoring (Policy Studio integration), console views (evidence browser, verification reports, issuer management), and CLI verbs (`stella attest sign|verify|list|fetch`).
|
||||
- **Phase 3 – Scan & VEX support**
|
||||
Accept SBOM, ScanResults, VEX, and PolicyEvaluation predicates; integrate with Scanner, Export Center, Excititor, and Policy Engine pipelines. Ensure AOC invariants on ingestion.
|
||||
- **Phase 4 – Transparency & keys**
|
||||
Add multi-log submission (primary + mirror), witness endorsements, KMS/HSM/FIDO2 drivers, key rotation/revocation workflows, and audit trails.
|
||||
- **Phase 5 – Bulk & air gap**
|
||||
Implement batch submission/verification, DSSE archival to CAS/object storage, export/import bundles for Offline Kit, and mirror transparency log snapshots.
|
||||
- **Phase 6 – Performance & hardening**
|
||||
Optimise cache usage, parallel verification (target ≥1 k envelopes/minute per worker), extend observability (metrics/logs/traces), fuzz parsers, and finalise incident playbooks.
|
||||
|
||||
## Work breakdown
|
||||
- **Attestor service & libraries**
|
||||
- DSSE validation pipeline (payload whitelist, signature verification, trust roots).
|
||||
- Rekor client with inclusion-proof acquisition, retry/backoff, mirroring controls.
|
||||
- Mongo repositories for entries, dedupe, audit; CAS storage for DSSE envelopes.
|
||||
- Batch submission/verification APIs, verification cache, deterministic serialization.
|
||||
- Observability hooks: metrics (`attestor_submission_total`, `attestor_verify_seconds`), structured logs, OpenTelemetry traces.
|
||||
- **Signer & Authority integration**
|
||||
- Enforce mTLS peer validation, Authority scope mapping (`attestor.write`, `attestor.verify`), and DPoP binding.
|
||||
- Provide signer identity attestation metadata consumed by Attestor.
|
||||
- **Policy & Console**
|
||||
- Extend Policy Studio with `VerificationPolicy` authoring, approvals, and simulated results.
|
||||
- Console workflows: Evidence browser, verification reports, chain-of-custody graph, key management UI, bulk verification screens.
|
||||
- **CLI & SDK**
|
||||
- `stella attest` command group (sign/verify/list/fetch/key management) with DSSE canonicalisation and cosign interoperability.
|
||||
- SDK helpers for DSSE envelope creation, verification, and proof inspection.
|
||||
- **Export Center & Offline Kit**
|
||||
- Export Center adapters for attestation bundles; CLI/Console flows to export & import evidence in air-gapped environments.
|
||||
- Offline Kit scripts for replaying verification, mirroring transparency logs, and reporting gaps.
|
||||
- **Security & key management**
|
||||
- KMS/HSM/FIDO2 driver abstraction, key rotation and revocation runbooks, witness endorsements, and revocation telemetry.
|
||||
- **Docs & training**
|
||||
- Update module dossier (overview, architecture, implementation plan), key management guides, transparency reference, CLI/Console documentation, and air-gap runbooks.
|
||||
|
||||
## Cross-module dependencies
|
||||
- **Policy Studio / Policy Engine:** verification policy artefacts, explain integration, remediation hints.
|
||||
- **Export Center:** attestation bundle export/import, provenance linking.
|
||||
- **Authority & Tenancy:** scopes, identity attestations, tenant-aware issuer catalogues.
|
||||
- **Notifications:** attestation success/failure events, key rotation alerts.
|
||||
- **Observability:** dashboards and alerting for signing/verification pipelines.
|
||||
|
||||
## Acceptance criteria
|
||||
- Service ingests DSSE envelopes for all supported predicate types, logs them to configured transparency logs, and returns proofs with deterministic hashes.
|
||||
- Verification APIs/CLI/UI validate signatures, inclusion proofs, and policy compliance; cached verification accelerates repeated checks.
|
||||
- Verification policies gate attestation usage, enforcing issuer, freshness, signature count, and witness requirements.
|
||||
- Export Center and Offline Kit workflows bundle attestations and replay verification offline.
|
||||
- Observability coverage includes metrics, traces, logs, audit events, and alert triggers for key compromise, log outages, and verification failure spikes.
|
||||
- Performance target met (≥1 k envelopes/minute per worker) with horizontal scaling.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Key compromise or leakage:** enforce hardware-backed keys, rotation procedures, revocation checks, and incident runbooks.
|
||||
- **Parser bugs / malformed DSSE:** fuzz DSSE and predicate schemas, strict schema validation, fail closed.
|
||||
- **Transparency outage:** mirror logs, support witness endorsements, queue submissions for retry with exponential backoff.
|
||||
- **Policy complexity:** ship curated starter policies, provide simulation tooling, and document common scenarios.
|
||||
- **Offline gaps:** archive bundles and proof material, surface gaps to operators, and document compensating controls.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** DSSE validation, Rekor client, dedupe logic, key drivers, policy enforcement.
|
||||
- **Integration:** submit/verify flows across predicate types, multi-log publishing, batch operations, CLI/UI end-to-end exercises.
|
||||
- **Security:** tenant isolation, scope enforcement, key rotation regression, tamper detection.
|
||||
- **Performance:** throughput benchmarks, cache hit-rate monitoring, large batch verification.
|
||||
- **Chaos:** inject Rekor outages, network failures, corrupt bundles; ensure graceful degradation and auditable alerts.
|
||||
|
||||
## Definition of done
|
||||
- Phased milestones delivered with telemetry, documentation, and runbooks in place.
|
||||
- CLI/Console parity verified; Offline Kit procedures validated in sealed environment.
|
||||
- Cross-module dependencies acknowledged in ./TASKS.md and ../../TASKS.md.
|
||||
- Documentation set refreshed (overview, architecture, key management, transparency, CLI/UI) with imposed rule statement.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs sprint: `docs/implplan/SPRINT_0313_0001_0001_docs_modules_attestor.md`; statuses mirrored in `docs/modules/attestor/TASKS.md`.
|
||||
- Observability evidence stub lives in `operations/observability.md` with Grafana placeholder under `operations/dashboards/`; finalize after next demo outputs.
|
||||
- ATTESTOR-OPS-0001 remains BLOCKED until next demo provides observability data; update sprint/TASKS when available.
|
||||
|
||||
---
|
||||
|
||||
## Sprint readiness tracker
|
||||
|
||||
> Last updated: 2025-11-27 (ATTESTOR-ENG-0001)
|
||||
|
||||
This section maps delivery phases to implementation sprints and tracks readiness checkpoints.
|
||||
|
||||
### Phase 1 — Foundations
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| ATTEST-73-001 | ✅ DONE (2025-11-25) | SPRINT_110_ingestion_evidence | Attestation claims builder verified; TRX archived. |
|
||||
| ATTEST-73-002 | ✅ DONE (2025-11-25) | SPRINT_110_ingestion_evidence | Internal verify endpoint validated; TRX archived. |
|
||||
| ATTEST-PLAN-2001 | ✅ DONE (2025-11-24) | SPRINT_0200_0001_0001_attestation_coord | Coordination plan published at `docs/modules/attestor/prep/2025-11-24-attest-plan-2001.md`. |
|
||||
| ELOCKER-CONTRACT-2001 | ✅ DONE (2025-11-24) | SPRINT_0200_0001_0001_attestation_coord | Evidence Locker contract published. |
|
||||
| KMSI-73-001/002 | ✅ DONE (2025-11-03) | SPRINT_100_identity_signing | KMS key management and FIDO2 profile. |
|
||||
|
||||
**Checkpoint:** Foundations complete — service skeleton, DSSE ingestion, Rekor client, and cache layer operational.
|
||||
|
||||
### Phase 2 — Policies & UI
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| POLICY-ATTEST-73-001 | ⏳ BLOCKED | SPRINT_0123_0001_0001_policy_reasoning | VerificationPolicy schema/persistence; awaiting prep artefact finalization. |
|
||||
| POLICY-ATTEST-73-002 | ⏳ BLOCKED | SPRINT_0123_0001_0001_policy_reasoning | Editor DTOs/validation; depends on 73-001. |
|
||||
| POLICY-ATTEST-74-001 | ⏳ BLOCKED | SPRINT_0123_0001_0001_policy_reasoning | Surface attestation reports; depends on 73-002. |
|
||||
| POLICY-ATTEST-74-002 | ⏳ BLOCKED | SPRINT_0123_0001_0001_policy_reasoning | Console report integration; depends on 74-001. |
|
||||
| CLI-ATTEST-73-001 | ⏳ BLOCKED | SPRINT_0201_0001_0001_cli_i | `stella attest sign` command; blocked by scanner analyzer issues. |
|
||||
| CLI-ATTEST-73-002 | ⏳ BLOCKED | SPRINT_0201_0001_0001_cli_i | `stella attest verify` command; depends on 73-001. |
|
||||
| CLI-ATTEST-74-001 | ⏳ BLOCKED | SPRINT_0201_0001_0001_cli_i | `stella attest list` command; depends on 73-002. |
|
||||
| CLI-ATTEST-74-002 | ⏳ BLOCKED | SPRINT_0201_0001_0001_cli_i | `stella attest fetch` command; depends on 74-001. |
|
||||
|
||||
**Checkpoint:** Policy Studio integration and Console verification views blocked on upstream schema/API deliverables.
|
||||
|
||||
### Phase 3 — Scan & VEX support
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| ATTEST-01-003 | ✅ DONE (2025-11-23) | SPRINT_110_ingestion_evidence | Excititor attestation payloads shipped on frozen bundle v1. |
|
||||
| CONCELIER-ATTEST-73-001 | ✅ DONE (2025-11-25) | SPRINT_110_ingestion_evidence | Core/WebService attestation suites executed. |
|
||||
| CONCELIER-ATTEST-73-002 | ✅ DONE (2025-11-25) | SPRINT_110_ingestion_evidence | Attestation verify endpoint validated. |
|
||||
|
||||
**Checkpoint:** Scan/VEX attestation payloads integrated; ingestion flows verified.
|
||||
|
||||
### Phase 4 — Transparency & keys
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-ATTEST-74-001 | ✅ DONE (2025-11-16) | SPRINT_0171_0001_0001_notifier_i | Notification templates for verification/key events created. |
|
||||
| NOTIFY-ATTEST-74-002 | 📝 TODO | SPRINT_0171_0001_0001_notifier_i | Wire notifications to key rotation/revocation; blocked on payload localization freeze. |
|
||||
| ATTEST-REPLAY-187-003 | 📝 TODO | SPRINT_187_evidence_locker_cli_integration | Wire Attestor/Rekor anchoring for replay manifests. |
|
||||
|
||||
**Checkpoint:** Key event notifications partially complete; witness endorsements and rotation workflows pending.
|
||||
|
||||
### Phase 5 — Bulk & air gap
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| EXPORT-ATTEST-74-001 | ⏳ BLOCKED | SPRINT_0162_0001_0001_exportcenter_i | Export job producing attestation bundles; needs EvidenceLocker DSSE layout. |
|
||||
| EXPORT-ATTEST-74-002 | ⏳ BLOCKED | SPRINT_0162_0001_0001_exportcenter_i | CI/offline kit integration; depends on 74-001. |
|
||||
| EXPORT-ATTEST-75-001 | ⏳ BLOCKED | SPRINT_0162_0001_0001_exportcenter_i | CLI `stella attest bundle verify/import`; depends on 74-002. |
|
||||
| EXPORT-ATTEST-75-002 | ⏳ BLOCKED | SPRINT_0162_0001_0001_exportcenter_i | Offline kit integration; depends on 75-001. |
|
||||
|
||||
**Checkpoint:** Bulk/air-gap workflows blocked awaiting Export Center contracts.
|
||||
|
||||
### Phase 6 — Performance & hardening
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| ATTEST-73-003 | 📝 TODO | SPRINT_302_docs_tasks_md_ii | Evidence documentation; waiting on ATEL0102 evidence. |
|
||||
| ATTEST-73-004 | 📝 TODO | SPRINT_302_docs_tasks_md_ii | Extended documentation; depends on 73-003. |
|
||||
|
||||
**Checkpoint:** Performance benchmarks and incident playbooks pending; observability coverage to be validated.
|
||||
|
||||
---
|
||||
|
||||
### Overall readiness summary
|
||||
|
||||
| Phase | Status | Blocking items |
|
||||
|-------|--------|----------------|
|
||||
| **1 – Foundations** | ✅ Complete | — |
|
||||
| **2 – Policies & UI** | ⏳ Blocked | POLICY-ATTEST-73-001 prep; CLI build issues |
|
||||
| **3 – Scan & VEX** | ✅ Complete | — |
|
||||
| **4 – Transparency & keys** | 🔄 In progress | NOTIFY-ATTEST-74-002 payload freeze |
|
||||
| **5 – Bulk & air gap** | ⏳ Blocked | EXPORT-ATTEST-74-001 contract |
|
||||
| **6 – Performance** | 📝 Not started | Upstream phase completion |
|
||||
|
||||
### Next actions
|
||||
1. Track POLICY-ATTEST-73-001 prep artefact publication (Sprint 0123).
|
||||
2. Resolve CLI build blockers to unblock CLI-ATTEST-73-001 (Sprint 0201).
|
||||
3. Complete NOTIFY-ATTEST-74-002 wiring once payload localization freezes (Sprint 0171).
|
||||
4. Monitor Export Center contract finalization for Phase 5 tasks (Sprint 0162).
|
||||
@@ -48,3 +48,23 @@ Authority is the platform OIDC/OAuth2 control plane that mints short-lived, send
|
||||
- **Epic 2 – Policy Engine & Editor:** supply policy evaluation/principal scopes and short-lived tokens for evaluator workflows.
|
||||
- **Epic 4 – Policy Studio:** integrate approval/promotion signatures and policy registry access controls.
|
||||
- **Epic 14 – Identity & Tenancy:** deliver tenant isolation, RBAC hierarchies, and governance tooling for authentication.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
**Epic Milestones:**
|
||||
- Epic 1 (AOC enforcement) – Complete: OpTok scopes, guardrails, AOC role templates, and scope policies operational
|
||||
- Epic 2 (Policy Engine & Editor) – Complete: DPoP validation and mTLS sender-constraint flows operational
|
||||
- Epic 4 (Policy Studio) – Complete: pack signing policies, approval RBAC, CLI CI token scopes, audit logging
|
||||
- Epic 14 (Identity & Tenancy) – In progress: tenancy contract published, sovereign crypto provider integration ongoing
|
||||
- Future (Attestation support) – Not started: DSSE predicate types and verification helpers pending upstream dependencies
|
||||
|
||||
**Key Technical Decisions:**
|
||||
- DPoP validation on token grants with cnf.jkt inheritance for interactive tokens
|
||||
- Refresh grants enforce original client cert with x5t#S256 metadata persistence
|
||||
- Sealed-mode CI gating refuses tokens when sealed install lacks confirmation
|
||||
- Tenant-scope contract published for cross-module coordination
|
||||
|
||||
**Risks & Mitigations:**
|
||||
- Sovereign crypto keystore migration in progress, key-loading path updates required
|
||||
- DSSE predicate schema draft pending coordination with Signer guild
|
||||
- Provenance harness dependency for verification helpers
|
||||
|
||||
@@ -1,101 +0,0 @@
|
||||
# Implementation plan — Authority
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** deliver OpTok scopes, guardrails, and AOC verifier hooks for ingestion services.
|
||||
- **Epic 2 – Policy Engine & Editor:** support policy evaluator flows (device-code, client credentials, scope sandboxing).
|
||||
- **Epic 4 – Policy Studio:** provide registry/promotion signing, approvals, and fresh-auth prompts.
|
||||
- **Epic 14 – Identity & Tenancy:** implement tenant isolation, RBAC hierarchies, audit trails, and PoE integration.
|
||||
- Track additional work (DOCS-SEC-62-001, AUTH-POLICY-20-001/002) in ../../TASKS.md and src/Authority/**/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0314_0001_0001_docs_modules_authority.md`; statuses mirrored in `docs/modules/authority/TASKS.md`.
|
||||
- Observability assets remain in `operations/monitoring.md` with Grafana JSON `operations/grafana-dashboard.json` (offline import).
|
||||
- Authority readiness tracker (AUTHORITY-ENG-0001) delivered 2025-11-27; future updates should adjust both sprint and TASKS.
|
||||
|
||||
---
|
||||
|
||||
## Sprint readiness tracker
|
||||
|
||||
> Last updated: 2025-11-27 (AUTHORITY-ENG-0001)
|
||||
|
||||
This section maps epic milestones to implementation sprints and tracks readiness checkpoints.
|
||||
|
||||
### Epic 1 — AOC enforcement
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-SIG-26-001 | ✅ DONE (2025-10-29) | SPRINT_0143_0001_0001_signals | Signals scopes + AOC role templates; propagation validation complete. |
|
||||
| AUTH-AIRGAP-57-001 | ✅ DONE (2025-11-08) | SPRINT_100_identity_signing | Sealed-mode CI gating; refuses tokens when sealed install lacks confirmation. |
|
||||
|
||||
**Checkpoint:** AOC enforcement operational with guardrails and scope policies in place.
|
||||
|
||||
### Epic 2 — Policy Engine & Editor
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-DPOP-11-001 | ✅ DONE (2025-11-08) | SPRINT_100_identity_signing | DPoP validation on `/token` grants; interactive tokens inherit `cnf.jkt`. |
|
||||
| AUTH-MTLS-11-002 | ✅ DONE (2025-11-08) | SPRINT_100_identity_signing | Refresh grants enforce original client cert; `x5t#S256` metadata persisted. |
|
||||
|
||||
**Checkpoint:** DPoP and mTLS sender-constraint flows operational.
|
||||
|
||||
### Epic 4 — Policy Studio
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-PACKS-43-001 | ✅ DONE (2025-11-09) | SPRINT_100_identity_signing | Pack signing policies, approval RBAC, CLI CI token scopes, audit logging. |
|
||||
|
||||
**Checkpoint:** Pack signing and approval flows with fresh-auth prompts complete.
|
||||
|
||||
### Epic 14 — Identity & Tenancy
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-TEN-47-001 | ✅ Contract published | SPRINT_0115_0001_0004_concelier_iv | Tenant-scope contract at `docs/modules/authority/tenant-scope-47-001.md`. |
|
||||
| AUTH-CRYPTO-90-001 | 🔄 DOING | SPRINT_0514_0001_0001_sovereign_crypto | Sovereign signing provider; key-loading path migration in progress. |
|
||||
|
||||
**Checkpoint:** Tenancy contract published; sovereign crypto provider integration in progress.
|
||||
|
||||
### Future tasks
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-REACH-401-005 | 📝 TODO | SPRINT_0401_0001_0001_reachability_evidence_chain | DSSE predicate types for SBOM/Graph/VEX/Replay; blocked on predicate definitions. |
|
||||
| AUTH-VERIFY-186-007 | 📝 TODO | SPRINT_186_record_deterministic_execution | Verification helper for DSSE signatures and Rekor proofs; awaits provenance harness. |
|
||||
|
||||
**Checkpoint:** Attestation predicate support and verification helpers pending upstream dependencies.
|
||||
|
||||
---
|
||||
|
||||
### Overall readiness summary
|
||||
|
||||
| Epic | Status | Blocking items |
|
||||
|------|--------|----------------|
|
||||
| **1 – AOC enforcement** | ✅ Complete | — |
|
||||
| **2 – Policy Engine & Editor** | ✅ Complete | — |
|
||||
| **4 – Policy Studio** | ✅ Complete | — |
|
||||
| **14 – Identity & Tenancy** | 🔄 In progress | AUTH-CRYPTO-90-001 provider contract |
|
||||
| **Future (Attestation)** | 📝 Not started | DSSE predicate schema; provenance harness |
|
||||
|
||||
### Cross-module dependencies
|
||||
|
||||
| Dependency | Required by | Status |
|
||||
|------------|-------------|--------|
|
||||
| Signals scope propagation | AUTH-SIG-26-001 | ✅ Validated |
|
||||
| Sealed-mode CI evidence | AUTH-AIRGAP-57-001 | ✅ Implemented |
|
||||
| DSSE predicate definitions | AUTH-REACH-401-005 | Schema draft pending |
|
||||
| Provenance harness (PROB0101) | AUTH-VERIFY-186-007 | In progress |
|
||||
| Sovereign crypto keystore plan | AUTH-CRYPTO-90-001 | ✅ Prep published |
|
||||
|
||||
### Next actions
|
||||
1. Complete AUTH-CRYPTO-90-001 provider registry wiring (Sprint 0514).
|
||||
2. Coordinate DSSE predicate schema with Signer guild for AUTH-REACH-401-005 (Sprint 0401).
|
||||
3. Monitor PROB0101 provenance harness for AUTH-VERIFY-186-007 (Sprint 186).
|
||||
50
docs/modules/benchmark/README.md
Normal file
50
docs/modules/benchmark/README.md
Normal file
@@ -0,0 +1,50 @@
|
||||
# Benchmark
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/Bench/`
|
||||
**Owner:** Platform Team
|
||||
|
||||
## Purpose
|
||||
|
||||
Benchmark provides performance testing and regression analysis for StellaOps components. Ensures deterministic scan times, throughput validation, and performance profiling for critical paths (scanning, policy evaluation, SBOM generation).
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- `StellaOps.Bench` - Benchmarking harness with BenchmarkDotNet integration
|
||||
|
||||
**Key Features:**
|
||||
- Scanner performance benchmarks (per-analyzer, full-scan)
|
||||
- Policy engine evaluation latency tests
|
||||
- SBOM generation throughput tests
|
||||
- Database query performance profiling
|
||||
- Determinism validation (output stability)
|
||||
|
||||
## Configuration
|
||||
|
||||
Benchmark configuration via BenchmarkDotNet attributes and runtime parameters.
|
||||
|
||||
Key settings:
|
||||
- Benchmark filters and categories
|
||||
- Iterations and warmup counts
|
||||
- Memory profiling and allocation tracking
|
||||
- Export formats (JSON, HTML, Markdown)
|
||||
|
||||
## Dependencies
|
||||
|
||||
- BenchmarkDotNet framework
|
||||
- Scanner (benchmark targets)
|
||||
- Policy Engine (benchmark targets)
|
||||
- SbomService (benchmark targets)
|
||||
- Test fixtures and datasets
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- Scanner: `../scanner/`
|
||||
- Policy: `../policy/`
|
||||
- Operations: `./operations/` (if exists)
|
||||
|
||||
## Current Status
|
||||
|
||||
Implemented with BenchmarkDotNet harness. Provides performance baselines for scanner analyzers, policy evaluation, and SBOM generation. Used for regression detection in CI/CD pipeline.
|
||||
46
docs/modules/binaryindex/README.md
Normal file
46
docs/modules/binaryindex/README.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# BinaryIndex
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/BinaryIndex/`
|
||||
**Owner:** Scanner Guild + Concelier Guild
|
||||
|
||||
## Purpose
|
||||
|
||||
BinaryIndex provides vulnerable binary detection independent of package metadata. It addresses the gap where package version strings can lie (backports, custom builds, stripped metadata) through binary-first vulnerability identification using Build-IDs, hash catalogs, and function fingerprints.
|
||||
|
||||
## Components
|
||||
|
||||
**Libraries:**
|
||||
- `StellaOps.BinaryIndex.Core` - Core binary identity extraction and matching engine
|
||||
- `StellaOps.BinaryIndex.Corpus` - Binary-to-advisory mapping database
|
||||
- `StellaOps.BinaryIndex.Corpus.Debian` - Debian-specific corpus support
|
||||
- `StellaOps.BinaryIndex.Fingerprints` - Function fingerprint storage and matching (CFG/basic-block hashes)
|
||||
- `StellaOps.BinaryIndex.FixIndex` - Patch-aware backport handling
|
||||
- `StellaOps.BinaryIndex.Persistence` - Storage adapters for binary catalogs
|
||||
|
||||
## Configuration
|
||||
|
||||
Configuration is typically embedded in Scanner and Concelier module settings.
|
||||
|
||||
Key features:
|
||||
- Three-tier binary identification (package/version, Build-ID/hash, function fingerprints)
|
||||
- Binary identity extraction (Build-ID, PE CodeView GUID, Mach-O UUID)
|
||||
- Integration with Scanner.Worker for binary lookup
|
||||
- Offline-first design with deterministic outputs
|
||||
|
||||
## Dependencies
|
||||
|
||||
- PostgreSQL (integrated with Scanner/Concelier schemas)
|
||||
- Scanner.Analyzers.Native (for binary disassembly/analysis)
|
||||
- Concelier (for advisory-to-binary mapping)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- High-Level Architecture: `../../07_HIGH_LEVEL_ARCHITECTURE.md`
|
||||
- Scanner Architecture: `../scanner/architecture.md`
|
||||
- Concelier Architecture: `../concelier/architecture.md`
|
||||
|
||||
## Current Status
|
||||
|
||||
Library implementation complete with support for ELF (Build-ID), PE (CodeView GUID), and Mach-O (UUID) binary formats. Integrated into Scanner's native binary analysis pipeline.
|
||||
49
docs/modules/cartographer/README.md
Normal file
49
docs/modules/cartographer/README.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# Cartographer Module
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/Cartographer/`
|
||||
|
||||
## Purpose
|
||||
|
||||
The Cartographer service materializes immutable SBOM property graphs, precomputes layout tiles, and hydrates policy and VEX overlays so other services (API, UI, CLI) can navigate and reason about dependency relationships with context.
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- **StellaOps.Cartographer** - Core graph materialization, overlay management, and tile serving
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Graph Materialization** - Convert normalized SBOMs (CycloneDX/SPDX) into immutable, versioned graph snapshots
|
||||
- **Property Graph Generation** - Build dependency relationship graphs with context-aware nodes and edges
|
||||
- **Overlay Hydration** - Merge Policy Engine findings and VEX metadata onto graph nodes and edges
|
||||
- **Layout Tiles** - Precomputed viewport tiles for efficient UI navigation
|
||||
- **Path Relevance** - Compute path importance within the dependency graph
|
||||
- **Graph Diffing** - Compare SBOM versions to track changes
|
||||
- **Tenant-Aware Storage** - Per-tenant graph isolation and versioning
|
||||
|
||||
## API Capabilities
|
||||
|
||||
- Viewport tile serving for large graphs (≥50k nodes)
|
||||
- Path exploration and filtering
|
||||
- Graph export and simulation overlays
|
||||
- RBAC-enforced access control via Authority
|
||||
|
||||
## Dependencies
|
||||
|
||||
- **PostgreSQL** - Graph and overlay storage
|
||||
- **Policy Engine** - Effective findings computation
|
||||
- **SBom Service** - Normalized SBOM projections
|
||||
- **Excititor** - VEX metadata ingestion
|
||||
- **Authority** - Authentication and RBAC enforcement (scopes: `graph:*`, `sbom:read`, `findings:read`)
|
||||
- **Scheduler** - Overlay update coordination
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **Architecture Charter:** See `src/Cartographer/StellaOps.Cartographer/AGENTS.md` for charter and responsibilities
|
||||
- **Sprint Plan:** Check `docs/implplan/SPRINT_*.md` for current development status
|
||||
- **Tasks:** Completed tasks documented in `src/Cartographer/StellaOps.Cartographer/TASKS.completed.md`
|
||||
|
||||
## Current Status
|
||||
|
||||
Active development. Materializes immutable SBOM property graphs with overlay hydration, deterministic snapshots, and optimized tile serving for dependency navigation.
|
||||
@@ -17,10 +17,10 @@ CI module collects reproducible pipeline recipes for builds, tests, and release
|
||||
## Operational notes
|
||||
- Encourage reuse through templated YAML/JSON fragments.
|
||||
|
||||
## Related resources
|
||||
- ./recipes.md
|
||||
- ./TASKS.md (status mirror)
|
||||
- ../../implplan/SPRINT_0315_0001_0001_docs_modules_ci.md (sprint tracker)
|
||||
## Related resources
|
||||
- ./recipes.md
|
||||
- ./TASKS.md (status mirror)
|
||||
- ../../implplan/SPRINT_0315_0001_0001_docs_modules_ci.md (sprint tracker)
|
||||
|
||||
## Backlog references
|
||||
- CI recipes refresh tracked in ../../TASKS.md under DOCS-CI stories.
|
||||
@@ -29,3 +29,21 @@ CI module collects reproducible pipeline recipes for builds, tests, and release
|
||||
- **Epic 1 – AOC enforcement:** bake ingestion/verifier guardrails into CI recipes.
|
||||
- **Epic 10 – Export Center:** provide pipeline snippets for export packaging, signing, and Offline Kit publication.
|
||||
- **Epic 11 – Notifications Studio:** offer CI hooks for notification previews/tests where relevant.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
**Epic Milestones:**
|
||||
- Epic 1 (AOC enforcement) – Ensure pipelines enforce schemas, provenance, and verifier jobs
|
||||
- Epic 10 (Export Center) – Add export/signing/Offline Kit automation templates
|
||||
- Epic 11 (Notifications Studio) – Document CI hooks for notification previews and tests
|
||||
|
||||
**Key Deliverables:**
|
||||
- Reproducible pipeline recipes for builds, tests, and release promotion
|
||||
- Ready-to-use snippets for ingestion, scanning, policy evaluation, and exports
|
||||
- Documentation of required secrets/scopes and deterministic build knobs
|
||||
- Offline-compatible workflows and cache strategies
|
||||
|
||||
**Operational Focus:**
|
||||
- Maintain deterministic behavior and offline parity across releases
|
||||
- Keep documentation, telemetry, and runbooks aligned with sprint outcomes
|
||||
- Preserve determinism and provenance requirements in all recipe additions
|
||||
|
||||
@@ -1,22 +0,0 @@
|
||||
# Implementation plan — CI Recipes
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** ensure pipelines enforce schemas, provenance, and verifier jobs.
|
||||
- **Epic 10 – Export Center:** add export/signing/Offline Kit automation templates.
|
||||
- **Epic 11 – Notifications Studio:** document CI hooks for notification previews/tests.
|
||||
- Track DOCS-CI stories in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Mirror task status changes in `./TASKS.md` and the owning sprint file.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change; record deterministic/offline considerations with each recipe addition.
|
||||
@@ -28,12 +28,12 @@ The `stella` CLI is the operator-facing Swiss army knife for scans, exports, pol
|
||||
- ./guides/cli-reference.md
|
||||
- ./guides/policy.md
|
||||
|
||||
## Backlog references
|
||||
- DOCS-CLI-OBS-52-001 / DOCS-CLI-FORENSICS-53-001 in ../../TASKS.md.
|
||||
- CLI-CORE-41-001 epic in `src/Cli/StellaOps.Cli/TASKS.md`.
|
||||
|
||||
## Current workstreams (Q4 2025)
|
||||
- Active docs sprint: `docs/implplan/SPRINT_0316_0001_0001_docs_modules_cli.md` — normalised sprint naming, doc sync, and upcoming ops/runbook refresh.
|
||||
## Backlog references
|
||||
- DOCS-CLI-OBS-52-001 / DOCS-CLI-FORENSICS-53-001 in ../../TASKS.md.
|
||||
- CLI-CORE-41-001 epic in `src/Cli/StellaOps.Cli/TASKS.md`.
|
||||
|
||||
## Current workstreams (Q4 2025)
|
||||
- Active docs sprint: `docs/implplan/SPRINT_0316_0001_0001_docs_modules_cli.md` — normalised sprint naming, doc sync, and upcoming ops/runbook refresh.
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver deterministic policy authoring, simulation, and explain verbs.
|
||||
@@ -41,3 +41,23 @@ The `stella` CLI is the operator-facing Swiss army knife for scans, exports, pol
|
||||
- **Epic 6 – Vulnerability Explorer:** surface triage and ledger operations.
|
||||
- **Epic 10 – Export Center:** orchestrate export requests, verification, and Offline Kit automation.
|
||||
- **Epic 11 – Notifications Studio:** manage notification authoring/previews from the command line.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
**Epic Milestones:**
|
||||
- Epic 2 (Policy Engine & Editor) – Deliver deterministic policy verbs, simulation, and explain outputs
|
||||
- Epic 4 (Policy Studio) – Add registry/promotion workflows, lint tooling, and approvals UX
|
||||
- Epic 6 (Vulnerability Explorer) – Integrate ledger/triage operations
|
||||
- Epic 10 (Export Center) – Automate export verification and Offline Kit flows
|
||||
- Epic 11 (Notifications Studio) – Manage rule/channel authoring and previews via CLI
|
||||
|
||||
**Key Responsibilities:**
|
||||
- Deterministic verbs for scan, diff, export, policy, and observability operations
|
||||
- Interactive and non-interactive authentication via Authority (device code, client credentials)
|
||||
- Offline kit workflows including bundle verification and seed installation
|
||||
- JSON outputs suitable for CI parity and golden tests
|
||||
|
||||
**Operational Focus:**
|
||||
- Maintain deterministic output fixtures and versioned command documentation
|
||||
- Support plugin catalogue for restart-only extensions
|
||||
- Keep documentation aligned with active sprint outcomes
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
# Implementation plan — CLI
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
- Documentation sync: keep module docs aligned with active sprint `docs/implplan/SPRINT_0316_0001_0001_docs_modules_cli.md`.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver deterministic policy verbs, simulation, and explain outputs.
|
||||
- **Epic 4 – Policy Studio:** add registry/promotion workflows, lint tooling, and approvals UX.
|
||||
- **Epic 6 – Vulnerability Explorer:** integrate ledger/triage operations.
|
||||
- **Epic 10 – Export Center:** automate export verification and Offline Kit flows.
|
||||
- **Epic 11 – Notifications Studio:** manage rule/channel authoring and previews via CLI.
|
||||
- Track CLI-specific work (e.g., CLI-CORE-41-001, DOCS-CLI-OBS-52-001) in ../../TASKS.md and src/Cli/**/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
@@ -1,6 +1,15 @@
|
||||
# StellaOps Concelier
|
||||
|
||||
Concelier ingests signed advisories from dozens of sources and converts them into immutable observations plus linksets under the Aggregation-Only Contract (AOC).
|
||||
Concelier ingests signed advisories from **32 advisory connectors** and converts them into immutable observations plus linksets under the Aggregation-Only Contract (AOC).
|
||||
|
||||
**Advisory Sources (32 connectors):**
|
||||
- **National CERTs (8):** ACSC (Australia), CCCS (Canada), CERT-Bund (Germany), CERT-CC (US), CERT-FR (France), CERT-IN (India), JVN (Japan), KISA (Korea)
|
||||
- **OS Distros (5):** Alpine SecDB, Debian Security Tracker, RedHat OVAL, SUSE OVAL, Ubuntu USN
|
||||
- **Vendors (7):** Apple, Adobe, Chromium, Cisco PSIRT, Microsoft MSRC, Oracle, VMware
|
||||
- **Standards (5):** CVE, NVD, GHSA (GitHub), OSV, EPSS v4
|
||||
- **Threat Intel (3):** KEV (CISA Exploited Vulns), CISA ICS, Kaspersky ICS
|
||||
- **Regional (3):** Russia BDU, Russia NKCKI, Plus regional mirrors
|
||||
- **Internal (1):** StellaOps internal mirror
|
||||
|
||||
## Responsibilities
|
||||
- Fetch and normalise vulnerability advisories via restart-time connectors.
|
||||
@@ -41,3 +50,28 @@ Concelier ingests signed advisories from dozens of sources and converts them int
|
||||
## Epic alignment
|
||||
- **Epic 1 – AOC enforcement:** uphold raw observation invariants, provenance requirements, linkset-only enrichment, and AOC verifier guardrails across every connector.
|
||||
- **Epic 10 – Export Center:** expose deterministic advisory exports and metadata required by JSON/Trivy/mirror bundles.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
**Delivery Phases:**
|
||||
- Phase 1 (Guardrails & schema) – PostgreSQL validators, AOCWriteGuard interceptor, deterministic linkset builders operational
|
||||
- Phase 2 (API & observability) – Ingestion/verification endpoints with Authority scopes, telemetry, Offline Kit packaging
|
||||
- Phase 3 (Experience polish) – CLI/Console affordances, Export Center hand-off metadata, CI enforcement
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- PostgreSQL validators and runtime guards reject forbidden fields and missing provenance with ERR_AOC_00x codes
|
||||
- Linksets and supersedes chains deterministic; identical payloads yield byte-identical documents
|
||||
- CLI `stella aoc verify` exits non-zero on violations, zero on clean datasets
|
||||
- Export Center consumes advisory datasets without legacy normalized fields
|
||||
- CI fails on lint violations or guard test regressions
|
||||
|
||||
**Key Risks & Mitigations:**
|
||||
- Collector drift: guard middleware + CI lint + schema validation; RFC required for linkset changes
|
||||
- Migration complexity: staged cutover with backup copies, temporary views for Policy Engine parity
|
||||
- Performance overhead: guard remains O(number of keys), index review for insert latency targets
|
||||
- Tenancy leakage: tenant required in schema, Authority claims enforced, observability alerts
|
||||
|
||||
**Recent Milestones:**
|
||||
- Sprint 110 attestation chain validated, evidence bundle tests green
|
||||
- Link-Not-Merge cache and console consumption docs frozen
|
||||
- Observation events transport reviewed, NATS/air-gap guidance updated
|
||||
|
||||
@@ -1,73 +0,0 @@
|
||||
# Implementation plan — Concelier
|
||||
|
||||
## Delivery timeline
|
||||
- **Phase 1 — Guardrails & schema**
|
||||
Stand up PostgreSQL JSON schema validators for `advisory_raw` and `vex_raw`, wire the `AOCWriteGuard` repository interceptor, and seed deterministic linkset builders. Freeze legacy normalisation paths and migrate callers to the new raw schema.
|
||||
- **Phase 2 — API & observability**
|
||||
Publish ingestion and verification endpoints (`POST /ingest/*`, `GET /advisories.raw`, `POST /aoc/verify`) with Authority scopes, expose telemetry (`aoc_violation_total`, guard spans, structured logs), and ensure Offline Kit packaging captures validator deployment steps.
|
||||
- **Phase 3 — Experience polish**
|
||||
Ship CLI/Console affordances (`stella sources ingest --dry-run`, dashboard tiles, violation drill-downs), finish Export Center hand-off metadata, and close out CI enforcement (`stella aoc verify` preflight, AST lint, seeded fixtures).
|
||||
|
||||
## Work breakdown by component
|
||||
- **Concelier WebService & worker**
|
||||
- Add PostgreSQL validators and unique indexes over `(tenant, source.vendor, upstream.upstream_id, upstream.content_hash)`.
|
||||
- Implement write interceptors rejecting forbidden fields, missing provenance, or merge attempts.
|
||||
- Deterministically compute linksets and persist canonical JSON payloads.
|
||||
- Introduce `/ingest/advisory`, `/advisories/raw*`, and `/aoc/verify` surfaces guarded by `advisory:*` and `aoc:verify` scopes.
|
||||
- Emit guard metrics/traces and surface supersedes/violation audit logs.
|
||||
- **Excititor (shared ingestion contract)**
|
||||
- Mirror Concelier guard and schema changes for `vex_raw`.
|
||||
- Maintain restart-time plug-in determinism and linkset extraction parity.
|
||||
- **Shared libraries**
|
||||
- Publish `StellaOps.Ingestion.AOC` (forbidden key catalog, guard middleware, provenance helpers, signature verification).
|
||||
- Share error codes (`ERR_AOC_00x`) and deterministic hashing utilities.
|
||||
- **Policy Engine integration**
|
||||
- Enforce `effective_finding_*` write exclusivity.
|
||||
- Consume only raw documents + linksets, removing any implicit normalisation.
|
||||
- **Authority scopes**
|
||||
- Provision `advisory:ingest|read`, `vex:ingest|read`, `aoc:verify`; propagate tenant claims to ingestion services.
|
||||
- **CLI & Console**
|
||||
- Implement `stella sources ingest --dry-run` and `stella aoc verify` (with exit codes mapped to `ERR_AOC_00x`).
|
||||
- Surface AOC dashboards, violation drill-down, and verification shortcuts in the Console.
|
||||
- **CI/CD**
|
||||
- Add Roslyn analyzer / AST lint to block forbidden writes.
|
||||
- Seed fixtures and run `stella aoc verify` against snapshots in pipeline gating.
|
||||
|
||||
## Documentation deliverables
|
||||
- Update `docs/aoc/aggregation-only-contract.md` with guard invariants, schemas, error codes, and migration guidance.
|
||||
- Refresh `docs/modules/concelier/operations/*.md` (mirror, conflict-resolution, authority audit) with validator rollouts and observability dashboards.
|
||||
- Cross-link Authority scope definitions, CLI reference, Console sources guide, and observability runbooks to the AOC guard changes.
|
||||
- Ensure Offline Kit documentation captures validator bootstrap and verify workflows.
|
||||
|
||||
## Acceptance criteria
|
||||
- PostgreSQL validators and runtime guards reject forbidden fields and missing provenance with the documented `ERR_AOC_00x` codes.
|
||||
- Linksets and supersedes chains are deterministic; rerunning ingestion over identical payloads yields byte-identical documents.
|
||||
- CLI `stella aoc verify` exits non-zero on seeded violations and zero on clean datasets; Console dashboards show real-time guard status.
|
||||
- Export Center consumes advisory datasets without relying on legacy normalised fields.
|
||||
- CI fails if lint rules detect forbidden writes or if seeded guard tests regress.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Collector drift introduces new forbidden keys.** Mitigated by guard middleware + CI lint + schema validation; RFC required for linkset changes.
|
||||
- **Migration complexity from legacy normalisation.** Staged cutover with `_backup_*` copies and temporary views to keep Policy Engine parity.
|
||||
- **Performance overhead during ingest.** Guard remains O(number of keys); index review ensures insert latency stays within warm (<5 s) / cold (<30 s) targets.
|
||||
- **Tenancy leakage.** `tenant` required in schema, Authority-supplied claims enforced per request, observability alerts fire on missing tenant identifiers.
|
||||
|
||||
## Test strategy
|
||||
- **Unit**: guard rejection paths, provenance enforcement, idempotent insertions, linkset determinism.
|
||||
- **Property**: fuzz upstream payloads to guarantee no forbidden fields emerge.
|
||||
- **Integration**: batch ingest (50k advisories, mixed VEX fixtures), verifying zero guard violations and consistent supersedes.
|
||||
- **Contract**: Policy Engine consumers verify raw-only reads; Export Center consumes canonical datasets.
|
||||
- **End-to-end**: ingest/verify flow with CLI + Console actions to confirm observability and guard reporting.
|
||||
|
||||
## Definition of done
|
||||
- Validators deployed and verified in staging/offline environments.
|
||||
- Runtime guards, CLI/Console workflows, and CI linting all active.
|
||||
- Observability dashboards and runbooks updated; metrics visible.
|
||||
- Documentation updates merged; Offline Kit instructions published.
|
||||
- ./TASKS.md reflects status transitions; cross-module dependencies acknowledged in ../../TASKS.md.
|
||||
|
||||
## Readiness checkpoints (2025-11-25)
|
||||
- Sprint 110 attestation chain validated: `/internal/attestations/verify` endpoint and evidence bundle tests green (`TestResults/concelier-attestation/web.trx`, `core.trx`).
|
||||
- Link-Not-Merge cache + console consumption docs frozen (see `operations/lnm-cache-plan.md`, `operations/console-lnm-consumption.md`); cache headers remain deterministic.
|
||||
- Observation events transport reviewed; backlog guardrails and NATS/air-gap guidance updated in `operations/observation-events.md`.
|
||||
- Next gating dependency: TaskRunner contract drop (sprint 0157 blockers) before wiring approvals/pack ingest flows into Concelier.
|
||||
49
docs/modules/cryptography/README.md
Normal file
49
docs/modules/cryptography/README.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# Cryptography
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/Cryptography/`
|
||||
**Owner:** Platform Team
|
||||
|
||||
## Purpose
|
||||
|
||||
Cryptography provides pluggable cryptographic primitives supporting regional standards (eIDAS, FIPS, GOST, SM, PQ). Enables sovereign operation with country-specific crypto requirements while maintaining deterministic signing operations.
|
||||
|
||||
## Components
|
||||
|
||||
**Libraries:**
|
||||
- `StellaOps.Cryptography` - Core cryptographic abstractions and plugin loader
|
||||
- `StellaOps.Cryptography.Profiles.Ecdsa` - ECDSA signing profile (NIST curves, secp256k1)
|
||||
- `StellaOps.Cryptography.Profiles.EdDsa` - EdDSA signing profile (Ed25519, Ed448)
|
||||
|
||||
**Plugin Architecture:**
|
||||
Additional profiles can be loaded for:
|
||||
- GOST R 34.10-2012 (Russian Federation)
|
||||
- SM2/SM3/SM4 (China)
|
||||
- Post-quantum signatures (experimental)
|
||||
|
||||
## Configuration
|
||||
|
||||
Cryptographic profiles are configured through module-specific settings (Signer, Attestor, Authority).
|
||||
|
||||
Key features:
|
||||
- Algorithm agility with deterministic output
|
||||
- Offline key management support
|
||||
- HSM/TPM integration capability
|
||||
- Signature scheme negotiation
|
||||
|
||||
## Dependencies
|
||||
|
||||
- .NET Cryptography APIs
|
||||
- Optional: Hardware Security Modules (HSM)
|
||||
- Optional: CryptoPro CSP (for GOST support)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Signer Module: `../signer/`
|
||||
- Attestor Module: `../attestor/`
|
||||
- Authority Module: `../authority/`
|
||||
- Air-Gap Operations: `../../24_OFFLINE_KIT.md`
|
||||
|
||||
## Current Status
|
||||
|
||||
Core ECDSA and EdDSA profiles implemented. Plugin architecture supports future regional crypto extensions. Integrated with Signer and Attestor modules for deterministic signing operations.
|
||||
@@ -40,3 +40,25 @@ The DevOps module captures release, deployment, and migration playbooks that kee
|
||||
- **Epic 9 – Orchestrator Dashboard:** support operational dashboards, job recovery runbooks, and rate-limit governance.
|
||||
- **Epic 10 – Export Center:** manage signing workflows, Offline Kit packaging, and release promotion for exports.
|
||||
- **Epic 15 – Observability & Forensics:** coordinate telemetry deployment, evidence retention, and forensic automation.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes
|
||||
|
||||
### Key Milestones
|
||||
- **Epic 1 – AOC enforcement:** ensure CI/CD guardrails, schema validation, and verifier pipelines are enforced
|
||||
- **Epic 9 – Orchestrator Dashboard:** deliver dashboards, recovery runbooks, and rate-limit governance
|
||||
- **Epic 10 – Export Center:** manage signing/promotions and Offline Kit bundle publishing
|
||||
- **Epic 15 – Observability & Forensics:** coordinate telemetry deployments, evidence retention, and forensic automation
|
||||
|
||||
### Workstreams
|
||||
- Backlog grooming: reconcile open stories with module roadmap
|
||||
- Implementation: collaborate with service owners to land feature work
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements
|
||||
|
||||
### Coordination
|
||||
- Review ./AGENTS.md before picking up new work
|
||||
- Sync with cross-cutting teams noted in sprint files
|
||||
- Update plan whenever scope, dependencies, or guardrails change
|
||||
|
||||
@@ -1,22 +0,0 @@
|
||||
# Implementation plan — DevOps
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** ensure CI/CD guardrails, schema validation, and verifier pipelines are enforced.
|
||||
- **Epic 9 – Orchestrator Dashboard:** deliver dashboards, recovery runbooks, and rate-limit governance.
|
||||
- **Epic 10 – Export Center:** manage signing/promotions and Offline Kit bundle publishing.
|
||||
- **Epic 15 – Observability & Forensics:** coordinate telemetry deployments, evidence retention, and forensic automation.
|
||||
- Track module runbooks (DEVOPS-LAUNCH-18-001/900) and telemetry automation via ../../TASKS.md and ops/devops/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
48
docs/modules/evidence-locker/README.md
Normal file
48
docs/modules/evidence-locker/README.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# EvidenceLocker
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/EvidenceLocker/`
|
||||
**Owner:** Platform Team
|
||||
|
||||
## Purpose
|
||||
|
||||
EvidenceLocker provides sealed, immutable storage for vulnerability scan evidence and audit logs. Ensures tamper-proof evidence chains for compliance and forensic analysis with content-addressable storage and cryptographic sealing.
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- `StellaOps.EvidenceLocker.WebService` - HTTP API for evidence submission and retrieval
|
||||
- `StellaOps.EvidenceLocker.Worker` - Background sealing and archival workers
|
||||
|
||||
**Libraries:**
|
||||
- `StellaOps.EvidenceLocker.Core` - Evidence sealing, verification, and chain validation
|
||||
- `StellaOps.EvidenceLocker.Infrastructure` - Storage adapters and evidence bundle management
|
||||
|
||||
## Configuration
|
||||
|
||||
See `etc/evidence-locker.yaml.sample` for configuration options (if available).
|
||||
|
||||
Key settings:
|
||||
- Storage backend (filesystem, object storage)
|
||||
- Sealing policy (immediate vs. batch)
|
||||
- Retention policies
|
||||
- Export destinations
|
||||
- Authority integration for access control
|
||||
|
||||
## Dependencies
|
||||
|
||||
- PostgreSQL (schema: `evidence_locker`)
|
||||
- Authority (authentication and authorization)
|
||||
- Signer (cryptographic sealing operations)
|
||||
- ExportCenter (evidence bundle export)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Operations: `./operations/` (if exists)
|
||||
- ExportCenter: `../export-center/`
|
||||
- Attestor: `../attestor/`
|
||||
- High-Level Architecture: `../../07_HIGH_LEVEL_ARCHITECTURE.md`
|
||||
|
||||
## Current Status
|
||||
|
||||
Implemented with WebService and Worker components. Supports sealed evidence storage with cryptographic verification. Integrated with ExportCenter for audit bundle generation.
|
||||
49
docs/modules/evidence/README.md
Normal file
49
docs/modules/evidence/README.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# Evidence
|
||||
|
||||
**Status:** Design/Planning
|
||||
**Source:** N/A (cross-cutting concept)
|
||||
**Owner:** Platform Team
|
||||
|
||||
## Purpose
|
||||
|
||||
Evidence defines the unified evidence model for vulnerability findings across StellaOps. Provides canonical data structures for evidence capture, aggregation, and scoring used by Signals, Policy Engine, and EvidenceLocker modules.
|
||||
|
||||
## Components
|
||||
|
||||
**Concept Documentation:**
|
||||
- `unified-model.md` - Unified evidence data model specification
|
||||
|
||||
**Evidence Types:**
|
||||
- Reachability evidence (call graph, data flow)
|
||||
- Runtime evidence (eBPF traces, dynamic observations)
|
||||
- Binary evidence (backport detection, fix validation)
|
||||
- Exploit evidence (EPSS scores, KEV flags, exploit-db entries)
|
||||
- VEX evidence (source trust, statement provenance)
|
||||
- Mitigation evidence (active mitigations, compensating controls)
|
||||
|
||||
## Implementation Locations
|
||||
|
||||
Evidence structures are implemented across multiple modules:
|
||||
- **Signals** - Evidence aggregation and normalization
|
||||
- **Policy Engine** - Reachability analysis and evidence generation
|
||||
- **EvidenceLocker** - Evidence storage and sealing
|
||||
- **Scanner** - Binary and vulnerability evidence capture
|
||||
- **Concelier** - Backport and exploit evidence enrichment
|
||||
|
||||
## Dependencies
|
||||
|
||||
- All evidence-producing modules (Scanner, Policy, Concelier, etc.)
|
||||
- Signals (evidence aggregation)
|
||||
- EvidenceLocker (evidence storage)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Unified Model: `./unified-model.md`
|
||||
- Signals: `../signals/`
|
||||
- Policy: `../policy/`
|
||||
- EvidenceLocker: `../evidence-locker/`
|
||||
- Data Schemas: `../../11_DATA_SCHEMAS.md`
|
||||
|
||||
## Current Status
|
||||
|
||||
Evidence model documented in `unified-model.md`. Implementation distributed across Signals (aggregation), Policy (reachability), EvidenceLocker (storage), and Scanner (capture) modules.
|
||||
@@ -48,3 +48,29 @@ Excititor converts heterogeneous VEX feeds into raw observations and linksets th
|
||||
- **Epic 1 – AOC enforcement:** maintain immutable VEX observations, provenance, and AOC verifier coverage.
|
||||
- **Epic 7 – VEX Consensus Lens:** supply trustworthy raw inputs, trust metadata, and consensus hooks for the lens computations.
|
||||
- **Epic 8 – Advisory AI:** expose citation-ready VEX payloads for the advisory assistant pipeline.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes
|
||||
|
||||
### Key Milestones
|
||||
- **Epic 1 – AOC enforcement:** enforce immutable VEX observation schema, provenance capture, and guardrails
|
||||
- **Epic 7 – VEX Consensus Lens:** provide lens-ready metadata (issuer trust, temporal scoping) and consensus APIs
|
||||
- **Epic 8 – Advisory AI:** guarantee citation-ready payloads and normalized context for AI summaries/explainers
|
||||
|
||||
### Recent Delivery Status
|
||||
- Chunk API documentation remains blocked until CI is green and a pinned OpenAPI spec with deterministic samples are available
|
||||
- Link-Not-Merge readiness and consensus beta completed with DSSE packaging guidance
|
||||
- Observability guide additions and policy/CLI follow-ups tracked in sprint files
|
||||
|
||||
### Workstreams
|
||||
- Backlog grooming: reconcile open stories with module roadmap
|
||||
- Implementation: collaborate with service owners to land feature work
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements
|
||||
|
||||
### Coordination
|
||||
- Review ./AGENTS.md before picking up new work
|
||||
- Sync with cross-cutting teams noted in sprint files
|
||||
- Update plan whenever scope, dependencies, or guardrails change
|
||||
|
||||
@@ -1,34 +0,0 @@
|
||||
# Implementation plan — Excititor
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** enforce immutable VEX observation schema, provenance capture, and guardrails.
|
||||
- **Epic 7 – VEX Consensus Lens:** provide lens-ready metadata (issuer trust, temporal scoping) and consensus APIs.
|
||||
- **Epic 8 – Advisory AI:** guarantee citation-ready payloads and normalized context for AI summaries/explainers.
|
||||
- Track DOCS-LNM-22-006/007 and CLI-EXC-25-001..002 in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
|
||||
| Sprint task | State (SPRINT_0333_0001_0001_docs_modules_excititor) | Notes |
|
||||
| --- | --- | --- |
|
||||
| EXCITOR-DOCS-0001 | DONE | README release alignment + consensus beta references refreshed (DSSE/export guidance). |
|
||||
| EXCITOR-ENG-0001 | DONE | Implementation plan mirrored to sprint status; TASKS board created. |
|
||||
| EXCITOR-OPS-0001 | DONE | Runbook/observability checklist added to `mirrors.md`. |
|
||||
| EXCITITOR-DOCS-0001 | BLOCKED | Waiting on chunk API CI validation + OpenAPI freeze. |
|
||||
| EXCITITOR-ENG-0001 | TODO | Will update engineering notes once DOCS unblock. |
|
||||
| EXCITITOR-OPS-0001 | TODO | Sync observability/runbook updates after OpenAPI freeze. |
|
||||
|
||||
See `/docs/implplan/SPRINT_0333_0001_0001_docs_modules_excititor.md` for the canonical status table.
|
||||
@@ -1,7 +1,7 @@
|
||||
# StellaOps Export Center
|
||||
|
||||
Export Center packages reproducible evidence bundles (JSON, Trivy DB, mirror) with provenance metadata and optional signing for offline or mirrored deployments.
|
||||
|
||||
# StellaOps Export Center
|
||||
|
||||
Export Center packages reproducible evidence bundles (JSON, Trivy DB, mirror) with provenance metadata and optional signing for offline or mirrored deployments.
|
||||
|
||||
## Latest updates (2025-11-30)
|
||||
- Sprint tracker `docs/implplan/SPRINT_0320_0001_0001_docs_modules_export_center.md` and module `TASKS.md` added to mirror status.
|
||||
- Observability runbook stub + dashboard placeholder added under `operations/` (offline import).
|
||||
@@ -12,7 +12,7 @@ Export Center packages reproducible evidence bundles (JSON, Trivy DB, mirror) wi
|
||||
- Assemble manifests, provenance documents, and cosign signatures.
|
||||
- Stream bundles via HTTP/OCI and stage them for Offline Kit uses.
|
||||
- Expose CLI/API surfaces for automation.
|
||||
|
||||
|
||||
## Key components
|
||||
- `StellaOps.ExportCenter.WebService` planner.
|
||||
- `StellaOps.ExportCenter.Worker` bundle builder.
|
||||
@@ -44,5 +44,31 @@ Export Center packages reproducible evidence bundles (JSON, Trivy DB, mirror) wi
|
||||
- DOCS-EXPORT-35-001 … DOCS-EXPORT-37-002 in ../../TASKS.md.
|
||||
- EXPORT-ATTEST-75-002 cross-team deliverable.
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 10 – Export Center:** deliver canonical JSON, Trivy DB, and mirror bundle workflows with provenance, signatures, and offline parity.
|
||||
## Epic alignment
|
||||
- **Epic 10 – Export Center:** deliver canonical JSON, Trivy DB, and mirror bundle workflows with provenance, signatures, and offline parity.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Delivery Phases
|
||||
- **Phase 1 – JSON & mirror foundations:** Stand up service + worker, deliver canonical JSON and mirror profiles, seed schema migrations, publish manifest/provenance formats
|
||||
- **Phase 2 – Trivy adapters & distribution:** Implement Trivy DB/Java DB adapters, wire OCI/object storage distribution, expose policy snapshot embedding + verification
|
||||
- **Phase 3 – Delta, encryption, scheduling:** Release mirror deltas, bundle encryption, advanced scheduling/automation, resumable downloads, CLI/Console verification workflows
|
||||
|
||||
### Acceptance Criteria
|
||||
- Operators can create, monitor, and download exports; verification succeeds against manifest + provenance
|
||||
- Trivy bundles import cleanly; mirror bundles run in Offline Kit reference environment (full + delta)
|
||||
- Policy snapshot runs reproduce deterministic decisions with embedded policyVersion + inputsHash
|
||||
- Tenant scoping and RBAC block unauthorized actions; encryption-enabled bundles lock data to recipient keys
|
||||
- Metrics and dashboards reflect live runs; alerts trigger on sustained failure rates
|
||||
- Retried runs remain idempotent with matching manifests, hashes, and distribution artefacts
|
||||
|
||||
### Key Risks & Mitigations
|
||||
- **Schema drift:** Versioned adapters with compatibility gates, CI integration tests, fail-fast with actionable errors
|
||||
- **Bundle bloat:** zstd compression, sharding, delta exports, OCI dedupe
|
||||
- **Data leakage:** Strict schema allowlists, tenancy filters, redaction enforcement, encryption options
|
||||
- **Non-determinism:** Embed policy snapshots, enforce deterministic ordering, include content hashes in manifest
|
||||
|
||||
### Recent Updates
|
||||
- Sprint tracker and module TASKS.md added to mirror status
|
||||
- Observability runbook stub + dashboard placeholder added under operations/
|
||||
- Bundle/profile/offline manifest guidance reaffirmed
|
||||
|
||||
@@ -1,71 +0,0 @@
|
||||
# Implementation plan — Export Center
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – JSON & mirror foundations**
|
||||
Stand up the Export Center service + worker, deliver canonical JSON (`json:raw`, `json:policy`) and `mirror:full` profiles as download-only bundles, seed schema migrations, and publish manifest/provenance formats.
|
||||
- **Phase 2 – Trivy adapters & distribution**
|
||||
Implement Trivy DB / Java DB adapters, wire OCI/object storage distribution paths, and expose policy snapshot embedding + verification tooling.
|
||||
- **Phase 3 – Delta, encryption, scheduling**
|
||||
Release mirror deltas, bundle encryption, advanced scheduling/automation, resumable downloads, and CLI/Console verification workflows.
|
||||
|
||||
## Component work breakdown
|
||||
- **Service & worker**
|
||||
- Define migrations for `export_profiles`, `export_runs`, `export_inputs`, `export_distributions`.
|
||||
- Implement planner, adapter host, signing/attestation layer, distribution engines, and deterministic manifests.
|
||||
- Enforce tenant quotas, concurrency controls, and audit logging for create/cancel/distribute events.
|
||||
- **Adapters**
|
||||
- JSON adapters: canonical JSONL writers, redaction guardrails, compression (zstd).
|
||||
- Trivy adapters: field mapping, schema compatibility gating, validation suite.
|
||||
- Mirror adapters: filesystem/OCI layout, delta computation, optional encryption with manifest updates.
|
||||
- **Integrations**
|
||||
- Findings Ledger streaming APIs for advisories, VEX, SBOMs, findings.
|
||||
- Policy Engine deterministic snapshot endpoint; VEX Lens consensus snapshot.
|
||||
- Export Center telemetry surfaced through Observability stack.
|
||||
- **Surfaces**
|
||||
- Console: profiles CRUD, run wizard, run detail + verification panel, distribution dashboards.
|
||||
- CLI: `stella export profile|run|download|verify` with resumable downloads and signature verification.
|
||||
- **Security / RBAC**
|
||||
- Scope enforcement per tenant, role matrix coverage, encryption key rotation tests, redaction filters.
|
||||
- **Docs & ops**
|
||||
- Author module dossier (overview, architecture, profiles, API, CLI, mirror bundles, Trivy adapter, provenance & signing).
|
||||
- Produce runbooks (`docs/operations/export-runbook.md`) and hardening guidance (`docs/security/export-hardening.md`).
|
||||
|
||||
## Documentation deliverables
|
||||
- `docs/modules/export-center/overview.md` — responsibilities, profiles, surfaces.
|
||||
- `docs/modules/export-center/architecture.md` — service topology, adapters, manifests, distribution flow.
|
||||
- `docs/modules/export-center/profiles.md`, `trivy-adapter.md`, `mirror-bundles.md`, `provenance-and-signing.md`, `api.md`, `cli.md` — keep aligned with shipped features.
|
||||
- Cross-link Orchestrator, Policy, VEX Lens, CLI, and Offline Kit docs whenever exports become dependencies.
|
||||
|
||||
## Acceptance criteria
|
||||
- Operators can create, monitor, and download an export; `cosign verify` (and CLI verify) succeeds against manifest + provenance, mapping back to source artifacts.
|
||||
- Trivy bundles import cleanly into Trivy across supported versions; mirror bundles run in Offline Kit reference environment (full + delta).
|
||||
- Policy snapshot runs reproduce deterministic decisions and include embedded `policyVersion` + `inputsHash`.
|
||||
- Tenant scoping and RBAC block unauthorized actions; encryption-enabled bundles lock data to recipient keys.
|
||||
- Metrics (`exporter_run_duration_seconds`, `exporter_bundle_bytes_total`, `exporter_run_failures_total`) and dashboards reflect live runs; alerts trigger on sustained failure rates.
|
||||
- Retried runs remain idempotent: manifests, hashes, and distribution artefacts match across identical inputs.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Schema drift (Trivy / policy):** versioned adapters with compatibility gates, CI integration tests, fail-fast with actionable errors.
|
||||
- **Bundle bloat:** zstd compression, sharding, delta exports, OCI dedupe.
|
||||
- **Data leakage:** strict schema allowlists, tenancy filters, redaction enforcement, encryption options.
|
||||
- **Non-determinism:** embed policy snapshots, enforce deterministic ordering, include content hashes in manifest.
|
||||
- **Operational slowness:** streaming downloads with range support, resumable CLI, concurrency limits, retry policies for workers.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** adapter mapping, manifest hashing, signing/attestation, delta computation, encryption round-trips.
|
||||
- **Integration:** end-to-end runs for every profile, verification workflows, OCI push/pull, resume/abort scenarios.
|
||||
- **Compatibility:** matrix tests for Trivy versions, mirror bundle import in Offline Kit sample environment.
|
||||
- **Security:** tenant fuzzing, RBAC coverage, redaction/PII filters, key rotation.
|
||||
- **Performance & chaos:** stress exports with large datasets, simulate worker/API failures mid-run, confirm deterministic recovery.
|
||||
|
||||
## Definition of done
|
||||
- Service, worker, and adapters deployed with telemetry & alerting.
|
||||
- CLI & Console workflows published, Offline Kit instructions updated.
|
||||
- Documentation set listed above refreshed; imposed rule statements appended where required.
|
||||
- CI pipelines include schema validation, profile verification, and determinism checks.
|
||||
- ./TASKS.md + ../../TASKS.md reflect current status for in-flight stories.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs sprint: `docs/implplan/SPRINT_0320_0001_0001_docs_modules_export_center.md`; statuses mirrored in `docs/modules/export-center/TASKS.md`.
|
||||
- Observability evidence stub lives in `operations/observability.md` with Grafana placeholder under `operations/dashboards/`.
|
||||
- Bundle/profile/offline manifest guidance maintained in `devportal-offline*.md`, `mirror-bundles.md`, and `provenance-and-signing.md`; update sprint/TASKS if these change.
|
||||
@@ -14,3 +14,25 @@ Immutable, append-only event ledger for tracking vulnerability findings, policy
|
||||
- Schema catalog (events/projections/exports): `schema-catalog.md`
|
||||
- Merkle & external anchor policy: `merkle-anchor-policy.md`
|
||||
- Tenant isolation & redaction manifest: `tenant-isolation-redaction.md`
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Delivery Phases
|
||||
- **Phase 1 – Observability baselines:** Instrument writer/projector with metrics, structured logs, OTLP exporters, Grafana dashboards + alert rules
|
||||
- **Phase 2 – Determinism harness:** Finalize NDJSON fixtures for ≥5M findings/tenant, implement replay harness CLI, add CI pipeline jobs
|
||||
- **Phase 3 – Deployment & backup collateral:** Integrate ledger service into Compose/Helm, automate PostgreSQL migrations, document backup cadence
|
||||
- **Phase 4 – Provenance & air-gap extensions:** Ingest orchestrator run export metadata, extend ledger events for bundle provenance, store attestation pointers
|
||||
|
||||
### Key Dependencies
|
||||
- AdvisoryAI Sprint 110.A completion (raw findings parity)
|
||||
- Observability schema approval to unblock Phase 1 instrumentation
|
||||
- QA lab capacity for 5M replay checkpoint
|
||||
- DevOps review of Compose/Helm overlays
|
||||
- Orchestrator export schema freeze for provenance linkage
|
||||
|
||||
### Acceptance Criteria
|
||||
- Metrics/logging/tracing implementation merged with dashboards exported
|
||||
- Harness CLI + fixtures + signed reports committed
|
||||
- Compose/Helm overlays + backup/restore runbooks validated
|
||||
- Air-gap provenance fields documented + implemented
|
||||
- Sprint tracker and release notes updated after each phase
|
||||
|
||||
@@ -1,45 +0,0 @@
|
||||
# Implementation Plan — Findings Ledger (Sprint 120)
|
||||
|
||||
## Phase 1 – Observability baselines (LEDGER-29-007)
|
||||
- Instrument writer/projector with metrics listed in `observability.md` (`ledger_write_latency_seconds`, `ledger_events_total`, `ledger_projection_lag_seconds`, etc.).
|
||||
- Emit structured logs (Serilog JSON) including chain/sequence/hash metadata.
|
||||
- Wire OTLP exporters, ensure `/metrics` endpoint exposes histogram buckets with exemplars.
|
||||
- Publish Grafana dashboards + alert rules (Policy SLO pack).
|
||||
- Deliver doc updates + sample Grafana JSON in repo (`docs/observability/dashboards/findings-ledger/`).
|
||||
|
||||
## Phase 2 – Determinism harness (LEDGER-29-008)
|
||||
- Finalize NDJSON fixtures for ≥5 M findings/tenant (per tenant/test scenario).
|
||||
- Implement `tools/LedgerReplayHarness` CLI as specified in `replay-harness.md`.
|
||||
- Add GitHub/Gitea pipeline job(s) running nightly (1 M) + weekly (5 M) harness plus DSSE signing.
|
||||
- Capture CPU/memory/latency metrics and commit signed reports for validation.
|
||||
- Provide runbook for QA + Ops to rerun harness in their environments.
|
||||
|
||||
## Phase 3 – Deployment & backup collateral (LEDGER-29-009)
|
||||
- Integrate ledger service into Compose (`docker-compose.prod.yaml`) and Helm values.
|
||||
- Automate PostgreSQL migrations (DatabaseMigrator invocation pre-start).
|
||||
- Document backup cadence (pg_dump + WAL archiving) and projection rebuild process (call harness).
|
||||
- Ensure Offline Kit packaging pulls binaries, migrations, harness, and default dashboards.
|
||||
|
||||
## Phase 4 – Provenance & air-gap extensions
|
||||
- LEDGER-34-101: ingest orchestrator run export metadata, index by artifact hash, expose audit endpoint.
|
||||
- LEDGER-AIRGAP-56/57/58: extend ledger events to capture bundle provenance, staleness metrics, timeline events.
|
||||
- LEDGER-ATTEST-73-001: store attestation pointers (DSSE IDs, Rekor metadata) for explainability.
|
||||
- For each extension, update schema doc + workflow inference doc to describe newly recorded fields and tenant-safe defaults.
|
||||
|
||||
## Dependencies & sequencing
|
||||
1. AdvisoryAI Sprint 110.A completion (raw findings parity).
|
||||
2. Observability schema approval (Nov 15) to unblock Phase 1 instrumentation.
|
||||
3. QA lab capacity for 5 M replay (Nov 18 checkpoint).
|
||||
4. DevOps review of Compose/Helm overlays (Nov 20).
|
||||
5. Orchestrator export schema freeze (Nov 25) for provenance linkage.
|
||||
|
||||
## Deliverables checklist
|
||||
- [ ] Metrics/logging/tracing implementation merged, dashboards exported.
|
||||
- [ ] Harness CLI + fixtures + signed reports committed.
|
||||
- [ ] Compose/Helm overlays + backup/restore runbooks validated.
|
||||
- [ ] Air-gap provenance fields documented + implemented.
|
||||
- [ ] Sprint tracker and release notes updated after each phase.
|
||||
|
||||
---
|
||||
|
||||
*Draft: 2025-11-13. Update when sequencing or dependencies change.*
|
||||
49
docs/modules/gateway/README.md
Normal file
49
docs/modules/gateway/README.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# Gateway
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/Gateway/`
|
||||
**Owner:** Platform Team
|
||||
|
||||
## Purpose
|
||||
|
||||
Gateway provides API routing, authentication enforcement, and transport abstraction for StellaOps services. Acts as the single entry point for external clients with support for HTTP/HTTPS and transport-agnostic messaging via Router module.
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- `StellaOps.Gateway.WebService` - API gateway with routing, middleware, and security
|
||||
|
||||
**Key Features:**
|
||||
- Route configuration and service discovery
|
||||
- Authorization middleware (Authority integration)
|
||||
- Request/response transformation
|
||||
- Rate limiting and throttling
|
||||
- Transport abstraction (HTTP, TCP/TLS, UDP, RabbitMQ, Valkey)
|
||||
|
||||
## Configuration
|
||||
|
||||
See `etc/policy-gateway.yaml.sample` for gateway configuration examples.
|
||||
|
||||
Key settings:
|
||||
- Service route mappings
|
||||
- Authority issuer and audience configuration
|
||||
- Transport protocols and endpoints
|
||||
- Security policies and CORS settings
|
||||
- Rate limiting rules
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Authority (authentication and authorization)
|
||||
- Router (transport-agnostic messaging)
|
||||
- All backend services (routing targets)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- Router Module: `../router/`
|
||||
- Authority Module: `../authority/`
|
||||
- API Reference: `../../09_API_CLI_REFERENCE.md`
|
||||
|
||||
## Current Status
|
||||
|
||||
Implemented with HTTP/HTTPS support. Integrated with Authority for token validation and authorization. Supports service routing and middleware composition.
|
||||
@@ -60,3 +60,31 @@ Graph Indexer + Graph API build the tenant-scoped knowledge graph that powers bl
|
||||
## Epic alignment
|
||||
- **Epic 5 – SBOM Graph Explorer:** Graph Indexer, Graph API, saved queries, overlays, Console/CLI experiences, Offline Kit parity.
|
||||
- Cross-epic ties: Policy reasoning (explain overlays), Scheduler recompute, Notify/Task Runner integration for graph incidents.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Delivery Phases
|
||||
- **Phase 1 – Graph Indexer foundations:** Stand up Graph Indexer service, node/edge schemas, ingestion from SBOM/Concelier/Excititor events, identity stability, snapshot materialisation
|
||||
- **Phase 2 – Graph API service:** Expose search, query, path, impact, diff, and overlay endpoints with RBAC, cost controls, streaming responses
|
||||
- **Phase 3 – Console & CLI experiences:** Ship Graph Explorer UI (WebGL canvas, filters, diff mode, overlays) and CLI for automation pipelines
|
||||
- **Phase 4 – Advanced analytics:** Implement clustering, centrality, saved queries, overlay caching, Policy Engine explain integration
|
||||
- **Phase 5 – Exports & offline:** Deliver GraphML/CSV/NDJSON exports, Offline Kit bundles with deterministic manifests
|
||||
- **Phase 6 – Observability & hardening:** Complete dashboards, alerts, runbooks, load/perf testing, a11y review
|
||||
|
||||
### Acceptance Criteria
|
||||
- Graph Indexer ingests SBOM/advisory/VEX events deterministically with tenant isolation and append-only provenance
|
||||
- Graph API serves endpoints within budgeted latency and enforces cost limits + RBAC
|
||||
- Console explorer visualises topology, overlays, diffs; CLI commands mirror functionality for automation
|
||||
- Exports and Offline Kit bundles reproduce snapshots and overlays with signed manifests
|
||||
- Observability dashboards/alerts detect ingest lag, query failures, cache churn, memory pressure; runbooks guide remediation
|
||||
- Policy/VEX overlays align with Policy Engine explain traces and VEX suppressions
|
||||
|
||||
### Key Risks & Mitigations
|
||||
- **Graph scale/complexity:** Adopt adjacency compression, cached overlays, streaming pagination, enforced query budgets
|
||||
- **Tenant bleed:** Strict tenant filters, fuzz tests, data masking, compliance reviews
|
||||
- **Runaway queries/visualization:** Cost planner, query timeout, UI hints, safe mode renders
|
||||
- **Cache poisoning:** Input validation, schema versioning, eviction policies
|
||||
- **Offline parity gaps:** Deterministic export pipeline, integration tests for Offline Kit import
|
||||
|
||||
### Current Active Sprint
|
||||
- Runtime & Signals 140.A: Clustering/centrality jobs, incremental/backfill pipeline, determinism tests, packaging
|
||||
|
||||
@@ -1,65 +0,0 @@
|
||||
# Implementation plan — Graph
|
||||
|
||||
## Delivery phases
|
||||
> Current active execution sprint: `docs/implplan/SPRINT_0141_0001_0001_graph_indexer.md` (Runtime & Signals 140.A).
|
||||
- **Phase 1 – Graph Indexer foundations**
|
||||
Stand up Graph Indexer service, node/edge schemas, ingestion from SBOM/Concelier/Excititor events, identity stability, and snapshot materialisation.
|
||||
- **Phase 2 – Graph API service**
|
||||
Expose search, query, path, impact, diff, and overlay endpoints with RBAC, cost controls, and streaming responses.
|
||||
- **Phase 3 – Console & CLI experiences**
|
||||
Ship Graph Explorer UI (WebGL canvas, filters, diff mode, overlays) and CLI (`stella sbom graph ...`) for automation pipelines.
|
||||
- **Phase 4 – Advanced analytics**
|
||||
Implement clustering, centrality, saved queries, overlay caching, and Policy Engine explain integration.
|
||||
- **Phase 5 – Exports & offline**
|
||||
Deliver GraphML/CSV/NDJSON exports, Offline Kit bundles (`nodes.jsonl`, `edges.jsonl`, overlays), and deterministic manifests.
|
||||
- **Phase 6 – Observability & hardening**
|
||||
Complete dashboards, alerts, runbooks, load/perf testing, and a11y/accessibility review.
|
||||
|
||||
## Work breakdown
|
||||
- **Services**
|
||||
- Graph Indexer: event consumers, node/edge builders, snapshot/version handling, aggregate metrics.
|
||||
- Graph API: validation, planner/cost guard, streaming tile engine, diff/overlay builder, exports.
|
||||
- Worker jobs: clustering, diff, overlay materialisation with backpressure awareness.
|
||||
- **Data model & storage**
|
||||
- Collections/tables (`graph_nodes`, `graph_edges`, `graph_snapshots`, `graph_saved_queries`, `graph_overlays_cache`), indexes, tenant partitioning, append-only change logs.
|
||||
- Evaluate document + adjacency vs graph DB abstraction; ensure deterministic serialization for exports.
|
||||
- **Console**
|
||||
- Feature module `graph-explorer` with routes, canvas renderer, panels, diff UI, saved queries, export workflows, a11y pass.
|
||||
- Telemetry instrumentation for user interactions and query budgets.
|
||||
- **CLI & SDK**
|
||||
- `stella sbom graph query|diff|impact|export`, with JSON schema and piping support.
|
||||
- SDK utilities for automation and CI pipelines.
|
||||
- **Policy & VEX integration**
|
||||
- Fetch explain traces for policy overlays, integrate VEX suppressions, align with Policy Engine & VEX Lens data models.
|
||||
- **Observability & Ops**
|
||||
- Metrics (ingest lag, query latency, cache hit rate), log/traces, dashboards, alerting for runaway queries and OOM.
|
||||
- Runbooks for incident classes (query denial, cache poisoning, degraded render).
|
||||
- **Documentation**
|
||||
- Maintain overview, API, query language, console guide, CLI reference, policy/VEX integration docs with compliance checklists.
|
||||
|
||||
## Acceptance criteria
|
||||
- Graph Indexer ingests SBOM/advisory/VEX events deterministically with tenant isolation and append-only provenance.
|
||||
- Graph API serves search/query/path/diff/overlay endpoints within budgeted latency and enforces cost limits + RBAC.
|
||||
- Console explorer visualises topology, overlays, diffs, saved queries; CLI commands mirror functionality for automation.
|
||||
- Exports (GraphML/CSV/NDJSON) and Offline Kit bundles reproduce snapshots and overlays with signed manifests.
|
||||
- Observability dashboards/alerts detect ingest lag, query failures, cache churn, and memory pressure; runbooks guide remediation.
|
||||
- Policy/VEX overlays align with Policy Engine explain traces and VEX suppressions.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Graph scale/complexity:** adopt adjacency compression, cached overlays, streaming pagination, enforced query budgets.
|
||||
- **Tenant bleed:** strict tenant filters, fuzz tests, data masking, compliance reviews.
|
||||
- **Runaway queries/visualization:** cost planner, query timeout, UI hints, safe mode renders.
|
||||
- **Cache poisoning:** input validation, schema versioning, eviction policies.
|
||||
- **Offline parity gaps:** deterministic export pipeline, integration tests for Offline Kit import.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** node/edge builders, identifier stability, overlay computations, query planner, diff engine.
|
||||
- **Integration:** end-to-end ingest + query flows across SBOM/advisory/VEX, saved query execution, diff exports.
|
||||
- **Performance:** large SBOM datasets, concurrency, memory profiling, WebGL rendering.
|
||||
- **Security:** tenant isolation tests, RBAC, query cost abuse.
|
||||
- **Offline:** export/import verification, manifest hashing, CLI replay.
|
||||
|
||||
## Definition of done
|
||||
- All phases delivered with telemetry, documentation, runbooks, and Offline Kit parity.
|
||||
- Console/CLI parity validated; a11y review complete.
|
||||
- ./TASKS.md and ../../TASKS.md updated; README/architecture/plan kept current with imposed rule references.
|
||||
44
docs/modules/issuer-directory/README.md
Normal file
44
docs/modules/issuer-directory/README.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# IssuerDirectory
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/IssuerDirectory/`
|
||||
**Owner:** VEX Guild
|
||||
|
||||
## Purpose
|
||||
|
||||
IssuerDirectory maintains a trust registry of CSAF publishers and VEX statement issuers. Provides discovery, validation, and trust scoring for upstream vulnerability advisories and VEX statements.
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- `StellaOps.IssuerDirectory` - Main service for issuer registry management and API
|
||||
|
||||
## Configuration
|
||||
|
||||
See `etc/issuer-directory.yaml.sample` for configuration options.
|
||||
|
||||
Key settings:
|
||||
- PostgreSQL connection (schema: `issuer_directory`)
|
||||
- Authority integration settings
|
||||
- Issuer discovery endpoints
|
||||
- Trust validation policies
|
||||
- CSAF provider metadata validation
|
||||
|
||||
## Dependencies
|
||||
|
||||
- PostgreSQL (schema: `issuer_directory`)
|
||||
- Authority (authentication)
|
||||
- Concelier (consumes issuer metadata)
|
||||
- VexHub (consumes issuer trust data)
|
||||
- VexLens (trust scoring integration)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- Concelier: `../concelier/`
|
||||
- VexHub: `../vexhub/`
|
||||
- VexLens: `../vex-lens/`
|
||||
|
||||
## Current Status
|
||||
|
||||
Implemented with CSAF publisher discovery and validation. Supports issuer metadata storage and trust registry queries. Integrated with VEX ingestion pipeline.
|
||||
46
docs/modules/mirror/README.md
Normal file
46
docs/modules/mirror/README.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# Mirror
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/Mirror/`
|
||||
**Owner:** Platform Team
|
||||
|
||||
## Purpose
|
||||
|
||||
Mirror creates offline package mirrors and advisory snapshots for air-gapped deployments. Enables deterministic, offline-first operation by capturing point-in-time snapshots of external package repositories and vulnerability feeds.
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- `StellaOps.Mirror.Creator` - Mirror creation and management tool
|
||||
|
||||
**Scripts:**
|
||||
- `make-thin-v1.sh` - Thin mirror creation script
|
||||
- `schedule-export-center-run.sh` - Integration with ExportCenter for scheduled exports
|
||||
|
||||
## Configuration
|
||||
|
||||
Configuration is typically provided via CLI arguments or environment variables.
|
||||
|
||||
Key features:
|
||||
- Package repository mirroring (distro mirrors, language ecosystems)
|
||||
- Advisory feed snapshots (CSAF, OSV, NVD)
|
||||
- Thin mirror creation (metadata-only for bandwidth optimization)
|
||||
- Integration with ExportCenter for bundle creation
|
||||
- Air-gap bundle generation
|
||||
|
||||
## Dependencies
|
||||
|
||||
- ExportCenter (for bundle packaging)
|
||||
- AirGap (for import/validation)
|
||||
- External package repositories and advisory feeds (as mirror sources)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- AirGap Module: `../airgap/`
|
||||
- ExportCenter: `../export-center/`
|
||||
- Offline Kit: `../../24_OFFLINE_KIT.md`
|
||||
- Operations: `./operations/` (if exists)
|
||||
|
||||
## Current Status
|
||||
|
||||
Mirror.Creator implemented with thin mirror support. Integrated with ExportCenter for air-gap bundle generation. Used for offline deployment preparation.
|
||||
@@ -50,3 +50,34 @@ Status for these items is tracked in `src/Notifier/StellaOps.Notifier/TASKS.md`
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 11 – Notifications Studio:** notifications workspace, preview tooling, immutable delivery ledger, throttling/digest controls, and forthcoming correlation/simulation features.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Delivery Phases
|
||||
- **Phase 1 – Core rules engine & delivery ledger:** Implement rules/channels schema, event ingestion, rule evaluation, idempotent deliveries, audit logging
|
||||
- **Phase 2 – Connectors & rendering:** Ship Slack/Teams/Email/Webhook connectors, template rendering, localization, throttling, retries, secret referencing
|
||||
- **Phase 3 – Console & CLI authoring:** Provide UI/CLI for rule authoring, previews, channel health, delivery browsing, digests, test sends
|
||||
- **Phase 4 – Governance & observability:** Add approvals, RBAC, tenant quotas, metrics/logs/traces, dashboards, alerts, runbooks
|
||||
- **Phase 5 – Offline & compliance:** Produce Offline Kit bundles (rules/channels/deploy scripts), signed exports, retention policies, auditing
|
||||
|
||||
### Acceptance Criteria
|
||||
- Rules evaluate deterministically per event; deliveries idempotent with audit trail and DSSE signatures
|
||||
- Channel connectors support retries, rate limits, health checks, previews; secrets referenced securely
|
||||
- Console/CLI support rule creation, testing, digests, delivery browsing, export/import workflows
|
||||
- Observability dashboards track delivery health; alerts fire for sustained failures or backlog; runbooks cover remediation
|
||||
- Offline Kit bundle contains configs, rules, digests, deployment scripts for air-gapped installs
|
||||
- Notify respects tenancy and RBAC; governance (approvals, change log) enforced for high-impact rules
|
||||
|
||||
### Key Risks & Mitigations
|
||||
- **Notification storms:** Throttling, digests, dedupe windows, preview/test gating
|
||||
- **Secret compromise:** Secret references only, rotation workflows, audit logging
|
||||
- **Connector API changes:** Versioned adapter layer, nightly health checks, fallback channels
|
||||
- **Noise vs signal:** Simulation previews, metrics, rule scoring, recommended defaults
|
||||
- **Offline parity:** Export/import of rules, connectors, digests with signed manifests
|
||||
|
||||
### Current Phase Progress
|
||||
- Phase 1: Core rules engine mostly complete; template dispatch/rendering in progress
|
||||
- Phase 2: Connector and rendering work not yet started; depends on Phase 1 completion
|
||||
- Phase 3: Console/CLI authoring work not started; depends on Phase 2 completion
|
||||
- Phase 4: Core observability complete; governance and risk notifications blocked on upstream dependencies
|
||||
- Phase 5: Offline basics complete; tenancy work blocked on upstream Sprint 0172
|
||||
|
||||
@@ -1,160 +0,0 @@
|
||||
# Implementation plan — Notify
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Core rules engine & delivery ledger**
|
||||
Implement rules/channels schema, event ingestion, rule evaluation, idempotent deliveries, and audit logging.
|
||||
- **Phase 2 – Connectors & rendering**
|
||||
Ship Slack/Teams/Email/Webhook connectors, template rendering, localization, throttling, retries, and secret referencing.
|
||||
- **Phase 3 – Console & CLI authoring**
|
||||
Provide UI/CLI for rule authoring, previews, channel health, delivery browsing, digests, and test sends.
|
||||
- **Phase 4 – Governance & observability**
|
||||
Add approvals, RBAC, tenant quotas, Notify metrics/logs/traces, dashboards, Notify-specific alerts, and Notify runbooks.
|
||||
- **Phase 5 – Offline & compliance**
|
||||
Produce Offline Kit bundles (rules/channels/deploy scripts), signed exports, retention policies, and auditing for regulated environments.
|
||||
|
||||
## Work breakdown
|
||||
- **Service & worker**
|
||||
- REST API for rules/channels/delivery history, idempotency middleware, digest scheduler.
|
||||
- Worker pipelines for event intake, rule matching, template rendering, delivery execution, retries, and throttling.
|
||||
- Delivery ledger capturing payload metadata, response, retry state, DSSE signatures.
|
||||
- **Connectors**
|
||||
- Slack/Teams/Email/Webhook plug-ins with configuration validation, rate limiting, error classification.
|
||||
- Secrets referenced via Authority/Secret store; no plaintext storage.
|
||||
- **Console & CLI**
|
||||
- Console module for rules builder, condition editor, preview, test send, delivery insights, digests and schedule configuration.
|
||||
- CLI (`stella notify rule|channel|delivery`) for automation, export/import.
|
||||
- **Integrations**
|
||||
- Event sources: Concelier, Excititor, Policy Engine, Vuln Explorer, Export Center, Attestor, Zastava, Scheduler.
|
||||
- Notify events to Notify (meta) for failure escalations, accepted-risk expiration reminders.
|
||||
- **Observability & ops**
|
||||
- Metrics: delivery success/failure, retry counts, throttle hits, digest generation, channel health.
|
||||
- Logs/traces with tenant, rule ID, channel, correlation ID; dashboards and alerts.
|
||||
- Runbooks for misconfigured channels, throttling, event backlog, incident digest.
|
||||
- **Docs & compliance**
|
||||
- Update Notifications Studio guides, channel runbooks, security/RBAC docs, Offline Kit instructions.
|
||||
- Provide compliance checklist (audit logging, retention, opt-out).
|
||||
|
||||
## Acceptance criteria
|
||||
- Rules evaluate deterministically per event; deliveries idempotent with audit trail and DSSE signatures.
|
||||
- Channel connectors support retries, rate limits, health checks, previews; secrets referenced securely.
|
||||
- Console/CLI support rule creation, testing, digests, delivery browsing, and export/import workflows.
|
||||
- Observability dashboards track delivery health; alerts fire for sustained failures or backlog; runbooks cover remediation.
|
||||
- Offline Kit bundle contains configs, rules, digests, and deployment scripts for air-gapped installs.
|
||||
- Notify respects tenancy and RBAC; governance (approvals, change log) enforced for high-impact rules.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Notification storms:** throttling, digests, dedupe windows, preview/test gating.
|
||||
- **Secret compromise:** secret references only, rotation workflows, audit logging.
|
||||
- **Connector API changes:** versioned adapter layer, nightly health checks, fallback channels.
|
||||
- **Noise vs signal:** simulation previews, metrics, rule scoring, recommended defaults.
|
||||
- **Offline parity:** export/import of rules, connectors, and digests with signed manifests.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** rule evaluation, template rendering, connector clients, throttling, digests.
|
||||
- **Integration:** end-to-end events from core services, multi-channel routing, retries, audit logging.
|
||||
- **Performance:** burst throttling, digest creation, large rule sets.
|
||||
- **Security:** RBAC tests, tenant isolation, secret reference validation, DSSE signature verification.
|
||||
- **Offline:** export/import round-trips, Offline Kit deployment, manual delivery replay.
|
||||
|
||||
## Definition of done
|
||||
- Notify service, workers, connectors, Console/CLI, observability, and Offline Kit assets shipped with documentation and runbooks.
|
||||
- Compliance checklist appended to docs; ./TASKS.md and ../../TASKS.md updated with progress.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs sprint: `docs/implplan/SPRINT_322_docs_modules_notify.md`; statuses mirrored in `docs/modules/notify/TASKS.md`.
|
||||
- Observability evidence stub: `operations/observability.md` and `operations/dashboards/notify-observability.json` (to be populated after next demo outputs).
|
||||
- NOTIFY-DOCS-0002 remains blocked pending NOTIFY-SVC-39-001..004 (correlation/digests/simulation/quiet hours); keep sprint/TASKS synced when those land.
|
||||
|
||||
---
|
||||
|
||||
## Sprint readiness tracker
|
||||
|
||||
> Last updated: 2025-11-27 (NOTIFY-ENG-0001)
|
||||
|
||||
This section maps delivery phases to implementation sprints and tracks readiness checkpoints.
|
||||
|
||||
### Phase 1 — Core rules engine & delivery ledger
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-SVC-37-001 | ✅ DONE (2025-11-24) | SPRINT_0172_0001_0002_notifier_ii | Pack approval contract published (OpenAPI schema, payloads). |
|
||||
| NOTIFY-SVC-37-002 | ✅ DONE (2025-11-24) | SPRINT_0172_0001_0002_notifier_ii | Ingestion endpoint with Mongo persistence, idempotent writes, audit trail. |
|
||||
| NOTIFY-SVC-37-003 | 🔄 DOING | SPRINT_0172_0001_0002_notifier_ii | Approval/policy templates, routing predicates; dispatch/rendering pending. |
|
||||
| NOTIFY-SVC-37-004 | ✅ DONE (2025-11-24) | SPRINT_0172_0001_0002_notifier_ii | Acknowledgement API, test harness, metrics. |
|
||||
| NOTIFY-OAS-61-001 | ✅ DONE (2025-11-17) | SPRINT_0171_0001_0001_notifier_i | OAS with rules/templates/incidents/quiet hours endpoints. |
|
||||
| NOTIFY-OAS-61-002 | ✅ DONE (2025-11-17) | SPRINT_0171_0001_0001_notifier_i | `/.well-known/openapi` discovery endpoint. |
|
||||
| NOTIFY-OAS-62-001 | ✅ DONE (2025-11-17) | SPRINT_0171_0001_0001_notifier_i | SDK examples for rule CRUD. |
|
||||
| NOTIFY-OAS-63-001 | ✅ DONE (2025-11-17) | SPRINT_0171_0001_0001_notifier_i | Deprecation headers and templates. |
|
||||
|
||||
**Checkpoint:** Core rules engine mostly complete; template dispatch/rendering in progress.
|
||||
|
||||
### Phase 2 — Connectors & rendering
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-SVC-38-002 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Channel adapters (email, chat webhook, generic webhook) with retry policies. |
|
||||
| NOTIFY-SVC-38-003 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Template service, renderer with redaction and localization. |
|
||||
| NOTIFY-SVC-38-004 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | REST + WS APIs for rules CRUD, templates preview, incidents. |
|
||||
| NOTIFY-DOC-70-001 | ✅ DONE (2025-11-02) | SPRINT_0171_0001_0001_notifier_i | Architecture docs for `src/Notify` vs `src/Notifier` split. |
|
||||
|
||||
**Checkpoint:** Connector and rendering work not yet started; depends on Phase 1 completion.
|
||||
|
||||
### Phase 3 — Console & CLI authoring
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-SVC-39-001 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Correlation engine with throttler, quiet hours, incident lifecycle. |
|
||||
| NOTIFY-SVC-39-002 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Digest generator with schedule runner. |
|
||||
| NOTIFY-SVC-39-003 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Simulation engine for dry-run rules against historical events. |
|
||||
| NOTIFY-SVC-39-004 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Quiet hour calendars with audit logging. |
|
||||
|
||||
**Checkpoint:** Console/CLI authoring work not started; depends on Phase 2 completion.
|
||||
|
||||
### Phase 4 — Governance & observability
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-SVC-40-001 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Escalations, on-call schedules, PagerDuty/OpsGenie adapters. |
|
||||
| NOTIFY-SVC-40-002 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Summary storm breaker, localization bundles. |
|
||||
| NOTIFY-SVC-40-003 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Security hardening (signed ack links, webhook HMAC). |
|
||||
| NOTIFY-SVC-40-004 | 📝 TODO | SPRINT_0172_0001_0002_notifier_ii | Observability metrics/traces, dead-letter handling, chaos tests. |
|
||||
| NOTIFY-OBS-51-001 | ✅ DONE (2025-11-22) | SPRINT_0171_0001_0001_notifier_i | SLO evaluator webhooks with templates/routing/suppression. |
|
||||
| NOTIFY-OBS-55-001 | ✅ DONE (2025-11-22) | SPRINT_0171_0001_0001_notifier_i | Incident mode templates with evidence/trace/retention context. |
|
||||
| NOTIFY-ATTEST-74-001 | ✅ DONE (2025-11-16) | SPRINT_0171_0001_0001_notifier_i | Templates for verification failures, key revocations, transparency. |
|
||||
| NOTIFY-ATTEST-74-002 | 📝 TODO | SPRINT_0171_0001_0001_notifier_i | Wire notifications to key rotation/revocation events. |
|
||||
| NOTIFY-RISK-66-001 | ⏳ BLOCKED | SPRINT_0171_0001_0001_notifier_i | Risk severity escalation triggers; needs POLICY-RISK-40-002. |
|
||||
| NOTIFY-RISK-67-001 | ⏳ BLOCKED | SPRINT_0171_0001_0001_notifier_i | Risk profile publish/deprecate notifications. |
|
||||
| NOTIFY-RISK-68-001 | ⏳ BLOCKED | SPRINT_0171_0001_0001_notifier_i | Per-profile routing, quiet hours, dedupe. |
|
||||
|
||||
**Checkpoint:** Core observability complete; governance and risk notifications blocked on upstream dependencies.
|
||||
|
||||
### Phase 5 — Offline & compliance
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| NOTIFY-AIRGAP-56-002 | ✅ DONE | SPRINT_0171_0001_0001_notifier_i | Bootstrap Pack with deterministic secrets and offline validation. |
|
||||
| NOTIFY-TEN-48-001 | ⏳ BLOCKED | SPRINT_0173_0001_0003_notifier_iii | Tenant-scope rules/templates; needs Sprint 0172 tenancy model. |
|
||||
|
||||
**Checkpoint:** Offline basics complete; tenancy work blocked on upstream Sprint 0172.
|
||||
|
||||
---
|
||||
|
||||
### Overall readiness summary
|
||||
|
||||
| Phase | Status | Blocking items |
|
||||
|-------|--------|----------------|
|
||||
| **1 – Core rules engine** | 🔄 In progress | NOTIFY-SVC-37-003 dispatch/rendering |
|
||||
| **2 – Connectors & rendering** | 📝 Not started | Phase 1 completion |
|
||||
| **3 – Console & CLI** | 📝 Not started | Phase 2 completion |
|
||||
| **4 – Governance & observability** | 🔄 Partial | POLICY-RISK-40-002 for risk notifications |
|
||||
| **5 – Offline & compliance** | 🔄 Partial | Sprint 0172 tenancy model |
|
||||
|
||||
### Cross-module dependencies
|
||||
|
||||
| Dependency | Required by | Status |
|
||||
|------------|-------------|--------|
|
||||
| Attestor payload localization | NOTIFY-ATTEST-74-002 | Freeze pending |
|
||||
| POLICY-RISK-40-002 export | NOTIFY-RISK-66/67/68 | BLOCKED |
|
||||
| Sprint 0172 tenancy model | NOTIFY-TEN-48-001 | In progress |
|
||||
| Telemetry SLO webhook schema | NOTIFY-OBS-51-001 | ✅ Published (`docs/modules/notify/slo-webhook-schema.md`) |
|
||||
|
||||
### Next actions
|
||||
1. Complete NOTIFY-SVC-37-003 dispatch/rendering wiring (Sprint 0172).
|
||||
2. Start NOTIFY-SVC-38-002 channel adapters once Phase 1 closes.
|
||||
3. Track POLICY-RISK-40-002 to unblock risk notification tasks.
|
||||
4. Monitor Sprint 0172 tenancy model for NOTIFY-TEN-48-001.
|
||||
@@ -33,6 +33,46 @@ The Orchestrator schedules, observes, and recovers ingestion and analysis jobs a
|
||||
- Log streaming: SSE/WS endpoints carry correlationId + tenant/project; buffer size and retention must be documented in runbooks.
|
||||
- When using `orch:quota` / `orch:backfill` scopes, capture reason/ticket fields in runbooks and audit checklists.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 1 – Core service & job ledger (Complete)
|
||||
- PostgreSQL schema with sources, runs, jobs, artifacts, DAG edges, quotas, schedules, incidents
|
||||
- Lease manager with heartbeats, retries, dead-letter queues
|
||||
- Token-bucket rate limiter per tenant/source.host with adaptive refill
|
||||
- Watermark/backfill orchestration for event-time windows
|
||||
|
||||
### Phase 2 – Worker SDK & artifact registry (Complete)
|
||||
- Claim/heartbeat/report contract with deterministic artifact hashing
|
||||
- Idempotency enforcement and worker SDKs for .NET/Rust/Go agents
|
||||
- Integrated with Concelier, Excititor, SBOM Service, Policy Engine
|
||||
|
||||
### Phase 3 – Observability & dashboard (In Progress)
|
||||
- Metrics: queue depth, job latency, failure classes, rate-limit hits, burn rate
|
||||
- Error clustering for HTTP 429/5xx, schema mismatches, parse errors
|
||||
- SSE/WebSocket feeds for Console updates, Gantt timeline/DAG JSON
|
||||
|
||||
### Phase 4 – Controls & resilience (Planned)
|
||||
- Pause/resume/throttle/retry/backfill tooling
|
||||
- Dead-letter review, circuit breakers, blackouts, backpressure handling
|
||||
- Automation hooks and control plane APIs
|
||||
|
||||
### Phase 5 – Offline & compliance (Planned)
|
||||
- Deterministic audit bundles (jobs.jsonl, history.jsonl, throttles.jsonl)
|
||||
- Provenance manifests and offline replay scripts
|
||||
- Tenant isolation validation and secret redaction
|
||||
|
||||
### Key Acceptance Criteria
|
||||
- Schedules all jobs with quotas, rate limits, idempotency; preserves provenance
|
||||
- Console reflects real-time DAG status, queue depth, SLO burn rate
|
||||
- Observability stack exposes metrics, logs, traces, incidents for stuck jobs and throttling
|
||||
- Offline audit bundles reproduce job history deterministically with verified signatures
|
||||
|
||||
### Technical Decisions & Risks
|
||||
- Backpressure/queue overload mitigated via adaptive token buckets, circuit breakers, dynamic concurrency
|
||||
- Upstream vendor throttles managed with visible state, automatic jitter and retry
|
||||
- Tenant leakage prevented through API/queue/storage filters, fuzz tests, redaction
|
||||
- Complex DAG errors handled with diagnostics, error clustering, partial replay tooling
|
||||
|
||||
## Epic alignment
|
||||
- Epic 9: Source & Job Orchestrator Dashboard.
|
||||
- ORCH stories in ../../TASKS.md.
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
# Implementation plan — Source & Job Orchestrator
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Core service & job ledger**
|
||||
Implement source registry, run/job tables, queue abstraction, lease management, token-bucket rate limiting, watchdogs, and API primitives (`/sources`, `/runs`, `/jobs`).
|
||||
- **Phase 2 – Worker SDK & artifact registry**
|
||||
Embed worker SDK in Conseiller, Excititor, SBOM, Policy Engine; capture artifact metadata + hashes, enforce idempotency, publish progress/metrics.
|
||||
- **Phase 3 – Observability & dashboard**
|
||||
Ship metrics, traces, incident logging, SSE/WebSocket feeds, and Console dashboard (DAG/timeline, heatmaps, error clustering, SLO burn rate).
|
||||
- **Phase 4 – Controls & resilience**
|
||||
Deliver pause/resume/throttle/retry/backfill tooling, dead-letter review, circuit breakers, blackouts, backpressure handling, and automation hooks.
|
||||
- **Phase 5 – Offline & compliance**
|
||||
Generate deterministic audit bundles (`jobs.jsonl`, `history.jsonl`, `throttles.jsonl`), provenance manifests, and offline replay scripts.
|
||||
|
||||
## Work breakdown
|
||||
- **Service & persistence**
|
||||
- Postgres schema (`sources`, `runs`, `jobs`, `artifacts`, `dag_edges`, `quotas`, `schedules`, `incidents`).
|
||||
- Lease manager with heartbeats, retries, and dead-letter queues.
|
||||
- Token-bucket rate limiter per `{tenant, source.host}`; adaptive refill on upstream throttles.
|
||||
- Watermark/backfill orchestration for event-time windows.
|
||||
- **Worker SDK**
|
||||
- Claim/heartbeat/report contract, deterministic artifact hashing, idempotency enforcement.
|
||||
- Library release for .NET workers plus language bindings for Rust/Go ingestion agents.
|
||||
- **Control plane APIs**
|
||||
- CRUD for sources, runs, jobs, quotas, schedules; control actions (retry, cancel, prioritize, pause/resume, backfill).
|
||||
- SSE/WebSocket stream for Console updates.
|
||||
- **Observability**
|
||||
- Metrics: queue depth, job latency, failure classes, rate-limit hits, burn rate.
|
||||
- Error clustering (HTTP 429/5xx, schema mismatch, parse errors), incident logging with reason codes.
|
||||
- Gantt timeline and DAG JSON for Console visualisation.
|
||||
- **Console & CLI**
|
||||
- Console app sections: overview, sources, runs, job detail, incidents, throttles.
|
||||
- CLI commands: `stella orchestrator sources|runs|jobs|throttle|backfill`.
|
||||
- **Compliance & offline**
|
||||
- Immutable audit bundles with signatures; exports via Export Center; Offline Kit instructions.
|
||||
- Tenant isolation validation and secret redaction for logs/UI.
|
||||
|
||||
## Acceptance criteria
|
||||
- Orchestrator schedules all advisory/VEX/SBOM/policy jobs with quotas, rate limits, and idempotency; retries and replay preserve provenance.
|
||||
- Console dashboard reflects real-time DAG status, queue depth, SLO burn rate, and allows pause/resume/throttle/backfill with audit trail.
|
||||
- Worker SDK integrated across producer services, emitting progress and artifact metadata.
|
||||
- Observability stack exposes metrics, logs, traces, incidents, and alerts for stuck jobs, throttling, and failure spikes.
|
||||
- Offline audit bundles reproduce job history deterministically and verify signatures.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Backpressure/queue overload:** adaptive token buckets, circuit breakers, dynamic concurrency; degrade gracefully.
|
||||
- **Upstream vendor throttles:** throttle management with user-visible state, automatic jitter and retry.
|
||||
- **Tenant leakage:** enforce tenant filters at API/queue/storage, fuzz tests, redaction.
|
||||
- **Complex DAG errors:** built-in diagnostics, error clustering, partial replay tooling.
|
||||
- **Operator error:** confirmation prompts, RBAC, runbook guidance, reason codes logged.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** scheduling, quota enforcement, lease renewals, token bucket, watermark arithmetic.
|
||||
- **Integration:** worker SDK with Conseiller/Excititor/SBOM pipelines, pause/resume/backfill flows, failure recovery.
|
||||
- **Performance:** high-volume job workloads, queue backpressure, concurrency caps, dashboard SSE load tests.
|
||||
- **Chaos:** simulate upstream outages, stuck workers, clock skew, Postgres failover.
|
||||
- **Compliance:** audit bundle generation, signature verification, offline replay.
|
||||
|
||||
## Definition of done
|
||||
- All phases delivered with telemetry, dashboards, and runbooks published.
|
||||
- Console + CLI parity validated; Offline Kit instructions complete.
|
||||
- ./TASKS.md and ../../TASKS.md updated with status; documentation (README/architecture/this plan) reflects latest behaviour.
|
||||
@@ -1,7 +1,7 @@
|
||||
# StellaOps Platform
|
||||
|
||||
Platform module describes cross-cutting architecture, contracts, and guardrails that bind the services together.
|
||||
|
||||
# StellaOps Platform
|
||||
|
||||
Platform module describes cross-cutting architecture, contracts, and guardrails that bind the services together.
|
||||
|
||||
## Latest updates (2025-11-30)
|
||||
- Sprint tracker `docs/implplan/SPRINT_0324_0001_0001_docs_modules_platform.md` and module `TASKS.md` added to mirror status.
|
||||
- README now points to architecture overview, AOC references, and offline guidance entry points.
|
||||
@@ -12,25 +12,55 @@ Platform module describes cross-cutting architecture, contracts, and guardrails
|
||||
- Capture Aggregation-Only Contract guidance and migration playbooks.
|
||||
- Document shared services such as API gateway, tenancy, quotas, and offline posture.
|
||||
- Coordinate platform-wide epics and compliance checklists.
|
||||
|
||||
|
||||
## Key components
|
||||
- Architecture overview in `architecture-overview.md`.
|
||||
- Platform architecture summary in `architecture.md`.
|
||||
- High-level reference: `../../07_HIGH_LEVEL_ARCHITECTURE.md`.
|
||||
|
||||
## Integrations & dependencies
|
||||
- All StellaOps services via shared contracts (AOC, telemetry, security).
|
||||
- DevOps for release governance.
|
||||
- Docs guild for cross-module onboarding.
|
||||
|
||||
|
||||
## Integrations & dependencies
|
||||
- All StellaOps services via shared contracts (AOC, telemetry, security).
|
||||
- DevOps for release governance.
|
||||
- Docs guild for cross-module onboarding.
|
||||
|
||||
## Operational notes
|
||||
- Docs-only module; focus is architectural governance and cross-module guardrails.
|
||||
- Glossaries and guardrails cross-linked across docs; keep AOC references current.
|
||||
- Status mirrors: sprint file and `docs/modules/platform/TASKS.md`.
|
||||
|
||||
## Backlog references
|
||||
- DOCS-AOC-19-002/003 in ../../TASKS.md.
|
||||
- Future platform epics under docs/implplan/SPRINTS.md.
|
||||
|
||||
## Epic alignment
|
||||
- Aligns with the Aggregation-Only Contract reference, Policy and Policy Studio guides, Graph/Vulnerability Explorer documentation, and the Orchestrator, Advisory AI, and Notifications implementation plans to keep platform guardrails consistent across services.
|
||||
|
||||
## Backlog references
|
||||
- DOCS-AOC-19-002/003 in ../../TASKS.md.
|
||||
- Future platform epics under docs/implplan/SPRINTS.md.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Current Objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases
|
||||
- Keep documentation, telemetry, and runbooks aligned with latest sprint outcomes
|
||||
- Coordinate cross-cutting contracts and guardrails across all modules
|
||||
|
||||
### Epic Milestones
|
||||
- Epic 1 – AOC enforcement: authoritative guardrail docs, schemas, verifier checklists (ongoing maintenance)
|
||||
- Epics 2 & 4 – Policy Engine/Studio: platform-wide governance, approvals, tenancy scopes (coordinated)
|
||||
- Epic 5 – SBOM Graph Explorer: shared contracts for graph indexing and overlays (defined)
|
||||
- Epics 6-11 – Cross-cutting contracts: Explorer, Lens, AI, Orchestrator, Notifications alignment (active)
|
||||
|
||||
### Coordination Approach
|
||||
- Review AGENTS.md before starting new work
|
||||
- Sync with cross-cutting teams via sprint files in docs/implplan/SPRINT_*.md
|
||||
- Update implementation plan when scope, dependencies, or guardrails change
|
||||
- Mirror status across sprint tracker and docs/modules/platform/TASKS.md
|
||||
|
||||
### Key Documentation Assets
|
||||
- architecture-overview.md: system-wide integration diagrams
|
||||
- AOC references: aggregation-only contract guidance and migration playbooks
|
||||
- Offline guidance: sealed-mode deployments and air-gap operation procedures
|
||||
- Glossaries and guardrails: cross-linked across all module documentation
|
||||
|
||||
### Technical Notes
|
||||
- Platform is docs-only; no runtime services
|
||||
- Focus on architectural governance and cross-module guardrails
|
||||
- Ensures discoverability of Offline Kit and AOC references from README/architecture
|
||||
|
||||
## Epic alignment
|
||||
- Aligns with the Aggregation-Only Contract reference, Policy and Policy Studio guides, Graph/Vulnerability Explorer documentation, and the Orchestrator, Advisory AI, and Notifications implementation plans to keep platform guardrails consistent across services.
|
||||
|
||||
@@ -1,27 +0,0 @@
|
||||
# Implementation plan — Platform
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 1 – AOC enforcement:** maintain authoritative guardrail docs, schemas, and verifier checklists.
|
||||
- **Epics 2 & 4 – Policy Engine/Studio:** coordinate platform-wide governance, approvals, and tenancy scopes.
|
||||
- **Epic 5 – SBOM Graph Explorer:** define shared contracts for graph indexing and overlays.
|
||||
- **Epics 6–11:** ensure cross-cutting contracts (Explorer, Lens, AI, Orchestrator, Notifications) stay aligned.
|
||||
- Track additional platform updates in ../../TASKS.md and docs/implplan/SPRINTS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs sprint: `docs/implplan/SPRINT_0324_0001_0001_docs_modules_platform.md`; statuses mirrored in `docs/modules/platform/TASKS.md`.
|
||||
- Keep links to `architecture-overview.md` and `../../07_HIGH_LEVEL_ARCHITECTURE.md` current; update both sprint and TASKS if platform guardrails change.
|
||||
- Platform is docs-only; ensure Offline Kit and AOC references remain discoverable from README/architecture.
|
||||
@@ -29,6 +29,55 @@ Policy Engine compiles and evaluates Stella DSL policies deterministically, prod
|
||||
- DOCS-POLICY-20-001 … DOCS-POLICY-20-012 (completed baseline).
|
||||
- DOCS-POLICY-23-007 (upcoming command updates).
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 1 – Deterministic evaluation core (Complete)
|
||||
- DSL compiler with caching, static analysis, runtime guardrails
|
||||
- Batch evaluator with deterministic ordering, change-stream inputs
|
||||
- Append-only effective findings ledger
|
||||
- Explain trace generation with evidence linking
|
||||
|
||||
### Phase 2 – Orchestration & incremental runs (In Progress)
|
||||
- Run scheduler with job leasing, fair-share per tenant/policy
|
||||
- Determinism hash verification and replay validation
|
||||
- Incremental delta processing with change-stream integration
|
||||
- Time-travel snapshots and resume cursors
|
||||
|
||||
### Phase 3 – Policy Studio workflows (Planned)
|
||||
- Policy registry with draft/review/approved lifecycle
|
||||
- Signed promotion pipeline with multi-step approvals
|
||||
- Console integration: editor, simulation, approvals, explain viewer
|
||||
- CLI parity for policy management operations
|
||||
|
||||
### Phase 4 – Simulation & approvals (Planned)
|
||||
- Diff/simulation APIs with rationale breakdown
|
||||
- Approval queues with change management workflows
|
||||
- Integration with CLI/Console for policy previews
|
||||
|
||||
### Phase 5 – Exports & offline parity (Planned)
|
||||
- Policy bundles with deterministic manifests
|
||||
- Explain archives for audit and review
|
||||
- Offline Kit assets with signed packages
|
||||
- Export Center integration
|
||||
|
||||
### Phase 6 – Observability & hardening (Planned)
|
||||
- Metrics: run duration, evaluation verdict counts, simulation latency
|
||||
- Guard violation detection and alerting
|
||||
- Incident response runbooks and compliance attestations
|
||||
|
||||
### Key Acceptance Criteria
|
||||
- Evaluation deterministic across runs; effective findings materialised only by Policy Engine
|
||||
- Incremental runs handle deltas within ≤5 min SLA; replay verification succeeds
|
||||
- Policy Studio supports full lifecycle with signed promotions and explain traces
|
||||
- Exports reproducible with signed manifests; Offline Kit delivers same tooling
|
||||
- Guardrails prevent forbidden IO; static analysis integrated into CI
|
||||
|
||||
### Technical Decisions & Risks
|
||||
- Non-determinism prevented via strict static analysis, runtime guards, replay tests
|
||||
- Policy drift managed through simulation previews, approval workflow, audit trail
|
||||
- Scaling handled via sharded workers, incremental deltas, caching, queue fairness
|
||||
- Guard bypass prevented by analyzers in CI and runtime rejection of forbidden operations
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver deterministic evaluation, DSL infrastructure, explain traces, and incremental runs.
|
||||
- **Epic 4 – Policy Studio:** integrate registry workflows, simulation at scale, approvals, and promotion semantics.
|
||||
|
||||
@@ -1,67 +0,0 @@
|
||||
# Implementation plan — Policy Engine
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Deterministic evaluation core**
|
||||
Finalise DSL compiler, runtime guardrails, evaluation workers, change-stream integration (advisories, VEX, SBOM), and append-only effective findings.
|
||||
- **Phase 2 – Orchestration & incremental runs**
|
||||
Implement run scheduler, incremental deltas, change-stream replay, simulation hooks, and determinism hashing.
|
||||
- **Phase 3 – Policy Studio workflows**
|
||||
Deliver policy registry, versioning, approvals, explain trace API, client editor integration, and signed promotion pipelines.
|
||||
- **Phase 4 – Simulation & approvals**
|
||||
Provide diff/simulation APIs, approval queues, change management, and integration with CLI/Console.
|
||||
- **Phase 5 – Exports & offline parity**
|
||||
Produce policy bundles, explain archives, Offline Kit assets, and deterministic manifests; integrate with Export Center.
|
||||
- **Phase 6 – Observability & hardening**
|
||||
Ship metrics, logs, traces, incident response runbooks, guardrail analyzers, and compliance attestations.
|
||||
|
||||
## Work breakdown
|
||||
- **Evaluation engine**
|
||||
- DSL compiler with caching, static analysis, and guard rails (no wall-clock/random/network outside allowlist).
|
||||
- Batch evaluator with deterministic ordering, change-stream inputs, policy IR caching.
|
||||
- Explain trace generation, evidence linking, storage in object store.
|
||||
- **Run orchestration**
|
||||
- Scheduler for incremental runs, job leasing, fair-share per tenant/policy.
|
||||
- Determinism hash + replay verification, time-travel snapshots, resume cursors.
|
||||
- Simulation endpoints returning diff summaries, rationale breakdown, exit codes.
|
||||
- **Policy Studio**
|
||||
- Policy registry (draft→review→approved), signed promotion pipeline, approvals workflow (multi-step).
|
||||
- Console integration (editor, simulation, approvals, explain viewer) and CLI parity.
|
||||
- **Integrations**
|
||||
- Inputs: Concelier, Excititor, SBOM Service, VEX Lens, runtime signals.
|
||||
- Outputs: Findings ledger, Vuln Explorer, Notify (policy events), Export Center (policy bundles).
|
||||
- Authority scopes, tenancy enforcement, RBAC for policy author/reviewer/operator.
|
||||
- **Observability & compliance**
|
||||
- Metrics: run duration, evaluation verdict counts, simulation latency, guard violations.
|
||||
- Logs/traces with trace ID propagation, policy version references, tenant scoping.
|
||||
- Guard analyzers (static + runtime), unit/property tests, compliance reports.
|
||||
- **Docs & tooling**
|
||||
- Update DSL guide, policy lifecycle/runbooks, simulation manual, CLI reference, Offline Kit instructions.
|
||||
- Provide sample policies, fixtures, and analyzer rules.
|
||||
|
||||
## Acceptance criteria
|
||||
- Evaluation engine deterministic across runs; effective findings materialised only by Policy Engine; guardrails prevent forbidden IO.
|
||||
- Incremental runs handle advisory/VEX/SBOM deltas within ≤5 min SLA; determinism hash and replay verification succeed.
|
||||
- Policy Studio supports draft/review/approval, signed promotions, simulation diffing, and explain traces in UI/CLI.
|
||||
- Exports (policy bundles, explain archives) reproducible with signed manifests; Offline Kit packages deliver same tooling.
|
||||
- Observability dashboards show run metrics, guard violations, simulation usage; alerts trigger on determinism hash mismatch or backlog.
|
||||
- CLI/Console parity for policy management, simulation, approvals, and export workflows.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Non-determinism:** strict static analysis, runtime guard, determinism hash, replay tests.
|
||||
- **Policy drift vs reality:** simulation diff previews, approval workflow, history/audit trail.
|
||||
- **Scaling evaluations:** sharded workers, incremental deltas, caching, job queue fairness.
|
||||
- **Guard bypass:** analyzers integrated into CI, runtime guard rejects forbidden operations.
|
||||
- **Offline compliance:** deterministic exports, manifest verification, documentation for sealed-mode deployments.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** DSL parsing, guard analyzer, evaluation pipeline, simulation diff calculations.
|
||||
- **Property:** randomised policy inputs verifying determinism and guard enforcement.
|
||||
- **Integration:** Concelier/Excititor/SBOM feeds → Policy Engine → findings ledger, simulation, approvals.
|
||||
- **Performance:** evaluation throughput, change-stream backlog recovery, simulation under load.
|
||||
- **Security/compliance:** RBAC/tenancy, analyzer enforcement, audit logging, signed promotions.
|
||||
- **Offline:** export/import of policy bundles, explain archives, CLI verification.
|
||||
|
||||
## Definition of done
|
||||
- Policy Engine core, orchestration, Policy Studio workflows, exports, and observability delivered with runbooks and Offline Kit parity.
|
||||
- Documentation suite (overview, architecture, DSL, lifecycle, Studio, simulation, CLI) updated with imposed rule statements.
|
||||
- ./TASKS.md and ../../TASKS.md reflect status; analyzers integrated into CI; compliance evidence captured.
|
||||
@@ -25,6 +25,33 @@ It exchanges an Authority-issued access token for a registry-compatible JWT afte
|
||||
- File: `etc/registry-token.yaml`
|
||||
- Environment variables: `REGISTRY_TOKEN_*`
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Current Objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases
|
||||
- Keep documentation, telemetry, and runbooks aligned with latest sprint outcomes
|
||||
|
||||
### Epic Milestones
|
||||
- Epic 10 – Export Center: signed registry token bundles for mirror/Offline Kit workflows (planned)
|
||||
- Epic 14 – Identity & Tenancy: tenant-aware scope validation, revocation, audit trails (planned)
|
||||
|
||||
### Core Capabilities
|
||||
- Docker registry token exchange with Authority validation
|
||||
- Plan/license constraint enforcement via claims inspection
|
||||
- Short-lived JWT tokens (default 5 minutes) signed by local RSA key
|
||||
- Revocation support via deny list and stellaops:license claim
|
||||
|
||||
### Technical Decisions
|
||||
- Token lifetime bounded to 5 minutes to minimize exposure window
|
||||
- Local RSA key signing avoids external dependencies
|
||||
- Plan catalogue enforcement ensures license compliance
|
||||
- Integration with Authority for caller identity and scope validation
|
||||
|
||||
### Coordination Approach
|
||||
- Review AGENTS.md before starting new work
|
||||
- Sync with cross-cutting teams via docs/implplan/SPRINT_*.md
|
||||
- Track follow-ups in ../../TASKS.md and src/Registry/TASKS.md
|
||||
|
||||
## Related docs
|
||||
|
||||
- Architecture: `docs/modules/registry/architecture.md`
|
||||
|
||||
@@ -1,20 +0,0 @@
|
||||
# Implementation plan — Registry Token Service
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 10 – Export Center:** deliver signed registry token bundles supporting mirror/Offline Kit workflows.
|
||||
- **Epic 14 – Identity & Tenancy:** integrate tenant-aware scope validation, revocation, and audit trails.
|
||||
- Track follow-ups in ../../TASKS.md and src/Registry/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
48
docs/modules/sbomservice/README.md
Normal file
48
docs/modules/sbomservice/README.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# SbomService
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/SbomService/`
|
||||
**Owner:** Scanner Guild
|
||||
|
||||
## Purpose
|
||||
|
||||
SbomService provides SBOM storage, versioning, and lineage tracking. Maintains the canonical SBOM repository with support for SPDX 3.0.1 and CycloneDX 1.6 formats, including temporal queries and dependency graph analysis.
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- `StellaOps.SbomService` - Main SBOM service with API and business logic
|
||||
|
||||
**Libraries:**
|
||||
- `StellaOps.SbomService.Storage.Postgres` - PostgreSQL storage adapter for SBOM persistence
|
||||
- `StellaOps.SbomService.Storage.Postgres.Tests` - Storage layer integration tests
|
||||
|
||||
## Configuration
|
||||
|
||||
Configuration is embedded in the service module settings.
|
||||
|
||||
Key settings:
|
||||
- PostgreSQL connection (schema: `sbom_service`)
|
||||
- Authority integration
|
||||
- SBOM format support (SPDX, CycloneDX)
|
||||
- Versioning and lineage policies
|
||||
- Retention settings
|
||||
|
||||
## Dependencies
|
||||
|
||||
- PostgreSQL (schema: `sbom_service`)
|
||||
- Authority (authentication)
|
||||
- Scanner (SBOM generation source)
|
||||
- Attestor (SBOM attestation integration)
|
||||
- ExportCenter (SBOM export and distribution)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- Scanner: `../scanner/`
|
||||
- Attestor: `../attestor/`
|
||||
- Data Schemas: `../../11_DATA_SCHEMAS.md`
|
||||
|
||||
## Current Status
|
||||
|
||||
Implemented with PostgreSQL storage backend. Supports SBOM ingestion, versioning, and lineage tracking. Provides API for SBOM queries and temporal analysis.
|
||||
@@ -44,7 +44,7 @@ Operational rules:
|
||||
## 3) APIs (first wave)
|
||||
- `GET /sbom/paths?purl=...&artifact=...&scope=...&env=...` — returns ordered paths with runtime_flag/blast_radius and nearest-safe-version hint; supports `cursor` pagination.
|
||||
- `GET /sbom/versions?artifact=...` – time-ordered SBOM version timeline for Advisory AI; include provenance and source bundle hash.
|
||||
- `POST /sbom/upload` – BYOS upload endpoint; validates/normalizes SPDX 2.3/3.0 or CycloneDX 1.4–1.7 and registers a ledger version.
|
||||
- `POST /sbom/upload` – BYOS upload endpoint; validates/normalizes SPDX 2.3/3.0.1 or CycloneDX 1.4–1.7 and registers a ledger version.
|
||||
- `GET /sbom/ledger/history` – list version history for an artifact (cursor pagination).
|
||||
- `GET /sbom/ledger/point` – resolve the SBOM version at a specific timestamp.
|
||||
- `GET /sbom/ledger/range` – query versions within a time range.
|
||||
@@ -77,7 +77,7 @@ Operational rules:
|
||||
- See `docs/modules/sbomservice/byos-ingestion.md` for supported formats and troubleshooting.
|
||||
|
||||
## 4) Ingestion & orchestrator integration
|
||||
- Ingest sources: Scanner pipeline (preferred) or uploaded SPDX 2.3/3.0 and CycloneDX 1.4–1.6 bundles.
|
||||
- Ingest sources: Scanner pipeline (preferred) or uploaded SPDX 2.3/3.0.1 and CycloneDX 1.4–1.7 bundles.
|
||||
- Orchestrator: register SBOM ingest/index jobs; worker SDK emits artifact hash + job metadata; honor pause/throttle; report backpressure metrics; support watermark-based backfill for idempotent replays.
|
||||
- Idempotency: combine `(tenant, artifactDigest, sbomVersion)` as primary key; duplicate ingests short-circuit.
|
||||
|
||||
|
||||
@@ -55,6 +55,59 @@ Scanner analyses container images layer-by-layer, producing deterministic SBOM f
|
||||
- DOCS-SCANNER updates tracked in ../../TASKS.md.
|
||||
- Analyzer parity work in src/Scanner/**/TASKS.md.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 1 – Control plane & job queue (Complete)
|
||||
- Scanner WebService with queue abstraction (Valkey/NATS)
|
||||
- Job leasing with retries and dead-letter handling
|
||||
- CAS layer cache and artifact catalog
|
||||
- REST API endpoints for scan management
|
||||
|
||||
### Phase 2 – Analyzer parity & SBOM assembly (In Progress)
|
||||
- OS analyzers: apk/dpkg/rpm with deterministic metadata
|
||||
- Language analyzers: Java, Node, Python, Go, .NET, Rust with lock file support
|
||||
- Native analyzers: ELF/PE/MachO for binary analysis
|
||||
- SBOM views: inventory/usage with CycloneDX/SPDX emitters
|
||||
- Entry trace resolution and dependency analysis
|
||||
|
||||
### Phase 3 – Diff & attestations (In Progress)
|
||||
- Three-way diff engine (base, target, runtime)
|
||||
- DSSE SBOM/report signing pipeline
|
||||
- Attestation hand-off to Signer/Attestor
|
||||
- Metadata for Export Center integration
|
||||
|
||||
### Phase 4 – Integrations & exports (Planned)
|
||||
- Policy Engine integration for evaluation
|
||||
- Vuln Explorer metadata delivery
|
||||
- Export Center artifact packaging
|
||||
- CLI/Console workflows and buildx plugin
|
||||
|
||||
### Phase 5 – Observability & resilience (Planned)
|
||||
- Metrics: queue depth, scan latency, cache hit/miss, analyzer timing
|
||||
- Queue backpressure handling and cache eviction
|
||||
- SLO dashboards and alerting
|
||||
- Smoke tests and runbooks
|
||||
|
||||
### Key Acceptance Criteria
|
||||
- Scans produce deterministic SBOM inventory/usage with stable component identity
|
||||
- Queue/worker pipeline handles retries, backpressure, offline kits
|
||||
- DSSE attestations exported for Signer/Attestor without transformation
|
||||
- CLI/Console parity for scan submission, diffing, exports, verification
|
||||
- Offline scanning supported with local caches and manifest verification
|
||||
|
||||
### Technical Decisions & Risks
|
||||
- Analyzer drift prevented via golden fixtures, hash-based regression tests, deterministic sorting
|
||||
- Queue overload mitigated with adaptive backpressure, worker scaling, priority lanes
|
||||
- Storage growth managed via CAS dedupe, ILM policies, offline bundle pruning
|
||||
- Lock file integration (npm/yarn/pnpm, pip/poetry, gradle) with declared-only components
|
||||
- Surface cache reuse for Linux OS analyzers with rootfs-relative evidence
|
||||
|
||||
### Recent Enhancements (2025-12-12)
|
||||
- Deterministic SBOM composition with DSSE fixtures and offline verification
|
||||
- Node/Python/Java lock file collectors with CLI validation commands
|
||||
- Platform events rollout with scanner.report.ready@1 and scanner.scan.completed@1
|
||||
- Surface-cache environment resolution with startup validation
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 6 – Vulnerability Explorer:** provide policy-aware scan outputs, explain traces, and findings ledger hooks for triage workflows.
|
||||
- **Epic 10 – Export Center:** generate export-ready artefacts, manifests, and DSSE metadata for bundles.
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
# Implementation plan — Scanner
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Control plane & job queue**
|
||||
Finalise Scanner WebService, queue abstraction (Valkey/NATS), job leasing, CAS layer cache, artifact catalog, and API endpoints.
|
||||
- **Phase 2 – Analyzer parity & SBOM assembly**
|
||||
Implement OS/Lang/Native analyzers, inventory/usage SBOM views, entry trace resolution, deterministic component identity.
|
||||
- **Phase 3 – Diff & attestations**
|
||||
Deliver three-way diff engine, DSSE SBOM/report signing pipeline, attestation hand-off (Signer→Attestor), metadata for Export Center.
|
||||
- **Phase 4 – Integrations & exports**
|
||||
Integrate with Policy Engine, Vuln Explorer, Export Center, CLI/Console; provide buildx plugin, CLI commands, and offline scanning support.
|
||||
- **Phase 5 – Observability & resilience**
|
||||
Metrics/logs/traces, queue backpressure handling, cache eviction, runbooks, smoke tests, SLO dashboards.
|
||||
|
||||
## Work breakdown
|
||||
- **Control plane**
|
||||
- REST API for scan requests, diff, catalog listing, artifact retrieval.
|
||||
- Queue service with idempotency, retries, dead-letter handling; worker scaling.
|
||||
- CAS storage (RustFS + S3 fallback), GC, ILM policies, offline mode.
|
||||
- **Analyzers**
|
||||
- OS (apk/dpkg/rpm), language (Java/Node/Python/Go/DotNet/Rust), native (ELF/PE/MachO).
|
||||
- Deterministic metadata (purl, version, source location), heuristics optional under flags.
|
||||
- Entry trace/usage analysis, dependency resolution, license detection.
|
||||
- **SBOM & diff**
|
||||
- Inventory/usage SBOM assembly, CycloneDX/SPDX emitters, schema validation.
|
||||
- Three-way diff (base, target, runtime), evidence linking, JSON export.
|
||||
- **Attestation & export**
|
||||
- DSSE bundle signing, attestation metadata for Signer/Attestor, provenance summary.
|
||||
- Export Center integration (SBOM/diff artifacts, manifests), CLI builder plugin (buildx).
|
||||
- **CLI/Console**
|
||||
- CLI commands `stella scan`, `stella sbom diff`, `stella sbom export`, offline caching.
|
||||
- Console flows for scan requests, diff viewer, SBOM downloads, attestation status.
|
||||
- **Observability & ops**
|
||||
- Metrics (queue depth, scan latency, cache hit/miss, analyzer timing), logs/traces with job IDs.
|
||||
- Alerts for backlog, failed scans, attestation issues, storage pressure.
|
||||
- Runbooks for stuck jobs, cache corruption, analyzer regressions, offline mode.
|
||||
|
||||
## Acceptance criteria
|
||||
- Scans produce deterministic SBOM inventory/usage views with component identity stability and reproducible diffs.
|
||||
- Queue/worker pipeline handles retries, backpressure, offline kits, and exports DSSE attestations for Signer/Attestor.
|
||||
- Export Center consumes SBOM/diff artifacts; Vuln Explorer receives metadata and explain traces.
|
||||
- CLI/Console parity for scan submission, diffing, exports, attestation verification.
|
||||
- Observability dashboards cover queue health, analyzer success rates, performance; alerts fire on SLO breaches.
|
||||
- Offline scanning (air-gapped) supported with local caches and manifest verification.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Analyzer drift/determinism:** golden fixtures, hash-based regression tests, deterministic sorting, strict identity rules.
|
||||
- **Queue overload:** adaptive backpressure, scaling workers, dead-letter review, priority lanes.
|
||||
- **Storage growth:** CAS dedupe, ILM policies, offline bundle pruning.
|
||||
- **Attestation failures:** retry with backoff, attestation health checks, Notify integration.
|
||||
- **Offline divergence:** packaging of analyzers/configs, manifest signatures, parity tests.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** analyzer parsers, component identity, diff calculations, API validation.
|
||||
- **Integration:** end-to-end scan/diff/attestation flows, Export Center integration, CLI automation.
|
||||
- **Performance:** large images, concurrent scans, cache stress, queue throughput.
|
||||
- **Determinism:** repeated scans/diffs across systems, hash comparisons, property tests.
|
||||
- **Security:** RBAC, tenant isolation, attestation key handling, path sanitisation.
|
||||
- **Offline:** air-gap scanning, manifest verification, CLI offline mode.
|
||||
|
||||
## Definition of done
|
||||
- Scanner services, analyzers, diffing, attestation pipeline, exports, and observability delivered with runbooks and Offline Kit parity.
|
||||
- Documentation (architecture, analyzer guides, CLI, offline mode, operations) updated with imposed rule statements.
|
||||
- ./TASKS.md and ../../TASKS.md updated with progress; regression fixtures maintained in repo.
|
||||
@@ -31,6 +31,41 @@ Scheduler detects advisory/VEX deltas, computes impact windows, and orchestrates
|
||||
- SCHED-MODELS-20-001 (policy run DTOs) and related tasks in ../../TASKS.md.
|
||||
- Scheduler observability follow-ups in src/Scheduler/**/TASKS.md.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Current Objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases
|
||||
- Keep documentation, telemetry, and runbooks aligned with latest sprint outcomes
|
||||
- Coordinate with Policy Engine for incremental re-evaluation workflows
|
||||
|
||||
### Epic Milestones
|
||||
- Epic 2 – Policy Engine & Editor: incremental policy run orchestration, change streams, explain trace propagation (in progress)
|
||||
- Epic 6 – Vulnerability Explorer: findings updates and remediation triggers integration (in progress)
|
||||
- Epic 9 – Orchestrator Dashboard: job telemetry and control surfaces for UI/CLI (planned)
|
||||
|
||||
### Core Capabilities
|
||||
- Impact cursor maintenance and queue management for re-scan/re-evaluate jobs
|
||||
- Change-stream detection for advisory/VEX/SBOM deltas
|
||||
- Policy-triggered recheck orchestration with runtime hooks
|
||||
- SLA-aware retry logic with deterministic evaluation windows
|
||||
- DSSE-backed completion events for downstream consumers
|
||||
|
||||
### Integration Points
|
||||
- PostgreSQL schema (scheduler) for impact models and job state
|
||||
- Valkey/NATS for queueing with idempotency
|
||||
- Policy Engine, Scanner, Notify for job coordination
|
||||
- Orchestrator for backfills and incident routing
|
||||
|
||||
### Operational Assets
|
||||
- Monitoring: worker-grafana-dashboard.json, worker-prometheus-rules.yaml
|
||||
- Runbooks: operations/worker.md
|
||||
- Observability: metrics, traces, structured logs with correlation IDs
|
||||
|
||||
### Technical Notes
|
||||
- Coordination approach: review AGENTS.md, sync via docs/implplan/SPRINT_*.md
|
||||
- Backlog tracking: SCHED-MODELS-20-001 and related tasks in ../../TASKS.md
|
||||
- Module tasks: src/Scheduler/**/TASKS.md
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 2 – Policy Engine & Editor:** orchestrate incremental re-evaluation and simulation runs when raw facts or policies change.
|
||||
- **Epic 6 – Vulnerability Explorer:** feed triage workflows with up-to-date job status, explain traces, and ledger hooks.
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
# Implementation plan — Scheduler
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver incremental policy run orchestration, change streams, and explain trace propagation.
|
||||
- **Epic 6 – Vulnerability Explorer:** ensure findings updates and remediation triggers integrate with scheduler outputs.
|
||||
- **Epic 9 – Orchestrator Dashboard:** provide job telemetry and control surfaces consumed by the orchestrator UI/CLI.
|
||||
- Track additional work (SCHED-MODELS-20-001, observability follow-ups) in ../../TASKS.md and src/Scheduler/**/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
49
docs/modules/sdk/README.md
Normal file
49
docs/modules/sdk/README.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# SDK
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/Sdk/`
|
||||
**Owner:** Platform Team
|
||||
|
||||
## Purpose
|
||||
|
||||
SDK provides code generation and client libraries for integrating with StellaOps APIs. Generates strongly-typed clients from OpenAPI specifications for multiple languages (.NET, TypeScript, Python, Go).
|
||||
|
||||
## Components
|
||||
|
||||
**Tools:**
|
||||
- `StellaOps.Sdk.Generator` - OpenAPI-based client generator
|
||||
- `StellaOps.Sdk.Release` - SDK packaging and release automation
|
||||
|
||||
**Generated Clients:**
|
||||
- .NET (C#) client library
|
||||
- TypeScript/JavaScript client
|
||||
- Python client (planned)
|
||||
- Go client (planned)
|
||||
|
||||
## Configuration
|
||||
|
||||
SDK generation configured via OpenAPI specs in `docs/api/`.
|
||||
|
||||
Key features:
|
||||
- Strongly-typed API clients
|
||||
- Authentication helpers (OAuth/OIDC, DPoP)
|
||||
- Retry and error handling
|
||||
- Request/response validation
|
||||
- Offline-compatible operation
|
||||
|
||||
## Dependencies
|
||||
|
||||
- OpenAPI specifications (from all API modules)
|
||||
- Authority (authentication flows)
|
||||
- Gateway (API routing)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- API Reference: `../../09_API_CLI_REFERENCE.md`
|
||||
- OpenAPI Specs: `../../api/` (if exists)
|
||||
- CLI: `../cli/`
|
||||
- Gateway: `../gateway/`
|
||||
|
||||
## Current Status
|
||||
|
||||
Generator and Release tools implemented. Supports .NET and TypeScript client generation from OpenAPI specs. Used for CLI implementation and Web frontend integration.
|
||||
58
docs/modules/signals/README.md
Normal file
58
docs/modules/signals/README.md
Normal file
@@ -0,0 +1,58 @@
|
||||
# Signals
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/Signals/`
|
||||
**Owner:** Policy Guild
|
||||
|
||||
## Purpose
|
||||
|
||||
Signals provides evidence-weighted scoring for vulnerability findings. Aggregates evidence from reachability analysis, runtime observations, backport detection, exploit intelligence, source trust, and mitigations into a deterministic 0-100 score for rapid triage.
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- `StellaOps.Signals` - Main service with evidence aggregation and scoring engine
|
||||
- `StellaOps.Signals.Scheduler` - Background scheduler for score recalculation
|
||||
|
||||
**Libraries:**
|
||||
- `StellaOps.Signals.Storage.Postgres` - PostgreSQL storage for evidence and scores
|
||||
- `StellaOps.Signals.Storage.Postgres.Tests` - Storage layer tests
|
||||
|
||||
**Evidence Normalizers:**
|
||||
- Reachability (RCH) - Code path reachability to vulnerable sink
|
||||
- Runtime (RTS) - Live observation strength (eBPF/dyld/ETW)
|
||||
- Backport (BKP) - Patch evidence from distro/changelog/binary
|
||||
- Exploit (XPL) - Exploit probability (EPSS + KEV)
|
||||
- Source Trust (SRC) - VEX source trustworthiness
|
||||
- Mitigation (MIT) - Active mitigation effectiveness
|
||||
|
||||
## Configuration
|
||||
|
||||
See `etc/signals.yaml.sample` for configuration options.
|
||||
|
||||
Key settings:
|
||||
- PostgreSQL connection (schema: `signals`)
|
||||
- Authority integration (audiences, scopes)
|
||||
- Storage driver (filesystem or rustfs)
|
||||
- Evidence weight policies
|
||||
- Score guardrails and thresholds
|
||||
|
||||
## Dependencies
|
||||
|
||||
- PostgreSQL (schema: `signals`)
|
||||
- Authority (authentication)
|
||||
- Policy Engine (reachability data)
|
||||
- Concelier (backport detection, exploit data)
|
||||
- Excititor (VEX source trust)
|
||||
- Scanner (vulnerability findings)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- Policy Engine: `../policy/`
|
||||
- VexLens: `../vex-lens/`
|
||||
- High-Level Architecture: `../../07_HIGH_LEVEL_ARCHITECTURE.md`
|
||||
|
||||
## Current Status
|
||||
|
||||
Implemented with six evidence normalizers and deterministic scoring algorithm. Integrated with Policy Engine for reachability data and VexLens for source trust. Supports evidence decomposition and transparency.
|
||||
@@ -48,6 +48,54 @@ Signer validates callers, enforces Proof-of-Entitlement, and produces signed DSS
|
||||
- Sprint 0401: `docs/implplan/SPRINT_0401_0001_0001_reachability_evidence_chain.md` (SIGN-VEX-401-018 DONE; AUTH-REACH-401-005 TODO).
|
||||
- SIG docs/tasks in ../../TASKS.md (e.g., DOCS-SIG-26-006).
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 1 – Core service & PoE (Complete)
|
||||
- OpTok validation with Authority DPoP/mTLS tokens and signer.sign scope
|
||||
- Proof-of-Entitlement (PoE) introspection with cloud licensing integration
|
||||
- Scanner release verification via OCI referrers
|
||||
- DSSE signing pipeline: keyless (Fulcio) and keyful (KMS/HSM/FIDO2)
|
||||
- KMS key management foundations (KMSI-73-001, KMSI-73-002)
|
||||
- DSSE/SLSA BuildDefinition models with canonical JSON (PROV-OBS-53-001/002)
|
||||
|
||||
### Phase 2 – Export Center integration (In Progress)
|
||||
- CryptoDsseSigner with ICryptoProviderRegistry (keyless + KMS modes)
|
||||
- SignerStatementBuilder refactored for StellaOps predicate types
|
||||
- PromotionAttestationBuilder with canonicalized payloads (PROV-OBS-53-003)
|
||||
- Cosign-compatible DSSE output with provenance manifests
|
||||
- Blocking: SIGN-CORE-186-004/005 crypto provider refactoring, replay manifest support
|
||||
|
||||
### Phase 3 – Attestor alignment (Not Started)
|
||||
- DSSE envelope metadata for Attestor ingestion
|
||||
- Extended predicate catalog: stella.ops/vexDecision@v1, stella.ops/graph@v1 (SIGN-VEX-401-018 complete)
|
||||
- Helper methods: IsVexRelatedType, IsReachabilityRelatedType, predicate validation
|
||||
- Blocking: AUTH-REACH-401-005 predicate definitions, verification library (PROV-OBS-54-001/002)
|
||||
|
||||
### Phase 4 – Observability & resilience (Not Started)
|
||||
- Metrics: signing latency, PoE failures, quota hits, key usage distribution
|
||||
- Structured logs with trace IDs, subject digests, issuer mode, decision outcomes
|
||||
- Alerts for PoE outages, key exhaustion, quota breaches, failure spikes
|
||||
- CLI commands: stella promotion attest/verify, stella forensic attest show
|
||||
|
||||
### Key Acceptance Criteria
|
||||
- Signs only requests satisfying OpTok, PoE, quota, scanner provenance checks
|
||||
- DSSE outputs verify with standard cosign tooling
|
||||
- Export Center receives signed bundles with provenance manifests
|
||||
- Audit logs capture every request with tenant, issuer, subject digest, PoE state
|
||||
- CLI/Offline workflows verify signatures using Offline Kit trust roots
|
||||
|
||||
### Technical Decisions & Risks
|
||||
- PoE/entitlement outages: cache last-known entitlement within TTL, emergency bypass with audit
|
||||
- Key compromise: hardware-backed keys, rotation cadence, immediate revocation, incident runbook
|
||||
- Release verification failures: allowlist for trusted scanner digests, manual approval fallback
|
||||
- Determinism: canonicalize JSON, lock timestamp sources, regression tests for DSSE hashing
|
||||
|
||||
### Recent Updates (Sprint 0186/0401 · 2025-11-26)
|
||||
- CryptoDsseSigner with ES256 signature generation via ICryptoProviderRegistry
|
||||
- PredicateTypes catalog extended with VEX/graph predicates
|
||||
- Integration tests upgraded with real crypto, fixture predicates (102 tests passing)
|
||||
- CryptoPro signer plugin in progress (SEC-CRYPTO-90-020)
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 10 – Export Center:** provide signing pipelines, cosign interoperability, and provenance manifests for bundle promotion.
|
||||
- **Epic 19 – Attestor Console:** supply DSSE payloads and Proof-of-Entitlement enforcement feeding attestation workflows described in `docs/modules/attestor/`.
|
||||
|
||||
@@ -1,136 +0,0 @@
|
||||
# Implementation plan — Signer
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Core service & PoE**
|
||||
Harden OpTok validation, Proof-of-Entitlement (PoE) checks, quota enforcement, scanner release verification, and DSSE signing pipeline (keyless + keyful).
|
||||
- **Phase 2 – Export Center integration**
|
||||
Produce signing bundles with provenance manifests for Export Center, deliver cosign-compatible outputs, and document verification workflows for offline exports.
|
||||
- **Phase 3 – Attestor alignment**
|
||||
Emit DSSE envelopes, metadata, and signer identity information required by Attestor (key metadata, certificate chains, bundle hashes); expose audit APIs.
|
||||
- **Phase 4 – Observability & resilience**
|
||||
Complete telemetry, throttling dashboards, audit trails, fallback key rotation, and offline kit packaging.
|
||||
|
||||
## Work breakdown
|
||||
- **Authentication & entitlement**
|
||||
- Enforce Authority-issued DPoP/mTLS tokens with `signer.sign` scope and tenant checks.
|
||||
- Integrate PoE introspection (cloud licensing) and plan quotas.
|
||||
- Validate scanner release signature via OCI referrers before signing reports/SBOMs.
|
||||
- **Signing pipeline**
|
||||
- Implement DSSE canonicalisation, keyless (Fulcio) and keyful (KMS/HSM/FIDO2) signing.
|
||||
- Support multi-signature output, certificate chain embedding, and deterministic bundle hashing.
|
||||
- Provide policy metadata (policy digest, view set) for report predicates.
|
||||
- **Integrations**
|
||||
- Coordinate with Attestor on bundle schema, signer identity payload, and error codes.
|
||||
- Expose Export Center-ready signing API (`/sign/export`) that returns manifest + signature tuple.
|
||||
- Surface CLI flows (`stella sign sbom/report`, offline verification helpers).
|
||||
- **Security & key management**
|
||||
- Key rotation and revocation runbooks; separation of signing keys per tenancy/plan.
|
||||
- Hardware-backed key support (HSM/FIDO2) with audit logging and attested builds.
|
||||
- **Observability**
|
||||
- Metrics: signing latency, PoE failures, quota hits, key usage distribution.
|
||||
- Structured logs with trace/context IDs, subject digests, issuer mode, decision outcome.
|
||||
- Alerts for PoE outages, key exhaustion, quota breach, signing failure spikes.
|
||||
- **Documentation & runbooks**
|
||||
- Update README/architecture/implementation plan, operator runbooks, offline verification guidance, and CLI reference.
|
||||
|
||||
## Acceptance criteria
|
||||
- Signer only signs requests that satisfy OpTok, PoE, quota, and scanner provenance checks.
|
||||
- DSSE outputs (keyless + keyful) verify with standard cosign tooling; Attestor logs them without additional transformation.
|
||||
- Export Center receives signed bundles, provenance manifests, and signature metadata to package exports deterministically.
|
||||
- Audit logs capture every signing request with tenant, issuer, subject digest, PoE state, and key source.
|
||||
- Observability dashboards/alerts reflect latency, failure rate, PoE status, and quota usage.
|
||||
- CLI/Offline workflows verify signatures using Offline Kit trust roots.
|
||||
|
||||
## Risks & mitigations
|
||||
- **PoE/entitlement outages:** cache last-known entitlement within TTL, provide emergency bypass toggles with audit trail.
|
||||
- **Key compromise:** enforce hardware-backed keys, rotation cadence, immediate revocation workflow, incident runbook.
|
||||
- **Release verification failures:** maintain allowlist for trusted scanner digests, fallback to manual approval with audit.
|
||||
- **Determinism drift:** canonicalise JSON, lock timestamp sources, regression tests for DSSE hashing.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** OpTok/PoE validation, quota enforcement, scanner signature verification, DSSE canonicalisation, multi-sig bundling.
|
||||
- **Integration:** end-to-end signing for SBOM, report, export artifacts; Attestor ingestion; Export Center bundle signing.
|
||||
- **Security:** fuzz signing inputs, simulate PoE tampering, ensure unauthorized actors are rejected.
|
||||
- **Performance:** signing throughput benchmarks (keyless vs keyful), quota pressure, concurrency checks.
|
||||
- **Offline:** verify signatures using Offline Kit trust roots and cosign CLI without network access.
|
||||
|
||||
## Definition of done
|
||||
- Signing pipeline deployed with observability and incident runbooks.
|
||||
- Export Center + Attestor dependencies validated; CLI parity confirmed.
|
||||
- Documentation updated (README, architecture, runbooks, CLI guides) with imposed rule compliance.
|
||||
- ./TASKS.md and ../../TASKS.md reflect the latest status transitions.
|
||||
|
||||
---
|
||||
|
||||
## Sprint readiness tracker
|
||||
|
||||
> Last updated: 2025-11-27 (SIGNER-ENG-0001)
|
||||
|
||||
This section maps delivery phases to implementation sprints and tracks readiness checkpoints.
|
||||
|
||||
### Phase 1 — Core service & PoE
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| KMSI-73-001 | ✅ DONE (2025-11-03) | SPRINT_100_identity_signing | KMS key management foundations with staffing + DSSE contract. |
|
||||
| KMSI-73-002 | ✅ DONE (2025-11-03) | SPRINT_100_identity_signing | FIDO2 profile integration. |
|
||||
| PROV-OBS-53-001 | ✅ DONE (2025-11-17) | SPRINT_0513_0001_0001_provenance | DSSE/SLSA BuildDefinition + BuildMetadata models with canonical JSON serializer. |
|
||||
| PROV-OBS-53-002 | ✅ DONE (2025-11-23) | SPRINT_0513_0001_0001_provenance | Signer abstraction (cosign/KMS/offline) with key rotation hooks and audit logging. |
|
||||
| SEC-CRYPTO-90-020 | 🔄 IN PROGRESS | SPRINT_0514_0001_0001_sovereign_crypto | CryptoPro signer plugin; Windows CSP runner pending. |
|
||||
|
||||
**Checkpoint:** Core signing infrastructure operational — KMS drivers, signer abstractions, and DSSE models delivered.
|
||||
|
||||
### Phase 2 — Export Center integration
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| PROV-OBS-53-003 | ✅ DONE (2025-11-23) | SPRINT_0513_0001_0001_provenance | PromotionAttestationBuilder feeding canonicalised payloads to Signer. |
|
||||
| SIGN-REPLAY-186-003 | 📝 TODO | SPRINT_186_record_deterministic_execution | Extend Signer/Authority DSSE flows for replay manifest/bundle payloads. |
|
||||
| SIGN-CORE-186-004 | 📝 TODO | SPRINT_186_record_deterministic_execution | Replace HMAC demo with StellaOps.Cryptography providers (keyless + KMS). |
|
||||
| SIGN-CORE-186-005 | 📝 TODO | SPRINT_186_record_deterministic_execution | Refactor SignerStatementBuilder for StellaOps predicate types. |
|
||||
| SIGN-TEST-186-006 | 📝 TODO | SPRINT_186_record_deterministic_execution | Upgrade signer integration tests with real crypto + fixture predicates. |
|
||||
|
||||
**Checkpoint:** Export Center signing APIs partially complete; replay manifest support and crypto provider refactoring pending.
|
||||
|
||||
### Phase 3 — Attestor alignment
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| AUTH-REACH-401-005 | 📝 TODO | SPRINT_0401_0001_0001_reachability_evidence_chain | DSSE predicate types for SBOM/Graph/VEX/Replay; blocked on predicate definitions. |
|
||||
| SIGN-VEX-401-018 | 📝 TODO | SPRINT_0401_0001_0001_reachability_evidence_chain | Extend predicate catalog with `stella.ops/vexDecision@v1`. |
|
||||
| PROV-OBS-54-001 | 📝 TODO | SPRINT_0513_0001_0001_provenance | Verification library for DSSE signatures, Merkle roots, timeline chain. |
|
||||
| PROV-OBS-54-002 | 📝 TODO | SPRINT_0513_0001_0001_provenance | .NET global tool for local verification + CLI `stella forensic verify`. |
|
||||
|
||||
**Checkpoint:** Attestor DSSE alignment pending; predicate catalog extension and verification library not started.
|
||||
|
||||
### Phase 4 — Observability & resilience
|
||||
| Task ID | Status | Sprint | Notes |
|
||||
|---------|--------|--------|-------|
|
||||
| DOCS-PROMO-70-001 | 📝 TODO | SPRINT_0304_0001_0004_docs_tasks_md_iv | Promotion attestations doc (CLI commands, Signer/Attestor integration, offline verification). |
|
||||
| CLI-PROMO-70-002 | 📝 TODO | SPRINT_203_cli_iii | `stella promotion attest` / `promotion verify` commands. |
|
||||
| CLI-FORENSICS-54-002 | 📝 TODO | SPRINT_202_cli_ii | `stella forensic attest show <artifact>` listing signer details. |
|
||||
|
||||
**Checkpoint:** Observability and CLI integration pending; waiting on upstream signing pipeline completion.
|
||||
|
||||
---
|
||||
|
||||
### Overall readiness summary
|
||||
|
||||
| Phase | Status | Blocking items |
|
||||
|-------|--------|----------------|
|
||||
| **1 – Core service & PoE** | ✅ Complete | — |
|
||||
| **2 – Export Center integration** | 🔄 In progress | SIGN-CORE-186-004/005 crypto provider refactoring |
|
||||
| **3 – Attestor alignment** | 📝 Not started | AUTH-REACH-401-005 predicate definitions |
|
||||
| **4 – Observability & resilience** | 📝 Not started | Upstream phase completion |
|
||||
|
||||
### Cross-module dependencies
|
||||
|
||||
| Dependency | Required by | Status |
|
||||
|------------|-------------|--------|
|
||||
| Attestor DSSE bundle schema | SIGN-VEX-401-018 | Documented in `docs/modules/attestor/architecture.md` §1 |
|
||||
| Provenance library canonicalisation | SIGN-CORE-186-005 | Available via PROV-OBS-53-001/002 |
|
||||
| Export Center bundle manifest | SIGN-REPLAY-186-003 | Pending Sprint 162/163 deliverables |
|
||||
| Authority predicate definitions | AUTH-REACH-401-005 | Schema draft pending |
|
||||
|
||||
### Next actions
|
||||
1. Complete CryptoPro signer plugin Windows smoke test (SEC-CRYPTO-90-020, Sprint 0514).
|
||||
2. Start SIGN-CORE-186-004 once replay bundle schema finalises (Sprint 186).
|
||||
3. Track AUTH-REACH-401-005 predicate schema draft for Attestor alignment (Sprint 401).
|
||||
4. Monitor PROV-OBS-54-001/002 for verification library availability.
|
||||
51
docs/modules/snapshot/README.md
Normal file
51
docs/modules/snapshot/README.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Snapshot
|
||||
|
||||
**Status:** Design/Planning
|
||||
**Source:** N/A (cross-cutting concept)
|
||||
**Owner:** Platform Team
|
||||
|
||||
## Purpose
|
||||
|
||||
Snapshot defines the knowledge snapshot model for deterministic, point-in-time captures of StellaOps data. Enables offline operation, merge preview, replay, and air-gap export with cryptographic integrity.
|
||||
|
||||
## Components
|
||||
|
||||
**Concept Documentation:**
|
||||
- `merge-preview.md` - Merge preview specification
|
||||
- `replay-yaml.md` - Replay YAML format and semantics
|
||||
|
||||
**Snapshot Types:**
|
||||
- Advisory snapshots (Concelier ingestion state)
|
||||
- VEX snapshots (VexHub distribution state)
|
||||
- SBOM snapshots (SbomService repository state)
|
||||
- Policy snapshots (Policy Engine rule state)
|
||||
- Task pack snapshots (PacksRegistry versions)
|
||||
|
||||
## Implementation Locations
|
||||
|
||||
Snapshot functionality is implemented across multiple modules:
|
||||
- **AirGap** - Snapshot export/import orchestration
|
||||
- **ExportCenter** - Snapshot bundle creation
|
||||
- **Replay** - Deterministic replay from snapshots
|
||||
- **Concelier** - Advisory snapshot merge preview
|
||||
- All data modules (snapshot sources)
|
||||
|
||||
## Dependencies
|
||||
|
||||
- AirGap (snapshot orchestration)
|
||||
- ExportCenter (bundle creation)
|
||||
- Replay (snapshot replay)
|
||||
- All data modules (snapshot sources)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Merge Preview: `./merge-preview.md`
|
||||
- Replay YAML: `./replay-yaml.md`
|
||||
- AirGap: `../airgap/`
|
||||
- ExportCenter: `../export-center/`
|
||||
- Replay: `../replay/` (if exists)
|
||||
- Offline Kit: `../../24_OFFLINE_KIT.md`
|
||||
|
||||
## Current Status
|
||||
|
||||
Snapshot concepts documented in merge-preview.md and replay-yaml.md. Implementation distributed across AirGap (export/import), ExportCenter (packaging), and Replay (playback) modules. Used for offline/air-gap operation.
|
||||
52
docs/modules/symbols/README.md
Normal file
52
docs/modules/symbols/README.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# Symbols Module
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/Symbols/`
|
||||
|
||||
## Purpose
|
||||
|
||||
The Symbols module provides debug symbol storage, resolution, and distribution for binary analysis and call-graph construction. It supports multi-tenant symbol management with DSSE-signed manifests and CAS-backed blob storage.
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- **StellaOps.Symbols.Server** - REST API for symbol management and resolution
|
||||
- **StellaOps.Symbols.Client** - Client SDK for Scanner/runtime integration
|
||||
- **StellaOps.Symbols.Ingestor.Cli** - CLI tool for symbol ingestion
|
||||
- **StellaOps.Symbols.Core** - Core abstractions, models, and interfaces
|
||||
- **StellaOps.Symbols.Infrastructure** - In-memory and production implementations
|
||||
- **StellaOps.Symbols.Bundle** - Offline bundle management
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Symbol Resolution** - Address-to-symbol mapping for binaries (ELF, PE, Mach-O)
|
||||
- **Multi-tenant Storage** - Tenant-isolated symbol manifests with BLAKE3 content hashing
|
||||
- **DSSE Integration** - Optional cryptographic signing of manifests for transparency
|
||||
- **CAS Blob Storage** - Content-addressed storage for symbol files and metadata
|
||||
- **Debug Identifiers** - Support for ELF Build-IDs, PDB GUIDs, and GNU build-ids
|
||||
|
||||
## API Endpoints
|
||||
|
||||
- `POST /v1/symbols/manifests` - Upload symbol manifest
|
||||
- `GET /v1/symbols/manifests/{id}` - Get manifest by ID
|
||||
- `GET /v1/symbols/manifests` - Query manifests with filters
|
||||
- `POST /v1/symbols/resolve` - Resolve addresses to symbols
|
||||
- `GET /v1/symbols/by-debug-id/{debugId}` - Get manifests by debug ID
|
||||
- `GET /health` - Health check (anonymous)
|
||||
|
||||
## Dependencies
|
||||
|
||||
- PostgreSQL (for production storage)
|
||||
- Authority (authentication and multi-tenancy)
|
||||
- Policy Engine (optional, for overlay integration)
|
||||
- Graph module (for call-graph construction support)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **Architecture:** See `src/Symbols/AGENTS.md` for detailed agent instructions
|
||||
- **Sprint Plan:** Check `docs/implplan/SPRINT_*.md` for current development status
|
||||
- **Fixtures:** Test data available in `tests/Symbols/fixtures/`
|
||||
|
||||
## Current Status
|
||||
|
||||
Active development. Supports symbol ingestion, resolution, and multi-tenant distribution with DSSE-signed manifests and CAS-backed storage.
|
||||
59
docs/modules/taskrunner/README.md
Normal file
59
docs/modules/taskrunner/README.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# TaskRunner
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/TaskRunner/`
|
||||
**Owner:** Platform Team
|
||||
|
||||
## Purpose
|
||||
|
||||
TaskRunner executes Task Packs deterministically with approvals, sealed-mode enforcement, and evidence capture. Provides orchestration for automated workflows with plan-hash binding, offline operation, and complete provenance generation (DSSE attestation + evidence bundle).
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- `StellaOps.TaskRunner.WebService` - HTTP API with plan hash validation and SSE log streaming
|
||||
- `StellaOps.TaskRunner.Worker` - Run orchestration with retries, artifact capture, and attestation
|
||||
|
||||
**Libraries:**
|
||||
- `StellaOps.TaskRunner.Core` - Execution graph builder, simulation engine, step state machine
|
||||
- `StellaOps.TaskRunner.Infrastructure` - Storage adapters (PostgreSQL, file), artifact/object store clients
|
||||
|
||||
## Configuration
|
||||
|
||||
See `etc/task-runner.yaml.sample` for configuration options.
|
||||
|
||||
Key settings:
|
||||
- PostgreSQL connection (schemas: `pack_runs`, `pack_run_logs`, `pack_artifacts`)
|
||||
- Authority integration (issuer, audiences, client credentials)
|
||||
- Telemetry and OTLP endpoint
|
||||
- Artifact storage paths
|
||||
- Approval timeout and retry policies
|
||||
- Sealed-mode network allowlists
|
||||
|
||||
## API Surface
|
||||
|
||||
- `POST /api/runs` - Submit pack run (requires manifest, inputs, tenant context)
|
||||
- `GET /api/runs/{runId}` - Retrieve run status and graph
|
||||
- `GET /api/runs/{runId}/logs` - SSE stream of ordered log events
|
||||
- `GET /api/runs/{runId}/artifacts` - List captured artifacts with digests
|
||||
- `POST /api/runs/{runId}/approve` - Record approval gate decision
|
||||
- `POST /api/runs/{runId}/cancel` - Cancel active run
|
||||
|
||||
## Dependencies
|
||||
|
||||
- PostgreSQL (schemas: `pack_runs`, `pack_run_logs`, `pack_artifacts`)
|
||||
- Authority (authentication and approval token claims)
|
||||
- Attestor (DSSE attestation generation)
|
||||
- PacksRegistry (task pack manifests and modules)
|
||||
- Scheduler (optional, for scheduled runs)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- Task Packs Spec: `../../task-packs/` (if exists)
|
||||
- Orchestrator: `../orchestrator/`
|
||||
- Attestor: `../attestor/`
|
||||
|
||||
## Current Status
|
||||
|
||||
Implemented with plan-hash binding and deterministic execution. Supports parallel/map steps, approval gates, policy gates, and the built-in `bundle.ingest` helper. Produces DSSE attestations for all completed runs.
|
||||
@@ -2,12 +2,12 @@
|
||||
|
||||
Telemetry module captures deployment and operations guidance for the shared observability stack (collectors, storage, dashboards).
|
||||
|
||||
## Latest updates (2025-11-30)
|
||||
- Sprint tracker `docs/implplan/SPRINT_0330_0001_0001_docs_modules_telemetry.md` and module `TASKS.md` added to mirror status.
|
||||
- Observability runbook stub + dashboard placeholder added under `operations/` (offline import).
|
||||
- Storage/isolation posture references updated; align with platform docs.
|
||||
|
||||
## Responsibilities
|
||||
## Latest updates (2025-11-30)
|
||||
- Sprint tracker `docs/implplan/SPRINT_0330_0001_0001_docs_modules_telemetry.md` and module `TASKS.md` added to mirror status.
|
||||
- Observability runbook stub + dashboard placeholder added under `operations/` (offline import).
|
||||
- Storage/isolation posture references updated; align with platform docs.
|
||||
|
||||
## Responsibilities
|
||||
- Deploy and operate OpenTelemetry collectors for StellaOps services.
|
||||
- Provide storage configuration for Prometheus/Tempo/Loki stacks.
|
||||
- Document smoke tests and offline bootstrapping steps.
|
||||
@@ -23,11 +23,11 @@ Telemetry module captures deployment and operations guidance for the shared obse
|
||||
- Module-specific dashboards (scheduler, scanner, etc.).
|
||||
- Security/Compliance for retention policies.
|
||||
|
||||
## Operational notes
|
||||
- Smoke script references (../../ops/devops/telemetry).
|
||||
- Bundle packaging instructions in ops/devops/telemetry.
|
||||
- Sprint 23 console security sign-off (2025-10-27) added the `console-security.json` Grafana board and burn-rate alert pack—ensure environments import the updated dashboards/alerts referenced in `docs/updates/2025-10-27-console-security-signoff.md`.
|
||||
- Observability assets for this sprint: `operations/observability.md` and `operations/dashboards/telemetry-observability.json` (offline import).
|
||||
## Operational notes
|
||||
- Smoke script references (../../ops/devops/telemetry).
|
||||
- Bundle packaging instructions in ops/devops/telemetry.
|
||||
- Sprint 23 console security sign-off (2025-10-27) added the `console-security.json` Grafana board and burn-rate alert pack—ensure environments import the updated dashboards/alerts referenced in `docs/updates/2025-10-27-console-security-signoff.md`.
|
||||
- Observability assets for this sprint: `operations/observability.md` and `operations/dashboards/telemetry-observability.json` (offline import).
|
||||
|
||||
## Related resources
|
||||
- ./operations/collector.md
|
||||
@@ -37,5 +37,64 @@ Telemetry module captures deployment and operations guidance for the shared obse
|
||||
- TELEMETRY-OBS-50-001 … 50-004 in ../../TASKS.md.
|
||||
- Collector/storage automation tracked in ops/devops/TASKS.md.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 1 – Collector & pipeline profiles (In Progress)
|
||||
- OpenTelemetry collector configs: default, forensic, airgap profiles
|
||||
- Ingest gateways with TLS/mTLS support
|
||||
- Attribute redaction policies and tenant isolation
|
||||
- CLI automation: stella telemetry deploy, stella telemetry profile diff
|
||||
|
||||
### Phase 2 – Storage backends & retention (Planned)
|
||||
- Prometheus/Tempo/Loki deployment with retention tiers
|
||||
- Bucket/object storage with deterministic manifest generation
|
||||
- Sealed-mode allowlists and offline bundle support
|
||||
- Remote-write configuration and archivers
|
||||
|
||||
### Phase 3 – Incident mode & forensic capture (Planned)
|
||||
- Incident toggles via CLI/API for sampling adjustments
|
||||
- Tail sampling to 100% during incidents
|
||||
- Forensic bundle generation: OTLP archives with manifest/signature
|
||||
- Notify hooks for incident escalation
|
||||
|
||||
### Phase 4 – Observability dashboards & automation (Planned)
|
||||
- Service SLO dashboards: queue depth, policy latency, ingestion violations
|
||||
- Alert rules: burn-rate, collector failure, exporter backlog
|
||||
- Grafana packages for core services
|
||||
- Self-observability metrics
|
||||
|
||||
### Phase 5 – Offline & compliance (Planned)
|
||||
- Offline Kit artifacts: collector binaries/configs, import scripts
|
||||
- Deterministic bundles with signed manifests
|
||||
- Replay tooling and compliance checklists
|
||||
- File-based exporters for air-gapped environments
|
||||
|
||||
### Phase 6 – Hardening & SOC handoff (Planned)
|
||||
- RBAC integration and audit logging
|
||||
- Incident response runbooks and performance tuning
|
||||
- Integration tests across services
|
||||
- SOC handoff package with control objectives
|
||||
|
||||
### Key Acceptance Criteria
|
||||
- Collectors ingest metrics/logs/traces with redaction rules and tenant isolation
|
||||
- Storage backends retain data per SLAs with deterministic manifests
|
||||
- Incident mode triggers forensic capture with signed bundles
|
||||
- Dashboards/alerts cover service SLOs and telemetry stack health
|
||||
- CLI automates config rollout, forensic capture, verification
|
||||
- Offline bundles replay telemetry in sealed environments
|
||||
|
||||
### Technical Decisions & Risks
|
||||
- PII leakage prevented via strict redaction processors, policy-managed allowlists
|
||||
- Collector overload managed with horizontal scaling, batching, circuit breakers
|
||||
- Storage cost controlled via tiered retention, compression, pruning, offline archiving
|
||||
- Air-gap drift mitigated with offline kit refresh schedule, manifest verification
|
||||
- Alert fatigue reduced with burn-rate alerts, deduping, SOC runbooks
|
||||
|
||||
### Operational Assets (Sprint 0330 · 2025-11-30)
|
||||
- Observability runbook: operations/observability.md
|
||||
- Dashboard placeholder: operations/dashboards/telemetry-observability.json
|
||||
- Console security dashboard: console-security.json (Sprint 23)
|
||||
- Burn-rate alert pack for environments
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 15 – Observability & Forensics:** deliver collector/storage deployments, forensic evidence retention, and observability bundles with deterministic configuration.
|
||||
|
||||
@@ -1,69 +0,0 @@
|
||||
# Implementation plan — Telemetry
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Collector & pipeline profiles**
|
||||
Publish OpenTelemetry collector configs (`default`, `forensic`, `airgap`), establish ingest gateways, TLS/mTLS, and attribute redaction policies.
|
||||
- **Phase 2 – Storage backends & retention**
|
||||
Deploy Prometheus/Tempo/Loki (or equivalents) with retention tiers, bucket/object storage, deterministic manifest generation, and sealed-mode allowlists.
|
||||
- **Phase 3 – Incident mode & forensic capture**
|
||||
Implement incident toggles (CLI/API), tail sampling adjustments, forensic bundle generation (OTLP archives, manifest/signature), and Notify hooks.
|
||||
- **Phase 4 – Observability dashboards & automation**
|
||||
Deliver dashboards (service SLOs, queue depth, policy latency), alert rules, Grafana packages, and CLI automation for deployment and capture.
|
||||
- **Phase 5 – Offline & compliance**
|
||||
Ship Offline Kit artefacts (collectors, configs, dashboards, replay tooling), signed bundles, and documentation for air-gapped review workflows.
|
||||
- **Phase 6 – Hardening & SOC handoff**
|
||||
Complete RBAC integration, audit logging, incident response runbooks, performance tuning, and integration tests across services.
|
||||
|
||||
## Work breakdown
|
||||
- **Collector configs**
|
||||
- Maintain config templates per profile with processors (redaction, batching, resource detection) and exporters.
|
||||
- CLI automation (`stella telemetry deploy`, `stella telemetry profile diff`), validation tests, and config signing.
|
||||
- **Storage & retention**
|
||||
- Provision Prometheus/Tempo/Loki (or vendor equivalents) with retention tiers (default, forensic, airgap).
|
||||
- Ensure determinism (chunk manifests, content hashing), remote-write allowlists, sealed/offline modes.
|
||||
- Implement archivers for forensic bundles (metrics/traces/logs) with cosign signatures.
|
||||
- **Incident mode**
|
||||
- API/CLI to toggle incident sampling, retention escalation, Notify signals, and auto bundle capture.
|
||||
- Hook into Orchestrator to respond to incidents and revert after cooldown.
|
||||
- **Dashboards & alerts**
|
||||
- Dashboard packages for core services (ingestion, policy, export, attestation).
|
||||
- Alert rules for SLO burn, collector failure, exporter backlog, bundle generation errors.
|
||||
- Self-observability metrics (`collector_export_failures_total`, `telemetry_incident_mode{}`).
|
||||
- **Offline support**
|
||||
- Offline Kit assets: collector binaries/configs, import scripts, dashboards, replay instructions, compliance checklists.
|
||||
- File-based exporters and manual transfer workflows with signed manifests.
|
||||
- **Docs & runbooks**
|
||||
- Update observability overview, forensic capture guide, incident response checklist, sealed-mode instructions, RBAC matrix.
|
||||
- SOC handoff package with control objectives and audit evidence.
|
||||
|
||||
## Acceptance criteria
|
||||
- Collectors ingest metrics/logs/traces across deployments, applying redaction rules and tenant isolation; profiles validate via CI.
|
||||
- Storage backends retain data per default/forensic/airgap SLAs with deterministic chunk manifests and sealed-mode compliance.
|
||||
- Incident mode toggles sampling to 100 %, extends retention, triggers Notify, and captures forensic bundles signed with cosign.
|
||||
- Dashboards and alerts cover service SLOs, queue depth, policy latency, ingestion violations, and telemetry stack health.
|
||||
- CLI commands (`stella telemetry deploy/capture/status`) automate config rollout, forensic capture, and verification.
|
||||
- Offline bundles replay telemetry in sealed environments using provided scripts and manifests.
|
||||
|
||||
## Risks & mitigations
|
||||
- **PII leakage:** strict redaction processors, policy-managed allowlists, audit tests.
|
||||
- **Collector overload:** horizontal scaling, batching, circuit breakers, incident mode throttling.
|
||||
- **Storage cost:** tiered retention, compression, pruning policies, offline archiving.
|
||||
- **Air-gap drift:** offline kit refresh schedule, deterministic manifest verification.
|
||||
- **Alert fatigue:** burn-rate alerts, deduping, SOC runbooks.
|
||||
|
||||
## Test strategy
|
||||
- **Config lint/tests:** schema validation, unit tests for processors/exporters, golden configs.
|
||||
- **Integration:** simulate service traces/logs/metrics, verify pipelines, incident toggles, bundle generation.
|
||||
- **Performance:** load tests with peak ingestion, long retention windows, failover scenarios.
|
||||
- **Security:** redaction verification, RBAC/tenant scoping, sealed-mode tests, signed config verification.
|
||||
- **Offline:** capture bundles, transfer, replay, compliance attestation.
|
||||
|
||||
## Definition of done
|
||||
- Collector profiles, storage backends, incident mode, dashboards, CLI, and offline kit delivered with telemetry and documentation.
|
||||
- Runbooks and SOC handoff packages published; compliance checklists appended.
|
||||
- ./TASKS.md and ../../TASKS.md updated; imposed rule statements confirmed in documentation.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0330_0001_0001_docs_modules_telemetry.md`; statuses mirrored in `docs/modules/telemetry/TASKS.md`.
|
||||
- Observability evidence lives in `operations/observability.md` with Grafana JSON stub under `operations/dashboards/`.
|
||||
- Keep future doc/ops updates mirrored across sprint, TASKS, and module front doors to avoid drift.
|
||||
51
docs/modules/triage/README.md
Normal file
51
docs/modules/triage/README.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Triage
|
||||
|
||||
**Status:** Design/Planning
|
||||
**Source:** N/A (cross-cutting concept)
|
||||
**Owner:** VulnExplorer Guild
|
||||
|
||||
## Purpose
|
||||
|
||||
Triage defines workflows and data structures for vulnerability triage, exploit path analysis, and proof bundle generation. Provides specifications for prioritization, evidence review, and decision capture used by VulnExplorer and Policy modules.
|
||||
|
||||
## Components
|
||||
|
||||
**Concept Documentation:**
|
||||
- `exploit-path-inbox.md` - Exploit path inbox specification for automated triage
|
||||
- `proof-bundle-spec.md` - Proof bundle format for evidence packaging
|
||||
|
||||
**Triage Workflows:**
|
||||
- Automated triage (Signals-based prioritization)
|
||||
- Manual triage (human review and decision)
|
||||
- Exploit path analysis (reachability to exploitable sinks)
|
||||
- Proof bundle generation (evidence packaging for decisions)
|
||||
|
||||
## Implementation Locations
|
||||
|
||||
Triage functionality is implemented across multiple modules:
|
||||
- **VulnExplorer** - Triage UI and workflow orchestration
|
||||
- **Signals** - Automated prioritization scoring
|
||||
- **Policy Engine** - Exploit path analysis
|
||||
- **EvidenceLocker** - Proof bundle storage
|
||||
- **Notify** - Triage alert distribution
|
||||
|
||||
## Dependencies
|
||||
|
||||
- VulnExplorer (triage UI)
|
||||
- Signals (prioritization)
|
||||
- Policy Engine (exploit paths)
|
||||
- EvidenceLocker (proof bundles)
|
||||
- Notify (alerts)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Exploit Path Inbox: `./exploit-path-inbox.md`
|
||||
- Proof Bundle Spec: `./proof-bundle-spec.md`
|
||||
- VulnExplorer: `../vuln-explorer/`
|
||||
- Signals: `../signals/`
|
||||
- Policy: `../policy/`
|
||||
- EvidenceLocker: `../evidence-locker/`
|
||||
|
||||
## Current Status
|
||||
|
||||
Triage workflows documented in exploit-path-inbox.md and proof-bundle-spec.md. Implementation distributed across VulnExplorer (UI/workflows), Signals (scoring), Policy (analysis), and EvidenceLocker (proof storage) modules.
|
||||
@@ -26,7 +26,7 @@ The Console presents operator dashboards for scans, policies, VEX evidence, runt
|
||||
- Auth smoke tests in `operations/auth-smoke.md`.
|
||||
- Observability runbook + dashboard stub in `operations/observability.md` and `operations/dashboards/console-ui-observability.json` (offline import).
|
||||
- Console architecture doc for layout and SSE fan-out.
|
||||
- Operator guide: `../../15_UI_GUIDE.md`. Accessibility: `../../accessibility.md`. Security: `../../security/`.
|
||||
- Operator guide: `../../15_UI_GUIDE.md`. Accessibility: `../../accessibility.md`. Security: `../../security/`.
|
||||
|
||||
## Related resources
|
||||
- ./operations/auth-smoke.md
|
||||
@@ -37,6 +37,51 @@ The Console presents operator dashboards for scans, policies, VEX evidence, runt
|
||||
- DOCS-CONSOLE-23-001 … DOCS-CONSOLE-23-003 baseline (done).
|
||||
- CONSOLE-OBS-52-001 tasks for observability updates.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Current Objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases
|
||||
- Keep documentation, telemetry, and runbooks aligned with latest sprint outcomes
|
||||
- Coordinate with backend services for feature delivery across epics
|
||||
|
||||
### Epic Milestones & Workstreams
|
||||
- Epic 2 – Policy Engine & Editor: policy editor simulation and explain UX (in progress)
|
||||
- Epic 4 – Policy Studio: registry, approvals, promotion experiences (planned)
|
||||
- Epic 5 – SBOM Graph Explorer: graph navigation, overlays, diff views (planned)
|
||||
- Epic 6 – Vulnerability Explorer: triage dashboards, findings ledger, audit exports (in progress)
|
||||
- Epic 8 – Advisory AI: advisory summaries, remediation hints with strict provenance (planned)
|
||||
- Epic 9 – Orchestrator Dashboard: job/source monitoring controls (planned)
|
||||
- Epic 11 – Notifications Studio: notifications workspace with previews, audit trails (planned)
|
||||
|
||||
### Core Capabilities
|
||||
- Angular 17 workspace with signals-based state management (@ngrx/signals)
|
||||
- Real-time status via SSE for ingestion, scanning, policy, exports
|
||||
- Authority integration: fresh-auth with DPoP-protected calls, scope enforcement
|
||||
- Accessibility compliance and offline bundle support
|
||||
- API client generator for type-safe backend integration
|
||||
|
||||
### Integration Points
|
||||
- Backend APIs: Scanner, Policy, Notify, Export Center, Attestor
|
||||
- Authority: DPoP tokens and scope validation
|
||||
- Telemetry streams: observability dashboards and SSE fan-out
|
||||
- Offline bundles: deterministic build outputs
|
||||
|
||||
### Operational Assets (Sprint 0331 · 2025-11-30)
|
||||
- Auth smoke tests: operations/auth-smoke.md
|
||||
- Observability runbook: operations/observability.md
|
||||
- Dashboard stub: operations/dashboards/console-ui-observability.json
|
||||
- Console architecture: console-architecture.md (layout, SSE fan-out)
|
||||
|
||||
### Access Control (2025-11-03)
|
||||
- Authority scopes verified before enabling uploads
|
||||
- Access-control guidance retained in docs/updates/2025-11-03-vuln-explorer-access-controls.md
|
||||
|
||||
### Coordination Approach
|
||||
- Review AGENTS.md before starting new work
|
||||
- Sync with cross-cutting teams via docs/implplan/SPRINT_*.md
|
||||
- Track tasks: DOCS-CONSOLE-23-001…003 (baseline done), CONSOLE-OBS-52-001 (observability)
|
||||
- Mirror status across sprint tracker and docs/modules/ui/TASKS.md
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver deterministic policy authoring, simulation, and explain UX.
|
||||
- **Epic 4 – Policy Studio:** implement registry workspace, approvals, and promotion workflows.
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
# Implementation plan — Console UI
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Epic milestones
|
||||
- **Epic 2 – Policy Engine & Editor:** deliver policy editor simulation and explain UX.
|
||||
- **Epic 4 – Policy Studio:** build registry, approvals, and promotion experiences.
|
||||
- **Epic 5 – SBOM Graph Explorer:** implement graph navigation, overlays, and diff views.
|
||||
- **Epic 6 – Vulnerability Explorer:** surface triage dashboards, findings ledger, and audit exports.
|
||||
- **Epic 8 – Advisory AI:** integrate advisory summaries and remediation hints with strict provenance.
|
||||
- **Epic 9 – Orchestrator Dashboard:** expose job/source monitoring controls.
|
||||
- **Epic 11 – Notifications Studio:** provide notifications workspace with previews and audit trails.
|
||||
- Track supporting tasks (e.g., DOCS-CONSOLE-23-001, CONSOLE-OBS-52-001) in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Current doc refresh tracked in `docs/implplan/SPRINT_0331_0001_0001_docs_modules_ui.md`; statuses mirrored in `docs/modules/ui/TASKS.md`.
|
||||
- Observability evidence for latest demo lives in `operations/observability.md` with Grafana JSON stub under `operations/dashboards/`.
|
||||
- Keep future doc/ops changes mirrored across sprint, TASKS, and module front doors to prevent drift.
|
||||
@@ -44,9 +44,62 @@ VEX Lens produces a deterministic, provenance-rich consensus view of VEX stateme
|
||||
- Traces/logs: `consensus.group`, `consensus.join`, `consensus.persist` spans with correlation IDs and issuer details; structured logs capture trust adjustments and reconciliation outcomes.
|
||||
- Offline bundles include `consensus.jsonl`, `conflicts.jsonl`, manifest + DSSE signatures, enabling mirror deployments and replay validation.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 1 – Core lens service (In Progress)
|
||||
- Normalization pipeline: CSAF/OpenVEX/CycloneDX format support
|
||||
- Product mapping library with conservative scope scoring
|
||||
- Trust weighting functions: issuer tier, freshness decay, scope quality
|
||||
- Consensus algorithm with deterministic digest computation
|
||||
- Persistence: vex_consensus, vex_consensus_history, vex_conflict_queue tables
|
||||
- Connector-supplied trust weights/tiers from Excititor vex.provenance.* contract
|
||||
|
||||
### Phase 2 – API & integrations (Planned)
|
||||
- REST endpoints: /vex/consensus (query/detail/simulate/export)
|
||||
- Policy Engine threshold integration and simulation support
|
||||
- Vuln Explorer UI chips for consensus signals
|
||||
- VEX Lens change events for downstream consumers
|
||||
|
||||
### Phase 3 – Issuer Directory & signatures (Planned)
|
||||
- Issuer registry with CRUD, audit logs, CSAF publisher import
|
||||
- Key management and signature verification
|
||||
- RBAC enforcement and tenant overrides
|
||||
- Revocation runbooks and trust recalculation
|
||||
|
||||
### Phase 4 – Console & CLI experiences (Planned)
|
||||
- Console module: evidence table, quorum bar, conflicts, simulation drawer
|
||||
- CLI commands: stella vex consensus list/show/simulate/export
|
||||
- Saved views, filters, JSON/CSV output support
|
||||
|
||||
### Phase 5 – Recompute & performance (Planned)
|
||||
- Recompute scheduling: policy activation, Excititor deltas
|
||||
- Caching strategy and load tests (10M records/tenant, P95 < 500ms)
|
||||
- Observability dashboards and Offline Kit exports
|
||||
- Backpressure handling and incident surfacing
|
||||
|
||||
### Key Acceptance Criteria
|
||||
- Consensus results reproducible across VEX formats with deterministic digests
|
||||
- Signature verification influences trust weights without pipeline failure
|
||||
- Policy simulations show quorum shifts without persisting state
|
||||
- Issuer Directory enforces RBAC, audit logs, key rotation
|
||||
- Recompute pipeline handles deltas with backpressure management
|
||||
- Performance: P95 < 500ms for 100-row pages at 10M records/tenant
|
||||
|
||||
### Technical Decisions & Risks
|
||||
- Product mapping ambiguity: conservative scoring, manual overrides, warnings, policy review
|
||||
- Issuer compromise: signature verification, trust weighting, tenant overrides, revocation runbooks
|
||||
- Evidence storms: batching, worker sharding, orchestrator rate limiting, priority queues
|
||||
- Performance: caching, indexing, load tests, quota enforcement
|
||||
- Offline gaps: deterministic exports, manifest hashes, Offline Kit tests
|
||||
|
||||
### Provenance-Aware Trust Weighting (Current Focus)
|
||||
- Connector metadata contract: vex.provenance.* fields with provider id/name/kind
|
||||
- Weight calculation: trust.weight baseline × freshness × justification scope
|
||||
- Integrity hints: cosign.* and pgp.fingerprints toggle signature-policy shortcuts
|
||||
- Policy exposure: original provenance in sources[] for explain workflows
|
||||
|
||||
## Key docs & references
|
||||
- [`architecture.md`](architecture.md) — implementation-ready blueprint covering inputs, algorithm, APIs, storage, observability, and exports.
|
||||
- [`implementation_plan.md`](implementation_plan.md) — phased delivery roadmap and acceptance criteria.
|
||||
- [`scoring.md`](scoring.md) — future risk scoring model and formula reference.
|
||||
- [`../../vex/aggregation.md`](../../vex/aggregation.md) — Aggregation-Only Contract boundaries for VEX ingestion and downstream consumers.
|
||||
- **Operations:** [`operations/deployment.md`](operations/deployment.md), [`operations/offline-kit.md`](operations/offline-kit.md) — deployment guides and offline bundle preparation.
|
||||
|
||||
@@ -1,68 +0,0 @@
|
||||
# Implementation plan — VEX Consensus Lens
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Core lens service**
|
||||
Build normalisation pipeline (CSAF/OpenVEX/CycloneDX), product mapping library, trust weighting functions, consensus algorithm, and persistence (`vex_consensus`, history, conflicts).
|
||||
- **Phase 2 – API & integrations**
|
||||
Expose `/vex/consensus` query/detail/simulate/export endpoints, integrate Policy Engine thresholds, Vuln Explorer UI chips, and VEX Lens change events.
|
||||
- **Phase 3 – Issuer Directory & signatures**
|
||||
Deliver issuer registry, key management, signature verification, RBAC, audit logs, and tenant overrides.
|
||||
- **Phase 4 – Console & CLI experiences**
|
||||
Ship Console module (lists, evidence table, quorum bar, conflicts, simulation drawer) and CLI commands (`stella vex consensus ...`) with export support.
|
||||
- **Phase 5 – Recompute & performance**
|
||||
Implement recompute scheduling (policy activation, Excitor deltas), caching, load tests (10M records/tenant), observability dashboards, and Offline Kit exports.
|
||||
|
||||
## Work breakdown
|
||||
- **VEX Lens service**
|
||||
- Normalise VEX payloads, maintain scope scores, compute consensus digest.
|
||||
- Trust weighting functions (issuer tier, freshness decay, scope quality) ingest the new `vex.provenance.*` contract emitted by Excititor connectors (provider weight/tier, cosign metadata, fingerprints) so connector-tuned trust flows all the way to consensus.
|
||||
- Idempotent workers for consensus projection and history tracking.
|
||||
- Conflict handling queue for manual review and notifications.
|
||||
- **Integrations**
|
||||
- Excitor: enrich VEX events with issuer hints, signatures, product trees, and now connector-supplied trust weights/tiers that Lens consumes directly.
|
||||
- Policy Engine: trust knobs, simulation endpoints, policy-driven recompute.
|
||||
- Vuln Explorer & Advisory AI: consensus badges, conflict surfacing.
|
||||
- **Issuer Directory**
|
||||
- CRUD for issuers/keys, audit logs, import CSAF publishers, tenant overrides.
|
||||
- Signature verification endpoints consumed by Lens.
|
||||
- **APIs & UX**
|
||||
- REST endpoints for query/detail/conflict export, trust weight updates.
|
||||
- Console module with filters, saved views, evidence table, simulation drawer.
|
||||
- CLI commands for list/show/simulate/export with JSON/CSV output.
|
||||
- **Observability & Ops**
|
||||
- Metrics (consensus latency, conflict rate, signature failures, cache hit rate), logs, traces.
|
||||
- Dashboards + runbooks for recompute storms, mapping failures, signature errors, quota breaches.
|
||||
- Offline exports for Export Center/Offline Kit.
|
||||
|
||||
## Acceptance criteria
|
||||
- Consensus results reproducible across supported VEX formats with deterministic digests and provenance.
|
||||
- Signature verification influences trust weights; unverifiable evidence is down-weighted without pipeline failure.
|
||||
- Policy simulations show quorum shifts without persisting state; Vuln Explorer consumes consensus signals.
|
||||
- Issuer Directory enforces RBAC, audit logs, and key rotation; CLI & Console parity achieved.
|
||||
- Recompute pipeline handles Excitor deltas and policy activations with backpressure and incident surfacing.
|
||||
- Observability dashboards/alerts cover ingestion lag, conflict spikes, signature failures, performance budgets (P95 < 500 ms for 100-row pages at 10M records/tenant).
|
||||
|
||||
## Risks & mitigations
|
||||
- **Product mapping ambiguity:** conservative scope scoring, manual overrides, surfaced warnings, policy review hooks.
|
||||
- **Issuer compromise:** signature verification, trust weighting, tenant overrides, revocation runbooks.
|
||||
- **Evidence storms:** batching, worker sharding, orchestrator rate limiting, priority queues.
|
||||
- **Performance degradation:** caching, indexing, load tests, quota enforcement.
|
||||
- **Offline gaps:** deterministic exports, manifest hashes, Offline Kit tests.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** normalisers, mapping, trust weights, consensus lattice, signature verification.
|
||||
- **Property:** randomised evidence sets verifying lattice commutativity and determinism.
|
||||
- **Integration:** Excitor → Lens → Policy/Vuln Explorer flow, issuer overrides, simulation.
|
||||
- **Performance:** large tenant datasets, cache behaviour, concurrency tests.
|
||||
- **Security:** RBAC, tenant scoping, signature tampering, issuer revocation.
|
||||
- **Offline:** export/import verification, CLI parity.
|
||||
|
||||
## Definition of done
|
||||
- Lens service, issuer directory, API/CLI/Console components deployed with telemetry and runbooks.
|
||||
- Documentation set (overview, algorithm, issuer directory, API, console, policy trust) updated with imposed rule statements.
|
||||
- ./TASKS.md and ../../TASKS.md reflect current status; Offline Kit parity confirmed.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0332_0001_0001_docs_modules_vex_lens.md`; statuses mirrored in `docs/modules/vex-lens/TASKS.md`.
|
||||
- Observability evidence lives in `runbooks/observability.md` with Grafana JSON stub under `runbooks/dashboards/`.
|
||||
- Keep future doc/ops updates mirrored across sprint, TASKS, and module front doors to avoid drift.
|
||||
59
docs/modules/vexhub/README.md
Normal file
59
docs/modules/vexhub/README.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# VexHub
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/VexHub/`
|
||||
**Owner:** VEX Guild
|
||||
|
||||
## Purpose
|
||||
|
||||
VexHub aggregates, validates, and distributes VEX statements from multiple upstream sources. Provides the canonical VEX distribution layer with deterministic outputs, conflict detection, and offline-friendly operation for internal services and external tools (Trivy/Grype).
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
- `StellaOps.VexHub.WebService` - Minimal API host for distribution endpoints and admin controls
|
||||
- `StellaOps.VexHub.Worker` - Background workers for scheduled ingestion and validation (planned)
|
||||
|
||||
**Libraries:**
|
||||
- Normalization Pipeline - Canonicalizes statements to OpenVEX-compatible structures
|
||||
- Validation Pipeline - Schema validation (OpenVEX/CycloneDX/CSAF) and signature checks
|
||||
|
||||
## Configuration
|
||||
|
||||
Configuration is typically embedded in module settings.
|
||||
|
||||
Key settings:
|
||||
- PostgreSQL connection (schema: `vexhub`)
|
||||
- Ingestion source connectors
|
||||
- Validation policies (schema, signature)
|
||||
- Export snapshot settings
|
||||
- Conflict detection rules
|
||||
|
||||
## API Surface
|
||||
|
||||
- `GET /api/v1/vex/cve/{cve-id}` - Query VEX statements by CVE
|
||||
- `GET /api/v1/vex/package/{purl}` - Query VEX statements by PURL
|
||||
- `GET /api/v1/vex/source/{source-id}` - Query statements from specific source
|
||||
- `GET /api/v1/vex/export` - Bulk OpenVEX feed (deterministic)
|
||||
- `GET /api/v1/vex/index` - VEX index metadata (vex-index.json)
|
||||
|
||||
All responses use deterministic ordering: `timestamp DESC`, then `source_id ASC`, then `statement_hash ASC`.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- PostgreSQL (schema: `vexhub` with tables: `statement`, `provenance`, `conflict`, `export_cursor`)
|
||||
- Authority (authentication)
|
||||
- IssuerDirectory (issuer trust registry)
|
||||
- VexLens (consumes VexHub data for consensus)
|
||||
- Excititor (VEX ingestion source)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md`
|
||||
- VexLens: `../vex-lens/`
|
||||
- IssuerDirectory: `../issuer-directory/`
|
||||
- Excititor: `../excititor/`
|
||||
|
||||
## Current Status
|
||||
|
||||
WebService implemented with distribution API. Normalization and validation pipelines in place. Supports conflict detection and deterministic exports. Worker component planned for scheduled ingestion.
|
||||
@@ -1,16 +1,16 @@
|
||||
# StellaOps Vulnerability Explorer
|
||||
|
||||
Vulnerability Explorer delivers policy-aware triage, investigation, and reporting surfaces for effective findings.
|
||||
|
||||
## Latest updates (2025-11-30)
|
||||
- Documentation refresh aligned to sprint 0334: added observability/runbook snapshot and cross-links to OpenAPI draft (`./api.md`) and schemas in `architecture.md`.
|
||||
- New offline-friendly observability runbook at `runbooks/observability.md` plus stub Grafana JSON in `runbooks/dashboards/`.
|
||||
- Retained 2025-11-03 access-control changes; verify Authority scopes before enabling attachment uploads (`docs/updates/2025-11-03-vuln-explorer-access-controls.md`).
|
||||
|
||||
## Responsibilities
|
||||
- Present policy-evaluated findings with advisory, VEX, SBOM, and runtime context.
|
||||
- Capture triage workflow in an immutable findings ledger with role-based access.
|
||||
- Provide pivots, exports, and reports for auditors and operations teams.
|
||||
Vulnerability Explorer delivers policy-aware triage, investigation, and reporting surfaces for effective findings.
|
||||
|
||||
## Latest updates (2025-11-30)
|
||||
- Documentation refresh aligned to sprint 0334: added observability/runbook snapshot and cross-links to OpenAPI draft (`./api.md`) and schemas in `architecture.md`.
|
||||
- New offline-friendly observability runbook at `runbooks/observability.md` plus stub Grafana JSON in `runbooks/dashboards/`.
|
||||
- Retained 2025-11-03 access-control changes; verify Authority scopes before enabling attachment uploads (`docs/updates/2025-11-03-vuln-explorer-access-controls.md`).
|
||||
|
||||
## Responsibilities
|
||||
- Present policy-evaluated findings with advisory, VEX, SBOM, and runtime context.
|
||||
- Capture triage workflow in an immutable findings ledger with role-based access.
|
||||
- Provide pivots, exports, and reports for auditors and operations teams.
|
||||
- Integrate explain traces, remediation notes, and offline bundles.
|
||||
|
||||
## Key components
|
||||
@@ -24,11 +24,71 @@ Vulnerability Explorer delivers policy-aware triage, investigation, and reportin
|
||||
- Scheduler for remediation/verification jobs.
|
||||
- Notify for triage notifications.
|
||||
|
||||
## Operational notes
|
||||
- Audit logging per Epic 6 requirements.
|
||||
- Offline-ready CSV/PDF exports with deterministic hashes.
|
||||
- Dashboards for MTTR and triage throughput.
|
||||
- Observability runbook and dashboard stub: see `runbooks/observability.md` and `runbooks/dashboards/vuln-explorer-observability.json` (import locally).
|
||||
## Operational notes
|
||||
- Audit logging per Epic 6 requirements.
|
||||
- Offline-ready CSV/PDF exports with deterministic hashes.
|
||||
- Dashboards for MTTR and triage throughput.
|
||||
- Observability runbook and dashboard stub: see `runbooks/observability.md` and `runbooks/dashboards/vuln-explorer-observability.json` (import locally).
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 1 – Findings Ledger & resolver (In Progress)
|
||||
- Append-only ledger with Merkle root anchoring
|
||||
- Projector to finding_records and finding_history tables
|
||||
- Ecosystem resolvers: npm/Maven/PyPI/Go/RPM/DEB with canonical advisory keys
|
||||
- Provenance hashing and time-travel snapshots
|
||||
- Idempotent event processing
|
||||
|
||||
### Phase 2 – API & simulation (Planned)
|
||||
- REST endpoints: /v1/findings (list/detail/grouping/simulation)
|
||||
- Batch evaluation with Policy Engine rationales
|
||||
- Export orchestrator for JSON/CSV/PDF
|
||||
- Simulation endpoint returning diffs without state mutation
|
||||
|
||||
### Phase 3 – Console & CLI workflows (Planned)
|
||||
- Triage UI: assignments, comments, remediation plans, simulation bar
|
||||
- Detail tabs: policy, evidence, paths, remediation
|
||||
- Keyboard accessibility, virtualization for large result sets
|
||||
- CLI commands: stella vuln list/show/simulate/assign/accept-risk/verify-fix/export
|
||||
|
||||
### Phase 4 – Automation & integrations (Planned)
|
||||
- Advisory AI hints integration
|
||||
- Zastava runtime exposure correlation
|
||||
- Notify rules for SLA breaches and deadlines
|
||||
- Scheduler follow-up scans and Graph Explorer deep links
|
||||
|
||||
### Phase 5 – Exports & offline parity (Planned)
|
||||
- Deterministic bundles: JSON, CSV, PDF formats
|
||||
- Offline Kit manifests with signed reports
|
||||
- Audit logs and compliance exports
|
||||
- Evidence bundle viewer
|
||||
|
||||
### Phase 6 – Observability & hardening (Planned)
|
||||
- Dashboards: projection lag, MTTR, accepted-risk cadence
|
||||
- Alerts: projector backlog, API 5xx, export failures, expiring accepted-risk
|
||||
- Performance tuning for 5M findings/tenant
|
||||
- Security/RBAC validation and attachment encryption
|
||||
|
||||
### Key Acceptance Criteria
|
||||
- Ledger/event sourcing reproduces historical states byte-for-byte with Merkle verification
|
||||
- Resolver respects ecosystem semantics, scope, runtime context
|
||||
- Triage workflows enforce justification/approval with audit records
|
||||
- Simulation returns policy diffs without mutating state; CLI/UI parity achieved
|
||||
- Exports reproducible with signed manifests and provenance
|
||||
- RBAC/ABAC validated; attachments encrypted; tenant isolation guaranteed
|
||||
|
||||
### Technical Decisions & Risks
|
||||
- Advisory identity collisions: strict canonicalization, linkset references, raw evidence access
|
||||
- Resolver inaccuracies: property-based tests, path verification, manual override workflows
|
||||
- Projection lag/backlog: autoscaling, queue backpressure, alerting, pause controls
|
||||
- Export size/performance: streaming NDJSON, size estimators, chunked downloads
|
||||
- User confusion on suppression: rationale tab, explicit badges, explain traces
|
||||
|
||||
### Operational Assets (Sprint 0334 · 2025-11-30)
|
||||
- Observability runbook: runbooks/observability.md
|
||||
- Dashboard stub: runbooks/dashboards/vuln-explorer-observability.json
|
||||
- OpenAPI draft: api.md and openapi/vuln-explorer.v1.yaml
|
||||
- Access controls: docs/updates/2025-11-03-vuln-explorer-access-controls.md
|
||||
|
||||
## Epic alignment
|
||||
- Epic 6: Vulnerability Explorer.
|
||||
|
||||
@@ -1,75 +0,0 @@
|
||||
# Implementation plan — Vulnerability Explorer
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Findings Ledger & resolver**
|
||||
Create append-only ledger, projector, ecosystem resolvers (npm/Maven/PyPI/Go/RPM/DEB), canonical advisory keys, and provenance hashing.
|
||||
- **Phase 2 – API & simulation**
|
||||
Ship Vuln Explorer API (list/detail/grouping/simulation), batch evaluation with Policy Engine rationales, and export orchestrator.
|
||||
- **Phase 3 – Console & CLI workflows**
|
||||
Deliver triage UI (assignments, comments, remediation plans, simulation bar), keyboard accessibility, and CLI commands (`stella vuln ...`) with JSON/CSV output.
|
||||
- **Phase 4 – Automation & integrations**
|
||||
Integrate Advisory AI hints, Zastava runtime exposure, Notify rules, Scheduler follow-up scans, and Graph Explorer deep links.
|
||||
- **Phase 5 – Exports & offline parity**
|
||||
Generate deterministic bundles (JSON, CSV, PDF, Offline Kit manifests), audit logs, and signed reports.
|
||||
- **Phase 6 – Observability & hardening**
|
||||
Complete dashboards (projection lag, MTTR, accepted-risk cadence), alerts, runbooks, performance tuning (5M findings/tenant), and security/RBAC validation.
|
||||
|
||||
## Work breakdown
|
||||
- **Findings Ledger**
|
||||
- Define event schema, Merkle root anchoring, append-only storage, history tables.
|
||||
- Projector to `finding_records` and `finding_history`, idempotent event processing, time travel snapshots.
|
||||
- Resolver pipelines referencing SBOM inventory deltas, policy outputs, VEX consensus, runtime signals.
|
||||
- **API & exports**
|
||||
- REST endpoints (`/v1/findings`, `/v1/findings/{id}`, `/actions`, `/reports`, `/exports`) with ABAC filters.
|
||||
- Simulation endpoint returning diffs, integration with Policy Engine batch evaluation.
|
||||
- Export jobs for JSON/CSV/PDF plus Offline Kit bundle assembly and signing.
|
||||
- **Console**
|
||||
- Feature module `vuln-explorer` with grid, filters, saved views, deep links, detail tabs (policy, evidence, paths, remediation).
|
||||
- Simulation drawer, delta chips, accepted-risk approvals, evidence bundle viewer.
|
||||
- Accessibility (keyboard navigation, ARIA), virtualization for large result sets.
|
||||
- **CLI**
|
||||
- Commands `stella vuln list|show|simulate|assign|accept-risk|verify-fix|export`.
|
||||
- Stable schemas for automation; piping support; tests for exit codes.
|
||||
- **Integrations**
|
||||
- Conseiller/Excitor: normalized advisory keys, linksets, evidence retrieval.
|
||||
- SBOM Service: inventory deltas with scope/runtime flags, safe version hints.
|
||||
- Notify: events for SLA breaches, accepted-risk expiries, remediation deadlines.
|
||||
- Scheduler: trigger rescans when remediation plan milestones complete.
|
||||
- **Observability & ops**
|
||||
- Metrics (open findings, MTTR, projection lag, export duration, SLA burn), logs/traces with correlation IDs.
|
||||
- Alerting on projector backlog, API 5xx spikes, export failures, accepted-risk nearing expiry.
|
||||
- Runbooks covering recompute storms, mapping errors, report issues.
|
||||
|
||||
## Acceptance criteria
|
||||
- Ledger/event sourcing reproduces historical states byte-for-byte; Merkle hashes verify integrity.
|
||||
- Resolver respects ecosystem semantics, scope, and runtime context; path evidence presented in UI/CLI.
|
||||
- Triage workflows (assignment, comments, accepted-risk) enforce justification and approval requirements with audit records.
|
||||
- Simulation returns policy diffs without mutating state; CLI/UI parity achieved for simulation and exports.
|
||||
- Exports and Offline Kit bundles reproducible with signed manifests and provenance; reports available in JSON/CSV/PDF.
|
||||
- Observability dashboards show green SLOs, alerts fire for projection lag or SLA burns, and runbooks documented.
|
||||
- RBAC/ABAC validated; attachments encrypted; tenant isolation guaranteed.
|
||||
|
||||
## Risks & mitigations
|
||||
- **Advisory identity collisions:** strict canonicalization, linkset references, raw evidence access.
|
||||
- **Resolver inaccuracies:** property-based tests, path verification, manual override workflows.
|
||||
- **Projection lag/backlog:** autoscaling, queue backpressure, alerting, pause controls.
|
||||
- **Export size/performance:** streaming NDJSON, size estimators, chunked downloads.
|
||||
- **User confusion on suppression:** rationale tab, explicit badges, explain traces.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** resolver algorithms, state machine transitions, policy mapping, export builders.
|
||||
- **Integration:** ingestion → ledger → projector → API flow, simulation, Notify notifications.
|
||||
- **E2E:** Console triage scenarios, CLI flows, accessibility tests.
|
||||
- **Performance:** 5M findings/tenant, projection rebuild, export generation.
|
||||
- **Security:** RBAC/ABAC matrix, CSRF, attachment encryption, signed URL expiry.
|
||||
- **Determinism:** time-travel snapshots, export manifest hashing, Offline Kit replay.
|
||||
|
||||
## Definition of done
|
||||
- Services, UI/CLI, integrations, exports, and observability deployed with runbooks and Offline Kit parity.
|
||||
- Documentation suite (overview, using-console, API, CLI, findings ledger, policy mapping, VEX/SBOM integration, telemetry, security, runbooks, install) updated with imposed rule statement.
|
||||
- ./TASKS.md and ../../TASKS.md reflect active progress; compliance checklists appended where required.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0334_0001_0001_docs_modules_vuln_explorer.md` (Docs/OPS/ENG streams).
|
||||
- Observability runbook evidence lives under `runbooks/observability.md` with stub dashboard JSON for offline demos.
|
||||
- API/OpenAPI references: `api.md` and `openapi/vuln-explorer.v1.yaml` are the current contract drafts; keep Console/CLI generators synced to these.
|
||||
69
docs/modules/web/README.md
Normal file
69
docs/modules/web/README.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# Web
|
||||
|
||||
**Status:** Implemented
|
||||
**Source:** `src/Web/`
|
||||
**Owner:** UI Guild
|
||||
|
||||
## Purpose
|
||||
|
||||
Web provides the Angular 17 single-page application (SPA) frontend for StellaOps. Delivers the user interface for vulnerability exploration, policy management, scan results, SBOM visualization, and administrative functions.
|
||||
|
||||
## Components
|
||||
|
||||
**Application:**
|
||||
- `StellaOps.Web` - Angular 17 application with TypeScript, routing, and component library
|
||||
|
||||
**Key Features:**
|
||||
- Dashboard and vulnerability overview
|
||||
- Container scan results and SBOM explorer
|
||||
- Policy editor and rule visualization
|
||||
- VEX statement review and approval workflows
|
||||
- Task pack execution monitoring
|
||||
- Admin console for configuration and user management
|
||||
|
||||
## Configuration
|
||||
|
||||
Angular configuration in `angular.json` and environment files.
|
||||
|
||||
Key settings:
|
||||
- Backend API URL (`STELLAOPS_BACKEND_URL`)
|
||||
- Authority OAuth/OIDC endpoints
|
||||
- Feature flags for module visibility
|
||||
- Telemetry and error reporting
|
||||
|
||||
## Build and Development
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
cd src/Web/StellaOps.Web
|
||||
npm install
|
||||
|
||||
# Development server
|
||||
npm start
|
||||
|
||||
# Production build
|
||||
npm run build
|
||||
|
||||
# Run tests
|
||||
npm test
|
||||
|
||||
# Run E2E tests (Playwright)
|
||||
npx playwright test
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Gateway (API access)
|
||||
- Authority (authentication and authorization)
|
||||
- All backend services (via Gateway routing)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- Architecture: `./architecture.md` (if exists)
|
||||
- UI Module: `../ui/` (shared UI components)
|
||||
- Gateway: `../gateway/`
|
||||
- Authority: `../authority/`
|
||||
|
||||
## Current Status
|
||||
|
||||
Angular 17 application implemented with routing, authentication, and integration with backend services. Supports vulnerability exploration, scan results, SBOM visualization, and policy management. Playwright E2E tests configured.
|
||||
@@ -2,13 +2,13 @@
|
||||
|
||||
Zastava monitors running workloads, verifies supply chain posture, and enforces runtime policy via Kubernetes admission webhooks.
|
||||
|
||||
## Latest updates (2025-12-02)
|
||||
- DSSE-signed schemas, thresholds, exports, and deterministic `zastava-kit` bundle published under `docs/modules/zastava`; verification via `kit/verify.sh` and hashes in `SHA256SUMS`.
|
||||
- Sprint tracker `docs/implplan/SPRINT_0335_0001_0001_docs_modules_zastava.md` and module `TASKS.md` added to mirror status.
|
||||
- Observability runbook stub + dashboard placeholder added under `operations/` (offline import).
|
||||
- Surface.Env/Surface.Secrets adoption remains pending platform contracts; align with platform docs before enabling sealed mode.
|
||||
|
||||
## Responsibilities
|
||||
## Latest updates (2025-12-02)
|
||||
- DSSE-signed schemas, thresholds, exports, and deterministic `zastava-kit` bundle published under `docs/modules/zastava`; verification via `kit/verify.sh` and hashes in `SHA256SUMS`.
|
||||
- Sprint tracker `docs/implplan/SPRINT_0335_0001_0001_docs_modules_zastava.md` and module `TASKS.md` added to mirror status.
|
||||
- Observability runbook stub + dashboard placeholder added under `operations/` (offline import).
|
||||
- Surface.Env/Surface.Secrets adoption remains pending platform contracts; align with platform docs before enabling sealed mode.
|
||||
|
||||
## Responsibilities
|
||||
- Observe node/container activity and emit runtime events.
|
||||
- Validate signatures, SBOM presence, and backend verdicts before allowing containers.
|
||||
- Buffer and replay events during disconnections.
|
||||
@@ -24,16 +24,61 @@ Zastava monitors running workloads, verifies supply chain posture, and enforces
|
||||
- Scanner/Scheduler for remediation triggers.
|
||||
- Notify/UI for runtime alerts and dashboards.
|
||||
|
||||
## Operational notes
|
||||
- Runbook `./operations/observability.md` (stub) plus dashboard placeholder `./operations/dashboards/zastava-observability.json`.
|
||||
- Legacy runtime runbook assets remain under ./operations if present; keep offline kit bundles deterministic.
|
||||
- DPoP/mTLS rotation guidance shared with Authority.
|
||||
## Operational notes
|
||||
- Runbook `./operations/observability.md` (stub) plus dashboard placeholder `./operations/dashboards/zastava-observability.json`.
|
||||
- Legacy runtime runbook assets remain under ./operations if present; keep offline kit bundles deterministic.
|
||||
- DPoP/mTLS rotation guidance shared with Authority.
|
||||
|
||||
## Related resources
|
||||
- ./operations/runtime.md
|
||||
- ./operations/runtime-grafana-dashboard.json
|
||||
- ./operations/runtime-prometheus-rules.yaml
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Current Objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases
|
||||
- Keep documentation, telemetry, and runbooks aligned with latest sprint outcomes
|
||||
- Coordinate with platform contracts before enabling sealed mode
|
||||
|
||||
### Core Capabilities
|
||||
- Runtime event observation: node/container activity monitoring
|
||||
- Admission control: signature validation, SBOM presence, backend verdict checks
|
||||
- Disconnection resilience: event buffering and replay during network outages
|
||||
- Delta scan triggering when runtime posture drifts
|
||||
|
||||
### Key Components
|
||||
- StellaOps.Zastava.Observer daemonset for runtime monitoring
|
||||
- StellaOps.Zastava.Webhook admission controller for policy enforcement
|
||||
- StellaOps.Zastava.Core shared contracts
|
||||
|
||||
### Integration Points
|
||||
- Authority: OpToks and mTLS for secure communication
|
||||
- Scanner/Scheduler: remediation trigger coordination
|
||||
- Notify/UI: runtime alerts and dashboard visualization
|
||||
- Platform contracts: Surface.Env/Surface.Secrets (pending alignment)
|
||||
|
||||
### Operational Assets (Sprint 0335 · 2025-12-02)
|
||||
- DSSE-signed schemas, thresholds, exports in docs/modules/zastava
|
||||
- Deterministic zastava-kit bundle with verification via kit/verify.sh
|
||||
- SHA256SUMS for bundle integrity validation
|
||||
- Observability runbook: operations/observability.md
|
||||
- Dashboard placeholder: operations/dashboards/zastava-observability.json
|
||||
- Legacy assets: operations/runtime.md, runtime-grafana-dashboard.json, runtime-prometheus-rules.yaml
|
||||
|
||||
### Technical Decisions
|
||||
- Deterministic offline kit bundles with signed manifests
|
||||
- DPoP/mTLS rotation guidance shared with Authority
|
||||
- Surface.Env/Surface.Secrets adoption pending platform contract finalization
|
||||
|
||||
### Coordination Approach
|
||||
- Review AGENTS.md before starting new work
|
||||
- Sync with cross-cutting teams via docs/implplan/SPRINT_*.md
|
||||
- Track backlog: ZASTAVA runtime tasks in ../../TASKS.md
|
||||
- Webhook smoke tests: src/Zastava/**/TASKS.md
|
||||
- Sprint tracker: docs/implplan/SPRINT_0335_0001_0001_docs_modules_zastava.md
|
||||
- Module status mirror: docs/modules/zastava/TASKS.md
|
||||
|
||||
## Backlog references
|
||||
- ZASTAVA runtime tasks in ../../TASKS.md.
|
||||
- Webhook smoke tests tracked in src/Zastava/**/TASKS.md.
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
# Implementation plan — Zastava
|
||||
|
||||
## Current objectives
|
||||
- Maintain deterministic behaviour and offline parity across releases.
|
||||
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes.
|
||||
|
||||
## Workstreams
|
||||
- Backlog grooming: reconcile open stories in ../../TASKS.md with this module's roadmap.
|
||||
- Implementation: collaborate with service owners to land feature work defined in SPRINTS/EPIC docs.
|
||||
- Validation: extend tests/fixtures to preserve determinism and provenance requirements.
|
||||
|
||||
## Backlog references
|
||||
- ZASTAVA runtime tasks in ../../TASKS.md.
|
||||
- Webhook smoke tests tracked in src/Zastava/**/TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0335_0001_0001_docs_modules_zastava.md`; statuses mirrored in `docs/modules/zastava/TASKS.md`.
|
||||
- Observability evidence lives in `operations/observability.md` with Grafana JSON stub under `operations/dashboards/`.
|
||||
- Surface.Env/Surface.Secrets contracts remain dependencies; align with platform docs before enabling sealed mode.
|
||||
Reference in New Issue
Block a user