docs consolidation work
This commit is contained in:
StellaOps Bot
@@ -44,9 +44,62 @@ VEX Lens produces a deterministic, provenance-rich consensus view of VEX stateme
|
||||
- Traces/logs: `consensus.group`, `consensus.join`, `consensus.persist` spans with correlation IDs and issuer details; structured logs capture trust adjustments and reconciliation outcomes.
|
||||
- Offline bundles include `consensus.jsonl`, `conflicts.jsonl`, manifest + DSSE signatures, enabling mirror deployments and replay validation.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Phase 1 – Core lens service (In Progress)
|
||||
- Normalization pipeline: CSAF/OpenVEX/CycloneDX format support
|
||||
- Product mapping library with conservative scope scoring
|
||||
- Trust weighting functions: issuer tier, freshness decay, scope quality
|
||||
- Consensus algorithm with deterministic digest computation
|
||||
- Persistence: vex_consensus, vex_consensus_history, vex_conflict_queue tables
|
||||
- Connector-supplied trust weights/tiers from Excititor vex.provenance.* contract
|
||||
|
||||
### Phase 2 – API & integrations (Planned)
|
||||
- REST endpoints: /vex/consensus (query/detail/simulate/export)
|
||||
- Policy Engine threshold integration and simulation support
|
||||
- Vuln Explorer UI chips for consensus signals
|
||||
- VEX Lens change events for downstream consumers
|
||||
|
||||
### Phase 3 – Issuer Directory & signatures (Planned)
|
||||
- Issuer registry with CRUD, audit logs, CSAF publisher import
|
||||
- Key management and signature verification
|
||||
- RBAC enforcement and tenant overrides
|
||||
- Revocation runbooks and trust recalculation
|
||||
|
||||
### Phase 4 – Console & CLI experiences (Planned)
|
||||
- Console module: evidence table, quorum bar, conflicts, simulation drawer
|
||||
- CLI commands: stella vex consensus list/show/simulate/export
|
||||
- Saved views, filters, JSON/CSV output support
|
||||
|
||||
### Phase 5 – Recompute & performance (Planned)
|
||||
- Recompute scheduling: policy activation, Excititor deltas
|
||||
- Caching strategy and load tests (10M records/tenant, P95 < 500ms)
|
||||
- Observability dashboards and Offline Kit exports
|
||||
- Backpressure handling and incident surfacing
|
||||
|
||||
### Key Acceptance Criteria
|
||||
- Consensus results reproducible across VEX formats with deterministic digests
|
||||
- Signature verification influences trust weights without pipeline failure
|
||||
- Policy simulations show quorum shifts without persisting state
|
||||
- Issuer Directory enforces RBAC, audit logs, key rotation
|
||||
- Recompute pipeline handles deltas with backpressure management
|
||||
- Performance: P95 < 500ms for 100-row pages at 10M records/tenant
|
||||
|
||||
### Technical Decisions & Risks
|
||||
- Product mapping ambiguity: conservative scoring, manual overrides, warnings, policy review
|
||||
- Issuer compromise: signature verification, trust weighting, tenant overrides, revocation runbooks
|
||||
- Evidence storms: batching, worker sharding, orchestrator rate limiting, priority queues
|
||||
- Performance: caching, indexing, load tests, quota enforcement
|
||||
- Offline gaps: deterministic exports, manifest hashes, Offline Kit tests
|
||||
|
||||
### Provenance-Aware Trust Weighting (Current Focus)
|
||||
- Connector metadata contract: vex.provenance.* fields with provider id/name/kind
|
||||
- Weight calculation: trust.weight baseline × freshness × justification scope
|
||||
- Integrity hints: cosign.* and pgp.fingerprints toggle signature-policy shortcuts
|
||||
- Policy exposure: original provenance in sources[] for explain workflows
|
||||
|
||||
## Key docs & references
|
||||
- [`architecture.md`](architecture.md) — implementation-ready blueprint covering inputs, algorithm, APIs, storage, observability, and exports.
|
||||
- [`implementation_plan.md`](implementation_plan.md) — phased delivery roadmap and acceptance criteria.
|
||||
- [`scoring.md`](scoring.md) — future risk scoring model and formula reference.
|
||||
- [`../../vex/aggregation.md`](../../vex/aggregation.md) — Aggregation-Only Contract boundaries for VEX ingestion and downstream consumers.
|
||||
- **Operations:** [`operations/deployment.md`](operations/deployment.md), [`operations/offline-kit.md`](operations/offline-kit.md) — deployment guides and offline bundle preparation.
|
||||
|
||||
@@ -1,68 +0,0 @@
|
||||
# Implementation plan — VEX Consensus Lens
|
||||
|
||||
## Delivery phases
|
||||
- **Phase 1 – Core lens service**
|
||||
Build normalisation pipeline (CSAF/OpenVEX/CycloneDX), product mapping library, trust weighting functions, consensus algorithm, and persistence (`vex_consensus`, history, conflicts).
|
||||
- **Phase 2 – API & integrations**
|
||||
Expose `/vex/consensus` query/detail/simulate/export endpoints, integrate Policy Engine thresholds, Vuln Explorer UI chips, and VEX Lens change events.
|
||||
- **Phase 3 – Issuer Directory & signatures**
|
||||
Deliver issuer registry, key management, signature verification, RBAC, audit logs, and tenant overrides.
|
||||
- **Phase 4 – Console & CLI experiences**
|
||||
Ship Console module (lists, evidence table, quorum bar, conflicts, simulation drawer) and CLI commands (`stella vex consensus ...`) with export support.
|
||||
- **Phase 5 – Recompute & performance**
|
||||
Implement recompute scheduling (policy activation, Excitor deltas), caching, load tests (10M records/tenant), observability dashboards, and Offline Kit exports.
|
||||
|
||||
## Work breakdown
|
||||
- **VEX Lens service**
|
||||
- Normalise VEX payloads, maintain scope scores, compute consensus digest.
|
||||
- Trust weighting functions (issuer tier, freshness decay, scope quality) ingest the new `vex.provenance.*` contract emitted by Excititor connectors (provider weight/tier, cosign metadata, fingerprints) so connector-tuned trust flows all the way to consensus.
|
||||
- Idempotent workers for consensus projection and history tracking.
|
||||
- Conflict handling queue for manual review and notifications.
|
||||
- **Integrations**
|
||||
- Excitor: enrich VEX events with issuer hints, signatures, product trees, and now connector-supplied trust weights/tiers that Lens consumes directly.
|
||||
- Policy Engine: trust knobs, simulation endpoints, policy-driven recompute.
|
||||
- Vuln Explorer & Advisory AI: consensus badges, conflict surfacing.
|
||||
- **Issuer Directory**
|
||||
- CRUD for issuers/keys, audit logs, import CSAF publishers, tenant overrides.
|
||||
- Signature verification endpoints consumed by Lens.
|
||||
- **APIs & UX**
|
||||
- REST endpoints for query/detail/conflict export, trust weight updates.
|
||||
- Console module with filters, saved views, evidence table, simulation drawer.
|
||||
- CLI commands for list/show/simulate/export with JSON/CSV output.
|
||||
- **Observability & Ops**
|
||||
- Metrics (consensus latency, conflict rate, signature failures, cache hit rate), logs, traces.
|
||||
- Dashboards + runbooks for recompute storms, mapping failures, signature errors, quota breaches.
|
||||
- Offline exports for Export Center/Offline Kit.
|
||||
|
||||
## Acceptance criteria
|
||||
- Consensus results reproducible across supported VEX formats with deterministic digests and provenance.
|
||||
- Signature verification influences trust weights; unverifiable evidence is down-weighted without pipeline failure.
|
||||
- Policy simulations show quorum shifts without persisting state; Vuln Explorer consumes consensus signals.
|
||||
- Issuer Directory enforces RBAC, audit logs, and key rotation; CLI & Console parity achieved.
|
||||
- Recompute pipeline handles Excitor deltas and policy activations with backpressure and incident surfacing.
|
||||
- Observability dashboards/alerts cover ingestion lag, conflict spikes, signature failures, performance budgets (P95 < 500 ms for 100-row pages at 10M records/tenant).
|
||||
|
||||
## Risks & mitigations
|
||||
- **Product mapping ambiguity:** conservative scope scoring, manual overrides, surfaced warnings, policy review hooks.
|
||||
- **Issuer compromise:** signature verification, trust weighting, tenant overrides, revocation runbooks.
|
||||
- **Evidence storms:** batching, worker sharding, orchestrator rate limiting, priority queues.
|
||||
- **Performance degradation:** caching, indexing, load tests, quota enforcement.
|
||||
- **Offline gaps:** deterministic exports, manifest hashes, Offline Kit tests.
|
||||
|
||||
## Test strategy
|
||||
- **Unit:** normalisers, mapping, trust weights, consensus lattice, signature verification.
|
||||
- **Property:** randomised evidence sets verifying lattice commutativity and determinism.
|
||||
- **Integration:** Excitor → Lens → Policy/Vuln Explorer flow, issuer overrides, simulation.
|
||||
- **Performance:** large tenant datasets, cache behaviour, concurrency tests.
|
||||
- **Security:** RBAC, tenant scoping, signature tampering, issuer revocation.
|
||||
- **Offline:** export/import verification, CLI parity.
|
||||
|
||||
## Definition of done
|
||||
- Lens service, issuer directory, API/CLI/Console components deployed with telemetry and runbooks.
|
||||
- Documentation set (overview, algorithm, issuer directory, API, console, policy trust) updated with imposed rule statements.
|
||||
- ./TASKS.md and ../../TASKS.md reflect current status; Offline Kit parity confirmed.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs refresh tracked in `docs/implplan/SPRINT_0332_0001_0001_docs_modules_vex_lens.md`; statuses mirrored in `docs/modules/vex-lens/TASKS.md`.
|
||||
- Observability evidence lives in `runbooks/observability.md` with Grafana JSON stub under `runbooks/dashboards/`.
|
||||
- Keep future doc/ops updates mirrored across sprint, TASKS, and module front doors to avoid drift.
|
||||
Reference in New Issue
Block a user