Align AOC tasks for Excititor and Concelier

This commit is contained in:
master
2025-10-31 18:50:15 +02:00
committed by root
parent 9e6d9fbae8
commit 8da4e12a90
334 changed files with 35528 additions and 34546 deletions

View File

@@ -1,22 +1,22 @@
# Vexer agent guide
## Mission
Vexer computes deterministic consensus across VEX claims, preserving conflicts and producing attestable evidence for policy suppression.
## Key docs
- [Module README](./README.md)
- [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md)
## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
# Vexer agent guide
## Mission
Vexer computes deterministic consensus across VEX claims, preserving conflicts and producing attestable evidence for policy suppression.
## Key docs
- [Module README](./README.md)
- [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md)
## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change.

View File

@@ -1,34 +1,34 @@
# StellaOps Vexer
Vexer computes deterministic consensus across VEX claims, preserving conflicts and producing attestable evidence for policy suppression.
## Responsibilities
- Ingest Excititor observations and compute per-product consensus snapshots.
- Provide APIs for querying canonical VEX positions and conflict sets.
- Publish exports and DSSE-ready digests for downstream consumption.
- Keep provenance weights and disagreement metadata.
## Key components
- Consensus engine and API host in `StellaOps.Vexer.*` (to-be-implemented).
- Storage schema for consensus graphs.
- Integration hooks for Policy Engine suppression logic.
## Integrations & dependencies
- Excititor for raw observations.
- Policy Engine and UI for suppression stories.
- CLI for evidence inspection.
## Operational notes
- Deterministic consensus algorithms (see architecture).
- Planned telemetry for disagreement counts and freshness.
- Offline exports aligning with Concelier/Excititor timelines.
## Related resources
- ./scoring.md
## Backlog references
- DOCS-VEXER backlog referenced in architecture doc.
- CLI parity tracked in ../../TASKS.md (CLI-GRAPH/VEX stories).
## Epic alignment
- **Epic 7 VEX Consensus Lens:** deliver trust-weighted consensus snapshots, disagreement metadata, and explain APIs.
# StellaOps Vexer
Vexer computes deterministic consensus across VEX claims, preserving conflicts and producing attestable evidence for policy suppression.
## Responsibilities
- Ingest Excititor observations and compute per-product consensus snapshots.
- Provide APIs for querying canonical VEX positions and conflict sets.
- Publish exports and DSSE-ready digests for downstream consumption.
- Keep provenance weights and disagreement metadata.
## Key components
- Consensus engine and API host in `StellaOps.Vexer.*` (to-be-implemented).
- Storage schema for consensus graphs.
- Integration hooks for Policy Engine suppression logic.
## Integrations & dependencies
- Excititor for raw observations.
- Policy Engine and UI for suppression stories.
- CLI for evidence inspection.
## Operational notes
- Deterministic consensus algorithms (see architecture).
- Planned telemetry for disagreement counts and freshness.
- Offline exports aligning with Concelier/Excititor timelines.
## Related resources
- ./scoring.md
## Backlog references
- DOCS-VEXER backlog referenced in architecture doc.
- CLI parity tracked in ../../TASKS.md (CLI-GRAPH/VEX stories).
## Epic alignment
- **Epic 7 VEX Consensus Lens:** deliver trust-weighted consensus snapshots, disagreement metadata, and explain APIs.

View File

@@ -1,9 +1,9 @@
# Task board — Vexer
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------|
| VEXER-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| VEXER-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| VEXER-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |
# Task board — Vexer
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------|
| VEXER-DOCS-0001 | DOING (2025-10-29) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| VEXER-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| VEXER-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |

View File

@@ -1,65 +1,65 @@
# Implementation plan — Vexer
## Delivery phases
- **Phase 1 Connectors & normalization**
Build connectors for OpenVEX, CSAF VEX, CycloneDX VEX, OCI attestations; capture provenance, signatures, and source metadata; normalise into `VexClaim`.
- **Phase 2 Mapping & trust registry**
Implement product mapping (CPE → purl/version), issuer registry (trust tiers, signatures), scope scoring, and justification taxonomy.
- **Phase 3 Consensus & projections**
Deliver consensus computation, conflict preservation, projections (`vex_consensus`, history, provider snapshots), and DSSE events.
- **Phase 4 APIs & integrations**
Expose REST/CLI endpoints for claims, consensus, conflicts, exports; integrate Policy Engine, Vuln Explorer, Advisory AI, Export Center.
- **Phase 5 Observability & offline**
Ship metrics, logs, traces, dashboards, incident runbooks, Offline Kit bundles, and performance tuning (10M claims/tenant).
## Work breakdown
- **Connectors**
- Fetchers for vendor feeds, CSAF repositories, OpenVEX docs, OCI referrers.
- Signature verification (PGP, cosign, PKI) per source; schema validation; rate limiting.
- Source configuration (trust tier, fetch cadence, blackout windows) stored in metadata registry.
- **Normalization**
- Canonical `VexClaim` schema with deterministic IDs, provenance, supersedes chains.
- Product tree parsing, mapping to canonical product keys and environments.
- Justification and scope scoring derived from source semantics.
- **Consensus & projections**
- Lattice join with precedence rules, conflict tracking, confidence scores, recency decay.
- Append-only history, conflict queue, DSSE events (`vex.consensus.updated`).
- Export-ready JSONL & DSSE bundles for Offline Kit and Export Center.
- **APIs & UX**
- REST endpoints (`/claims`, `/consensus`, `/conflicts`, `/providers`) with tenant RBAC.
- CLI commands `stella vex claims|consensus|conflicts|export`.
- Console modules (list/detail, conflict diagnostics, provider health, simulation hooks).
- **Integrations**
- Policy Engine trust knobs, Vuln Explorer consensus badges, Advisory AI narrative generation, Notify alerts for conflicts.
- Orchestrator jobs for recompute/backfill triggered by Excitator deltas.
- **Observability & Ops**
- Metrics (ingest latency, signature failure rate, conflict rate, consensus latency).
- Logs/traces with tenant/issuer/provenance context.
- Runbooks for mapping failures, signature errors, recompute storms, quota exhaustion.
## Acceptance criteria
- Connectors ingest validated VEX statements with signed provenance, deterministic mapping, and tenant isolation.
- Consensus outputs reproducible, include conflicts, and integrate with Policy Engine/Vuln Explorer/Export Center.
- CLI/Console provide evidence inspection, conflict analysis, and exports; Offline Kit bundles replay verification offline.
- Observability dashboards/alerts capture ingest health, trust anomalies, conflict spikes, and performance budgets.
- Recompute pipeline handles policy changes and new evidence without dropping deterministic outcomes.
## Risks & mitigations
- **Mapping ambiguity:** maintain scope scores, manual overrides, highlight warnings.
- **Signature trust gaps:** issuer registry with auditing, fallback trust policies, tenant overrides.
- **Evidence surges:** orchestrator backpressure, prioritised queues, shardable workers.
- **Performance regressions:** indexing, caching, load tests, budget enforcement.
- **Tenant leakage:** strict RBAC/filters, fuzz tests, compliance reviews.
## Test strategy
- **Unit:** connector parsers, normalization, mapping conversions, lattice operations.
- **Property:** randomised evidence ensuring commutative consensus and deterministic digests.
- **Integration:** end-to-end pipeline from Excitator to consensus export, policy simulation, conflict handling.
- **Performance:** large feed ingestion, recompute stress, CLI export throughput.
- **Security:** signature tampering, issuer revocation, RBAC.
- **Offline:** export/import verification, DSSE bundle validation.
## Definition of done
- Connectors, normalization, consensus, APIs, and integrations deployed with telemetry, runbooks, and Offline Kit parity.
- Documentation (overview, architecture, algorithm, issuer registry, API/CLI, runbooks) updated with imposed rule compliance.
- ./TASKS.md and ../../TASKS.md reflect active status and dependencies.
# Implementation plan — Vexer
## Delivery phases
- **Phase 1 Connectors & normalization**
Build connectors for OpenVEX, CSAF VEX, CycloneDX VEX, OCI attestations; capture provenance, signatures, and source metadata; normalise into `VexClaim`.
- **Phase 2 Mapping & trust registry**
Implement product mapping (CPE → purl/version), issuer registry (trust tiers, signatures), scope scoring, and justification taxonomy.
- **Phase 3 Consensus & projections**
Deliver consensus computation, conflict preservation, projections (`vex_consensus`, history, provider snapshots), and DSSE events.
- **Phase 4 APIs & integrations**
Expose REST/CLI endpoints for claims, consensus, conflicts, exports; integrate Policy Engine, Vuln Explorer, Advisory AI, Export Center.
- **Phase 5 Observability & offline**
Ship metrics, logs, traces, dashboards, incident runbooks, Offline Kit bundles, and performance tuning (10M claims/tenant).
## Work breakdown
- **Connectors**
- Fetchers for vendor feeds, CSAF repositories, OpenVEX docs, OCI referrers.
- Signature verification (PGP, cosign, PKI) per source; schema validation; rate limiting.
- Source configuration (trust tier, fetch cadence, blackout windows) stored in metadata registry.
- **Normalization**
- Canonical `VexClaim` schema with deterministic IDs, provenance, supersedes chains.
- Product tree parsing, mapping to canonical product keys and environments.
- Justification and scope scoring derived from source semantics.
- **Consensus & projections**
- Lattice join with precedence rules, conflict tracking, confidence scores, recency decay.
- Append-only history, conflict queue, DSSE events (`vex.consensus.updated`).
- Export-ready JSONL & DSSE bundles for Offline Kit and Export Center.
- **APIs & UX**
- REST endpoints (`/claims`, `/consensus`, `/conflicts`, `/providers`) with tenant RBAC.
- CLI commands `stella vex claims|consensus|conflicts|export`.
- Console modules (list/detail, conflict diagnostics, provider health, simulation hooks).
- **Integrations**
- Policy Engine trust knobs, Vuln Explorer consensus badges, Advisory AI narrative generation, Notify alerts for conflicts.
- Orchestrator jobs for recompute/backfill triggered by Excitator deltas.
- **Observability & Ops**
- Metrics (ingest latency, signature failure rate, conflict rate, consensus latency).
- Logs/traces with tenant/issuer/provenance context.
- Runbooks for mapping failures, signature errors, recompute storms, quota exhaustion.
## Acceptance criteria
- Connectors ingest validated VEX statements with signed provenance, deterministic mapping, and tenant isolation.
- Consensus outputs reproducible, include conflicts, and integrate with Policy Engine/Vuln Explorer/Export Center.
- CLI/Console provide evidence inspection, conflict analysis, and exports; Offline Kit bundles replay verification offline.
- Observability dashboards/alerts capture ingest health, trust anomalies, conflict spikes, and performance budgets.
- Recompute pipeline handles policy changes and new evidence without dropping deterministic outcomes.
## Risks & mitigations
- **Mapping ambiguity:** maintain scope scores, manual overrides, highlight warnings.
- **Signature trust gaps:** issuer registry with auditing, fallback trust policies, tenant overrides.
- **Evidence surges:** orchestrator backpressure, prioritised queues, shardable workers.
- **Performance regressions:** indexing, caching, load tests, budget enforcement.
- **Tenant leakage:** strict RBAC/filters, fuzz tests, compliance reviews.
## Test strategy
- **Unit:** connector parsers, normalization, mapping conversions, lattice operations.
- **Property:** randomised evidence ensuring commutative consensus and deterministic digests.
- **Integration:** end-to-end pipeline from Excitator to consensus export, policy simulation, conflict handling.
- **Performance:** large feed ingestion, recompute stress, CLI export throughput.
- **Security:** signature tampering, issuer revocation, RBAC.
- **Offline:** export/import verification, DSSE bundle validation.
## Definition of done
- Connectors, normalization, consensus, APIs, and integrations deployed with telemetry, runbooks, and Offline Kit parity.
- Documentation (overview, architecture, algorithm, issuer registry, API/CLI, runbooks) updated with imposed rule compliance.
- ./TASKS.md and ../../TASKS.md reflect active status and dependencies.

View File

@@ -1,83 +1,83 @@
## Status
This document tracks the future-looking risk scoring model for Vexer. The calculation below is not active yet; Sprint 7 work will add the required schema fields, policy controls, and services. Until that ships, Vexer emits consensus statuses without numeric scores.
## Scoring model (target state)
**S = Gate(VEX_status) × W_trust(source) × [Severity_base × (1 + α·KEV + β·EPSS)]**
* **Gate(VEX_status)**: `affected`/`under_investigation` → 1, `not_affected`/`fixed` → 0. A trusted “not affected” or “fixed” still zeroes the score.
* **W_trust(source)**: normalized policy weight (baseline 01). Policies may opt into >1 boosts for signed vendor feeds once Phase 1 closes.
* **Severity_base**: canonical numeric severity from Feedser (CVSS or org-defined scale).
* **KEV flag**: 0/1 boost when CISA Known Exploited Vulnerabilities applies.
* **EPSS**: probability [0,1]; bounded multiplier.
* **α, β**: configurable coefficients (default α=0.25, β=0.5) stored in policy.
Safeguards: freeze boosts when product identity is unknown, clamp outputs ≥0, and log every factor in the audit trail.
## Implementation roadmap
| Phase | Scope | Artifacts |
| --- | --- | --- |
| **Phase 1 Schema foundations** | Extend Vexer consensus/claims and Feedser canonical advisories with severity, KEV, EPSS, and expose α/β + weight ceilings in policy. | Sprint 7 tasks `VEXER-CORE-02-001`, `VEXER-POLICY-02-001`, `VEXER-STORAGE-02-001`, `FEEDCORE-ENGINE-07-001`. |
| **Phase 2 Deterministic score engine** | Implement a scoring component that executes alongside consensus and persists score envelopes with hashes. | Planned task `VEXER-CORE-02-002` (backlog). |
| **Phase 3 Surfacing & enforcement** | Expose scores via WebService/CLI, integrate with Feedser noise priors, and enforce policy-based suppressions. | To be scheduled after Phase 2. |
## Data model (after Phase 1)
```json
{
"vulnerabilityId": "CVE-2025-12345",
"product": "pkg:name@version",
"consensus": {
"status": "affected",
"policyRevisionId": "rev-12",
"policyDigest": "0D9AEC…"
},
"signals": {
"severity": {"scheme": "CVSS:3.1", "score": 7.5},
"kev": true,
"epss": 0.40
},
"policy": {
"weight": 1.15,
"alpha": 0.25,
"beta": 0.5
},
"score": {
"value": 10.8,
"generatedAt": "2025-11-05T14:12:30Z",
"audit": [
"gate:affected",
"weight:1.15",
"severity:7.5",
"kev:1",
"epss:0.40"
]
}
}
```
## Operational guidance
* **Inputs**: Feedser delivers severity/KEV/EPSS via the advisory event log; Vexer connectors load VEX statements. Policy owns trust tiers and coefficients.
* **Processing**: the scoring engine (Phase 2) runs next to consensus, storing results with deterministic hashes so exports and attestations can reference them.
* **Consumption**: WebService/CLI will return consensus plus score; scanners may suppress findings only when policy-authorized VEX gating and signed score envelopes agree.
## Pseudocode (Phase 2 preview)
```python
def risk_score(gate, weight, severity, kev, epss, alpha, beta, freeze_boosts=False):
if gate == 0:
return 0
if freeze_boosts:
kev, epss = 0, 0
boost = 1 + alpha * kev + beta * epss
return max(0, weight * severity * boost)
```
## FAQ
* **Can operators opt out?** Set α=β=0 or keep weights ≤1.0 via policy.
* **What about missing signals?** Treat them as zero and log the omission.
* **When will this ship?** Phase 1 is planned for Sprint 7; later phases depend on connector coverage and attestation delivery.
## Status
This document tracks the future-looking risk scoring model for Vexer. The calculation below is not active yet; Sprint 7 work will add the required schema fields, policy controls, and services. Until that ships, Vexer emits consensus statuses without numeric scores.
## Scoring model (target state)
**S = Gate(VEX_status) × W_trust(source) × [Severity_base × (1 + α·KEV + β·EPSS)]**
* **Gate(VEX_status)**: `affected`/`under_investigation` → 1, `not_affected`/`fixed` → 0. A trusted “not affected” or “fixed” still zeroes the score.
* **W_trust(source)**: normalized policy weight (baseline 01). Policies may opt into >1 boosts for signed vendor feeds once Phase 1 closes.
* **Severity_base**: canonical numeric severity from Feedser (CVSS or org-defined scale).
* **KEV flag**: 0/1 boost when CISA Known Exploited Vulnerabilities applies.
* **EPSS**: probability [0,1]; bounded multiplier.
* **α, β**: configurable coefficients (default α=0.25, β=0.5) stored in policy.
Safeguards: freeze boosts when product identity is unknown, clamp outputs ≥0, and log every factor in the audit trail.
## Implementation roadmap
| Phase | Scope | Artifacts |
| --- | --- | --- |
| **Phase 1 Schema foundations** | Extend Vexer consensus/claims and Feedser canonical advisories with severity, KEV, EPSS, and expose α/β + weight ceilings in policy. | Sprint 7 tasks `VEXER-CORE-02-001`, `VEXER-POLICY-02-001`, `VEXER-STORAGE-02-001`, `FEEDCORE-ENGINE-07-001`. |
| **Phase 2 Deterministic score engine** | Implement a scoring component that executes alongside consensus and persists score envelopes with hashes. | Planned task `VEXER-CORE-02-002` (backlog). |
| **Phase 3 Surfacing & enforcement** | Expose scores via WebService/CLI, integrate with Feedser noise priors, and enforce policy-based suppressions. | To be scheduled after Phase 2. |
## Data model (after Phase 1)
```json
{
"vulnerabilityId": "CVE-2025-12345",
"product": "pkg:name@version",
"consensus": {
"status": "affected",
"policyRevisionId": "rev-12",
"policyDigest": "0D9AEC…"
},
"signals": {
"severity": {"scheme": "CVSS:3.1", "score": 7.5},
"kev": true,
"epss": 0.40
},
"policy": {
"weight": 1.15,
"alpha": 0.25,
"beta": 0.5
},
"score": {
"value": 10.8,
"generatedAt": "2025-11-05T14:12:30Z",
"audit": [
"gate:affected",
"weight:1.15",
"severity:7.5",
"kev:1",
"epss:0.40"
]
}
}
```
## Operational guidance
* **Inputs**: Feedser delivers severity/KEV/EPSS via the advisory event log; Vexer connectors load VEX statements. Policy owns trust tiers and coefficients.
* **Processing**: the scoring engine (Phase 2) runs next to consensus, storing results with deterministic hashes so exports and attestations can reference them.
* **Consumption**: WebService/CLI will return consensus plus score; scanners may suppress findings only when policy-authorized VEX gating and signed score envelopes agree.
## Pseudocode (Phase 2 preview)
```python
def risk_score(gate, weight, severity, kev, epss, alpha, beta, freeze_boosts=False):
if gate == 0:
return 0
if freeze_boosts:
kev, epss = 0, 0
boost = 1 + alpha * kev + beta * epss
return max(0, weight * severity * boost)
```
## FAQ
* **Can operators opt out?** Set α=β=0 or keep weights ≤1.0 via policy.
* **What about missing signals?** Treat them as zero and log the omission.
* **When will this ship?** Phase 1 is planned for Sprint 7; later phases depend on connector coverage and attestation delivery.