feat(docs): Add comprehensive documentation for Vexer, Vulnerability Explorer, and Zastava modules
- Introduced AGENTS.md, README.md, TASKS.md, and implementation_plan.md for Vexer, detailing mission, responsibilities, key components, and operational notes. - Established similar documentation structure for Vulnerability Explorer and Zastava modules, including their respective workflows, integrations, and observability notes. - Created risk scoring profiles documentation outlining the core workflow, factor model, governance, and deliverables. - Ensured all modules adhere to the Aggregation-Only Contract and maintain determinism and provenance in outputs.
This commit is contained in:
		
							
								
								
									
										67
									
								
								docs/modules/concelier/implementation_plan.md
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										67
									
								
								docs/modules/concelier/implementation_plan.md
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,67 @@ | ||||
| # Implementation plan — Concelier | ||||
|  | ||||
| ## Delivery timeline | ||||
| - **Phase 1 — Guardrails & schema**   | ||||
|   Stand up Mongo JSON validators for `advisory_raw` and `vex_raw`, wire the `AOCWriteGuard` repository interceptor, and seed deterministic linkset builders. Freeze legacy normalisation paths and migrate callers to the new raw schema. | ||||
| - **Phase 2 — API & observability**   | ||||
|   Publish ingestion and verification endpoints (`POST /ingest/*`, `GET /advisories.raw`, `POST /aoc/verify`) with Authority scopes, expose telemetry (`aoc_violation_total`, guard spans, structured logs), and ensure Offline Kit packaging captures validator deployment steps. | ||||
| - **Phase 3 — Experience polish**   | ||||
|   Ship CLI/Console affordances (`stella sources ingest --dry-run`, dashboard tiles, violation drill-downs), finish Export Center hand-off metadata, and close out CI enforcement (`stella aoc verify` preflight, AST lint, seeded fixtures). | ||||
|  | ||||
| ## Work breakdown by component | ||||
| - **Concelier WebService & worker** | ||||
|   - Add Mongo validators and unique indexes over `(tenant, source.vendor, upstream.upstream_id, upstream.content_hash)`. | ||||
|   - Implement write interceptors rejecting forbidden fields, missing provenance, or merge attempts. | ||||
|   - Deterministically compute linksets and persist canonical JSON payloads. | ||||
|   - Introduce `/ingest/advisory`, `/advisories/raw*`, and `/aoc/verify` surfaces guarded by `advisory:*` and `aoc:verify` scopes. | ||||
|   - Emit guard metrics/traces and surface supersedes/violation audit logs. | ||||
| - **Excititor (shared ingestion contract)** | ||||
|   - Mirror Concelier guard and schema changes for `vex_raw`. | ||||
|   - Maintain restart-time plug-in determinism and linkset extraction parity. | ||||
| - **Shared libraries** | ||||
|   - Publish `StellaOps.Ingestion.AOC` (forbidden key catalog, guard middleware, provenance helpers, signature verification). | ||||
|   - Share error codes (`ERR_AOC_00x`) and deterministic hashing utilities. | ||||
| - **Policy Engine integration** | ||||
|   - Enforce `effective_finding_*` write exclusivity. | ||||
|   - Consume only raw documents + linksets, removing any implicit normalisation. | ||||
| - **Authority scopes** | ||||
|   - Provision `advisory:ingest|read`, `vex:ingest|read`, `aoc:verify`; propagate tenant claims to ingestion services. | ||||
| - **CLI & Console** | ||||
|   - Implement `stella sources ingest --dry-run` and `stella aoc verify` (with exit codes mapped to `ERR_AOC_00x`). | ||||
|   - Surface AOC dashboards, violation drill-down, and verification shortcuts in the Console. | ||||
| - **CI/CD** | ||||
|   - Add Roslyn analyzer / AST lint to block forbidden writes. | ||||
|   - Seed fixtures and run `stella aoc verify` against snapshots in pipeline gating. | ||||
|  | ||||
| ## Documentation deliverables | ||||
| - Update `docs/ingestion/aggregation-only-contract.md` with guard invariants, schemas, error codes, and migration guidance. | ||||
| - Refresh `docs/modules/concelier/operations/*.md` (mirror, conflict-resolution, authority audit) with validator rollouts and observability dashboards. | ||||
| - Cross-link Authority scope definitions, CLI reference, Console sources guide, and observability runbooks to the AOC guard changes. | ||||
| - Ensure Offline Kit documentation captures validator bootstrap and verify workflows. | ||||
|  | ||||
| ## Acceptance criteria | ||||
| - Mongo validators and runtime guards reject forbidden fields and missing provenance with the documented `ERR_AOC_00x` codes. | ||||
| - Linksets and supersedes chains are deterministic; rerunning ingestion over identical payloads yields byte-identical documents. | ||||
| - CLI `stella aoc verify` exits non-zero on seeded violations and zero on clean datasets; Console dashboards show real-time guard status. | ||||
| - Export Center consumes advisory datasets without relying on legacy normalised fields. | ||||
| - CI fails if lint rules detect forbidden writes or if seeded guard tests regress. | ||||
|  | ||||
| ## Risks & mitigations | ||||
| - **Collector drift introduces new forbidden keys.** Mitigated by guard middleware + CI lint + schema validation; RFC required for linkset changes. | ||||
| - **Migration complexity from legacy normalisation.** Staged cutover with `_backup_*` copies and temporary views to keep Policy Engine parity. | ||||
| - **Performance overhead during ingest.** Guard remains O(number of keys); index review ensures insert latency stays within warm (<5 s) / cold (<30 s) targets. | ||||
| - **Tenancy leakage.** `tenant` required in schema, Authority-supplied claims enforced per request, observability alerts fire on missing tenant identifiers. | ||||
|  | ||||
| ## Test strategy | ||||
| - **Unit**: guard rejection paths, provenance enforcement, idempotent insertions, linkset determinism. | ||||
| - **Property**: fuzz upstream payloads to guarantee no forbidden fields emerge. | ||||
| - **Integration**: batch ingest (50k advisories, mixed VEX fixtures), verifying zero guard violations and consistent supersedes. | ||||
| - **Contract**: Policy Engine consumers verify raw-only reads; Export Center consumes canonical datasets. | ||||
| - **End-to-end**: ingest/verify flow with CLI + Console actions to confirm observability and guard reporting. | ||||
|  | ||||
| ## Definition of done | ||||
| - Validators deployed and verified in staging/offline environments. | ||||
| - Runtime guards, CLI/Console workflows, and CI linting all active. | ||||
| - Observability dashboards and runbooks updated; metrics visible. | ||||
| - Documentation updates merged; Offline Kit instructions published. | ||||
| - ./TASKS.md reflects status transitions; cross-module dependencies acknowledged in ../../TASKS.md. | ||||
		Reference in New Issue
	
	Block a user