git.stella-ops.org/docs/modules/concelier/implementation_plan.md

# Implementation plan — Concelier

## Delivery timeline
- **Phase 1 — Guardrails & schema**  
  Stand up Mongo JSON validators for `advisory_raw` and `vex_raw`, wire the `AOCWriteGuard` repository interceptor, and seed deterministic linkset builders. Freeze legacy normalisation paths and migrate callers to the new raw schema.
- **Phase 2 — API & observability**  
  Publish ingestion and verification endpoints (`POST /ingest/*`, `GET /advisories.raw`, `POST /aoc/verify`) with Authority scopes, expose telemetry (`aoc_violation_total`, guard spans, structured logs), and ensure Offline Kit packaging captures validator deployment steps.
- **Phase 3 — Experience polish**  
  Ship CLI/Console affordances (`stella sources ingest --dry-run`, dashboard tiles, violation drill-downs), finish Export Center hand-off metadata, and close out CI enforcement (`stella aoc verify` preflight, AST lint, seeded fixtures).

## Work breakdown by component
- **Concelier WebService & worker**
  - Add Mongo validators and unique indexes over `(tenant, source.vendor, upstream.upstream_id, upstream.content_hash)`.
  - Implement write interceptors rejecting forbidden fields, missing provenance, or merge attempts.
  - Deterministically compute linksets and persist canonical JSON payloads.
  - Introduce `/ingest/advisory`, `/advisories/raw*`, and `/aoc/verify` surfaces guarded by `advisory:*` and `aoc:verify` scopes.
  - Emit guard metrics/traces and surface supersedes/violation audit logs.
- **Excititor (shared ingestion contract)**
  - Mirror Concelier guard and schema changes for `vex_raw`.
  - Maintain restart-time plug-in determinism and linkset extraction parity.
- **Shared libraries**
  - Publish `StellaOps.Ingestion.AOC` (forbidden key catalog, guard middleware, provenance helpers, signature verification).
  - Share error codes (`ERR_AOC_00x`) and deterministic hashing utilities.
- **Policy Engine integration**
  - Enforce `effective_finding_*` write exclusivity.
  - Consume only raw documents + linksets, removing any implicit normalisation.
- **Authority scopes**
  - Provision `advisory:ingest|read`, `vex:ingest|read`, `aoc:verify`; propagate tenant claims to ingestion services.
- **CLI & Console**
  - Implement `stella sources ingest --dry-run` and `stella aoc verify` (with exit codes mapped to `ERR_AOC_00x`).
  - Surface AOC dashboards, violation drill-down, and verification shortcuts in the Console.
- **CI/CD**
  - Add Roslyn analyzer / AST lint to block forbidden writes.
  - Seed fixtures and run `stella aoc verify` against snapshots in pipeline gating.

## Documentation deliverables
- Update `docs/ingestion/aggregation-only-contract.md` with guard invariants, schemas, error codes, and migration guidance.
- Refresh `docs/modules/concelier/operations/*.md` (mirror, conflict-resolution, authority audit) with validator rollouts and observability dashboards.
- Cross-link Authority scope definitions, CLI reference, Console sources guide, and observability runbooks to the AOC guard changes.
- Ensure Offline Kit documentation captures validator bootstrap and verify workflows.

## Acceptance criteria
- Mongo validators and runtime guards reject forbidden fields and missing provenance with the documented `ERR_AOC_00x` codes.
- Linksets and supersedes chains are deterministic; rerunning ingestion over identical payloads yields byte-identical documents.
- CLI `stella aoc verify` exits non-zero on seeded violations and zero on clean datasets; Console dashboards show real-time guard status.
- Export Center consumes advisory datasets without relying on legacy normalised fields.
- CI fails if lint rules detect forbidden writes or if seeded guard tests regress.

## Risks & mitigations
- **Collector drift introduces new forbidden keys.** Mitigated by guard middleware + CI lint + schema validation; RFC required for linkset changes.
- **Migration complexity from legacy normalisation.** Staged cutover with `_backup_*` copies and temporary views to keep Policy Engine parity.
- **Performance overhead during ingest.** Guard remains O(number of keys); index review ensures insert latency stays within warm (<5 s) / cold (<30 s) targets.
- **Tenancy leakage.** `tenant` required in schema, Authority-supplied claims enforced per request, observability alerts fire on missing tenant identifiers.

## Test strategy
- **Unit**: guard rejection paths, provenance enforcement, idempotent insertions, linkset determinism.
- **Property**: fuzz upstream payloads to guarantee no forbidden fields emerge.
- **Integration**: batch ingest (50k advisories, mixed VEX fixtures), verifying zero guard violations and consistent supersedes.
- **Contract**: Policy Engine consumers verify raw-only reads; Export Center consumes canonical datasets.
- **End-to-end**: ingest/verify flow with CLI + Console actions to confirm observability and guard reporting.

## Definition of done
- Validators deployed and verified in staging/offline environments.
- Runtime guards, CLI/Console workflows, and CI linting all active.
- Observability dashboards and runbooks updated; metrics visible.
- Documentation updates merged; Offline Kit instructions published.
- ./TASKS.md reflects status transitions; cross-module dependencies acknowledged in ../../TASKS.md.

## Readiness checkpoints (2025-11-25)
- Sprint 110 attestation chain validated: `/internal/attestations/verify` endpoint and evidence bundle tests green (`TestResults/concelier-attestation/web.trx`, `core.trx`).
- Link-Not-Merge cache + console consumption docs frozen (see `operations/lnm-cache-plan.md`, `operations/console-lnm-consumption.md`); cache headers remain deterministic.
- Observation events transport reviewed; backlog guardrails and NATS/air-gap guidance updated in `operations/observation-events.md`.
- Next gating dependency: TaskRunner contract drop (sprint 0157 blockers) before wiring approvals/pack ingest flows into Concelier.