stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

docs consolidation work

Browse Source
This commit is contained in:
StellaOps Bot
2025-12-25 10:53:53 +02:00
parent b9f71fc7e9
commit deb82b4f03
117 changed files with 852 additions and 847 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/modules/concelier/AGENTS.md
View File

@@ -16,7 +16,7 @@ Concelier ingests signed advisories from dozens of sources and converts them int
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Honour the Aggregation-Only Contract where applicable (see ../../aoc/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change.

2
docs/modules/concelier/README.md
View File

@@ -15,7 +15,7 @@ Concelier ingests signed advisories from dozens of sources and converts them int
- Exporter packages (`StellaOps.Concelier.Exporter.*`).
## Recent updates
- **2025-11-07:** Paragraph-anchored `/advisories/{advisoryKey}/chunks` endpoint shipped for Advisory AI paragraph retrieval. Details and rollout notes live in [`../../updates/2025-11-07-concelier-advisory-chunks.md`](../../updates/2025-11-07-concelier-advisory-chunks.md).
- **2025-11-07:** Paragraph-anchored `/advisories/{advisoryKey}/chunks` endpoint shipped for Advisory AI paragraph retrieval. Details and rollout notes live in [`../../implplan/archived/updates/2025-11-07-concelier-advisory-chunks.md`](../../implplan/archived/updates/2025-11-07-concelier-advisory-chunks.md).
## Integrations & dependencies
- PostgreSQL (schema `vuln`) for canonical observations and schedules.

40
docs/modules/concelier/implementation_plan.md
View File

@@ -1,8 +1,8 @@
# Implementation plan — Concelier
## Delivery timeline
- **Phase 1 — Guardrails & schema**
Stand up Mongo JSON validators for `advisory_raw` and `vex_raw`, wire the `AOCWriteGuard` repository interceptor, and seed deterministic linkset builders. Freeze legacy normalisation paths and migrate callers to the new raw schema.
- **Phase 1 — Guardrails & schema**
Stand up PostgreSQL JSON schema validators for `advisory_raw` and `vex_raw`, wire the `AOCWriteGuard` repository interceptor, and seed deterministic linkset builders. Freeze legacy normalisation paths and migrate callers to the new raw schema.
- **Phase 2 — API & observability**
Publish ingestion and verification endpoints (`POST /ingest/*`, `GET /advisories.raw`, `POST /aoc/verify`) with Authority scopes, expose telemetry (`aoc_violation_total`, guard spans, structured logs), and ensure Offline Kit packaging captures validator deployment steps.
- **Phase 3 — Experience polish**
@@ -10,7 +10,7 @@
## Work breakdown by component
- **Concelier WebService & worker**
- Add Mongo validators and unique indexes over `(tenant, source.vendor, upstream.upstream_id, upstream.content_hash)`.
- Add PostgreSQL validators and unique indexes over `(tenant, source.vendor, upstream.upstream_id, upstream.content_hash)`.
- Implement write interceptors rejecting forbidden fields, missing provenance, or merge attempts.
- Deterministically compute linksets and persist canonical JSON payloads.
- Introduce `/ingest/advisory`, `/advisories/raw*`, and `/aoc/verify` surfaces guarded by `advisory:*` and `aoc:verify` scopes.
@@ -34,13 +34,13 @@
- Seed fixtures and run `stella aoc verify` against snapshots in pipeline gating.
## Documentation deliverables
- Update `docs/ingestion/aggregation-only-contract.md` with guard invariants, schemas, error codes, and migration guidance.
- Update `docs/aoc/aggregation-only-contract.md` with guard invariants, schemas, error codes, and migration guidance.
- Refresh `docs/modules/concelier/operations/*.md` (mirror, conflict-resolution, authority audit) with validator rollouts and observability dashboards.
- Cross-link Authority scope definitions, CLI reference, Console sources guide, and observability runbooks to the AOC guard changes.
- Ensure Offline Kit documentation captures validator bootstrap and verify workflows.
## Acceptance criteria
- Mongo validators and runtime guards reject forbidden fields and missing provenance with the documented `ERR_AOC_00x` codes.
- PostgreSQL validators and runtime guards reject forbidden fields and missing provenance with the documented `ERR_AOC_00x` codes.
- Linksets and supersedes chains are deterministic; rerunning ingestion over identical payloads yields byte-identical documents.
- CLI `stella aoc verify` exits non-zero on seeded violations and zero on clean datasets; Console dashboards show real-time guard status.
- Export Center consumes advisory datasets without relying on legacy normalised fields.
@@ -56,18 +56,18 @@
- **Unit**: guard rejection paths, provenance enforcement, idempotent insertions, linkset determinism.
- **Property**: fuzz upstream payloads to guarantee no forbidden fields emerge.
- **Integration**: batch ingest (50k advisories, mixed VEX fixtures), verifying zero guard violations and consistent supersedes.
- **Contract**: Policy Engine consumers verify raw-only reads; Export Center consumes canonical datasets.
- **End-to-end**: ingest/verify flow with CLI + Console actions to confirm observability and guard reporting.
## Definition of done
- Validators deployed and verified in staging/offline environments.
- Runtime guards, CLI/Console workflows, and CI linting all active.
- Observability dashboards and runbooks updated; metrics visible.
- Documentation updates merged; Offline Kit instructions published.
- ./TASKS.md reflects status transitions; cross-module dependencies acknowledged in ../../TASKS.md.
## Readiness checkpoints (2025-11-25)
- Sprint 110 attestation chain validated: `/internal/attestations/verify` endpoint and evidence bundle tests green (`TestResults/concelier-attestation/web.trx`, `core.trx`).
- Link-Not-Merge cache + console consumption docs frozen (see `operations/lnm-cache-plan.md`, `operations/console-lnm-consumption.md`); cache headers remain deterministic.
- Observation events transport reviewed; backlog guardrails and NATS/air-gap guidance updated in `operations/observation-events.md`.
- Next gating dependency: TaskRunner contract drop (sprint 0157 blockers) before wiring approvals/pack ingest flows into Concelier.
- **Contract**: Policy Engine consumers verify raw-only reads; Export Center consumes canonical datasets.
- **End-to-end**: ingest/verify flow with CLI + Console actions to confirm observability and guard reporting.
## Definition of done
- Validators deployed and verified in staging/offline environments.
- Runtime guards, CLI/Console workflows, and CI linting all active.
- Observability dashboards and runbooks updated; metrics visible.
- Documentation updates merged; Offline Kit instructions published.
- ./TASKS.md reflects status transitions; cross-module dependencies acknowledged in ../../TASKS.md.
## Readiness checkpoints (2025-11-25)
- Sprint 110 attestation chain validated: `/internal/attestations/verify` endpoint and evidence bundle tests green (`TestResults/concelier-attestation/web.trx`, `core.trx`).
- Link-Not-Merge cache + console consumption docs frozen (see `operations/lnm-cache-plan.md`, `operations/console-lnm-consumption.md`); cache headers remain deterministic.
- Observation events transport reviewed; backlog guardrails and NATS/air-gap guidance updated in `operations/observation-events.md`.
- Next gating dependency: TaskRunner contract drop (sprint 0157 blockers) before wiring approvals/pack ingest flows into Concelier.

8
docs/modules/concelier/link-not-merge-schema.md
View File

@@ -11,7 +11,7 @@ _Frozen v1 (add-only) — approved 2025-11-17 for CONCELIER-LNM-21-001/002/101._
- Frozen v1 as of 2025-11-17; further schema changes must go through ADR + sprint gating (CONCELIER-LNM-22x+).
- Canonical JSON Schemas + signed manifest live in `docs/modules/concelier/schemas/` (advisory observation, linkset, offline bundle). Verify with `openssl dgst -sha256 -verify schema-signing-pub.pem -signature schema.manifest.sig schema.manifest.json`.
## Observation document (Mongo JSON Schema excerpt)
## Observation document (PostgreSQL JSON Schema excerpt)
```json
{
"bsonType": "object",
@@ -152,11 +152,11 @@ When an advisory source publishes a revised version of an advisory:
- Deterministic sort: observations sorted by `source, advisoryId, fetchedAt` before hashing.
- Conflicts are additive only and now carry optional `sourceIds[]` to trace which upstream sources produced divergent values.
## Indexes (Mongo)
## Indexes (PostgreSQL)
- Observations: `{ tenantId:1, source:1, advisoryId:1, provenance.fetchedAt:-1 }` (compound for ingest); `{ provenance.sourceArtifactSha:1 }` unique to avoid dup writes.
- Linksets: `{ tenantId:1, advisoryId:1, source:1 }` unique; `{ observations:1 }` sparse for reverse lookups.
## Collections
## Tables
- `advisory_observations` — raw per-source docs (immutable).
- `advisory_linksets` — derived normalized aggregates with observation pointers and hashes.
@@ -170,7 +170,7 @@ See `docs/samples/lnm/observation-ghsa.json` and `docs/samples/lnm/linkset-ghsa.
## Approval path
1) Architecture + Concelier Core review this document.
2) If accepted, freeze JSON Schema and roll into `src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo` migrations.
2) If accepted, freeze JSON Schema and roll into `src/Concelier/__Libraries/StellaOps.Concelier.Storage.Postgres` migrations.
3) Update consumers (policy/CLI/export) to read from linksets only; deprecate Merge endpoints.
---