feat: Add new projects to solution and implement contract testing documentation
- Added "StellaOps.Policy.Engine", "StellaOps.Cartographer", and "StellaOps.SbomService" projects to the StellaOps solution. - Created AGENTS.md to outline the Contract Testing Guild Charter, detailing mission, scope, and definition of done. - Established TASKS.md for the Contract Testing Task Board, outlining tasks for Sprint 62 and Sprint 63 related to mock servers and replay testing.
This commit is contained in:
master
@@ -1,28 +1,86 @@
|
||||
# TASKS
|
||||
| Task | Owner(s) | Depends on | Notes |
|
||||
|---|---|---|---|
|
||||
|FEEDWEB-EVENTS-07-001 Advisory event replay API|Concelier WebService Guild|FEEDCORE-ENGINE-07-001|**DONE (2025-10-19)** – Added `/concelier/advisories/{vulnerabilityKey}/replay` endpoint with optional `asOf`, hex hashes, and conflict payloads; integration covered via `dotnet test src/StellaOps.Concelier.WebService.Tests/StellaOps.Concelier.WebService.Tests.csproj`.|
|
||||
|Bind & validate ConcelierOptions|BE-Base|WebService|DONE – options bound/validated with failure logging.|
|
||||
|Mongo service wiring|BE-Base|Storage.Mongo|DONE – wiring delegated to `AddMongoStorage`.|
|
||||
|Bootstrapper execution on start|BE-Base|Storage.Mongo|DONE – startup calls `MongoBootstrapper.InitializeAsync`.|
|
||||
|Plugin host options finalization|BE-Base|Plugins|DONE – default plugin directories/search patterns configured.|
|
||||
|Jobs API contract tests|QA|Core|DONE – WebServiceEndpointsTests now cover success payloads, filtering, and trigger outcome mapping.|
|
||||
|Health/Ready probes|DevOps|Ops|DONE – `/health` and `/ready` endpoints implemented.|
|
||||
|Serilog + OTEL integration hooks|BE-Base|Observability|DONE – `TelemetryExtensions` wires Serilog + OTEL with configurable exporters.|
|
||||
|Register built-in jobs (sources/exporters)|BE-Base|Core|DONE – AddBuiltInConcelierJobs adds fallback scheduler definitions for core connectors and exporters via reflection.|
|
||||
|HTTP problem details consistency|BE-Base|WebService|DONE – API errors now emit RFC7807 responses with trace identifiers and typed problem categories.|
|
||||
|Request logging and metrics|BE-Base|Observability|DONE – Serilog request logging enabled with enriched context and web.jobs counters published via OpenTelemetry.|
|
||||
|Endpoint smoke tests (health/ready/jobs error paths)|QA|WebService|DONE – WebServiceEndpointsTests assert success and problem responses for health, ready, and job trigger error paths.|
|
||||
|Batch job definition last-run lookup|BE-Base|Core|DONE – definitions endpoint now precomputes kinds array and reuses batched last-run dictionary; manual smoke verified via local GET `/jobs/definitions`.|
|
||||
|Add no-cache headers to health/readiness/jobs APIs|BE-Base|WebService|DONE – helper applies Cache-Control/Pragma/Expires on all health/ready/jobs endpoints; awaiting automated probe tests once connector fixtures stabilize.|
|
||||
|Authority configuration parity (FSR1)|DevEx/Concelier|Authority options schema|**DONE (2025-10-10)** – Options post-config loads clientSecretFile fallback, validators normalize scopes/audiences, and sample config documents issuer/credential/bypass settings.|
|
||||
|Document authority toggle & scope requirements|Docs/Concelier|Authority integration|**DONE (2025-10-21)** – Quickstart now documents staging flag, client credentials, env overrides; operator guide refresh merged. Remaining copy polishing is tracked under `DOCS-CONCELIER-07-201` in `docs/TASKS.md`.|
|
||||
|Plumb Authority client resilience options|BE-Base|Auth libraries LIB5|**DONE (2025-10-12)** – `Program.cs` wires `authority.resilience.*` + client scopes into `AddStellaOpsAuthClient`; new integration test asserts binding and retries.|
|
||||
|Author ops guidance for resilience tuning|Docs/Concelier|Plumb Authority client resilience options|**DONE (2025-10-12)** – `docs/21_INSTALL_GUIDE.md` + `docs/ops/concelier-authority-audit-runbook.md` document resilience profiles for connected vs air-gapped installs and reference monitoring cues.|
|
||||
|Document authority bypass logging patterns|Docs/Concelier|FSR3 logging|**DONE (2025-10-12)** – Updated operator guides clarify `Concelier.Authorization.Audit` fields (route/status/subject/clientId/scopes/bypass/remote) and SIEM triggers.|
|
||||
|Update Concelier operator guide for enforcement cutoff|Docs/Concelier|FSR1 rollout|**DONE (2025-10-12)** – Installation guide emphasises disabling `allowAnonymousFallback` before 2025-12-31 UTC and connects audit signals to the rollout checklist.|
|
||||
|Rename plugin drop directory to namespaced path|BE-Base|Plugins|**DONE (2025-10-19)** – Build outputs now target `StellaOps.Concelier.PluginBinaries`/`StellaOps.Authority.PluginBinaries`, plugin host defaults updated, config/docs refreshed, and `dotnet test src/StellaOps.Concelier.WebService.Tests/StellaOps.Concelier.WebService.Tests.csproj --no-restore` covers the change.|
|
||||
|Authority resilience adoption|Concelier WebService, Docs|Plumb Authority client resilience options|**BLOCKED (2025-10-10)** – Roll out retry/offline knobs to deployment docs and confirm CLI parity once LIB5 lands; unblock after resilience options wired and tested.|
|
||||
|CONCELIER-WEB-08-201 – Mirror distribution endpoints|Concelier WebService Guild|CONCELIER-EXPORT-08-201, DEVOPS-MIRROR-08-001|**DONE (2025-10-20)** – Mirror endpoints now enforce per-domain rate limits, emit cache headers, honour Authority/WWW-Authenticate, and docs cover auth + smoke workflows.|
|
||||
> Remark (2025-10-20): Updated ops runbook with token/rate-limit checks and added API tests for Retry-After + unauthorized flows.|
|
||||
|Wave 0B readiness checkpoint|Team WebService & Authority|Wave 0A completion|BLOCKED (2025-10-19) – FEEDSTORAGE-MONGO-08-001 closed, but remaining Wave 0A items (AUTH-DPOP-11-001, AUTH-MTLS-11-002, PLUGIN-DI-08-001) still open; maintain current DOING workstreams only.|
|
||||
# TASKS — Epic 1: Aggregation-Only Contract
|
||||
> **AOC Reminder:** service links and exposes raw data only—no precedence, severity, or hint computation inside Concelier APIs.
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|---|---|---|---|---|
|
||||
| CONCELIER-WEB-AOC-19-001 `Raw ingestion endpoints` | TODO | Concelier WebService Guild | CONCELIER-CORE-AOC-19-001, CONCELIER-STORE-AOC-19-001 | Implement `POST /ingest/advisory`, `GET /advisories/raw*`, and `POST /aoc/verify` minimal API endpoints. Enforce new Authority scopes, inject tenant claims, and surface `AOCWriteGuard` to repository calls. |
|
||||
| CONCELIER-WEB-AOC-19-002 `AOC observability` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-WEB-AOC-19-001 | Emit `ingestion_write_total`, `aoc_violation_total`, latency histograms, and tracing spans (`ingest.fetch/transform/write`, `aoc.guard`). Wire structured logging to include tenant, source vendor, upstream id, and content hash. |
|
||||
| CONCELIER-WEB-AOC-19-003 `Schema/guard unit tests` | TODO | QA Guild | CONCELIER-WEB-AOC-19-001 | Add unit tests covering schema validation failures, forbidden field rejections (`ERR_AOC_001/002/006/007`), idempotent upserts, and supersedes chains using deterministic fixtures. |
|
||||
| CONCELIER-WEB-AOC-19-004 `End-to-end ingest verification` | TODO | Concelier WebService Guild, QA Guild | CONCELIER-WEB-AOC-19-003, CONCELIER-CORE-AOC-19-002 | Create integration tests ingesting large advisory batches (cold/warm) validating linkset enrichment, metrics emission, and reproducible outputs. Capture load-test scripts + doc notes for Offline Kit dry runs. |
|
||||
|
||||
## Policy Engine v2
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|----|--------|----------|------------|-------|
|
||||
| CONCELIER-POLICY-20-001 `Policy selection endpoints` | TODO | Concelier WebService Guild | WEB-POLICY-20-001, CONCELIER-CORE-AOC-19-004 | Add batch advisory lookup APIs (`/policy/select/advisories`, `/policy/select/vex`) optimized for PURL/ID lists with pagination, tenant scoping, and explain metadata. |
|
||||
|
||||
## StellaOps Console (Sprint 23)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|----|--------|----------|------------|-------|
|
||||
| CONCELIER-CONSOLE-23-001 `Advisory aggregation views` | TODO | Concelier WebService Guild, BE-Base Platform Guild | CONCELIER-LNM-21-201, CONCELIER-LNM-21-202 | Expose `/console/advisories` endpoints returning aggregation groups (per linkset) with source chips, severity summaries, and provenance metadata for Console list + dashboard cards. Support filters by source, ecosystem, published/modified window, tenant enforcement. |
|
||||
| CONCELIER-CONSOLE-23-002 `Dashboard deltas API` | TODO | Concelier WebService Guild | CONCELIER-CONSOLE-23-001, CONCELIER-LNM-21-203 | Provide aggregated advisory delta counts (new, modified, conflicting) for Console dashboard + live status ticker; emit structured events for queue lag metrics. Ensure deterministic counts across repeated queries. |
|
||||
| CONCELIER-CONSOLE-23-003 `Search fan-out helpers` | TODO | Concelier WebService Guild | CONCELIER-CONSOLE-23-001 | Deliver fast lookup endpoints for CVE/GHSA/purl search (linksets, observations) returning evidence fragments for Console global search; implement caching + scope guards. |
|
||||
|
||||
## Graph Explorer v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|----|--------|----------|------------|-------|
|
||||
| CONCELIER-GRAPH-21-003 `SBOM projection API` | TODO | Concelier WebService Guild, Cartographer Guild | CONCELIER-GRAPH-21-001 | Expose normalized SBOM projection endpoint (`/sboms/{id}/projections/graph`) with pagination, tenant guard, and versioning metadata for Cartographer builds. |
|
||||
| CONCELIER-GRAPH-21-004 `Entry point registry` | TODO | Concelier WebService Guild | CONCELIER-GRAPH-21-003, SBOM-SERVICE-21-001 | Provide entrypoint/service node lookup API for Cartographer path relevance (configurable overrides, tagging) and document contract. |
|
||||
|
||||
## Link-Not-Merge v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|----|--------|----------|------------|-------|
|
||||
| CONCELIER-LNM-21-201 `Observation APIs` | TODO | Concelier WebService Guild, BE-Base Platform Guild | CONCELIER-LNM-21-001 | Add REST endpoints for advisory observations (`GET /advisories/observations`) with filters (alias, purl, source), pagination, and tenancy enforcement. |
|
||||
| CONCELIER-LNM-21-202 `Linkset APIs` | TODO | Concelier WebService Guild | CONCELIER-LNM-21-002, CONCELIER-LNM-21-003 | Implement linkset read/export endpoints (`/advisories/linksets/{id}`, `/advisories/by-purl/{purl}`, `/advisories/linksets/{id}/export`, `/evidence`) with correlation/conflict payloads and `ERR_AGG_*` mapping. |
|
||||
| CONCELIER-LNM-21-203 `Ingest events` | TODO | Concelier WebService Guild, Platform Events Guild | CONCELIER-LNM-21-005 | Publish NATS/Redis events for new observations/linksets and ensure idempotent consumer contracts; document event schemas. |
|
||||
|
||||
## Graph & Vuln Explorer v1
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|----|--------|----------|------------|-------|
|
||||
| CONCELIER-GRAPH-24-101 `Advisory summary API` | TODO | Concelier WebService Guild | CONCELIER-GRAPH-24-001 | Expose `/advisories/summary` returning raw linkset/observation metadata for overlay services; no derived severity or fix hints. |
|
||||
| CONCELIER-GRAPH-24-102 `Evidence batch API` | TODO | Concelier WebService Guild | CONCELIER-LNM-21-201 | Add batch fetch for advisory observations/linksets keyed by component sets to feed Graph overlay tooltips efficiently. |
|
||||
|
||||
## Vulnerability Explorer (Sprint 29)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|----|--------|----------|------------|-------|
|
||||
| CONCELIER-VULN-29-001 `Advisory key normalization` | TODO | Concelier WebService Guild, Data Integrity Guild | CONCELIER-LNM-21-001 | Normalize advisory identifiers (CVE/GHSA/vendor) into canonical `advisory_key`, persist `links[]`, expose raw payload snapshots for Explorer evidence tabs. Include migration/backfill scripts. |
|
||||
| CONCELIER-VULN-29-002 `Evidence retrieval API` | TODO | Concelier WebService Guild | CONCELIER-VULN-29-001, VULN-API-29-003 | Provide `/vuln/evidence/advisories/{advisory_key}` returning raw advisory docs with provenance, filtering by tenant and source. |
|
||||
| CONCELIER-VULN-29-004 `Observability enhancements` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-VULN-29-001 | Instrument metrics/logs for advisory normalization (key collisions, withdrawn flags), emit events consumed by Vuln Explorer resolver. |
|
||||
|
||||
## Advisory AI (Sprint 31)
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|----|--------|----------|------------|-------|
|
||||
| CONCELIER-AIAI-31-001 `Paragraph anchors` | TODO | Concelier WebService Guild | CONCELIER-VULN-29-001 | Expose advisory chunk API returning paragraph anchors, section metadata, and token-safe text for Advisory AI retrieval. |
|
||||
| CONCELIER-AIAI-31-002 `Structured fields` | TODO | Concelier WebService Guild | CONCELIER-AIAI-31-001 | Ensure normalized advisories expose workaround/fix/CVSS fields via API; add caching for summary queries. |
|
||||
| CONCELIER-AIAI-31-003 `Advisory AI telemetry` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-AIAI-31-001 | Emit metrics/logs for chunk requests, cache hits, and guardrail blocks triggered by advisory payloads. |
|
||||
|
||||
## Observability & Forensics (Epic 15)
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|----|--------|----------|------------|-------|
|
||||
| CONCELIER-WEB-OBS-50-001 `Telemetry adoption` | TODO | Concelier WebService Guild | TELEMETRY-OBS-50-001, CONCELIER-OBS-50-001 | Adopt telemetry core in web service host, ensure ingest + read endpoints emit trace/log fields (`tenant_id`, `route`, `decision_effect`), and add correlation IDs to responses. |
|
||||
| CONCELIER-WEB-OBS-51-001 `Observability APIs` | TODO | Concelier WebService Guild | CONCELIER-WEB-OBS-50-001, WEB-OBS-51-001 | Surface ingest health metrics, queue depth, and SLO status via `/obs/concelier/health` endpoint for Console widgets, with caching and tenant partitioning. |
|
||||
| CONCELIER-WEB-OBS-52-001 `Timeline streaming` | TODO | Concelier WebService Guild | CONCELIER-WEB-OBS-50-001, TIMELINE-OBS-52-003 | Provide SSE stream `/obs/concelier/timeline` bridging to Timeline Indexer with paging tokens, guardrails, and audit logging. |
|
||||
| CONCELIER-WEB-OBS-53-001 `Evidence locker integration` | TODO | Concelier WebService Guild, Evidence Locker Guild | CONCELIER-OBS-53-001, EVID-OBS-53-003 | Add `/evidence/advisories/*` routes invoking evidence locker snapshots, verifying tenant scopes (`evidence:read`), and returning signed manifest metadata. |
|
||||
| CONCELIER-WEB-OBS-54-001 `Attestation exposure` | TODO | Concelier WebService Guild | CONCELIER-OBS-54-001, PROV-OBS-54-001 | Provide `/attestations/advisories/*` read APIs surfacing DSSE status, verification summary, and provenance chain for Console/CLI. |
|
||||
| CONCELIER-WEB-OBS-55-001 `Incident mode toggles` | TODO | Concelier WebService Guild, DevOps Guild | CONCELIER-OBS-55-001, WEB-OBS-55-001 | Implement incident mode toggle endpoints, propagate to orchestrator/locker, and document cooldown/backoff semantics. |
|
||||
|
||||
## Air-Gapped Mode (Epic 16)
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|----|--------|----------|------------|-------|
|
||||
| CONCELIER-WEB-AIRGAP-56-001 `Mirror import APIs` | TODO | Concelier WebService Guild | AIRGAP-IMP-58-001, CONCELIER-AIRGAP-56-001 | Extend ingestion endpoints to register mirror bundle sources, expose bundle catalog queries, and block external feed URLs in sealed mode. |
|
||||
| CONCELIER-WEB-AIRGAP-56-002 `Airgap status surfaces` | TODO | Concelier WebService Guild | CONCELIER-AIRGAP-57-002, AIRGAP-CTL-56-002 | Add staleness metadata and bundle provenance to advisory APIs (`/advisories/observations`, `/advisories/linksets`). |
|
||||
| CONCELIER-WEB-AIRGAP-57-001 `Error remediation` | TODO | Concelier WebService Guild, AirGap Policy Guild | AIRGAP-POL-56-001 | Map sealed-mode violations to `AIRGAP_EGRESS_BLOCKED` responses with user guidance. |
|
||||
| CONCELIER-WEB-AIRGAP-58-001 `Import timeline emission` | TODO | Concelier WebService Guild, AirGap Importer Guild | CONCELIER-WEB-AIRGAP-56-001, TIMELINE-OBS-53-001 | Emit timeline events for bundle ingestion operations with bundle ID, scope, and actor metadata. |
|
||||
|
||||
## SDKs & OpenAPI (Epic 17)
|
||||
| ID | Status | Owner(s) | Depends on | Notes |
|
||||
|----|--------|----------|------------|-------|
|
||||
| CONCELIER-WEB-OAS-61-001 `/.well-known/openapi` | TODO | Concelier WebService Guild | OAS-61-001 | Implement discovery endpoint emitting Concelier spec with version metadata and ETag. |
|
||||
| CONCELIER-WEB-OAS-61-002 `Error envelope migration` | TODO | Concelier WebService Guild | APIGOV-61-001 | Ensure all API responses use standardized error envelope; update controllers/tests. |
|
||||
| CONCELIER-WEB-OAS-62-001 `Examples expansion` | TODO | Concelier WebService Guild | CONCELIER-OAS-61-002 | Add curated examples for advisory observations/linksets/conflicts; integrate into dev portal. |
|
||||
| CONCELIER-WEB-OAS-63-001 `Deprecation headers` | TODO | Concelier WebService Guild, API Governance Guild | APIGOV-63-001 | Add Sunset/Deprecation headers for retiring endpoints and update documentation/notifications. |
|
||||
|
||||
Reference in New Issue
Block a user