Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Added `LedgerMetrics` class to record write latency and total events for ledger operations. - Created comprehensive tests for Ruby packages endpoints, covering scenarios for missing inventory, successful retrieval, and identifier handling. - Introduced `TestSurfaceSecretsScope` for managing environment variables during tests. - Developed `ProvenanceMongoExtensions` for attaching DSSE provenance and trust information to event documents. - Implemented `EventProvenanceWriter` and `EventWriter` classes for managing event provenance in MongoDB. - Established MongoDB indexes for efficient querying of events based on provenance and trust. - Added models and JSON parsing logic for DSSE provenance and trust information.
11 KiB
11 KiB
Sprint 112 · Concelier.I — Canonical Evidence & Provenance (Rebaseline 2025-11-13)
Phase 110.B keeps Concelier focused on ingestion fidelity and evidence APIs. All active work here assumes Advisory AI consumes canonical advisory documents (no merge transforms) and that every field we emit carries exact provenance anchors.
Canonical Model Commitments
- Single source of truth:
/advisories/{key}/chunksmust render from the canonicalAdvisoryaggregate (document id + latest observation set), never from derived cache copies. - Provenance anchors: Each structured field cites both the Mongo
_idof the backing observation document and the JSON Pointer into that observation (observationPath). This mirrors how GHSA’s GraphQLsecurityAdvisory.referencesand Cisco PSIRT’sopenVulnfeeds expose source handles, so downstream tooling can reconcile fields deterministically. - Deterministic ordering: Sort structured entries by
(fieldType, observationPath, sourceId)to keep cache keys and telemetry stable across nodes. We are keeping this policy “as-is” for now to avoid churn in Advisory AI prompts. - External parity: Continue mapping fields named in competitor docs (GitHub Security Advisory GraphQL, Red Hat CVE data API, Cisco PSIRT openVuln) so migrations remain predictable.
Workstream A — Advisory AI Structured Fields (AIAI-31)
| Task ID | State | Exit criteria | Owners |
|---|---|---|---|
CONCELIER-AIAI-31-002 Structured fields |
DOING | 1) Program.cs endpoint fully rewritten to resolve the canonical advisory (via IAdvisoryStore/IAliasStore) and issue structured field entries. 2) Cache key = tenant + AdvisoryFingerprint. 3) Responses contain {chunkId, fingerprint, entries[], provenance.documentId, provenance.observationPath} with deterministic ordering. 4) Tests updated (StatementProvenanceEndpointAttachesMetadata, new structured chunk fixture) and Mongo2Go coverage passes. |
Concelier WebService Guild (src/Concelier/StellaOps.Concelier.WebService) |
CONCELIER-AIAI-31-003 Advisory AI telemetry |
DONE (2025-11-12) | OTEL counters (advisory_ai_chunk_requests_total, advisory_ai_chunk_cache_hits_total, advisory_ai_guardrail_blocks_total) tagged with tenant/result/cache. Nothing further planned unless guardrail policy changes. |
Concelier WebService Guild · Observability Guild |
Implementation checklist (kept inline until CONCELIER-AIAI-31-002 ships)
- Add
ResolveAdvisoryAsynchelper with alias fallback + tenant guard. - Update
AdvisoryChunkCacheKeyto includeAdvisoryFingerprint. - Rewrite
/advisories/{key}/chunkshandler to call the structured builder and emit provenance anchors. - Refresh telemetry tests to assert
Response.Entries.Count. - Extend docs (
docs/provenance/inline-dsse.md+ Advisory AI API reference) with the structured schema mirroring GHSA / Cisco references.
Workstream B — Mirror & Offline Provenance (AIRGAP-56/57/58)
| Task ID | State | Exit criteria / notes | Owners |
|---|---|---|---|
CONCELIER-AIRGAP-56-001 Mirror ingestion adapters |
TODO | Implement read paths for Offline Kit bundles, persist bundleId, merkleRoot, and maintain append-only ledger comparisons. |
Concelier Core Guild (src/Concelier/__Libraries/StellaOps.Concelier.Core) |
CONCELIER-AIRGAP-56-002 Bundle catalog linking |
TODO | Every observation/linkset stores {bundleId, merkleRoot, observationPath} so exported evidence can cite provenance exactly once; depends on 56-001. |
Concelier Core Guild · AirGap Importer Guild |
CONCELIER-AIRGAP-57-001 Sealed-mode source restrictions |
TODO | Feature flag + policy that rejects non-mirror connectors with actionable diagnostics; depends on 56-001. | Concelier Core Guild · AirGap Policy Guild |
CONCELIER-AIRGAP-57-002 Staleness annotations |
TODO | Compute fetchedAt/publishedAt/clockSource deltas per bundle and expose via observation APIs without mutating evidence; depends on 56-002. |
Concelier Core Guild · AirGap Time Guild |
CONCELIER-AIRGAP-58-001 Portable advisory evidence |
TODO | Package advisory observations/linksets + provenance notes (document id + observationPath) into timeline-bound portable bundles with verifier instructions; depends on 57-002. | Concelier Core Guild · Evidence Locker Guild |
Workstream C — Transparency & Attestor (ATTEST-73)
| Task ID | State | Exit criteria / notes | Owners |
|---|---|---|---|
CONCELIER-ATTEST-73-001 ScanResults attestation inputs |
TODO | Emit {observationDigest, linksetDigest, documentId} pairs required by Attestor so DSSE bundles include the same provenance anchors Advisory AI emits. |
Concelier Core Guild · Attestor Service Guild |
CONCELIER-ATTEST-73-002 Transparency metadata |
TODO | Read APIs expose bundleId, Rekor references, and observation paths for external transparency explorers; depends on 73-001. |
Concelier Core Guild |
Workstream D — Console & Search Surfaces (CONSOLE-23)
| Task ID | State | Exit criteria / notes | Owners |
|---|---|---|---|
CONCELIER-CONSOLE-23-001 Advisory aggregation views |
TODO | /console/advisories returns grouped linksets with per-source severity/status chips plus {documentId, observationPath} provenance references (matching GHSA + Red Hat CVE browser expectations); depends on CONCELIER-LNM-21-201/202. |
Concelier WebService Guild · BE-Base Platform Guild |
CONCELIER-CONSOLE-23-002 Dashboard deltas API |
TODO | Deterministic “new/modified/conflicting” sets referencing linkset IDs and field paths rather than computed verdicts; depends on 23-001. | Concelier WebService Guild |
CONCELIER-CONSOLE-23-003 Search fan-out helpers |
TODO | CVE/GHSA/PURL lookups return observation excerpts, provenance anchors, and cache hints so tenants can preview evidence safely; reuse structured field taxonomy from Workstream A. | Concelier WebService Guild |
Workstream E — Tenant Scope & AOC Guardrails
| Task ID | State | Exit criteria / notes | Owners |
|---|---|---|---|
CONCELIER-CORE-AOC-19-013 Authority tenant scope smoke coverage |
TODO | Expand smoke/e2e suites so Authority tokens + tenant headers are mandatory for ingest/read paths (including the new provenance endpoint). Must assert no merge-side effects and that provenance anchors always round-trip. | Concelier Core Guild (src/Concelier/__Libraries/StellaOps.Concelier.Core) |
Recent Updates
- 2025-11-12: CONCELIER-AIAI-31-003 shipped OTEL counters for Advisory AI chunk traffic; dashboards now display cache hit ratios and guardrail blocks per tenant.
- 2025-11-13: Sprint rebaseline complete; structured field scope locked to canonical model + provenance anchors, matching competitor schemas for short-term parity.
Current status (2025-11-13)
| Workstream | State | Notes |
|---|---|---|
| A – Advisory AI structured fields | 🔶 DOING | CONCELIER-AIAI-31-002 code work in progress; schema locked, telemetry landed, release blocked on Link-Not-Merge + CARTO schemas. |
| B – Mirror & offline provenance | 🔴 BLOCKED | No work can start until MIRROR-CRT-56-001 staffing and Offline Kit bundle contracts finalize. |
| C – Transparency & Attestor | 🔴 BLOCKED | Waiting on Workstream A output plus attestation backlog sequencing (Sprint 110/Excititor). |
| D – Console & search surfaces | 🔶 WATCHING | Scoped but dependencies on Link-Not-Merge + Console backlog; preparing schema docs in parallel. |
| E – Tenant scope & AOC guardrails | 🔶 WATCHING | Requires Authority smoke coverage; no active engineering yet but tests ready to clone once structured endpoint stabilizes. |
Blockers & dependencies
| Dependency | Impacted work | Owner(s) | Status |
|---|---|---|---|
Link-Not-Merge schema (CONCELIER-LNM-21-*, CARTO-GRAPH-21-002) |
Workstream A release, Workstream D APIs | Concelier Core · Cartographer Guild · Platform Events Guild | Review scheduled 2025-11-14; approval required before shipping structured fields/console APIs. |
| MIRROR-CRT-56-001 staffing | Workstream B (AIRGAP-56/57/58) | Mirror Creator Guild · Exporter Guild · AirGap Time Guild | Owner not assigned (per Sprint 110); kickoff on 2025-11-15 must resolve. |
| Evidence Locker attestation contract | Workstream C (ATTEST-73) | Evidence Locker Guild · Concelier Core | Needs alignment with Excititor attestation plan on 2025-11-15. |
Authority scope smoke coverage (CONCELIER-CORE-AOC-19-013) |
Workstream E | Concelier Core · Authority Guild | Waiting on structured endpoint readiness + AUTH-SIG-26-001 validation. |
Next actions (target: 2025-11-16)
| Workstream | Owner(s) | Action | Status |
|---|---|---|---|
| A | Concelier WebService Guild | Finish ResolveAdvisoryAsync, cache key update, and structured response builder; prep PR for review once schema approved. |
In progress |
| A | Docs Guild | Draft structured field schema appendix referencing provenance anchors for Advisory AI docs. | Pending |
| B | Concelier Core + Mirror leadership | Join 2025-11-15 kickoff, capture MIRROR-CRT-56-001 owner, and align bundle metadata contract. | Pending |
| C | Concelier Core + Evidence Locker | Produce attestation payload outline so ATTEST-73-001 can start immediately after sequencing meeting. | Pending |
| D | Concelier WebService Guild | Prepare /console/advisories API spec (field list, provenance references) so implementation can begin once Link-Not-Merge clears. |
Drafting |
| E | Concelier Core | Clone Authority smoke suites to cover new structured endpoint once Workstream A enters review. | Pending |
Standup prompts
- Has Link-Not-Merge schema review resolved all blocking comments? If not, what fields remain at risk?
- Who will own MIRROR-CRT-56-001 after the 2025-11-15 kickoff, and do we have staffing for follow-on AIRGAP tasks?
- Did Evidence Locker accept the attestation contract draft, enabling ATTEST-73-001 to move forward?
- Are Authority/AOC smoke tests ready to clone once structured fields release, or do we need additional scope from AUTH-SIG-26-001?
Risks (snapshot 2025-11-13)
| Risk | Impact | Mitigation / owner |
|---|---|---|
| Link-Not-Merge schema slips past 2025-11-14 | Structured fields + console APIs stay unreleased, blocking Advisory AI and Console surfaces. | Push for schema sign-off during 2025-11-14 review; prep fallback adapter if necessary. |
| Mirror staffing unresolved | AirGap provenance work (AIRGAP-56/57/58) cannot start, delaying Offline Kit parity. | Escalate at 2025-11-15 kickoff; consider borrowing engineers from Evidence Locker or Export guilds. |
| Evidence Locker contract delay | ATTEST-73 work cannot begin, leaving Advisory AI without attested provenance. | Align with Excititor/Evidence Locker owners during 2025-11-15 sequencing session; draft interim spec. |
| Authority smoke coverage gap | AOC guardrails may regress when structured endpoint ships. | Schedule paired testing with Authority guild once Workstream A PR is ready. |