Implement MongoDB-based storage for Pack Run approval, artifact, log, and state management
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Added MongoPackRunApprovalStore for managing approval states with MongoDB. - Introduced MongoPackRunArtifactUploader for uploading and storing artifacts. - Created MongoPackRunLogStore to handle logging of pack run events. - Developed MongoPackRunStateStore for persisting and retrieving pack run states. - Implemented unit tests for MongoDB stores to ensure correct functionality. - Added MongoTaskRunnerTestContext for setting up MongoDB test environment. - Enhanced PackRunStateFactory to correctly initialize state with gate reasons.
This commit is contained in:
master
@@ -3,8 +3,15 @@
|
||||
Excititor converts heterogeneous VEX feeds into raw observations and linksets that honour the Aggregation-Only Contract.
|
||||
|
||||
## Latest updates (2025-11-05)
|
||||
- Link-Not-Merge readiness: release note [Excitor consensus beta](../../updates/2025-11-05-excitor-consensus-beta.md) captures how Excititor feeds power the Excitor consensus beta (sample payload in [consensus JSON](../../vex/consensus-json.md)).
|
||||
- Link-Not-Merge readiness: release note [Excitor consensus beta](../../updates/2025-11-05-excitor-consensus-beta.md) captures how Excititor feeds power the Excititor consensus beta (sample payload in [consensus JSON](../../vex/consensus-json.md)).
|
||||
- README now points policy/UI teams to the upcoming consensus integration work.
|
||||
- DSSE packaging for consensus bundles and Export Center hooks are documented in the [beta release note](../../updates/2025-11-05-excitor-consensus-beta.md); operators mirroring Excititor exports must verify detached JWS artefacts (`bundle.json.jws`) alongside each bundle.
|
||||
- Follow-ups called out in the release note (Policy weighting knobs `POLICY-ENGINE-30-101`, CLI verb `CLI-VEX-30-002`) remain in-flight and are tracked in `/docs/implplan/SPRINT_200_documentation_process.md`.
|
||||
|
||||
## Release references
|
||||
- Consensus beta payload reference: [docs/vex/consensus-json.md](../../vex/consensus-json.md)
|
||||
- Export Center offline packaging: [docs/modules/export-center/devportal-offline.md](../export-center/devportal-offline.md)
|
||||
- Historical release log: [docs/updates/](../../updates/)
|
||||
|
||||
## Responsibilities
|
||||
- Fetch OpenVEX/CSAF/CycloneDX statements via restart-only connectors.
|
||||
|
||||
@@ -4,6 +4,6 @@
|
||||
|
||||
| ID | Status | Owner(s) | Description | Notes |
|
||||
|----|--------|----------|-------------|-------|
|
||||
| EXCITITOR-DOCS-0001 | DONE (2025-11-05) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | README now links to the [Excitor consensus beta release note](../../updates/2025-11-05-excitor-consensus-beta.md) and [consensus JSON sample](../../vex/consensus-json.md). |
|
||||
| EXCITITOR-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
|
||||
| EXCITITOR-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against `/docs/implplan/SPRINT_*.md`. | Update status via ./AGENTS.md workflow |
|
||||
| EXCITITOR-DOCS-0001 | DONE (2025-11-07) | Docs Guild | Validate that ./README.md aligns with the latest release notes. | README now includes DSSE/export references + release-note cross-links for the consensus beta. |
|
||||
| EXCITITOR-OPS-0001 | DONE (2025-11-07) | Ops Guild | Review runbooks/observability assets after next sprint demo. | Added runbook/observability checklist (metrics, alerts, incident steps) to `docs/modules/excititor/mirrors.md`. |
|
||||
| EXCITITOR-ENG-0001 | DONE (2025-11-07) | Module Team | Cross-check implementation plan milestones against `/docs/implplan/SPRINT_*.md`. | Implementation plan now mirrors SPRINT_200 statuses in a new sprint-alignment table. |
|
||||
|
||||
@@ -15,7 +15,17 @@
|
||||
- **Epic 8 – Advisory AI:** guarantee citation-ready payloads and normalized context for AI summaries/explainers.
|
||||
- Track DOCS-LNM-22-006/007 and CLI-EXC-25-001..002 in ../../TASKS.md.
|
||||
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
## Coordination
|
||||
- Review ./AGENTS.md before picking up new work.
|
||||
- Sync with cross-cutting teams noted in `/docs/implplan/SPRINT_*.md`.
|
||||
- Update this plan whenever scope, dependencies, or guardrails change.
|
||||
|
||||
## Sprint alignment (2025-11-07)
|
||||
|
||||
| Sprint task | State (SPRINT_200) | Notes |
|
||||
| --- | --- | --- |
|
||||
| EXCITITOR-DOCS-0001 | DONE | README release alignment + consensus beta references refreshed (DSSE/export guidance). |
|
||||
| EXCITITOR-ENG-0001 | DONE | Implementation plan now mirrors `SPRINT_200_documentation_process.md` through this table. |
|
||||
| EXCITITOR-OPS-0001 | DONE | Runbook/observability checklist added to `docs/modules/excititor/mirrors.md`. |
|
||||
|
||||
See `/docs/implplan/SPRINT_200_documentation_process.md` for the canonical status table.
|
||||
|
||||
@@ -156,9 +156,40 @@ Downstream automation reads `manifest.json`/`bundle.json` directly, while `/exci
|
||||
|
||||
---
|
||||
|
||||
## 6) Future alignment
|
||||
|
||||
* Replace manual export definitions with generated mirror bundle manifests once `EXCITITOR-EXPORT-01-007` ships.
|
||||
* Extend `/index` payload with quiet-provenance when `EXCITITOR-EXPORT-01-006` adds that metadata.
|
||||
* Integrate domain manifests with DevOps mirror profiles (`DEVOPS-MIRROR-08-001`) so helm/compose overlays can enable or disable domains declaratively.
|
||||
## 6) Future alignment
|
||||
|
||||
* Replace manual export definitions with generated mirror bundle manifests once `EXCITITOR-EXPORT-01-007` ships.
|
||||
* Extend `/index` payload with quiet-provenance when `EXCITITOR-EXPORT-01-006` adds that metadata.
|
||||
* Integrate domain manifests with DevOps mirror profiles (`DEVOPS-MIRROR-08-001`) so helm/compose overlays can enable or disable domains declaratively.
|
||||
|
||||
---
|
||||
|
||||
## 7) Runbook & observability checklist (Sprint 22 demo refresh · 2025-11-07)
|
||||
|
||||
### Daily / on-call checks
|
||||
1. **Index freshness** – watch `excitor_mirror_export_latency_seconds` (p95 < 180) grouped by `domainId`. If latency grows past 10 minutes, verify the export worker queue (`stellaops-export-worker` logs) and ensure Mongo `vex_exports` has entries newer than `now()-10m`.
|
||||
2. **Quota exhaustion** – alert on `excitor_mirror_quota_exhausted_total{scope="download"}` increases. When triggered, inspect structured logs (`MirrorDomainId`, `QuotaScope`, `RemoteIp`) and either raise limits or throttle abusive clients.
|
||||
3. **Bundle signature health** – metric `excitor_mirror_bundle_signature_verified_total` should match download counts when signing enabled. Deltas indicate missing `.jws` files; rebuild the bundle via export job or copy artefacts from the authority mirror cache.
|
||||
4. **HTTP errors** – dashboards should track 4xx/5xx rates split by route; repeated `503` statuses imply misconfigured exports. Check `mirror/index` logs for `status=misconfigured`.
|
||||
|
||||
### Incident steps
|
||||
1. Use `GET /excititor/mirror/domains/{id}/index` to capture current manifests. Attach the response to the incident log for reproducibility.
|
||||
2. For quota incidents, temporarily raise `maxIndexRequestsPerHour`/`maxDownloadRequestsPerHour` via the `Excititor:Mirror:Domains` config override, redeploy, then work with the consuming team on caching.
|
||||
3. For stale exports, trigger the export job (`Excititor.ExportRunner`) and confirm the artefacts are written to `outputRoot/<domain>`.
|
||||
4. Validate DSSE artefacts by running `cosign verify-blob --certificate-rekor-url=<rekor> --bundle <domain>/bundle.json --signature <domain>/bundle.json.jws`.
|
||||
|
||||
### Logging fields (structured)
|
||||
| Field | Description |
|
||||
| --- | --- |
|
||||
| `MirrorDomainId` | Domain handling the request (matches `id` in config). |
|
||||
| `QuotaScope` | `index` / `download`, useful when alerting on quota events. |
|
||||
| `ExportKey` | Included in download logs to pinpoint misconfigured exports. |
|
||||
| `BundleDigest` | SHA-256 of the artefact; compare with index payload when debugging corruption. |
|
||||
|
||||
### OTEL signals
|
||||
- **Counters:** `excitor.mirror.requests`, `excitor.mirror.quota_blocked`, `excitor.mirror.signature.failures`.
|
||||
- **Histograms:** `excitor.mirror.download.duration`, `excitor.mirror.export.latency`.
|
||||
- **Spans:** `mirror.index`, `mirror.download` include attributes `mirror.domain`, `mirror.export.key`, and `mirror.quota.remaining`.
|
||||
|
||||
Add these instruments via the `MirrorEndpoints` middleware; see `StellaOps.Excititor.WebService/Telemetry/MirrorMetrics.cs`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user