feat: Implement Policy Engine Evaluation Service and Cache with unit tests
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

Temp commit to debug
This commit is contained in:
master
2025-11-05 07:35:53 +00:00
parent 40e7f827da
commit 9253620833
125 changed files with 18735 additions and 17215 deletions

View File

@@ -323,17 +323,40 @@ Accept: application/json
**Response 200**:
```json
{
"scanId": "2f6c17f9b3f548e2a28b9c412f4d63f8",
"status": "Pending",
"image": {
"reference": "registry.example.com/acme/app:1.2.3",
"digest": null
},
"createdAt": "2025-10-18T20:15:12.482Z",
"updatedAt": "2025-10-18T20:15:12.482Z",
"failureReason": null
}
{
"scanId": "2f6c17f9b3f548e2a28b9c412f4d63f8",
"status": "Pending",
"image": {
"reference": "registry.example.com/acme/app:1.2.3",
"digest": "sha256:cafecafecafecafecafecafecafecafecafecafecafecafecafecafecafecafe"
},
"createdAt": "2025-10-18T20:15:12.482Z",
"updatedAt": "2025-10-18T20:15:12.482Z",
"failureReason": null,
"surface": {
"tenant": "default",
"generatedAt": "2025-10-18T20:15:12.482Z",
"manifestDigest": "sha256:8b4ddf1a9d3565eb7c2b176a0a64a970795e5ec373dbea3aaebb4208f9759b44",
"manifestUri": "cas://scanner-artifacts/scanner/surface/manifests/default/sha256/8b/4d/8b4ddf1a9d3565eb7c2b176a0a64a970795e5ec373dbea3aaebb4208f9759b44.json",
"manifest": {
"schema": "stellaops.surface.manifest@1",
"tenant": "default",
"imageDigest": "sha256:cafecafecafecafecafecafecafecafecafecafecafecafecafecafecafecafe",
"generatedAt": "2025-10-18T20:15:12.482Z",
"artifacts": [
{
"kind": "sbom-inventory",
"uri": "cas://scanner-artifacts/scanner/images/cafecafecafecafecafecafecafecafecafecafecafecafecafecafecafecafe/sbom.cdx.json",
"digest": "sha256:deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef",
"mediaType": "application/vnd.cyclonedx+json; version=1.6; view=inventory",
"format": "cdx-json",
"sizeBytes": 2048,
"view": "inventory"
}
]
}
}
}
```
Statuses: `Pending`, `Running`, `Succeeded`, `Failed`, `Cancelled`.
@@ -445,8 +468,40 @@ Request body mirrors policy preview inputs (image digest plus findings). The ser
"reachability": "runtime"
}
],
"issues": []
},
"issues": [],
"surface": {
"tenant": "default",
"generatedAt": "2025-10-23T15:32:22Z",
"manifestDigest": "sha256:1f3c5d7a8e4b921a0c2f6b8de0bb5aa8f5aa51b62e7d7d1864f9c826bfb44d91",
"manifestUri": "cas://scanner-artifacts/scanner/surface/manifests/default/sha256/1f/3c/1f3c5d7a8e4b921a0c2f6b8de0bb5aa8f5aa51b62e7d7d1864f9c826bfb44d91.json",
"manifest": {
"schema": "stellaops.surface.manifest@1",
"tenant": "default",
"imageDigest": "sha256:7dbe0c9a5d4f1c8184007e9d94dbe55928f8a2db5ab9c1c2d4a2f7bbcdfe1234",
"generatedAt": "2025-10-23T15:32:22Z",
"artifacts": [
{
"kind": "sbom-inventory",
"uri": "cas://scanner-artifacts/scanner/images/7dbe0c9a5d4f1c8184007e9d94dbe55928f8a2db5ab9c1c2d4a2f7bbcdfe1234/sbom.cdx.json",
"digest": "sha256:2b8ce7dd0037e59f0f93e4a5cff45b1eb305a511a1c9e2895d2f4ecdf616d3da",
"mediaType": "application/vnd.cyclonedx+json; version=1.6; view=inventory",
"format": "cdx-json",
"sizeBytes": 3072,
"view": "inventory"
},
{
"kind": "sbom-usage",
"uri": "cas://scanner-artifacts/scanner/images/7dbe0c9a5d4f1c8184007e9d94dbe55928f8a2db5ab9c1c2d4a2f7bbcdfe1234/sbom.cdx.pb",
"digest": "sha256:74e4d9f8ab0f2a1772e5768e15a5a9d7b662b849b1f223c8d6f3b184e4ac7780",
"mediaType": "application/vnd.cyclonedx+protobuf; version=1.6; view=usage",
"format": "cdx-protobuf",
"sizeBytes": 12800,
"view": "usage"
}
]
}
}
},
"dsse": {
"payloadType": "application/vnd.stellaops.report+json",
"payload": "eyJyZXBvcnQiOnsicmVwb3J0SWQiOiJyZXBvcnQtOWY4Y2RlMjFhYWI1NDMyMSJ9fQ==",
@@ -461,7 +516,7 @@ Request body mirrors policy preview inputs (image digest plus findings). The ser
}
```
- The `report` object omits null fields and is deterministic (ISO timestamps, sorted keys) while surfacing `unknownConfidence`, `confidenceBand`, and `unknownAgeDays` for auditability.
- The `report` object omits null fields and is deterministic (ISO timestamps, sorted keys) while surfacing `unknownConfidence`, `confidenceBand`, `unknownAgeDays`, and a `surface` block containing the manifest digest and CAS URIs for downstream tooling.
- `dsse` follows the DSSE (Dead Simple Signing Envelope) shape; `payload` is the canonical UTF-8 JSON and `signatures[0].signature` is the base64 HMAC/Ed25519 value depending on configuration.
- Full offline samples live at `samples/policy/policy-report-unknown.json` (request + response) and `samples/api/reports/report-sample.dsse.json` (envelope fixture) for tooling tests or signature verification.

View File

@@ -326,7 +326,80 @@ Produces sealed bundle for determinism verification; returns location of bundle.
---
## 7·Effective Findings APIs
## 7·Batch Evaluation API
Deterministic evaluator for downstream services (Findings Ledger, replay tooling, offline exporters). Consumers submit ledger event payloads and receive policy verdicts with rationale lists; no state is persisted in Policy Engine.
```
POST /api/policy/eval/batch
Scopes: policy:simulate (service identities only)
Headers: X-Stella-Tenant, Idempotency-Key (optional)
```
**Request**
```jsonc
{
"tenantId": "acme",
"policyVersion": "sha256:1fb2…",
"items": [
{
"findingId": "acme::artifact-1::CVE-2024-12345",
"eventId": "5d1fcc61-6903-42ef-9285-7f4d3d8f7f69",
"event": { ... canonical ledger payload ... },
"currentProjection": {
"status": "triaged",
"severity": 3.4,
"labels": { "exposure": "runtime" },
"explainRef": "policy://explain/123",
"rationale": ["policy://explain/123"]
}
}
]
}
```
| Field | Description |
|-------|-------------|
| `tenantId` | Must match the `X-Stella-Tenant` header. |
| `policyVersion` | Deterministic policy digest (for example `sha256:<hex>`). Required for caching. |
| `event` | Canonical ledger event payload (`ledger_events.event_body`). |
| `currentProjection` | Optional snapshot of the existing finding projection. Null values are ignored. |
**Response 200**
```jsonc
{
"items": [
{
"findingId": "acme::artifact-1::CVE-2024-12345",
"status": "affected",
"severity": 7.5,
"labels": { "exposure": "runtime" },
"explainRef": "policy://explain/123",
"rationale": [
"policy://explain/123",
"policy://remediation/321"
]
}
],
"cost": {
"units": 1,
"budgetRemaining": 999
}
}
```
Notes:
- Items that cannot be evaluated return `status: null` with an `error` object. Callers should fall back to inline evaluation.
- Policy Engine enforces per-tenant cost budgets; batches that exceed the remaining allowance receive `429 Too Many Requests`.
- Responses are deterministic; clients may cache results by `(tenantId, policyVersion, eventHash, projectionHash)` to support replay/offline parity.
- Standard `ERR_POL_*` payloads surface errors; `ERR_POL_006` indicates the evaluator aborted the batch.
---
## 8·Effective Findings APIs
### 7.1 List Findings
@@ -378,7 +451,7 @@ Returns rule hit sequence:
---
## 8·Events & Webhooks
## 9·Events & Webhooks
- `policy.run.completed` emitted with `runId`, `policyId`, `mode`, `stats`, `determinismHash`.
- `policy.run.failed` includes error code, retry count, guidance.
@@ -387,7 +460,7 @@ Returns rule hit sequence:
---
## 9·Compliance Checklist
## 10·Compliance Checklist
- [ ] **Scopes enforced:** Endpoint access requires correct Authority scope mapping (see `/src/Authority/StellaOps.Authority/TASKS.md`).
- [ ] **Schemas current:** JSON examples align with Scheduler Models (`SCHED-MODELS-20-001`) and Policy Engine DTOs; update when contracts change.

View File

@@ -12,6 +12,24 @@ This guide captures the minimum wiring required for connectors and Merge coordin
4. Verify with `dotnet test` that the connector snapshot fixtures now include the `normalizedVersions` array and update fixtures by setting the connector-specific `UPDATE_*_FIXTURES=1` environment variable.
5. Tail Merge logs (or the test output) for the new warning `Normalized version rules missing for {AdvisoryKey}`; an empty warning stream means the connector/merge artefacts are ready to close FEEDMERGE-COORD-02-901/902.
### 1.1 Rollout status (2025-11-04)
| Connector | Status | Notes / next steps |
|-----------|--------|--------------------|
| `vendor.acsc` | ⚠️ Pending | Upstream feed only supplies vendor/product strings. Waiting on ASD/ACSC feed update to expose explicit ranges. Track via FEEDCONN-ACSC-02-010. |
| `vendor.cccs` | ⚠️ Pending | Advisory payloads lack structured version ranges; all entries remain vendor identifiers. Coordinate with CCCS to obtain machine-readable version metadata or fall back to Model heuristics. |
| `certbund` | ⚠️ Pending | `product.Versions` contains natural-language German ranges. Parser spec drafted (`CERTBUND-NORM-01`); awaiting implementation before emitting normalized rules. |
| `vendor.cisco` | ⚠️ Pending | Current API exposes product IDs only. Engage Cisco PSIRT (FEEDCONN-CISCO-02-014) to surface affected version expressions. |
| `vendor.apple` | ✅ Done | SemVer-style helpers in `AppleMapper` emit normalized rules with provenance annotations. |
| `vendor.msrc` | ✅ Done | `MsrcMapper` maps KB build numbers to exact rules; normalized output guarded by fixtures. |
| `vendor.ghsa` | ✅ Done | SemVer + vendor fallback rules emitted (`CreateSemVerVersionArtifacts`). |
| `vendor.kisa` | ✅ Partial | Normalized SemVer rules for structured firmware ranges. Fallback vendor strings remain for prose-only advisories; continue capturing new patterns in fixtures. |
| `ics.cisa` | ✅ Done | Firmware helper emits normalized range matrix using `SemVerRangeRuleBuilder`. |
| `certcc` | ✅ Done | Vendor comparator transforms range fragments to normalized rules. |
| `cve.nvd` | ✅ Done | Full SemVer builder and provenance mapping rolled out (FEEDCONN-CVE-02-015). |
| `ru.bdu` | ⚠️ Pending | Feed only includes product codenames. Normalized rules blocked until Roskomnadzor publishes range schema. |
| `ru.nkcki` | ✅ Done | SemVer-style range parser covers vendor firmware records; remaining prose ranges logged with `Normalized version rules missing`. |
## 2. Code snippet: SemVer connector (CCCS/Cisco/ICS-CISA)
```csharp

View File

@@ -1,101 +1,141 @@
{
"eventId": "6d2d1b77-f3c3-4f70-8a9d-6f2d0c8801ab",
"kind": "scanner.event.report.ready",
"version": 1,
"tenant": "tenant-alpha",
"occurredAt": "2025-10-19T12:34:56Z",
"recordedAt": "2025-10-19T12:34:57Z",
"source": "scanner.webservice",
"idempotencyKey": "scanner.event.report.ready:tenant-alpha:report-abc",
"correlationId": "report-abc",
"traceId": "0af7651916cd43dd8448eb211c80319c",
"spanId": "b7ad6b7169203331",
"scope": {
"namespace": "acme/edge",
"repo": "api",
"digest": "sha256:feedface"
},
"attributes": {
"reportId": "report-abc",
"policyRevisionId": "rev-42",
"policyDigest": "digest-123",
"verdict": "blocked"
},
"payload": {
"reportId": "report-abc",
"scanId": "report-abc",
"imageDigest": "sha256:feedface",
"generatedAt": "2025-10-19T12:34:56Z",
"verdict": "fail",
"summary": {
"total": 1,
"blocked": 1,
"warned": 0,
"ignored": 0,
"quieted": 0
},
"delta": {
"newCritical": 1,
"kev": [
"CVE-2024-9999"
]
},
"quietedFindingCount": 0,
"policy": {
"digest": "digest-123",
"revisionId": "rev-42"
},
"links": {
"report": {
"ui": "https://scanner.example/ui/reports/report-abc",
"api": "https://scanner.example/api/v1/reports/report-abc"
{
"eventId": "6d2d1b77-f3c3-4f70-8a9d-6f2d0c8801ab",
"kind": "scanner.event.report.ready",
"version": 1,
"tenant": "tenant-alpha",
"occurredAt": "2025-10-19T12:34:56+00:00",
"recordedAt": "2025-10-19T12:34:57+00:00",
"source": "scanner.webservice",
"idempotencyKey": "scanner.event.report.ready:tenant-alpha:report-abc",
"correlationId": "report-abc",
"traceId": "0af7651916cd43dd8448eb211c80319c",
"spanId": "b7ad6b7169203331",
"scope": {
"namespace": "acme/edge",
"repo": "api",
"digest": "sha256:feedface"
},
"payload": {
"reportId": "report-abc",
"scanId": "report-abc",
"imageDigest": "sha256:feedface",
"generatedAt": "2025-10-19T12:34:56+00:00",
"verdict": "fail",
"summary": {
"total": 1,
"blocked": 1,
"warned": 0,
"ignored": 0,
"quieted": 0
},
"delta": {
"newCritical": 1,
"kev": [
"CVE-2024-9999"
]
},
"quietedFindingCount": 0,
"policy": {
"ui": "https://scanner.example/ui/policy/revisions/rev-42",
"api": "https://scanner.example/api/v1/policy/revisions/rev-42"
"revisionId": "rev-42",
"digest": "digest-123"
},
"attestation": {
"ui": "https://scanner.example/ui/attestations/report-abc",
"api": "https://scanner.example/api/v1/reports/report-abc/attestation"
"links": {
"report": {
"ui": "https://scanner.example/ui/reports/report-abc",
"api": "https://scanner.example/api/v1/reports/report-abc"
},
"policy": {
"ui": "https://scanner.example/ui/policy/revisions/rev-42",
"api": "https://scanner.example/api/v1/policy/revisions/rev-42"
},
"attestation": {
"ui": "https://scanner.example/ui/attestations/report-abc",
"api": "https://scanner.example/api/v1/reports/report-abc/attestation"
}
},
"dsse": {
"payloadType": "application/vnd.stellaops.report+json",
"payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJyZWFjaGFiaWxpdHkiOiJydW50aW1lIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwic3RhdHVzIjoiQmxvY2tlZCJ9XSwiaXNzdWVzIjpbXSwic3VyZmFjZSI6eyJ0ZW5hbnQiOiJ0ZW5hbnQtYWxwaGEiLCJnZW5lcmF0ZWRBdCI6IjIwMjUtMTAtMTlUMTI6MzQ6NTYrMDA6MDAiLCJtYW5pZmVzdERpZ2VzdCI6InNoYTI1Njo0ZmVlODdkMTg2MjkxZGRmYmJjYzJjNTZjOGVkMGU4Mjg1MjBiOGY1MmUxY2RlMGUxM2JiYTA4MmYxMDkxOGQ3IiwibWFuaWZlc3RVcmkiOiJjYXM6Ly9zY2FubmVyLWFydGlmYWN0cy9zY2FubmVyL3N1cmZhY2UvbWFuaWZlc3RzL3RlbmFudC1hbHBoYS9zaGEyNTYvNGYvZWUvNGZlZTg3ZDE4NjI5MWRkZmJiY2MyYzU2YzhlZDBlODI4NTIwYjhmNTJlMWNkZTBlMTNiYmEwODJmMTA5MThkNy5qc29uIiwibWFuaWZlc3QiOnsic2NoZW1hIjoic3RlbGxhb3BzLnN1cmZhY2UubWFuaWZlc3RAMSIsInRlbmFudCI6InRlbmFudC1hbHBoYSIsImltYWdlRGlnZXN0Ijoic2hhMjU2OmZlZWRmYWNlIiwiZ2VuZXJhdGVkQXQiOiIyMDI1LTEwLTE5VDEyOjM0OjU2KzAwOjAwIiwiYXJ0aWZhY3RzIjpbeyJraW5kIjoiZW50cnktdHJhY2UiLCJ1cmkiOiJjYXM6Ly9zY2FubmVyLWFydGlmYWN0cy9zY2FubmVyL2VudHJ5LXRyYWNlL2YwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwL2VudHJ5LXRyYWNlLmpzb24iLCJkaWdlc3QiOiJzaGEyNTY6ZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMCIsIm1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL2pzb24iLCJmb3JtYXQiOiJqc29uIiwic2l6ZUJ5dGVzIjo0MDk2fSx7ImtpbmQiOiJzYm9tLWludmVudG9yeSIsInVyaSI6ImNhczovL3NjYW5uZXItYXJ0aWZhY3RzL3NjYW5uZXIvaW1hZ2VzL2ZlZWRmYWNlL3Nib20uY2R4Lmpzb24iLCJkaWdlc3QiOiJzaGEyNTY6MTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMSIsIm1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5jeWNsb25lZHgranNvbjt2ZXJzaW9uPTEuNjt2aWV3PWludmVudG9yeSIsImZvcm1hdCI6ImNkeC1qc29uIiwic2l6ZUJ5dGVzIjoyNDU3NiwidmlldyI6ImludmVudG9yeSJ9LHsia2luZCI6InNib20tdXNhZ2UiLCJ1cmkiOiJjYXM6Ly9zY2FubmVyLWFydGlmYWN0cy9zY2FubmVyL2ltYWdlcy9mZWVkZmFjZS9zYm9tLXVzYWdlLmNkeC5qc29uIiwiZGlnZXN0Ijoic2hhMjU2OjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIiLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuY3ljbG9uZWR4K2pzb247dmVyc2lvbj0xLjY7dmlldz11c2FnZSIsImZvcm1hdCI6ImNkeC1qc29uIiwic2l6ZUJ5dGVzIjoxNjM4NCwidmlldyI6InVzYWdlIn1dfX19",
"signatures": [
{
"keyId": "test-key",
"algorithm": "hs256",
"signature": "signature-value"
}
]
},
"report": {
"reportId": "report-abc",
"imageDigest": "sha256:feedface",
"generatedAt": "2025-10-19T12:34:56+00:00",
"verdict": "blocked",
"policy": {
"revisionId": "rev-42",
"digest": "digest-123"
},
"summary": {
"total": 1,
"blocked": 1,
"warned": 0,
"ignored": 0,
"quieted": 0
},
"verdicts": [
{
"findingId": "finding-1",
"reachability": "runtime",
"score": 47.5,
"sourceTrust": "NVD",
"status": "Blocked"
}
],
"issues": [],
"surface": {
"tenant": "tenant-alpha",
"generatedAt": "2025-10-19T12:34:56+00:00",
"manifestDigest": "sha256:4fee87d186291ddfbbcc2c56c8ed0e828520b8f52e1cde0e13bba082f10918d7",
"manifestUri": "cas://scanner-artifacts/scanner/surface/manifests/tenant-alpha/sha256/4f/ee/4fee87d186291ddfbbcc2c56c8ed0e828520b8f52e1cde0e13bba082f10918d7.json",
"manifest": {
"schema": "stellaops.surface.manifest@1",
"tenant": "tenant-alpha",
"imageDigest": "sha256:feedface",
"generatedAt": "2025-10-19T12:34:56+00:00",
"artifacts": [
{
"kind": "entry-trace",
"uri": "cas://scanner-artifacts/scanner/entry-trace/f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0/entry-trace.json",
"digest": "sha256:f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0",
"mediaType": "application/json",
"format": "json",
"sizeBytes": 4096
},
{
"kind": "sbom-inventory",
"uri": "cas://scanner-artifacts/scanner/images/feedface/sbom.cdx.json",
"digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111",
"mediaType": "application/vnd.cyclonedx+json;version=1.6;view=inventory",
"format": "cdx-json",
"sizeBytes": 24576,
"view": "inventory"
},
{
"kind": "sbom-usage",
"uri": "cas://scanner-artifacts/scanner/images/feedface/sbom-usage.cdx.json",
"digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222",
"mediaType": "application/vnd.cyclonedx+json;version=1.6;view=usage",
"format": "cdx-json",
"sizeBytes": 16384,
"view": "usage"
}
]
}
}
}
},
"dsse": {
"payloadType": "application/vnd.stellaops.report+json",
"payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJzdGF0dXMiOiJCbG9ja2VkIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwicmVhY2hhYmlsaXR5IjoicnVudGltZSJ9XSwiaXNzdWVzIjpbXX0=",
"signatures": [
{
"keyId": "test-key",
"algorithm": "hs256",
"signature": "signature-value"
}
]
},
"report": {
"reportId": "report-abc",
"generatedAt": "2025-10-19T12:34:56Z",
"imageDigest": "sha256:feedface",
"policy": {
"digest": "digest-123",
"revisionId": "rev-42"
},
"summary": {
"total": 1,
"blocked": 1,
"warned": 0,
"ignored": 0,
"quieted": 0
},
"verdict": "blocked",
"verdicts": [
{
"findingId": "finding-1",
"status": "Blocked",
"score": 47.5,
"sourceTrust": "NVD",
"reachability": "runtime"
}
],
"issues": []
}
}
}
"attributes": {
"policyDigest": "digest-123",
"policyRevisionId": "rev-42",
"reportId": "report-abc",
"verdict": "blocked"
}
}

View File

@@ -1,56 +1,50 @@
{
"eventId": "08a6de24-4a94-4d14-8432-9d14f36f6da3",
"kind": "scanner.event.scan.completed",
"version": 1,
"tenant": "tenant-alpha",
"occurredAt": "2025-10-19T12:34:56Z",
"recordedAt": "2025-10-19T12:34:57Z",
"source": "scanner.webservice",
"idempotencyKey": "scanner.event.scan.completed:tenant-alpha:report-abc",
"correlationId": "report-abc",
"traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
"scope": {
"namespace": "acme/edge",
"repo": "api",
"digest": "sha256:feedface"
},
"attributes": {
"reportId": "report-abc",
"policyRevisionId": "rev-42",
"policyDigest": "digest-123",
"verdict": "blocked"
},
"payload": {
"reportId": "report-abc",
"scanId": "report-abc",
"imageDigest": "sha256:feedface",
"verdict": "fail",
"summary": {
"total": 1,
"blocked": 1,
"warned": 0,
"ignored": 0,
"quieted": 0
},
"delta": {
"newCritical": 1,
"kev": [
"CVE-2024-9999"
]
},
"policy": {
"digest": "digest-123",
"revisionId": "rev-42"
},
"findings": [
{
"id": "finding-1",
"severity": "Critical",
"cve": "CVE-2024-9999",
"purl": "pkg:docker/acme/edge-api@sha256-feedface",
"reachability": "runtime"
}
],
{
"eventId": "08a6de24-4a94-4d14-8432-9d14f36f6da3",
"kind": "scanner.event.scan.completed",
"version": 1,
"tenant": "tenant-alpha",
"occurredAt": "2025-10-19T12:34:56+00:00",
"recordedAt": "2025-10-19T12:34:57+00:00",
"source": "scanner.webservice",
"idempotencyKey": "scanner.event.scan.completed:tenant-alpha:report-abc",
"correlationId": "report-abc",
"traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
"scope": {
"namespace": "acme/edge",
"repo": "api",
"digest": "sha256:feedface"
},
"payload": {
"reportId": "report-abc",
"scanId": "report-abc",
"imageDigest": "sha256:feedface",
"verdict": "fail",
"summary": {
"total": 1,
"blocked": 1,
"warned": 0,
"ignored": 0,
"quieted": 0
},
"delta": {
"newCritical": 1,
"kev": [
"CVE-2024-9999"
]
},
"policy": {
"revisionId": "rev-42",
"digest": "digest-123"
},
"findings": [
{
"id": "finding-1",
"severity": "Critical",
"cve": "CVE-2024-9999",
"purl": "pkg:docker/acme/edge-api@sha256-feedface",
"reachability": "runtime"
}
],
"links": {
"report": {
"ui": "https://scanner.example/ui/reports/report-abc",
@@ -65,43 +59,89 @@
"api": "https://scanner.example/api/v1/reports/report-abc/attestation"
}
},
"dsse": {
"payloadType": "application/vnd.stellaops.report+json",
"payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJzdGF0dXMiOiJCbG9ja2VkIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwicmVhY2hhYmlsaXR5IjoicnVudGltZSJ9XSwiaXNzdWVzIjpbXX0=",
"signatures": [
{
"keyId": "test-key",
"algorithm": "hs256",
"signature": "signature-value"
}
]
},
"report": {
"reportId": "report-abc",
"generatedAt": "2025-10-19T12:34:56Z",
"imageDigest": "sha256:feedface",
"policy": {
"digest": "digest-123",
"revisionId": "rev-42"
},
"summary": {
"total": 1,
"blocked": 1,
"warned": 0,
"ignored": 0,
"quieted": 0
},
"verdict": "blocked",
"verdicts": [
{
"findingId": "finding-1",
"status": "Blocked",
"score": 47.5,
"sourceTrust": "NVD",
"reachability": "runtime"
}
],
"issues": []
}
}
}
"dsse": {
"payloadType": "application/vnd.stellaops.report+json",
"payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJyZWFjaGFiaWxpdHkiOiJydW50aW1lIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwic3RhdHVzIjoiQmxvY2tlZCJ9XSwiaXNzdWVzIjpbXSwic3VyZmFjZSI6eyJ0ZW5hbnQiOiJ0ZW5hbnQtYWxwaGEiLCJnZW5lcmF0ZWRBdCI6IjIwMjUtMTAtMTlUMTI6MzQ6NTYrMDA6MDAiLCJtYW5pZmVzdERpZ2VzdCI6InNoYTI1Njo0ZmVlODdkMTg2MjkxZGRmYmJjYzJjNTZjOGVkMGU4Mjg1MjBiOGY1MmUxY2RlMGUxM2JiYTA4MmYxMDkxOGQ3IiwibWFuaWZlc3RVcmkiOiJjYXM6Ly9zY2FubmVyLWFydGlmYWN0cy9zY2FubmVyL3N1cmZhY2UvbWFuaWZlc3RzL3RlbmFudC1hbHBoYS9zaGEyNTYvNGYvZWUvNGZlZTg3ZDE4NjI5MWRkZmJiY2MyYzU2YzhlZDBlODI4NTIwYjhmNTJlMWNkZTBlMTNiYmEwODJmMTA5MThkNy5qc29uIiwibWFuaWZlc3QiOnsic2NoZW1hIjoic3RlbGxhb3BzLnN1cmZhY2UubWFuaWZlc3RAMSIsInRlbmFudCI6InRlbmFudC1hbHBoYSIsImltYWdlRGlnZXN0Ijoic2hhMjU2OmZlZWRmYWNlIiwiZ2VuZXJhdGVkQXQiOiIyMDI1LTEwLTE5VDEyOjM0OjU2KzAwOjAwIiwiYXJ0aWZhY3RzIjpbeyJraW5kIjoiZW50cnktdHJhY2UiLCJ1cmkiOiJjYXM6Ly9zY2FubmVyLWFydGlmYWN0cy9zY2FubmVyL2VudHJ5LXRyYWNlL2YwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwL2VudHJ5LXRyYWNlLmpzb24iLCJkaWdlc3QiOiJzaGEyNTY6ZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMGYwZjBmMCIsIm1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL2pzb24iLCJmb3JtYXQiOiJqc29uIiwic2l6ZUJ5dGVzIjo0MDk2fSx7ImtpbmQiOiJzYm9tLWludmVudG9yeSIsInVyaSI6ImNhczovL3NjYW5uZXItYXJ0aWZhY3RzL3NjYW5uZXIvaW1hZ2VzL2ZlZWRmYWNlL3Nib20uY2R4Lmpzb24iLCJkaWdlc3QiOiJzaGEyNTY6MTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMTExMSIsIm1lZGlhVHlwZSI6ImFwcGxpY2F0aW9uL3ZuZC5jeWNsb25lZHgranNvbjt2ZXJzaW9uPTEuNjt2aWV3PWludmVudG9yeSIsImZvcm1hdCI6ImNkeC1qc29uIiwic2l6ZUJ5dGVzIjoyNDU3NiwidmlldyI6ImludmVudG9yeSJ9LHsia2luZCI6InNib20tdXNhZ2UiLCJ1cmkiOiJjYXM6Ly9zY2FubmVyLWFydGlmYWN0cy9zY2FubmVyL2ltYWdlcy9mZWVkZmFjZS9zYm9tLXVzYWdlLmNkeC5qc29uIiwiZGlnZXN0Ijoic2hhMjU2OjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIiLCJtZWRpYVR5cGUiOiJhcHBsaWNhdGlvbi92bmQuY3ljbG9uZWR4K2pzb247dmVyc2lvbj0xLjY7dmlldz11c2FnZSIsImZvcm1hdCI6ImNkeC1qc29uIiwic2l6ZUJ5dGVzIjoxNjM4NCwidmlldyI6InVzYWdlIn1dfX19",
"signatures": [
{
"keyId": "test-key",
"algorithm": "hs256",
"signature": "signature-value"
}
]
},
"report": {
"reportId": "report-abc",
"imageDigest": "sha256:feedface",
"generatedAt": "2025-10-19T12:34:56+00:00",
"verdict": "blocked",
"policy": {
"revisionId": "rev-42",
"digest": "digest-123"
},
"summary": {
"total": 1,
"blocked": 1,
"warned": 0,
"ignored": 0,
"quieted": 0
},
"verdicts": [
{
"findingId": "finding-1",
"reachability": "runtime",
"score": 47.5,
"sourceTrust": "NVD",
"status": "Blocked"
}
],
"issues": [],
"surface": {
"tenant": "tenant-alpha",
"generatedAt": "2025-10-19T12:34:56+00:00",
"manifestDigest": "sha256:4fee87d186291ddfbbcc2c56c8ed0e828520b8f52e1cde0e13bba082f10918d7",
"manifestUri": "cas://scanner-artifacts/scanner/surface/manifests/tenant-alpha/sha256/4f/ee/4fee87d186291ddfbbcc2c56c8ed0e828520b8f52e1cde0e13bba082f10918d7.json",
"manifest": {
"schema": "stellaops.surface.manifest@1",
"tenant": "tenant-alpha",
"imageDigest": "sha256:feedface",
"generatedAt": "2025-10-19T12:34:56+00:00",
"artifacts": [
{
"kind": "entry-trace",
"uri": "cas://scanner-artifacts/scanner/entry-trace/f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0/entry-trace.json",
"digest": "sha256:f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0f0",
"mediaType": "application/json",
"format": "json",
"sizeBytes": 4096
},
{
"kind": "sbom-inventory",
"uri": "cas://scanner-artifacts/scanner/images/feedface/sbom.cdx.json",
"digest": "sha256:1111111111111111111111111111111111111111111111111111111111111111",
"mediaType": "application/vnd.cyclonedx+json;version=1.6;view=inventory",
"format": "cdx-json",
"sizeBytes": 24576,
"view": "inventory"
},
{
"kind": "sbom-usage",
"uri": "cas://scanner-artifacts/scanner/images/feedface/sbom-usage.cdx.json",
"digest": "sha256:2222222222222222222222222222222222222222222222222222222222222222",
"mediaType": "application/vnd.cyclonedx+json;version=1.6;view=usage",
"format": "cdx-json",
"sizeBytes": 16384,
"view": "usage"
}
]
}
}
}
},
"attributes": {
"policyDigest": "digest-123",
"policyRevisionId": "rev-42",
"reportId": "report-abc",
"verdict": "blocked"
}
}

View File

@@ -129,7 +129,8 @@ Follow the sprint files below in order. Update task status in both `SPRINTS` and
> 2025-11-02: DOCS-SCANNER-BENCH-62-012 marked DONE (Docs Guild, Language Analyzer Guild) Dart coverage section fleshed out with detection strategies.
> 2025-11-02: DOCS-SCANNER-BENCH-62-013 marked DONE (Docs Guild, Swift Analyzer Guild) Swift analyzer roadmap captured with policy hooks.
> 2025-11-02: DOCS-SCANNER-BENCH-62-014 marked DONE (Docs Guild, Runtime Guild) Kubernetes/VM alignment section published.
> 2025-11-02: DOCS-SCANNER-BENCH-62-015 marked DONE (Docs Guild, Export Center Guild) DSSE/Rekor enablement guidance appended to gap doc.
> 2025-11-02: DOCS-SCANNER-BENCH-62-015 marked DONE (Docs Guild, Export Center Guild) DSSE/Rekor enablement guidance appended to gap doc.
> 2025-11-05: SCANNER-SURFACE-02 marked DONE (Scanner WebService Guild) WebService now persists `surface` manifest pointers in scan/report APIs, orchestrator samples and DSSE fixtures refreshed, and readiness tests updated with Surface validators stubbed for deterministic health checks.
> 2025-11-02: SCANNER-ENG-0009 moved to DOING (Ruby Analyzer Guild) drafting Ruby analyzer parity design package.
> 2025-11-02: SCANNER-ENG-0016 added (Ruby Analyzer Guild) implementing Ruby lock collector & vendor cache ingestion.
> 2025-11-02: SCANNER-ENG-0016 moved to DOING (Ruby Analyzer Guild) lockfile parser skeleton committed with initial Gemfile.lock parsing.

View File

@@ -105,10 +105,10 @@ SEC3.PLG | BLOCKED (2025-10-21) | Ensure lockout responses and rate-limit metada
SEC5.PLG | BLOCKED (2025-10-21) | Address plugin-specific mitigations (bootstrap user handling, password policy docs) in threat model backlog. <br>⛔ Final documentation depends on AUTH-DPOP-11-001 / AUTH-MTLS-11-002 / PLUGIN-DI-08-001 outcomes. | Security Guild (src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md)
PLG7.IMPL-001 | DONE (2025-11-03) | Scaffold `StellaOps.Authority.Plugin.Ldap` + tests, bind configuration (client certificate, trust-store, insecure toggle) with validation and docs samples. | BE-Auth Plugin (src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md)
> 2025-11-03: Initial `StellaOps.Authority.Plugin.Ldap` project/tests scaffolded with configuration options + registrar; sample manifest (`etc/authority.plugins/ldap.yaml`) updated to new schema (client certificate, trust store, insecure toggle).
PLG7.IMPL-002 | DOING (2025-11-03) | Implement LDAP credential store with TLS/mutual TLS enforcement, deterministic retry/backoff, and structured logging/metrics. | BE-Auth Plugin, Security Guild (src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md)
PLG7.IMPL-002 | DONE (2025-11-04) | Implement LDAP credential store with TLS/mutual TLS enforcement, deterministic retry/backoff, and structured logging/metrics. | BE-Auth Plugin, Security Guild (src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Standard/TASKS.md)
> 2025-11-03: Review concluded; RFC accepted with audit/mTLS/mapping decisions recorded in `docs/notes/2025-11-03-authority-plugin-ldap-review.md`. Follow-up implementation tasks PLG7.IMPL-001..005 added to plugin board.
> 2025-11-04: Updated connection factory to negotiate StartTLS via `StartTransportLayerSecurity(null)` and normalized LDAP result-code handling (invalid credentials + transient codes) against `System.DirectoryServices.Protocols` 8.0. Plugin unit suite (`dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Ldap.Tests/StellaOps.Authority.Plugin.Ldap.Tests.csproj`) now passes again after the retry/error-path fixes.
> 2025-11-04: PLG7.IMPL-002 progress enforced TLS/client certificate validation, expanded LDAP audit properties and retry telemetry, warned when cipher lists are unsupported, refreshed sample config, and reran `dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Ldap.Tests/StellaOps.Authority.Plugin.Ldap.Tests.csproj --no-restore`.
> 2025-11-04: PLG7.IMPL-002 DONE deterministic credential store retries now emit metrics + structured audit context, DirectoryServices factory enforces TLS/mTLS settings (trust store + client cert), and configuration samples/docs refreshed. Tests: `dotnet test src/Authority/StellaOps.Authority/StellaOps.Authority.Plugin.Ldap.Tests/StellaOps.Authority.Plugin.Ldap.Tests.csproj --no-restore`.
[Identity & Signing] 100.C) IssuerDirectory
Summary: Identity & Signing focus on IssuerDirectory.

View File

@@ -2,7 +2,7 @@
## Status Snapshot (2025-11-04)
- **Advisory AI** 5 of 11 tasks are DONE (AIAI-31-001, AIAI-31-002, AIAI-31-003, AIAI-31-010, AIAI-31-011); orchestration pipeline (AIAI-31-004) and host wiring (AIAI-31-004A) remain DOING while downstream guardrails, CLI, and observability tracks (AIAI-31-004B/004C and AIAI-31-005 through AIAI-31-009) stay TODO pending cache/guardrail implementation and WebService/Worker hardening.
- **Advisory AI** 5 of 11 tasks are DONE (AIAI-31-001, AIAI-31-002, AIAI-31-003, AIAI-31-010, AIAI-31-011); orchestration pipeline (AIAI-31-004) and host wiring (AIAI-31-004A) remain TODO while downstream guardrails, CLI, and observability tracks (AIAI-31-004B/004C and AIAI-31-005 through AIAI-31-009) stay TODO pending cache/guardrail implementation and WebService/Worker hardening.
- 2025-11-04: AIAI-31-002 and AIAI-31-003 shipped with deterministic SBOM context client wiring (`AddSbomContext` typed HTTP client) and toolset integration; WebService/Worker now invoke the orchestrator with SBOM-backed simulations and emit initial metrics.
- 2025-11-03: AIAI-31-002 landed the configurable HTTP client + DI defaults; retriever now resolves data via `/v1/sbom/context`, retaining a null fallback until SBOM service ships.
- 2025-11-03: Follow-up: SBOM guild to deliver base URL/API key and run an Advisory AI smoke retrieval once SBOM-AIAI-31-001 endpoints are live.
@@ -25,8 +25,8 @@ Task ID | State | Task description | Owners (Source)
AIAI-31-001 | DONE (2025-11-02) | Implement structured and vector retrievers for advisories/VEX with paragraph anchors and citation metadata. Dependencies: CONCELIER-VULN-29-001, EXCITITOR-VULN-29-001. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
AIAI-31-002 | DONE (2025-11-04) | Build SBOM context retriever (purl version timelines, dependency paths, env flags, blast radius estimator). Dependencies: SBOM-VULN-29-001. | Advisory AI Guild, SBOM Service Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
AIAI-31-003 | DONE (2025-11-04) | Implement deterministic toolset (version comparators, range checks, dependency analysis, policy lookup) exposed via orchestrator. Dependencies: AIAI-31-001..002. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
AIAI-31-004 | DOING | Build orchestration pipeline for Summary/Conflict/Remediation tasks (prompt templates, tool calls, token budgets, caching). Dependencies: AIAI-31-001..003, AUTH-VULN-29-001. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
AIAI-31-004A | DOING (2025-11-04) | Wire orchestrator into WebService/Worker, expose API + queue contract, emit metrics, stub cache. Dependencies: AIAI-31-004, AIAI-31-002. | Advisory AI Guild, Platform Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
AIAI-31-004 | DONE (2025-11-04) | Build orchestration pipeline for Summary/Conflict/Remediation tasks (prompt templates, tool calls, token budgets, caching). Dependencies: AIAI-31-001..003, AUTH-VULN-29-001. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
AIAI-31-004A | DONE (2025-11-04) | Wire orchestrator into WebService/Worker, expose API + queue contract, emit metrics, stub cache. Dependencies: AIAI-31-004, AIAI-31-002. | Advisory AI Guild, Platform Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
> 2025-11-03: WebService/Worker scaffolds created with in-memory cache/queue, minimal APIs (`/api/v1/advisory/plan`, `/api/v1/advisory/queue`), metrics counters, and plan cache instrumentation; worker processes queue using orchestrator.
> 2025-11-04: SBOM base address now flows via `SbomContextClientOptions.BaseAddress`, worker emits queue/plan metrics, and orchestrator cache keys expanded to cover SBOM hash inputs.
AIAI-31-004B | TODO | Implement prompt assembler, guardrails, cache persistence, DSSE provenance, golden outputs. Dependencies: AIAI-31-004A, DOCS-AIAI-31-003, AUTH-AIAI-31-004. | Advisory AI Guild, Security Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
@@ -41,6 +41,7 @@ DOCS-AIAI-31-007 | BLOCKED (2025-11-03) | Write `/docs/security/assistant-guardr
DOCS-AIAI-31-008 | BLOCKED (2025-11-03) | Publish `/docs/sbom/remediation-heuristics.md` (feasibility scoring, blast radius). Dependencies: SBOM-AIAI-31-001. | Docs Guild, SBOM Service Guild (docs/TASKS.md)
DOCS-AIAI-31-009 | BLOCKED (2025-11-03) | Create `/docs/runbooks/assistant-ops.md` for warmup, cache priming, model outages, scaling. Dependencies: DEVOPS-AIAI-31-001. | Docs Guild, DevOps Guild (docs/TASKS.md)
> 2025-11-03: DOCS-AIAI-31-003 moved to DOING drafting Advisory AI API reference (endpoints, rate limits, error model) for sprint 110.
> 2025-11-04: AIAI-31-005 DONE guardrail pipeline redacts secrets, enforces citation/injection policies, emits block counters, and tests (`AdvisoryGuardrailPipelineTests`) cover redaction + citation validation.
> 2025-11-03: DOCS-AIAI-31-003 marked DONE `docs/advisory-ai/api.md` published with scopes, request/response schemas, rate limits, and error catalogue (Docs Guild).
> 2025-11-03: DOCS-AIAI-31-001 marked DONE `docs/advisory-ai/overview.md` published with value, personas, guardrails, observability, and roadmap checklists (Docs Guild).
> 2025-11-03: DOCS-AIAI-31-002 marked DONE `docs/advisory-ai/architecture.md` published describing pipeline, deterministic tooling, caching, and profile governance (Docs Guild).
@@ -50,8 +51,8 @@ DOCS-AIAI-31-009 | BLOCKED (2025-11-03) | Create `/docs/runbooks/assistant-ops.m
> 2025-11-03: DOCS-AIAI-31-007 marked BLOCKED Guardrail implementation (AIAI-31-005) incomplete.
> 2025-11-03: DOCS-AIAI-31-008 marked BLOCKED Waiting on SBOM heuristics delivery (SBOM-AIAI-31-001).
> 2025-11-03: DOCS-AIAI-31-009 marked BLOCKED DevOps runbook inputs (DEVOPS-AIAI-31-001) outstanding.
AIAI-31-005 | DOING (2025-11-03) | Implement guardrails (redaction, injection defense, output validation, citation enforcement) and fail-safe handling. Dependencies: AIAI-31-004. | Advisory AI Guild, Security Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
AIAI-31-006 | DOING (2025-11-03) | Expose REST API endpoints (`/advisory/ai/*`) with RBAC, rate limits, OpenAPI schemas, and batching support. Dependencies: AIAI-31-004..005. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
AIAI-31-005 | DONE (2025-11-04) | Implement guardrails (redaction, injection defense, output validation, citation enforcement) and fail-safe handling. Dependencies: AIAI-31-004. | Advisory AI Guild, Security Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
AIAI-31-006 | DONE (2025-11-04) | Expose REST API endpoints (`/advisory/ai/*`) with RBAC, rate limits, OpenAPI schemas, and batching support. Dependencies: AIAI-31-004..005. | Advisory AI Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
> 2025-11-03: Shipped `/api/v1/advisory/{task}` execution and `/api/v1/advisory/outputs/{cacheKey}` retrieval endpoints with guardrail integration, provenance hashes, and metrics (RBAC & rate limiting still pending Authority scope delivery).
AIAI-31-007 | TODO | Instrument metrics (`advisory_ai_latency`, `guardrail_blocks`, `validation_failures`, `citation_coverage`), logs, and traces; publish dashboards/alerts. Dependencies: AIAI-31-004..006. | Advisory AI Guild, Observability Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
AIAI-31-008 | TODO | Package inference on-prem container, remote inference toggle, Helm/Compose manifests, scaling guidance, offline kit instructions. Dependencies: AIAI-31-006..007. | Advisory AI Guild, DevOps Guild (src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md)
@@ -64,6 +65,7 @@ AIAI-31-009 | TODO | Develop unit/golden/property/perf tests, injection harness,
> 2025-11-02: AIAI-31-004 kicked off orchestration pipeline design establishing deterministic task sequence (summary/conflict/remediation) and cache key strategy.
> 2025-11-02: AIAI-31-004 orchestration prerequisites documented in docs/modules/advisory-ai/orchestration-pipeline.md (tasks 004A/004B/004C).
> 2025-11-02: AIAI-31-003 moved to DOING beginning deterministic tooling (comparators, dependency analysis) while awaiting SBOM context client. Semantic & EVR comparators shipped; toolset interface published for orchestrator adoption.
> 2025-11-04: AIAI-31-004 DONE orchestrator composes evidence (structured/vector/SBOM) with stable cache keys, metadata, and hashing; tests keep determinism enforced.
> 2025-11-02: Structured + vector retrievers landed with deterministic CSAF/OSV/Markdown chunkers, deterministic hash embeddings, and unit coverage for sample advisories.
> 2025-11-02: SBOM context request/result models finalized; retriever tests now validate environment-flag toggles and dependency-path dedupe. SBOM guild to wire real context service client.
> 2025-11-04: AIAI-31-002 completed `AddSbomContext` typed client registered in WebService/Worker, BaseAddress/tenant headers sourced from configuration, and retriever HTTP-mapping tests extended.
@@ -194,12 +196,12 @@ FEEDCONN-CERTBUND-02-010 Version range provenance | BE-Conn-CERTBUND | **TODO (d
FEEDCONN-CISCO-02-009 SemVer range provenance | BE-Conn-Cisco | **TODO (due 2025-10-21)** Emit Cisco SemVer ranges into `advisory_observations.affected.versions[]` with provenance identifiers (`cisco:{productId}`) and deterministic comparison keys. Update mapper/tests for the Link-Not-Merge schema and replace legacy merge counter checks with observation/linkset validation. | CONCELIER-LNM-21-001 (src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md)
FEEDCONN-ICSCISA-02-012 Version range provenance | BE-Conn-ICS-CISA | **DONE (2025-11-03)** Promote existing firmware/semver data into `advisory_observations.affected.versions[]` entries with deterministic comparison keys and provenance identifiers (`ics-cisa:{advisoryId}:{product}`). Add regression coverage for mixed firmware strings and raise a Models ticket only when observation schema needs a new comparison helper.<br>2025-10-29: Follow `docs/dev/normalized-rule-recipes.md` §2 to build observation version entries and log failures without invoking the retired merge helpers.<br>2025-11-03: Completed connector now normalizes semver ranges with provenance notes, RSS fallback content clears the AOC guard, and end-to-end Fetch/Parse/Map integration tests pass. | CONCELIER-LNM-21-001 (src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md)
FEEDCONN-KISA-02-008 Firmware range provenance | BE-Conn-KISA, Models | **DONE (2025-11-04)** Define comparison helpers for Hangul-labelled firmware ranges (`XFU 1.0.1.0084 ~ 2.0.1.0034`) and map them into `advisory_observations.affected.versions[]` with provenance tags. Coordinate with Models only if a new comparison scheme is required, then update localisation notes and fixtures for the Link-Not-Merge schema.<br>2025-11-03: Analysis in progress auditing existing mapper output/fixtures ahead of implementing firmware range normalization and provenance wiring.<br>2025-11-03: SemVer normalization helper wired through `KisaMapper` with provenance slugs + vendor extensions; integration tests updated and green, follow-up capture for additional Hangul exclusivity markers queued before completion.<br>2025-11-03: Extended connector tests to cover single-ended (`이상`, `초과`, `이하`, `미만`) and non-numeric phrases, verifying normalized rule types (`gt`, `gte`, `lt`, `lte`) and fallback behaviour; broader corpus review remains before transitioning to DONE.<br>2025-11-03: Captured the top 10 `detailDos.do?IDX=` pages into `seed-data/kisa/html/` via `scripts/kisa_capture_html.py`; JSON endpoint (`rssDetailData.do?IDX=…`) now returns error pages, so connector updates must parse the embedded HTML or secure authenticated API access before closing.<br>2025-11-04: Fetch + parse pipeline now consumes the HTML detail pages end to end (metadata persisted, DOM parser extracts vendor/product ranges); fixtures/tests operate on the HTML snapshots to guard normalized SemVer + vendor extension expectations and severity extraction. | CONCELIER-LNM-21-001 (src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.md)
FEEDCONN-SHARED-STATE-003 Source state seeding helper | Tools Guild, BE-Conn-MSRC | **DOING (2025-10-19)** Provide a reusable CLI/utility to seed `pendingDocuments`/`pendingMappings` for connectors (MSRC backfills require scripted CVRF + detail injection). Coordinate with MSRC team for expected JSON schema and handoff once prototype lands. Prereqs confirmed none (2025-10-19). | Tools (src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.md)
FEEDCONN-SHARED-STATE-003 Source state seeding helper | Tools Guild, BE-Conn-MSRC | **DONE (2025-11-04)** Delivered `SourceStateSeeder` CLI + processor APIs, Mongo fixtures, and MSRC runbook updates. Seeds raw docs + cursor state deterministically; tests cover happy/path/idempotent flows (`dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Common.Tests/...` note: requires `libcrypto.so.1.1` when running Mongo2Go locally). | Tools (src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.md)
FEEDMERGE-COORD-02-901 Connector deadline check-ins | BE-Merge | **TODO (due 2025-10-21)** Confirm Cccs/Cisco version-provenance updates land, capture `LinksetVersionCoverage` dashboard snapshots (expect zero missing-range warnings), and update coordination docs with the results.<br>2025-10-29: Observation metrics now surface `version_entries_total`/`missing_version_entries_total`; include screenshots for both when closing this task. | FEEDMERGE-COORD-02-900 (src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md)
FEEDMERGE-COORD-02-902 ICS-CISA version comparison support | BE-Merge, Models | **TODO (due 2025-10-23)** Review ICS-CISA sample advisories, validate reuse of existing comparison helpers, and pre-stage Models ticket template only if a new firmware comparator is required. Document the outcome and observation coverage logs in coordination docs + tracker files.<br>2025-10-29: `docs/dev/normalized-rule-recipes.md` (§2§3) now covers observation entries; attach decision summary + log sample when handing off to Models. Dependencies: FEEDMERGE-COORD-02-901. | FEEDMERGE-COORD-02-900 (src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md)
FEEDMERGE-COORD-02-903 KISA firmware scheme review | BE-Merge, Models | **TODO (due 2025-10-24)** Pair with KISA team on proposed firmware comparison helper (`kisa.build` or variant), ensure observation mapper alignment, and open Models ticket only if a new comparator is required. Log the final helper signature and observation coverage metrics in coordination docs + tracker files. Dependencies: FEEDMERGE-COORD-02-902. | FEEDMERGE-COORD-02-900 (src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md)
Fixture validation sweep | QA | **DOING (2025-10-19)** Prereqs confirmed none; continuing RHSA fixture regeneration and diff review alongside mapper provenance updates.<br>2025-10-29: Added `scripts/update-redhat-fixtures.sh` to regenerate golden snapshots with `UPDATE_GOLDENS=1`; run it before reviews to capture CSAF contract deltas. | None (src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.md)
Link-Not-Merge version provenance coordination | BE-Merge | **DOING** Coordinate remaining connectors (`Acsc`, `Cccs`, `CertBund`, `CertCc`, `Cve`, `Ghsa`, `Ics.Cisa`, `Kisa`, `Ru.Bdu`, `Ru.Nkcki`, `Vndr.Apple`, `Vndr.Cisco`, `Vndr.Msrc`) so they emit `advisory_observations.affected.versions[]` entries with provenance tags and deterministic comparison keys. Track rollout status in `docs/dev/normalized-rule-recipes.md` (now updated for Link-Not-Merge) and retire the legacy merge counters as coverage transitions to linkset validation metrics.<br>2025-10-29: Added new guidance in the doc for recording observation version metadata and logging gaps via `LinksetVersionCoverage` warnings to replace prior `concelier.merge.normalized_rules*` alerts. Dependencies: CONCELIER-LNM-21-203. | CONCELIER-LNM-21-001 (src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md)
Fixture validation sweep | QA | **DONE (2025-11-04)** Regenerated RHSA CSAF goldens via `scripts/update-redhat-fixtures.sh` (sets `UPDATE_GOLDENS=1`) and re-ran connector tests `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Distro.RedHat.Tests/StellaOps.Concelier.Connector.Distro.RedHat.Tests.csproj --no-restore` to confirm snapshot parity. | None (src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.md)
Link-Not-Merge version provenance coordination | BE-Merge | **DONE (2025-11-04)** Published connector status tracker + follow-up IDs in `docs/dev/normalized-rule-recipes.md`, enabled `Normalized version rules missing` diagnostics in Merge, and aligned dashboards on `LinksetVersionCoverage`. Remaining gaps (ACSC/CCCS/CERTBUND/Cisco/RU-BDU) documented as upstream data deficiencies awaiting feed updates. Dependencies: CONCELIER-LNM-21-203. | CONCELIER-LNM-21-001 (src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md)
MERGE-LNM-21-001 | DONE (2025-11-03) | Draft `no-merge` migration playbook, documenting backfill strategy, feature flag rollout, and rollback steps for legacy merge pipeline deprecation.<br>2025-11-03: Authored `docs/migration/no-merge.md` covering rollout phases, backfill/validation checklists, and rollback guidance; shared artefact owners. | BE-Merge, Architecture Guild (src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md)
@@ -358,3 +360,7 @@ MIRROR-CRT-58-002 | TODO | Integrate with Export Center scheduling to automate m
If all tasks are done - read next sprint section - SPRINT_120_policy_reasoning.md
> 2025-11-04: AIAI-31-004A DONE WebService/Worker wiring plus filesystem queue operational; metrics/logs added; tests executed via `dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj --no-restore`.
> 2025-11-04: AIAI-31-006 DONE REST endpoints enforce scope headers, apply rate limits, sanitize prompts through guardrails, and enqueue execution with cached metadata.

View File

@@ -21,7 +21,7 @@ Task ID | State | Task description | Owners (Source)
LEDGER-29-001 | DONE (2025-11-03) | Design ledger & projection schemas (tables/indexes), canonical JSON format, hashing strategy, and migrations. Publish schema doc + fixtures.<br>2025-11-03: Initial migration, canonical fixtures, and schema doc alignment delivered (LEDGER-29-001). | Findings Ledger Guild (src/Findings/StellaOps.Findings.Ledger/TASKS.md)
LEDGER-29-002 | DONE (2025-11-03) | Implement ledger write API (`POST /vuln/ledger/events`) with validation, idempotency, hash chaining, and Merkle root computation job.<br>2025-11-03: Web service + domain scaffolding landed with canonical hashing helpers, in-memory repository, Merkle scheduler stub, request/response contracts, and unit tests covering hashing & conflict flows. Dependencies: LEDGER-29-001. | Findings Ledger Guild (src/Findings/StellaOps.Findings.Ledger/TASKS.md)
LEDGER-29-003 | DONE (2025-11-03) | Build projector worker that derives `findings_projection` rows from ledger events + policy determinations; ensure idempotent replay keyed by `(tenant,finding_id,policy_version)`. <br>2025-11-03: Postgres projection services landed with replay checkpoints, fixtures, and unit coverage (LEDGER-29-003). Dependencies: LEDGER-29-002. | Findings Ledger Guild, Scheduler Guild (src/Findings/StellaOps.Findings.Ledger/TASKS.md)
LEDGER-29-004 | DOING (2025-11-03) | Integrate Policy Engine batch evaluation (baseline + simulate) with projector; cache rationale references.<br>2025-11-04: Reducer+worker now store `policy_rationale` via inline evaluation; Postgres schema/fixtures/tests updated, pending real Policy Engine client wiring. Dependencies: LEDGER-29-003. | Findings Ledger Guild, Policy Guild (src/Findings/StellaOps.Findings.Ledger/TASKS.md)
LEDGER-29-004 | DONE (2025-11-04) | Integrate Policy Engine batch evaluation (baseline + simulate) with projector; cache rationale references.<br>2025-11-04: Ledger service now calls `/api/policy/eval/batch` with resilient HttpClient, shared cache, and inline fallback; documentation/config samples updated; ledger tests executed (`dotnet test src/Findings/__Tests/StellaOps.Findings.Ledger.Tests/StellaOps.Findings.Ledger.Tests.csproj --no-restore`). Dependencies: LEDGER-29-003. | Findings Ledger Guild, Policy Guild (src/Findings/StellaOps.Findings.Ledger/TASKS.md)
LEDGER-29-005 | TODO | Implement workflow mutation handlers (assign, comment, accept-risk, target-fix, verify-fix, reopen) producing ledger events with validation and attachments metadata. Dependencies: LEDGER-29-004. | Findings Ledger Guild (src/Findings/StellaOps.Findings.Ledger/TASKS.md)
LEDGER-29-006 | TODO | Integrate attachment encryption (KMS envelope), signed URL issuance, CSRF protection hooks for Console. Dependencies: LEDGER-29-005. | Findings Ledger Guild, Security Guild (src/Findings/StellaOps.Findings.Ledger/TASKS.md)
LEDGER-29-007 | TODO | Instrument metrics (`ledger_write_latency`, `projection_lag_seconds`, `ledger_events_total`), structured logs, and Merkle anchoring alerts; publish dashboards. Dependencies: LEDGER-29-006. | Findings Ledger Guild, Observability Guild (src/Findings/StellaOps.Findings.Ledger/TASKS.md)

View File

@@ -154,7 +154,7 @@ SCANNER-ENG-0025 | TODO | Implement WinSxS manifest collector per `design/window
SCANNER-ENG-0026 | TODO | Implement Windows Chocolatey & registry collectors per `design/windows-analyzer.md` §3.33.4. | Scanner Guild (docs/modules/scanner/TASKS.md)
SCANNER-ENG-0027 | TODO | Deliver Windows policy/offline integration per `design/windows-analyzer.md` §56. | Scanner Guild, Policy Guild, Offline Kit Guild (docs/modules/scanner/TASKS.md)
SCANNER-SURFACE-01 | DOING (2025-11-02) | Persist Surface.FS manifests after analyzer stages, including layer CAS metadata and EntryTrace fragments.<br>2025-11-02: Worker pipeline emitting draft Surface.FS manifests for sample scans; determinism checks running. | Scanner Worker Guild (src/Scanner/StellaOps.Scanner.Worker/TASKS.md)
SCANNER-SURFACE-02 | DOING (2025-11-02) | Publish Surface.FS pointers (CAS URIs, manifests) via scan/report APIs and update attestation metadata. Dependencies: SCANNER-SURFACE-01.<br>2025-11-02: WebService responses now include preview CAS URIs; attestation metadata updates staged for review. | Scanner WebService Guild (src/Scanner/StellaOps.Scanner.WebService/TASKS.md)
SCANNER-SURFACE-02 | DONE (2025-11-05) | Publish Surface.FS pointers (CAS URIs, manifests) via scan/report APIs and update attestation metadata. Dependencies: SCANNER-SURFACE-01.<br>2025-11-05: Surface pointer projection wired through WebService endpoints, orchestrator samples & DSSE fixtures refreshed with `surface` manifest block, and regression suite (platform events, report sample, ready check) updated. | Scanner WebService Guild (src/Scanner/StellaOps.Scanner.WebService/TASKS.md)
SCANNER-SURFACE-03 | TODO | Push layer manifests and entry fragments into Surface.FS during build-time SBOM generation. Dependencies: SCANNER-SURFACE-02. | BuildX Plugin Guild (src/Scanner/StellaOps.Scanner.Sbomer.BuildXPlugin/TASKS.md)
[Scanner & Surface] 130.A) Scanner.VIII

View File

@@ -1,139 +1,139 @@
# Advisory AI architecture
> Captures the retrieval, guardrail, and inference packaging requirements defined in the Advisory AI implementation plan and related module guides.
## 1) Goals
- Summarise advisories/VEX evidence into operator-ready briefs with citations.
- Explain conflicting statements with provenance and trust weights (using VEX Lens & Excititor data).
- Suggest remediation plans aligned with Offline Kit deployment models and scheduler follow-ups.
- Operate deterministically where possible; cache generated artefacts with digests for audit.
## 2) Pipeline overview
```
+---------------------+
Concelier/VEX Lens | Evidence Retriever |
Policy Engine ----> | (vector + keyword) | ---> Context Pack (JSON)
Zastava runtime +---------------------+
|
v
+-------------+
| Prompt |
| Assembler |
+-------------+
|
v
+-------------+
| Guarded LLM |
| (local/host)|
+-------------+
|
v
+-----------------+
| Citation & |
| Validation |
+-----------------+
|
v
+----------------+
| Output cache |
| (hash, bundle) |
+----------------+
```
## 3) Retrieval & context
- Hybrid search: vector embeddings (SBERT-compatible) + keyword filters for advisory IDs, PURLs, CVEs.
- Context packs include:
- Advisory raw excerpts with highlighted sections and source URLs.
- VEX statements (normalized tuples + trust metadata).
- Policy explain traces for the affected finding.
- Runtime/impact hints from Zastava (exposure, entrypoints).
- Export-ready remediation data (fixed versions, patches).
- **SBOM context retriever** (AIAI-31-002) hydrates:
- Version timelines (first/last observed, status, fix availability).
- Dependency paths (runtime vs build/test, deduped by coordinate chain).
- Tenant environment flags (prod/stage toggles) with optional blast radius summary.
- Service-side clamps: max 500 timeline entries, 200 dependency paths, with client-provided toggles for env/blast data.
- `AddSbomContextHttpClient(...)` registers the typed HTTP client that calls `/v1/sbom/context`, while `NullSbomContextClient` remains the safe default for environments that have not yet exposed the SBOM service.
**Sample configuration** (wire real SBOM base URL + API key):
```csharp
services.AddSbomContextHttpClient(options =>
{
options.BaseAddress = new Uri("https://sbom-service.internal");
options.Endpoint = "/v1/sbom/context";
options.ApiKey = configuration["SBOM_SERVICE_API_KEY"];
options.UserAgent = "stellaops-advisoryai/1.0";
options.Tenant = configuration["TENANT_ID"];
});
services.AddAdvisoryPipeline();
```
After configuration, issue a smoke request (e.g., `ISbomContextRetriever.RetrieveAsync`) during deployment validation to confirm end-to-end connectivity and credentials before enabling Advisory AI endpoints.
Retriever requests and results are trimmed/normalized before hashing; metadata (counts, provenance keys) is returned for downstream guardrails. Unit coverage ensures deterministic ordering and flag handling.
All context references include `content_hash` and `source_id` enabling verifiable citations.
## 4) Guardrails
- Prompt templates enforce structure: summary, conflicts, remediation, references.
- Response validator ensures:
- No hallucinated advisories (every fact must map to input context).
- Citations follow `[n]` indexing referencing actual sources.
- Remediation suggestions only cite policy-approved sources (fixed versions, vendor hotfixes).
- Moderation/PII filters prevent leaking secrets; responses failing validation are rejected and logged.
- Pre-flight guardrails redact secrets (AWS keys, generic API tokens, PEM blobs), block "ignore previous instructions"-style prompt injection attempts, enforce citation presence, and cap prompt payload length (default 16kB). Guardrail outcomes and redaction counts surface via `advisory_guardrail_blocks` / `advisory_outputs_stored` metrics.
## 5) Deterministic tooling
- **Version comparators** — offline semantic version + RPM EVR parsers with range evaluators. Supports chained constraints (`>=`, `<=`, `!=`) used by remediation advice and blast radius calcs.
- Registered via `AddAdvisoryDeterministicToolset` for reuse across orchestrator, CLI, and services.
- **Orchestration pipeline** — see `orchestration-pipeline.md` for prerequisites, task breakdown, and cross-guild responsibilities before wiring the execution flows.
- **Planned extensions** — NEVRA/EVR comparators, ecosystem-specific normalisers, dependency chain scorers (AIAI-31-003 scope).
- Exposed via internal interfaces to allow orchestrator/toolchain reuse; all helpers stay side-effect free and deterministic for golden testing.
## 6) Output persistence
- Cached artefacts stored in `advisory_ai_outputs` with fields:
- `output_hash` (sha256 of JSON response).
- `input_digest` (hash of context pack).
- `summary`, `conflicts`, `remediation`, `citations`.
- `generated_at`, `model_id`, `profile` (Sovereign/FIPS etc.).
- `signatures` (optional DSSE if run in deterministic mode).
- Offline bundle format contains `summary.md`, `citations.json`, `context_manifest.json`, `signatures/`.
## 7) Profiles & sovereignty
- **Profiles:** `default`, `fips-local` (FIPS-compliant local model), `gost-local`, `cloud-openai` (optional, disabled by default). Each profile defines allowed models, key management, and telemetry endpoints.
- **CryptoProfile/RootPack integration:** generated artefacts can be signed using configured CryptoProfile to satisfy procurement/trust requirements.
## 8) APIs
- `POST /api/v1/advisory/{task}` — executes Summary/Conflict/Remediation pipeline (`task` ∈ `summary|conflict|remediation`). Requests accept `{advisoryKey, artifactId?, policyVersion?, profile, preferredSections?, forceRefresh}` and return sanitized prompt payloads, citations, guardrail metadata, provenance hash, and cache hints.
- `GET /api/v1/advisory/outputs/{cacheKey}?taskType=SUMMARY&profile=default` — retrieves cached artefacts for downstream consumers (Console, CLI, Export Center). Guardrail state and provenance hash accompany results.
All endpoints accept `profile` parameter (default `fips-local`) and return `output_hash`, `input_digest`, and `citations` for verification.
## 9) Observability
- Metrics: `advisory_ai_requests_total{profile,type}`, `advisory_ai_latency_seconds`, `advisory_ai_validation_failures_total`.
- Logs: include `output_hash`, `input_digest`, `profile`, `model_id`, `tenant`, `artifacts`. Sensitive context is not logged.
- Traces: spans for retrieval, prompt assembly, model inference, validation, cache write.
## 10) Operational controls
- Feature flags per tenant (`ai.summary.enabled`, `ai.remediation.enabled`).
- Rate limits (per tenant, per profile) enforced by Orchestrator to prevent runaway usage.
- Offline/air-gapped deployments run local models packaged with Offline Kit; model weights validated via manifest digests.
## 11) Hosting surfaces
- **WebService** — exposes `/v1/advisory-ai/pipeline/{task}` to materialise plans and enqueue execution messages.
- **Worker** — background service draining the advisory pipeline queue (file-backed stub) pending integration with shared transport.
- Both hosts register `AddAdvisoryAiCore`, which wires the SBOM context client, deterministic toolset, pipeline orchestrator, and queue metrics.
- SBOM base address + tenant metadata are configured via `AdvisoryAI:SbomBaseAddress` and propagated through `AddSbomContext`.
# Advisory AI architecture
> Captures the retrieval, guardrail, and inference packaging requirements defined in the Advisory AI implementation plan and related module guides.
## 1) Goals
- Summarise advisories/VEX evidence into operator-ready briefs with citations.
- Explain conflicting statements with provenance and trust weights (using VEX Lens & Excititor data).
- Suggest remediation plans aligned with Offline Kit deployment models and scheduler follow-ups.
- Operate deterministically where possible; cache generated artefacts with digests for audit.
## 2) Pipeline overview
```
+---------------------+
Concelier/VEX Lens | Evidence Retriever |
Policy Engine ----> | (vector + keyword) | ---> Context Pack (JSON)
Zastava runtime +---------------------+
|
v
+-------------+
| Prompt |
| Assembler |
+-------------+
|
v
+-------------+
| Guarded LLM |
| (local/host)|
+-------------+
|
v
+-----------------+
| Citation & |
| Validation |
+-----------------+
|
v
+----------------+
| Output cache |
| (hash, bundle) |
+----------------+
```
## 3) Retrieval & context
- Hybrid search: vector embeddings (SBERT-compatible) + keyword filters for advisory IDs, PURLs, CVEs.
- Context packs include:
- Advisory raw excerpts with highlighted sections and source URLs.
- VEX statements (normalized tuples + trust metadata).
- Policy explain traces for the affected finding.
- Runtime/impact hints from Zastava (exposure, entrypoints).
- Export-ready remediation data (fixed versions, patches).
- **SBOM context retriever** (AIAI-31-002) hydrates:
- Version timelines (first/last observed, status, fix availability).
- Dependency paths (runtime vs build/test, deduped by coordinate chain).
- Tenant environment flags (prod/stage toggles) with optional blast radius summary.
- Service-side clamps: max 500 timeline entries, 200 dependency paths, with client-provided toggles for env/blast data.
- `AddSbomContextHttpClient(...)` registers the typed HTTP client that calls `/v1/sbom/context`, while `NullSbomContextClient` remains the safe default for environments that have not yet exposed the SBOM service.
**Sample configuration** (wire real SBOM base URL + API key):
```csharp
services.AddSbomContextHttpClient(options =>
{
options.BaseAddress = new Uri("https://sbom-service.internal");
options.Endpoint = "/v1/sbom/context";
options.ApiKey = configuration["SBOM_SERVICE_API_KEY"];
options.UserAgent = "stellaops-advisoryai/1.0";
options.Tenant = configuration["TENANT_ID"];
});
services.AddAdvisoryPipeline();
```
After configuration, issue a smoke request (e.g., `ISbomContextRetriever.RetrieveAsync`) during deployment validation to confirm end-to-end connectivity and credentials before enabling Advisory AI endpoints.
Retriever requests and results are trimmed/normalized before hashing; metadata (counts, provenance keys) is returned for downstream guardrails. Unit coverage ensures deterministic ordering and flag handling.
All context references include `content_hash` and `source_id` enabling verifiable citations.
## 4) Guardrails
- Prompt templates enforce structure: summary, conflicts, remediation, references.
- Response validator ensures:
- No hallucinated advisories (every fact must map to input context).
- Citations follow `[n]` indexing referencing actual sources.
- Remediation suggestions only cite policy-approved sources (fixed versions, vendor hotfixes).
- Moderation/PII filters prevent leaking secrets; responses failing validation are rejected and logged.
- Pre-flight guardrails redact secrets (AWS keys, generic API tokens, PEM blobs), block "ignore previous instructions"-style prompt injection attempts, enforce citation presence, and cap prompt payload length (default 16kB). Guardrail outcomes and redaction counts surface via `advisory_guardrail_blocks` / `advisory_outputs_stored` metrics.
## 5) Deterministic tooling
- **Version comparators** — offline semantic version + RPM EVR parsers with range evaluators. Supports chained constraints (`>=`, `<=`, `!=`) used by remediation advice and blast radius calcs.
- Registered via `AddAdvisoryDeterministicToolset` for reuse across orchestrator, CLI, and services.
- **Orchestration pipeline** — see `orchestration-pipeline.md` for prerequisites, task breakdown, and cross-guild responsibilities before wiring the execution flows.
- **Planned extensions** — NEVRA/EVR comparators, ecosystem-specific normalisers, dependency chain scorers (AIAI-31-003 scope).
- Exposed via internal interfaces to allow orchestrator/toolchain reuse; all helpers stay side-effect free and deterministic for golden testing.
## 6) Output persistence
- Cached artefacts stored in `advisory_ai_outputs` with fields:
- `output_hash` (sha256 of JSON response).
- `input_digest` (hash of context pack).
- `summary`, `conflicts`, `remediation`, `citations`.
- `generated_at`, `model_id`, `profile` (Sovereign/FIPS etc.).
- `signatures` (optional DSSE if run in deterministic mode).
- Offline bundle format contains `summary.md`, `citations.json`, `context_manifest.json`, `signatures/`.
## 7) Profiles & sovereignty
- **Profiles:** `default`, `fips-local` (FIPS-compliant local model), `gost-local`, `cloud-openai` (optional, disabled by default). Each profile defines allowed models, key management, and telemetry endpoints.
- **CryptoProfile/RootPack integration:** generated artefacts can be signed using configured CryptoProfile to satisfy procurement/trust requirements.
## 8) APIs
- `POST /api/v1/advisory/{task}` — executes Summary/Conflict/Remediation pipeline (`task` ∈ `summary|conflict|remediation`). Requests accept `{advisoryKey, artifactId?, policyVersion?, profile, preferredSections?, forceRefresh}` and return sanitized prompt payloads, citations, guardrail metadata, provenance hash, and cache hints.
- `GET /api/v1/advisory/outputs/{cacheKey}?taskType=SUMMARY&profile=default` — retrieves cached artefacts for downstream consumers (Console, CLI, Export Center). Guardrail state and provenance hash accompany results.
All endpoints accept `profile` parameter (default `fips-local`) and return `output_hash`, `input_digest`, and `citations` for verification.
## 9) Observability
- Metrics: `advisory_ai_requests_total{profile,type}`, `advisory_ai_latency_seconds`, `advisory_ai_validation_failures_total`.
- Logs: include `output_hash`, `input_digest`, `profile`, `model_id`, `tenant`, `artifacts`. Sensitive context is not logged.
- Traces: spans for retrieval, prompt assembly, model inference, validation, cache write.
## 10) Operational controls
- Feature flags per tenant (`ai.summary.enabled`, `ai.remediation.enabled`).
- Rate limits (per tenant, per profile) enforced by Orchestrator to prevent runaway usage.
- Offline/air-gapped deployments run local models packaged with Offline Kit; model weights validated via manifest digests.
## 11) Hosting surfaces
- **WebService** — exposes `/v1/advisory-ai/pipeline/{task}` to materialise plans and enqueue execution messages.
- **Worker** — background service draining the advisory pipeline queue (file-backed stub) pending integration with shared transport.
- Both hosts register `AddAdvisoryAiCore`, which wires the SBOM context client, deterministic toolset, pipeline orchestrator, and queue metrics.
- SBOM base address + tenant metadata are configured via `AdvisoryAI:SbomBaseAddress` and propagated through `AddSbomContext`.

View File

@@ -1,96 +1,96 @@
# Advisory AI Orchestration Pipeline (Planning Notes)
> **Status:** In progress orchestration metadata and cache-key wiring underway for AIAI-31-004.
> **Audience:** Advisory AI guild, WebService/Worker guilds, CLI guild, Docs/QA support teams.
## 1. Goal
Wire the deterministic pipeline (Summary / Conflict / Remediation flows) into the Advisory AI service, workers, and CLI with deterministic caching, prompt preparation, and guardrail fallback. This document captures the pre-integration checklist and task breakdown so each guild understands their responsibilities before coding begins.
## 2. Prerequisites
| Area | Requirement | Owner | Status |
|------|-------------|-------|--------|
| **Toolset** | Deterministic comparators, dependency analyzer (`IDeterministicToolset`, `AdvisoryPipelineOrchestrator`) | Advisory AI | ✅ landed (AIAI-31-003) |
| **SBOM context** | Real SBOM context client delivering timelines + dependency paths | SBOM Service Guild | ✅ typed client and DI helper ready; supply host BaseAddress at integration time |
| **Prompt artifacts** | Liquid/Handlebars prompt templates for summary/conflict/remediation | Advisory AI Docs Guild | ⏳ authoring needed |
| **Cache strategy** | Decision on DSSE or hash-only cache entries, TTLs, and eviction policy | Advisory AI + Platform | ⏳ hash-only plan keys implemented; persistence decision outstanding |
| **Auth scopes** | Confirm service account scopes for new API endpoints/worker-to-service calls | Authority Guild | 🔲 define |
**Blocking risk:** SBOM client and prompt templates must exist (even stubbed) before the orchestrator can produce stable plans.
## 3. Integration plan (high-level)
1. **Service layer (WebService / Worker)**
- Inject `IAdvisoryPipelineOrchestrator` via `AddAdvisoryPipeline`.
- Define REST endpoint `POST /v1/advisories/{key}/pipeline/{task}` (task ∈ summary/conflict/remediation).
- Worker consumes queue messages (`advisory.pipeline.execute`) -> fetches plan -> executes prompt -> persists output & provenance.
- Add metrics: `advisory_pipeline_requests_total`, `advisory_pipeline_plan_cache_hits_total`, `advisory_pipeline_latency_seconds`.
2. **CLI**
- New command `stella advise run <task>` with flags for artifact id, profile, policy version, `--force-refresh`.
- Render JSON/Markdown outputs; handle caching hints (print cache key, status).
3. **Caching / storage**
- Choose storage (Mongo collection vs existing DSSE output store).
- Persist `AdvisoryTaskPlan` metadata + generated output keyed by cache key + policy version.
- Expose TTL/force-refresh semantics.
4. **Docs & QA**
- Publish API spec (`docs/advisory-ai/api.md`) + CLI docs.
- Add golden outputs for deterministic runs; property tests for cache key stability (unit coverage landed for cache hashing + option clamps).
## 4. Task Breakdown
### AIAI-31-004A (Service orchestration wiring)
- **Scope:** WebService/Worker injection, REST/queue plumbing, metrics counters, basic cache stub.
- **Dependencies:** `AddAdvisoryPipeline`, SBOM client stub.
- **Exit:** API responds with plan metadata + queue message; worker logs execution attempt; metrics emitted.
### AIAI-31-004B (Prompt assembly & cache persistence)
- **Scope:** Implement prompt assembler, connect to guardrails, persist cache entries w/ DSSE metadata.
- **Dependencies:** Prompt templates, cache storage decision, guardrail interface.
- **Exit:** Deterministic outputs stored; force-refresh honoured; tests cover prompt assembly + caching.
> 2025-11-03: Prompt assembler now emits deterministic JSON payloads, guardrail pipeline wiring is stubbed for upcoming security hardening, and outputs persist with DSSE-ready provenance metadata plus golden test coverage.
### AIAI-31-004C (CLI integration & docs)
- **Scope:** CLI command + output renderer, docs updates, CLI tests (golden outputs).
- **Dependencies:** Service endpoints stable, caching semantics documented.
- **Exit:** CLI command produces deterministic output, docs updated, smoke tests recorded.
### AIAI-31-006 (Service API surface)
- **Scope:** Expose REST endpoints for summary/conflict/remediation execution plus cached output retrieval (`POST /api/v1/advisory/{task}`, `GET /api/v1/advisory/outputs/{cacheKey}`). Include guardrail execution, provenance hashing, metrics, and stubs for RBAC/rate limits.
- **Dependencies:** Guardrail enforcement (AIAI-31-005), Authority scope wiring (`advisory-ai:view` / `advisory-ai:operate`), Offline kit docs.
- **Exit:** Endpoints return sanitized prompts with citations, guardrail metadata, DSSE hash, and plan cache indicators; OpenAPI description updated; rate-limit hooks ready for Authority integration.
> 2025-11-03: Initial REST surface shipped direct execution runs through guardrail pipeline, outputs persist with DSSE-ready provenance, metrics `advisory_outputs_stored`/`advisory_guardrail_blocks` emit, and cache retrieval endpoint exposes stored artefacts (RBAC/header enforcement pending scope delivery).
### Supporting tasks (other guilds)
- **AUTH-AIAI-31-004** Update scopes and DSSE policy (Authority guild).
- **DOCS-AIAI-31-003** Publish API documentation, CLI guide updates (Docs guild).
- **QA-AIAI-31-004** Golden/properties/perf suite for pipeline (QA guild).
## 5. Acceptance checklist (per task)
| Item | Notes |
|------|-------|
| Cache key stability | `AdvisoryPipelineOrchestrator` hash must remain stable under re-run of identical inputs. |
| Metrics & logging | Request id, cache key, task type, profile, latency; guardrail results logged without sensitive prompt data. |
| Offline readiness | All prompt templates bundled with Offline Kit; CLI works in air-gapped mode with cached data. |
| Policy awareness | Plans encode policy version used; outputs reference policy digest for audit. |
| Testing | Unit tests (plan generation, cache keys, DI), integration (service endpoint, worker, CLI), deterministic golden outputs. |
## 6. Next steps
1. Finalize SBOM context client (AIAI-31-002) and prompt templates.
2. Create queue schema spec (`docs/modules/advisory-ai/queue-contracts.md`) if not already available.
3. Schedule cross-guild kickoff to agree on cache store & DSSE policy.
## 7. Recent updates
- 2025-11-04 — Orchestrator metadata now captures SBOM environment flags, blast-radius metrics, and dependency analysis details; cache-key normalization covers ordering.
- 2025-11-04 — Unit tests added for SBOM-absent requests, option-limit enforcement, and cache-key stability.
- 2025-11-04 — `AddSbomContext` DI helper enforces tenant header + base address wiring for downstream hosts.
_Last updated: 2025-11-04_
# Advisory AI Orchestration Pipeline (Planning Notes)
> **Status:** In progress orchestration metadata and cache-key wiring underway for AIAI-31-004.
> **Audience:** Advisory AI guild, WebService/Worker guilds, CLI guild, Docs/QA support teams.
## 1. Goal
Wire the deterministic pipeline (Summary / Conflict / Remediation flows) into the Advisory AI service, workers, and CLI with deterministic caching, prompt preparation, and guardrail fallback. This document captures the pre-integration checklist and task breakdown so each guild understands their responsibilities before coding begins.
## 2. Prerequisites
| Area | Requirement | Owner | Status |
|------|-------------|-------|--------|
| **Toolset** | Deterministic comparators, dependency analyzer (`IDeterministicToolset`, `AdvisoryPipelineOrchestrator`) | Advisory AI | ✅ landed (AIAI-31-003) |
| **SBOM context** | Real SBOM context client delivering timelines + dependency paths | SBOM Service Guild | ✅ typed client and DI helper ready; supply host BaseAddress at integration time |
| **Prompt artifacts** | Liquid/Handlebars prompt templates for summary/conflict/remediation | Advisory AI Docs Guild | ⏳ authoring needed |
| **Cache strategy** | Decision on DSSE or hash-only cache entries, TTLs, and eviction policy | Advisory AI + Platform | ⏳ hash-only plan keys implemented; persistence decision outstanding |
| **Auth scopes** | Confirm service account scopes for new API endpoints/worker-to-service calls | Authority Guild | 🔲 define |
**Blocking risk:** SBOM client and prompt templates must exist (even stubbed) before the orchestrator can produce stable plans.
## 3. Integration plan (high-level)
1. **Service layer (WebService / Worker)**
- Inject `IAdvisoryPipelineOrchestrator` via `AddAdvisoryPipeline`.
- Define REST endpoint `POST /v1/advisories/{key}/pipeline/{task}` (task ∈ summary/conflict/remediation).
- Worker consumes queue messages (`advisory.pipeline.execute`) -> fetches plan -> executes prompt -> persists output & provenance.
- Add metrics: `advisory_pipeline_requests_total`, `advisory_pipeline_plan_cache_hits_total`, `advisory_pipeline_latency_seconds`.
2. **CLI**
- New command `stella advise run <task>` with flags for artifact id, profile, policy version, `--force-refresh`.
- Render JSON/Markdown outputs; handle caching hints (print cache key, status).
3. **Caching / storage**
- Choose storage (Mongo collection vs existing DSSE output store).
- Persist `AdvisoryTaskPlan` metadata + generated output keyed by cache key + policy version.
- Expose TTL/force-refresh semantics.
4. **Docs & QA**
- Publish API spec (`docs/advisory-ai/api.md`) + CLI docs.
- Add golden outputs for deterministic runs; property tests for cache key stability (unit coverage landed for cache hashing + option clamps).
## 4. Task Breakdown
### AIAI-31-004A (Service orchestration wiring)
- **Scope:** WebService/Worker injection, REST/queue plumbing, metrics counters, basic cache stub.
- **Dependencies:** `AddAdvisoryPipeline`, SBOM client stub.
- **Exit:** API responds with plan metadata + queue message; worker logs execution attempt; metrics emitted.
### AIAI-31-004B (Prompt assembly & cache persistence)
- **Scope:** Implement prompt assembler, connect to guardrails, persist cache entries w/ DSSE metadata.
- **Dependencies:** Prompt templates, cache storage decision, guardrail interface.
- **Exit:** Deterministic outputs stored; force-refresh honoured; tests cover prompt assembly + caching.
> 2025-11-03: Prompt assembler now emits deterministic JSON payloads, guardrail pipeline wiring is stubbed for upcoming security hardening, and outputs persist with DSSE-ready provenance metadata plus golden test coverage.
### AIAI-31-004C (CLI integration & docs)
- **Scope:** CLI command + output renderer, docs updates, CLI tests (golden outputs).
- **Dependencies:** Service endpoints stable, caching semantics documented.
- **Exit:** CLI command produces deterministic output, docs updated, smoke tests recorded.
### AIAI-31-006 (Service API surface)
- **Scope:** Expose REST endpoints for summary/conflict/remediation execution plus cached output retrieval (`POST /api/v1/advisory/{task}`, `GET /api/v1/advisory/outputs/{cacheKey}`). Include guardrail execution, provenance hashing, metrics, and stubs for RBAC/rate limits.
- **Dependencies:** Guardrail enforcement (AIAI-31-005), Authority scope wiring (`advisory-ai:view` / `advisory-ai:operate`), Offline kit docs.
- **Exit:** Endpoints return sanitized prompts with citations, guardrail metadata, DSSE hash, and plan cache indicators; OpenAPI description updated; rate-limit hooks ready for Authority integration.
> 2025-11-03: Initial REST surface shipped direct execution runs through guardrail pipeline, outputs persist with DSSE-ready provenance, metrics `advisory_outputs_stored`/`advisory_guardrail_blocks` emit, and cache retrieval endpoint exposes stored artefacts (RBAC/header enforcement pending scope delivery).
### Supporting tasks (other guilds)
- **AUTH-AIAI-31-004** Update scopes and DSSE policy (Authority guild).
- **DOCS-AIAI-31-003** Publish API documentation, CLI guide updates (Docs guild).
- **QA-AIAI-31-004** Golden/properties/perf suite for pipeline (QA guild).
## 5. Acceptance checklist (per task)
| Item | Notes |
|------|-------|
| Cache key stability | `AdvisoryPipelineOrchestrator` hash must remain stable under re-run of identical inputs. |
| Metrics & logging | Request id, cache key, task type, profile, latency; guardrail results logged without sensitive prompt data. |
| Offline readiness | All prompt templates bundled with Offline Kit; CLI works in air-gapped mode with cached data. |
| Policy awareness | Plans encode policy version used; outputs reference policy digest for audit. |
| Testing | Unit tests (plan generation, cache keys, DI), integration (service endpoint, worker, CLI), deterministic golden outputs. |
## 6. Next steps
1. Finalize SBOM context client (AIAI-31-002) and prompt templates.
2. Create queue schema spec (`docs/modules/advisory-ai/queue-contracts.md`) if not already available.
3. Schedule cross-guild kickoff to agree on cache store & DSSE policy.
## 7. Recent updates
- 2025-11-04 — Orchestrator metadata now captures SBOM environment flags, blast-radius metrics, and dependency analysis details; cache-key normalization covers ordering.
- 2025-11-04 — Unit tests added for SBOM-absent requests, option-limit enforcement, and cache-key stability.
- 2025-11-04 — `AddSbomContext` DI helper enforces tenant header + base address wiring for downstream hosts.
_Last updated: 2025-11-04_

View File

@@ -1,298 +1,298 @@
# CLI AOC Commands Reference
> **Audience:** DevEx engineers, operators, and CI authors integrating the `stella` CLI with Aggregation-Only Contract (AOC) workflows.
> **Scope:** Command synopsis, options, exit codes, and offline considerations for `stella sources ingest --dry-run` and `stella aoc verify` as introduced in Sprint19.
# CLI AOC Commands Reference
> **Audience:** DevEx engineers, operators, and CI authors integrating the `stella` CLI with Aggregation-Only Contract (AOC) workflows.
> **Scope:** Command synopsis, options, exit codes, and offline considerations for `stella sources ingest --dry-run` and `stella aoc verify` as introduced in Sprint19.
Both commands are designed to enforce the AOC guardrails documented in the [aggregation-only reference](../../../ingestion/aggregation-only-contract.md) and the [architecture overview](../architecture.md). They consume Authority-issued tokens with tenant scopes and never mutate ingestion stores.
---
## 1·Prerequisites
- CLI version: `stella`0.19.0 (AOC feature gate enabled).
- Required scopes (DPoP-bound):
- `advisory:read` for Concelier sources.
- `vex:read` for Excititor sources (optional but required for VEX checks).
- `aoc:verify` to invoke guard verification endpoints.
- `tenant:select` if your deployment uses tenant switching.
- Connectivity: direct access to Concelier/Excititor APIs or Offline Kit snapshot (see §4).
- Environment: set `STELLA_AUTHORITY_URL`, `STELLA_TENANT`, and export a valid OpTok via `stella auth login` or existing token cache.
---
## 2·`stella sources ingest --dry-run`
### 2.1Synopsis
```bash
stella sources ingest --dry-run \
--source <source-key> \
--input <path-or-uri> \
[--tenant <tenant-id>] \
[--format json|table] \
[--no-color] \
[--output <file>]
```
### 2.2Description
Previews an ingestion write without touching MongoDB. The command loads an upstream advisory or VEX document, computes the would-write payload, runs it through the `AOCWriteGuard`, and reports any forbidden fields, provenance gaps, or idempotency issues. Use it during connector development, CI validation, or while triaging incidents.
### 2.3Options
| Option | Description |
|--------|-------------|
| `--source <source-key>` | Logical source name (`redhat`, `ubuntu`, `osv`, etc.). Mirrors connector configuration. |
| `--input <path-or-uri>` | Path to local CSAF/OSV/VEX file or HTTPS URI. CLI normalises transport (gzip/base64) before guard evaluation. |
| `--tenant <tenant-id>` | Overrides default tenant for multi-tenant deployments. Mandatory when `STELLA_TENANT` is not set. |
| `--format json|table` | Output format. `table` (default) prints summary with highlighted violations; `json` emits machine-readable report (see below). |
| `--no-color` | Disables ANSI colour output for CI logs. |
| `--output <file>` | Writes the JSON report to file while still printing human-readable summary to stdout. |
### 2.4Output schema (JSON)
```json
{
"source": "redhat",
"tenant": "default",
"guardVersion": "1.0.0",
"status": "ok",
"document": {
"contentHash": "sha256:…",
"supersedes": null,
"provenance": {
"signature": { "format": "pgp", "present": true }
}
},
"violations": []
}
```
When violations exist, `status` becomes `error` and `violations` contains entries with `code` (`ERR_AOC_00x`), a short `message`, and JSON Pointer `path` values indicating offending fields.
### 2.5Exit codes
| Exit code | Meaning |
|-----------|---------|
| `0` | Guard passed; would-write payload is AOC compliant. |
| `11` | `ERR_AOC_001` Forbidden field (`severity`, `cvss`, etc.) detected. |
| `12` | `ERR_AOC_002` Merge attempt (multiple upstream sources fused). |
| `13` | `ERR_AOC_003` Idempotency violation (duplicate without supersedes). |
| `14` | `ERR_AOC_004` Missing provenance fields. |
| `15` | `ERR_AOC_005` Signature/checksum mismatch. |
| `16` | `ERR_AOC_006` Effective findings present (Policy-only data). |
| `17` | `ERR_AOC_007` Unknown top-level fields / schema violation. |
| `70` | Transport error (network, auth, malformed input). |
> Exit codes map directly to the `ERR_AOC_00x` table for scripting consistency. Multiple violations yield the highest-priority code (e.g., 11 takes precedence over 14).
### 2.6Examples
Dry-run a local CSAF file:
```bash
stella sources ingest --dry-run \
--source redhat \
--input ./fixtures/redhat/RHSA-2025-1234.json
```
Stream from HTTPS and emit JSON for CI:
```bash
stella sources ingest --dry-run \
--source osv \
--input https://osv.dev/vulnerability/GHSA-aaaa-bbbb \
--format json \
--output artifacts/osv-dry-run.json
cat artifacts/osv-dry-run.json | jq '.violations'
```
### 2.7Offline notes
When operating in sealed/offline mode:
- Use `--input` paths pointing to Offline Kit snapshots (`offline-kit/advisories/*.json`).
- Provide `--tenant` explicitly if the offline bundle contains multiple tenants.
- The command does not attempt network access when given a file path.
- Store reports with `--output` to include in transfer packages for policy review.
---
## 3·`stella aoc verify`
### 3.1Synopsis
```bash
stella aoc verify \
[--since <iso8601|duration>] \
[--limit <count>] \
[--sources <list>] \
[--codes <ERR_AOC_00x,...>] \
[--format table|json] \
[--export <file>] \
[--tenant <tenant-id>] \
[--no-color]
```
### 3.2Description
Replays the AOC guard against stored raw documents. By default it checks all advisories and VEX statements ingested in the last 24hours for the active tenant, reporting totals, top violation codes, and sample documents. Use it in CI pipelines, scheduled verifications, or during incident response.
### 3.3Options
| Option | Description |
|--------|-------------|
| `--since <value>` | Verification window. Accepts ISO8601 timestamp (`2025-10-25T12:00:00Z`) or duration (`48h`, `7d`). Defaults to `24h`. |
| `--limit <count>` | Maximum number of violations to display (per code). `0` means show all. Defaults to `20`. |
| `--sources <list>` | Comma-separated list of sources (`redhat,ubuntu,osv`). Filters both advisories and VEX entries. |
| `--codes <list>` | Restricts output to specific `ERR_AOC_00x` codes. Useful for regression tracking. |
| `--format table|json` | `table` (default) prints summary plus top violations; `json` outputs machine-readable report identical to the `/aoc/verify` API. |
| `--export <file>` | Writes the JSON report to disk (useful for audits/offline uploads). |
| `--tenant <tenant-id>` | Overrides tenant context. Required for cross-tenant verifications when run by platform operators. |
| `--no-color` | Disables ANSI colours. |
`table` mode prints a summary showing the active tenant, evaluated window, counts of checked advisories/VEX statements, the active limit, total writes/violations, and whether the page was truncated. Status is colour-coded as `ok`, `violations`, or `truncated`. When violations exist the detail table lists the code, total occurrences, first sample document (`source` + `documentId` + `contentHash`), and JSON pointer path.
### 3.4Report structure (JSON)
```json
{
"tenant": "default",
"window": {
"from": "2025-10-25T12:00:00Z",
"to": "2025-10-26T12:00:00Z"
},
"checked": {
"advisories": 482,
"vex": 75
},
"violations": [
{
"code": "ERR_AOC_001",
"count": 2,
"examples": [
{
"source": "redhat",
"documentId": "advisory_raw:redhat:RHSA-2025:1",
"contentHash": "sha256:…",
"path": "/content/raw/cvss"
}
]
}
],
"metrics": {
"ingestion_write_total": 557,
"aoc_violation_total": 2
},
"truncated": false
}
```
### 3.5Exit codes
| Exit code | Meaning |
|-----------|---------|
| `0` | Verification succeeded with zero violations. |
| `11…17` | Same mapping as §2.5 when violations are detected. Highest-priority code returned. |
| `18` | Verification ran but results truncated (limit reached) treat as warning; rerun with higher `--limit`. |
| `70` | Transport/authentication error. |
| `71` | CLI misconfiguration (missing tenant, invalid `--since`, etc.). |
### 3.6Examples
Daily verification across all sources:
```bash
stella aoc verify --since 24h --format table
```
CI pipeline focusing on errant sources and exporting evidence:
```bash
stella aoc verify \
--sources redhat,ubuntu \
--codes ERR_AOC_001,ERR_AOC_004 \
--format json \
--limit 100 \
--export artifacts/aoc-verify.json
jq '.violations[] | {code, count}' artifacts/aoc-verify.json
```
Air-gapped verification using Offline Kit snapshot (example script):
```bash
stella aoc verify \
--since 7d \
--format json \
--export /mnt/offline/aoc-verify-$(date +%F).json
sha256sum /mnt/offline/aoc-verify-*.json > /mnt/offline/checksums.txt
```
### 3.7Automation tips
- Schedule with `cron` or platform scheduler and fail the job when exit code ≥11.
- Pair with `stella sources ingest --dry-run` for pre-flight validation before re-enabling a paused source.
- Push JSON exports to observability pipelines for historical tracking of violation counts.
### 3.8Offline notes
- Works against Offline Kit Mongo snapshots when CLI is pointed at the local API gateway included in the bundle.
- When fully disconnected, run against exported `aoc verify` reports generated on production and replay them using `--format json --export` (automation recipe above).
- Include verification output in compliance packages alongside Offline Kit manifests.
---
## 4·Global exit-code reference
| Code | Summary |
|------|---------|
| `0` | Success / no violations. |
| `11` | `ERR_AOC_001` Forbidden field present. |
| `12` | `ERR_AOC_002` Merge attempt detected. |
| `13` | `ERR_AOC_003` Idempotency violation. |
| `14` | `ERR_AOC_004` Missing provenance/signature metadata. |
| `15` | `ERR_AOC_005` Signature/checksum mismatch. |
| `16` | `ERR_AOC_006` Effective findings in ingestion payload. |
| `17` | `ERR_AOC_007` Schema violation / unknown fields. |
| `18` | Partial verification (limit reached). |
| `70` | Transport or HTTP failure. |
| `71` | CLI usage error (invalid arguments, missing tenant). |
Use these codes in CI to map outcomes to build statuses or alert severities.
---
## 4·`stella vuln observations` (Overlay paging)
`stella vuln observations` lists raw advisory observations for downstream overlays (Graph Explorer, Policy simulations, Console). Large tenants can now page through results deterministically.
| Option | Description |
|--------|-------------|
| `--limit <count>` | Caps the number of observations returned in a single call. Defaults to `200`; values above `500` are clamped server-side. |
| `--cursor <token>` | Opaque continuation token produced by the previous page (`nextCursor` in JSON output). Pass it back to resume iteration. |
Additional notes:
- Table mode prints a hint when `hasMore` is `true`:
`[yellow]More observations available. Continue with --cursor <token>[/]`.
- JSON mode returns `nextCursor` and `hasMore` alongside the observation list so automation can loop until `hasMore` is `false`.
- Supplying a non-positive limit falls back to the default (`200`). Invalid/expired cursors yield `400 Bad Request`; restart without `--cursor` to begin a fresh iteration.
---
## 5·Related references
---
## 1·Prerequisites
- CLI version: `stella`0.19.0 (AOC feature gate enabled).
- Required scopes (DPoP-bound):
- `advisory:read` for Concelier sources.
- `vex:read` for Excititor sources (optional but required for VEX checks).
- `aoc:verify` to invoke guard verification endpoints.
- `tenant:select` if your deployment uses tenant switching.
- Connectivity: direct access to Concelier/Excititor APIs or Offline Kit snapshot (see §4).
- Environment: set `STELLA_AUTHORITY_URL`, `STELLA_TENANT`, and export a valid OpTok via `stella auth login` or existing token cache.
---
## 2·`stella sources ingest --dry-run`
### 2.1Synopsis
```bash
stella sources ingest --dry-run \
--source <source-key> \
--input <path-or-uri> \
[--tenant <tenant-id>] \
[--format json|table] \
[--no-color] \
[--output <file>]
```
### 2.2Description
Previews an ingestion write without touching MongoDB. The command loads an upstream advisory or VEX document, computes the would-write payload, runs it through the `AOCWriteGuard`, and reports any forbidden fields, provenance gaps, or idempotency issues. Use it during connector development, CI validation, or while triaging incidents.
### 2.3Options
| Option | Description |
|--------|-------------|
| `--source <source-key>` | Logical source name (`redhat`, `ubuntu`, `osv`, etc.). Mirrors connector configuration. |
| `--input <path-or-uri>` | Path to local CSAF/OSV/VEX file or HTTPS URI. CLI normalises transport (gzip/base64) before guard evaluation. |
| `--tenant <tenant-id>` | Overrides default tenant for multi-tenant deployments. Mandatory when `STELLA_TENANT` is not set. |
| `--format json|table` | Output format. `table` (default) prints summary with highlighted violations; `json` emits machine-readable report (see below). |
| `--no-color` | Disables ANSI colour output for CI logs. |
| `--output <file>` | Writes the JSON report to file while still printing human-readable summary to stdout. |
### 2.4Output schema (JSON)
```json
{
"source": "redhat",
"tenant": "default",
"guardVersion": "1.0.0",
"status": "ok",
"document": {
"contentHash": "sha256:…",
"supersedes": null,
"provenance": {
"signature": { "format": "pgp", "present": true }
}
},
"violations": []
}
```
When violations exist, `status` becomes `error` and `violations` contains entries with `code` (`ERR_AOC_00x`), a short `message`, and JSON Pointer `path` values indicating offending fields.
### 2.5Exit codes
| Exit code | Meaning |
|-----------|---------|
| `0` | Guard passed; would-write payload is AOC compliant. |
| `11` | `ERR_AOC_001` Forbidden field (`severity`, `cvss`, etc.) detected. |
| `12` | `ERR_AOC_002` Merge attempt (multiple upstream sources fused). |
| `13` | `ERR_AOC_003` Idempotency violation (duplicate without supersedes). |
| `14` | `ERR_AOC_004` Missing provenance fields. |
| `15` | `ERR_AOC_005` Signature/checksum mismatch. |
| `16` | `ERR_AOC_006` Effective findings present (Policy-only data). |
| `17` | `ERR_AOC_007` Unknown top-level fields / schema violation. |
| `70` | Transport error (network, auth, malformed input). |
> Exit codes map directly to the `ERR_AOC_00x` table for scripting consistency. Multiple violations yield the highest-priority code (e.g., 11 takes precedence over 14).
### 2.6Examples
Dry-run a local CSAF file:
```bash
stella sources ingest --dry-run \
--source redhat \
--input ./fixtures/redhat/RHSA-2025-1234.json
```
Stream from HTTPS and emit JSON for CI:
```bash
stella sources ingest --dry-run \
--source osv \
--input https://osv.dev/vulnerability/GHSA-aaaa-bbbb \
--format json \
--output artifacts/osv-dry-run.json
cat artifacts/osv-dry-run.json | jq '.violations'
```
### 2.7Offline notes
When operating in sealed/offline mode:
- Use `--input` paths pointing to Offline Kit snapshots (`offline-kit/advisories/*.json`).
- Provide `--tenant` explicitly if the offline bundle contains multiple tenants.
- The command does not attempt network access when given a file path.
- Store reports with `--output` to include in transfer packages for policy review.
---
## 3·`stella aoc verify`
### 3.1Synopsis
```bash
stella aoc verify \
[--since <iso8601|duration>] \
[--limit <count>] \
[--sources <list>] \
[--codes <ERR_AOC_00x,...>] \
[--format table|json] \
[--export <file>] \
[--tenant <tenant-id>] \
[--no-color]
```
### 3.2Description
Replays the AOC guard against stored raw documents. By default it checks all advisories and VEX statements ingested in the last 24hours for the active tenant, reporting totals, top violation codes, and sample documents. Use it in CI pipelines, scheduled verifications, or during incident response.
### 3.3Options
| Option | Description |
|--------|-------------|
| `--since <value>` | Verification window. Accepts ISO8601 timestamp (`2025-10-25T12:00:00Z`) or duration (`48h`, `7d`). Defaults to `24h`. |
| `--limit <count>` | Maximum number of violations to display (per code). `0` means show all. Defaults to `20`. |
| `--sources <list>` | Comma-separated list of sources (`redhat,ubuntu,osv`). Filters both advisories and VEX entries. |
| `--codes <list>` | Restricts output to specific `ERR_AOC_00x` codes. Useful for regression tracking. |
| `--format table|json` | `table` (default) prints summary plus top violations; `json` outputs machine-readable report identical to the `/aoc/verify` API. |
| `--export <file>` | Writes the JSON report to disk (useful for audits/offline uploads). |
| `--tenant <tenant-id>` | Overrides tenant context. Required for cross-tenant verifications when run by platform operators. |
| `--no-color` | Disables ANSI colours. |
`table` mode prints a summary showing the active tenant, evaluated window, counts of checked advisories/VEX statements, the active limit, total writes/violations, and whether the page was truncated. Status is colour-coded as `ok`, `violations`, or `truncated`. When violations exist the detail table lists the code, total occurrences, first sample document (`source` + `documentId` + `contentHash`), and JSON pointer path.
### 3.4Report structure (JSON)
```json
{
"tenant": "default",
"window": {
"from": "2025-10-25T12:00:00Z",
"to": "2025-10-26T12:00:00Z"
},
"checked": {
"advisories": 482,
"vex": 75
},
"violations": [
{
"code": "ERR_AOC_001",
"count": 2,
"examples": [
{
"source": "redhat",
"documentId": "advisory_raw:redhat:RHSA-2025:1",
"contentHash": "sha256:…",
"path": "/content/raw/cvss"
}
]
}
],
"metrics": {
"ingestion_write_total": 557,
"aoc_violation_total": 2
},
"truncated": false
}
```
### 3.5Exit codes
| Exit code | Meaning |
|-----------|---------|
| `0` | Verification succeeded with zero violations. |
| `11…17` | Same mapping as §2.5 when violations are detected. Highest-priority code returned. |
| `18` | Verification ran but results truncated (limit reached) treat as warning; rerun with higher `--limit`. |
| `70` | Transport/authentication error. |
| `71` | CLI misconfiguration (missing tenant, invalid `--since`, etc.). |
### 3.6Examples
Daily verification across all sources:
```bash
stella aoc verify --since 24h --format table
```
CI pipeline focusing on errant sources and exporting evidence:
```bash
stella aoc verify \
--sources redhat,ubuntu \
--codes ERR_AOC_001,ERR_AOC_004 \
--format json \
--limit 100 \
--export artifacts/aoc-verify.json
jq '.violations[] | {code, count}' artifacts/aoc-verify.json
```
Air-gapped verification using Offline Kit snapshot (example script):
```bash
stella aoc verify \
--since 7d \
--format json \
--export /mnt/offline/aoc-verify-$(date +%F).json
sha256sum /mnt/offline/aoc-verify-*.json > /mnt/offline/checksums.txt
```
### 3.7Automation tips
- Schedule with `cron` or platform scheduler and fail the job when exit code ≥11.
- Pair with `stella sources ingest --dry-run` for pre-flight validation before re-enabling a paused source.
- Push JSON exports to observability pipelines for historical tracking of violation counts.
### 3.8Offline notes
- Works against Offline Kit Mongo snapshots when CLI is pointed at the local API gateway included in the bundle.
- When fully disconnected, run against exported `aoc verify` reports generated on production and replay them using `--format json --export` (automation recipe above).
- Include verification output in compliance packages alongside Offline Kit manifests.
---
## 4·Global exit-code reference
| Code | Summary |
|------|---------|
| `0` | Success / no violations. |
| `11` | `ERR_AOC_001` Forbidden field present. |
| `12` | `ERR_AOC_002` Merge attempt detected. |
| `13` | `ERR_AOC_003` Idempotency violation. |
| `14` | `ERR_AOC_004` Missing provenance/signature metadata. |
| `15` | `ERR_AOC_005` Signature/checksum mismatch. |
| `16` | `ERR_AOC_006` Effective findings in ingestion payload. |
| `17` | `ERR_AOC_007` Schema violation / unknown fields. |
| `18` | Partial verification (limit reached). |
| `70` | Transport or HTTP failure. |
| `71` | CLI usage error (invalid arguments, missing tenant). |
Use these codes in CI to map outcomes to build statuses or alert severities.
---
## 4·`stella vuln observations` (Overlay paging)
`stella vuln observations` lists raw advisory observations for downstream overlays (Graph Explorer, Policy simulations, Console). Large tenants can now page through results deterministically.
| Option | Description |
|--------|-------------|
| `--limit <count>` | Caps the number of observations returned in a single call. Defaults to `200`; values above `500` are clamped server-side. |
| `--cursor <token>` | Opaque continuation token produced by the previous page (`nextCursor` in JSON output). Pass it back to resume iteration. |
Additional notes:
- Table mode prints a hint when `hasMore` is `true`:
`[yellow]More observations available. Continue with --cursor <token>[/]`.
- JSON mode returns `nextCursor` and `hasMore` alongside the observation list so automation can loop until `hasMore` is `false`.
- Supplying a non-positive limit falls back to the default (`200`). Invalid/expired cursors yield `400 Bad Request`; restart without `--cursor` to begin a fresh iteration.
---
## 5·Related references
- [Aggregation-Only Contract reference](../../../ingestion/aggregation-only-contract.md)
- [Architecture overview](../../platform/architecture-overview.md)
- [Console AOC dashboard](../../../ui/console.md)
- [Authority scopes](../../authority/architecture.md)
- [Task Pack CLI profiles](./packs-profiles.md)
---
---
## 6·Compliance checklist
- [ ] Usage documented for both table and JSON formats.
@@ -320,7 +320,7 @@ All publish/promote operations require interactive identities with `policy:publi
*Last updated: 2025-11-03 (Sprint100).*
## 13. Authority configuration quick reference
| Setting | Purpose | How to set |
|---------|---------|------------|
| `StellaOps:Authority:OperatorReason` | Incident/change description recorded with `orch:operate` tokens. | CLI flag `--Authority:OperatorReason=...` or env `STELLAOPS_ORCH_REASON`. |
@@ -332,4 +332,4 @@ All publish/promote operations require interactive identities with `policy:publi
| `StellaOps:Authority:Scope` | Default scope string requested during `stella auth login`. | CLI flag `--Authority:Scope=\"packs.read packs.run\"` or env `STELLAOPS_AUTHORITY_SCOPE`; see `docs/modules/cli/guides/packs-profiles.md` for common Task Pack profiles. |
> Tokens requesting `orch:operate` fail with `invalid_request` unless both operator values are present. `orch:quota` tokens require `quota_reason` (≤256 chars) and accept an optional `quota_ticket` (≤128 chars). `orch:backfill` tokens require both `backfill_reason` (≤256 chars) and `backfill_ticket` (≤128 chars). Avoid embedding secrets in any value.

View File

@@ -1,35 +1,35 @@
# DevOps agent guide
## Mission
The DevOps module captures release, deployment, and migration playbooks that keep StellaOps deterministic across environments.
## Key docs
- [Module README](./README.md)
- [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md)
- [Task Runner simulation notes](./task-runner-simulation.md)
## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change.
## Required Reading
- `docs/modules/devops/README.md`
- `docs/modules/devops/architecture.md`
- `docs/modules/devops/implementation_plan.md`
- `docs/modules/platform/architecture-overview.md`
## Working Agreement
- 1. Update task status to `DOING`/`DONE` in both `docs/implplan/SPRINTS.md` and the local `TASKS.md` when you start or finish work.
- 2. Review this charter and the Required Reading documents before coding; confirm prerequisites are met.
- 3. Keep changes deterministic (stable ordering, timestamps, hashes) and align with offline/air-gap expectations.
- 4. Coordinate doc updates, tests, and cross-guild communication whenever contracts or workflows change.
- 5. Revert to `TODO` if you pause the task without shipping changes; leave notes in commit/PR descriptions for context.
# DevOps agent guide
## Mission
The DevOps module captures release, deployment, and migration playbooks that keep StellaOps deterministic across environments.
## Key docs
- [Module README](./README.md)
- [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md)
- [Task Runner simulation notes](./task-runner-simulation.md)
## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change.
## Required Reading
- `docs/modules/devops/README.md`
- `docs/modules/devops/architecture.md`
- `docs/modules/devops/implementation_plan.md`
- `docs/modules/platform/architecture-overview.md`
## Working Agreement
- 1. Update task status to `DOING`/`DONE` in both `docs/implplan/SPRINTS.md` and the local `TASKS.md` when you start or finish work.
- 2. Review this charter and the Required Reading documents before coding; confirm prerequisites are met.
- 3. Keep changes deterministic (stable ordering, timestamps, hashes) and align with offline/air-gap expectations.
- 4. Coordinate doc updates, tests, and cross-guild communication whenever contracts or workflows change.
- 5. Revert to `TODO` if you pause the task without shipping changes; leave notes in commit/PR descriptions for context.

View File

@@ -1,42 +1,42 @@
# StellaOps DevOps
The DevOps module captures release, deployment, and migration playbooks that keep StellaOps deterministic across environments.
## Responsibilities
- Maintain CI pipelines, signing workflows, and release packaging steps.
- Operate shared runbooks for launch readiness, upgrades, and NuGet previews.
- Provide offline kit assembly instructions and tooling integration.
- Wrap observability/telemetry bootstrap flows for platform teams.
## Key components
- Runbooks under ./runbooks/ (launch, deployment, nuget).
- Migration guidance under ./migrations/.
- Architecture overview bridging CI/CD & infrastructure concerns.
## Integrations & dependencies
- Ops pipelines (Gitea, GitHub Actions) and artifact registries.
- Authority/Signer for supply chain signing.
- Telemetry stack bootstrap scripts.
## Operational notes
- Offline bundle packaging guidance in docs/modules/export-center/operations/runbook.md.
- Dashboards for launch cutover rehearsals.
- Coordination with Security for enforced guardrails.
## Related resources
- ./runbooks/launch-readiness.md
- ./runbooks/launch-cutover.md
- ./runbooks/deployment-upgrade.md
- ./runbooks/nuget-preview-bootstrap.md
- ./migrations/semver-style.md
- ./task-runner-simulation.md
## Backlog references
- DEVOPS-LAUNCH-18-001 / 18-900 runbooks in ../../TASKS.md.
- Telemetry bootstrap automation tracked in `ops/devops/TASKS.md`.
## Epic alignment
- **Epic 1 AOC enforcement:** bake AOC verifier steps, CI guards, and schema validation into pipelines.
- **Epic 9 Orchestrator Dashboard:** support operational dashboards, job recovery runbooks, and rate-limit governance.
- **Epic 10 Export Center:** manage signing workflows, Offline Kit packaging, and release promotion for exports.
- **Epic 15 Observability & Forensics:** coordinate telemetry deployment, evidence retention, and forensic automation.
# StellaOps DevOps
The DevOps module captures release, deployment, and migration playbooks that keep StellaOps deterministic across environments.
## Responsibilities
- Maintain CI pipelines, signing workflows, and release packaging steps.
- Operate shared runbooks for launch readiness, upgrades, and NuGet previews.
- Provide offline kit assembly instructions and tooling integration.
- Wrap observability/telemetry bootstrap flows for platform teams.
## Key components
- Runbooks under ./runbooks/ (launch, deployment, nuget).
- Migration guidance under ./migrations/.
- Architecture overview bridging CI/CD & infrastructure concerns.
## Integrations & dependencies
- Ops pipelines (Gitea, GitHub Actions) and artifact registries.
- Authority/Signer for supply chain signing.
- Telemetry stack bootstrap scripts.
## Operational notes
- Offline bundle packaging guidance in docs/modules/export-center/operations/runbook.md.
- Dashboards for launch cutover rehearsals.
- Coordination with Security for enforced guardrails.
## Related resources
- ./runbooks/launch-readiness.md
- ./runbooks/launch-cutover.md
- ./runbooks/deployment-upgrade.md
- ./runbooks/nuget-preview-bootstrap.md
- ./migrations/semver-style.md
- ./task-runner-simulation.md
## Backlog references
- DEVOPS-LAUNCH-18-001 / 18-900 runbooks in ../../TASKS.md.
- Telemetry bootstrap automation tracked in `ops/devops/TASKS.md`.
## Epic alignment
- **Epic 1 AOC enforcement:** bake AOC verifier steps, CI guards, and schema validation into pipelines.
- **Epic 9 Orchestrator Dashboard:** support operational dashboards, job recovery runbooks, and rate-limit governance.
- **Epic 10 Export Center:** manage signing workflows, Offline Kit packaging, and release promotion for exports.
- **Epic 15 Observability & Forensics:** coordinate telemetry deployment, evidence retention, and forensic automation.

View File

@@ -1,139 +1,139 @@
# StellaOps Architecture Overview (Sprint19)
> **Ownership:** Architecture Guild • Docs Guild
> **Audience:** Service owners, platform engineers, solution architects
> **Related:** [High-Level Architecture](../../07_HIGH_LEVEL_ARCHITECTURE.md), [Concelier Architecture](../concelier/architecture.md), [Policy Engine Architecture](../policy/architecture.md), [Aggregation-Only Contract](../../ingestion/aggregation-only-contract.md)
This dossier summarises the end-to-end runtime topology after the Aggregation-Only Contract (AOC) rollout. It highlights where raw facts live, how ingest services enforce guardrails, and how downstream components consume those facts to derive policy decisions and user-facing experiences.
---
## 1·System landscape
```mermaid
graph TD
subgraph Edge["Clients & Automation"]
CLI[stella CLI]
UI[Console SPA]
APIClients[CI / API Clients]
end
Gateway[API Gateway<br/>(JWT + DPoP scopes)]
subgraph Scanner["Fact Collection"]
ScannerWeb[Scanner.WebService]
ScannerWorkers[Scanner.Workers]
Agent[Agent Runtime]
end
subgraph Ingestion["Aggregation-Only Ingestion (AOC)"]
Concelier[Concelier.WebService]
Excititor[Excititor.WebService]
RawStore[(MongoDB<br/>advisory_raw / vex_raw)]
end
subgraph Derivation["Policy & Overlay"]
Policy[Policy Engine]
Scheduler[Scheduler Services]
Notify[Notifier]
end
subgraph Experience["UX & Export"]
UIService[Console Backend]
Exporters[Export / Offline Kit]
end
Observability[Telemetry Stack]
CLI --> Gateway
UI --> Gateway
APIClients --> Gateway
Gateway --> ScannerWeb
ScannerWeb --> ScannerWorkers
ScannerWorkers --> Concelier
ScannerWorkers --> Excititor
Concelier --> RawStore
Excititor --> RawStore
RawStore --> Policy
Policy --> Scheduler
Policy --> Notify
Policy --> UIService
Scheduler --> UIService
UIService --> Exporters
Exporters --> CLI
Exporters --> Offline[Offline Kit]
Observability -.-> ScannerWeb
Observability -.-> Concelier
Observability -.-> Excititor
Observability -.-> Policy
Observability -.-> Scheduler
Observability -.-> Notify
```
Key boundaries:
- **AOC border.** Everything inside the Ingestion subgraph writes only immutable raw facts plus link hints. Derived severity, consensus, and risk remain outside the border.
- **Policy-only derivation.** Policy Engine materialises `effective_finding_*` collections and emits overlays; other services consume but never mutate them.
- **Tenant enforcement.** Authority-issued DPoP scopes flow through Gateway to every service; raw stores and overlays include `tenant` strictly.
---
## 2·Aggregation-Only Contract focus
### 2.1 Responsibilities at the boundary
| Area | Services | Responsibilities under AOC | Forbidden under AOC |
|------|----------|-----------------------------|---------------------|
| **Ingestion (Concelier / Excititor)** | `StellaOps.Concelier.WebService`, `StellaOps.Excititor.WebService` | Fetch upstream advisories/VEX, verify signatures, compute linksets, append immutable documents to `advisory_raw` / `vex_raw`, emit observability signals, expose raw read APIs. | Computing severity, consensus, suppressions, or policy hints; merging upstream sources into a single derived record; mutating existing documents. |
| **Policy & Overlay** | `StellaOps.Policy.Engine`, Scheduler | Join SBOM inventory with raw advisories/VEX, evaluate policies, issue `effective_finding_*` overlays, drive remediation workflows. | Writing to raw collections; bypassing guard scopes; running without recorded provenance. |
| **Experience layers** | Console, CLI, Exporters | Surface raw facts + policy overlays; run `stella aoc verify`; render AOC dashboards and reports. | Accepting ingestion payloads that lack provenance or violate guard results. |
### 2.2 Raw stores
| Collection | Purpose | Key fields | Notes |
|------------|---------|------------|-------|
| `advisory_raw` | Immutable vendor/ecosystem advisory documents. | `_id`, `tenant`, `source.*`, `upstream.*`, `content.raw`, `linkset`, `supersedes`. | Idempotent by `(source.vendor, upstream.upstream_id, upstream.content_hash)`. |
| `vex_raw` | Immutable vendor VEX statements. | Mirrors `advisory_raw`; `identifiers.statements` summarises affected components. | Maintains supersedes chain identical to advisory flow. |
| Change streams (`advisory_raw_stream`, `vex_raw_stream`) | Feed Policy Engine and Scheduler. | `operationType`, `documentKey`, `fullDocument`, `tenant`, `traceId`. | Scope filtered per tenant before delivery. |
### 2.3 Guarded ingestion sequence
```mermaid
sequenceDiagram
participant Upstream as Upstream Source
participant Connector as Concelier/Excititor Connector
participant Guard as AOCWriteGuard
participant Mongo as MongoDB (advisory_raw / vex_raw)
participant Stream as Change Stream
participant Policy as Policy Engine
Upstream-->>Connector: CSAF / OSV / VEX document
Connector->>Connector: Normalize transport, compute content_hash
Connector->>Guard: Candidate raw doc (source + upstream + content + linkset)
Guard-->>Connector: ERR_AOC_00x on violation
Guard->>Mongo: Append immutable document (with tenant & supersedes)
Mongo-->>Stream: Change event (tenant scoped)
Stream->>Policy: Raw delta payload
Policy->>Policy: Evaluate policies, compute effective findings
```
---
### 2.4 Authority scopes & tenancy
| Scope | Holder | Purpose | Notes |
|-------|--------|---------|-------|
| `advisory:ingest` / `vex:ingest` | Concelier / Excititor collectors | Append raw documents through ingestion endpoints. | Paired with tenant claims; requests without tenant are rejected. |
| `advisory:read` / `vex:read` | DevOps verify identity, CLI | Run `stella aoc verify` or call `/aoc/verify`. | Read-only; cannot mutate raw docs. |
| `effective:write` | Policy Engine | Materialise `effective_finding_*` overlays. | Only Policy Engine identity may hold; ingestion contexts receive `ERR_AOC_006` if they attempt. |
| `findings:read` | Console, CLI, exports | Consume derived findings. | Enforced by Gateway and downstream services. |
---
## 3·Data & control flow highlights
1. **Ingestion:** Concelier / Excititor connectors fetch upstream documents, compute linksets, and hand payloads to `AOCWriteGuard`. Guards validate schema, provenance, forbidden fields, supersedes pointers, and append-only rules before writing to Mongo.
2. **Verification:** `stella aoc verify` (CLI/CI) and `/aoc/verify` endpoints replay guard checks against stored documents, mapping `ERR_AOC_00x` codes to exit codes for automation.
3. **Policy evaluation:** Mongo change streams deliver tenant-scoped raw deltas. Policy Engine joins SBOM inventory (via BOM Index), executes deterministic policies, writes overlays, and emits events to Scheduler/Notify.
4. **Experience surfaces:** Console renders an AOC dashboard showing ingestion latency, guard violations, and supersedes depth. CLI exposes raw-document fetch helpers for auditing. Offline Kit bundles raw collections alongside guard configs to keep air-gapped installs verifiable.
5. **Observability:** All services emit `ingestion_write_total`, `aoc_violation_total{code}`, `ingestion_latency_seconds`, and trace spans `ingest.fetch`, `ingest.transform`, `ingest.write`, `aoc.guard`. Logs correlate via `traceId`, `tenant`, `source.vendor`, and `content_hash`.
---
# StellaOps Architecture Overview (Sprint19)
> **Ownership:** Architecture Guild • Docs Guild
> **Audience:** Service owners, platform engineers, solution architects
> **Related:** [High-Level Architecture](../../07_HIGH_LEVEL_ARCHITECTURE.md), [Concelier Architecture](../concelier/architecture.md), [Policy Engine Architecture](../policy/architecture.md), [Aggregation-Only Contract](../../ingestion/aggregation-only-contract.md)
This dossier summarises the end-to-end runtime topology after the Aggregation-Only Contract (AOC) rollout. It highlights where raw facts live, how ingest services enforce guardrails, and how downstream components consume those facts to derive policy decisions and user-facing experiences.
---
## 1·System landscape
```mermaid
graph TD
subgraph Edge["Clients & Automation"]
CLI[stella CLI]
UI[Console SPA]
APIClients[CI / API Clients]
end
Gateway[API Gateway<br/>(JWT + DPoP scopes)]
subgraph Scanner["Fact Collection"]
ScannerWeb[Scanner.WebService]
ScannerWorkers[Scanner.Workers]
Agent[Agent Runtime]
end
subgraph Ingestion["Aggregation-Only Ingestion (AOC)"]
Concelier[Concelier.WebService]
Excititor[Excititor.WebService]
RawStore[(MongoDB<br/>advisory_raw / vex_raw)]
end
subgraph Derivation["Policy & Overlay"]
Policy[Policy Engine]
Scheduler[Scheduler Services]
Notify[Notifier]
end
subgraph Experience["UX & Export"]
UIService[Console Backend]
Exporters[Export / Offline Kit]
end
Observability[Telemetry Stack]
CLI --> Gateway
UI --> Gateway
APIClients --> Gateway
Gateway --> ScannerWeb
ScannerWeb --> ScannerWorkers
ScannerWorkers --> Concelier
ScannerWorkers --> Excititor
Concelier --> RawStore
Excititor --> RawStore
RawStore --> Policy
Policy --> Scheduler
Policy --> Notify
Policy --> UIService
Scheduler --> UIService
UIService --> Exporters
Exporters --> CLI
Exporters --> Offline[Offline Kit]
Observability -.-> ScannerWeb
Observability -.-> Concelier
Observability -.-> Excititor
Observability -.-> Policy
Observability -.-> Scheduler
Observability -.-> Notify
```
Key boundaries:
- **AOC border.** Everything inside the Ingestion subgraph writes only immutable raw facts plus link hints. Derived severity, consensus, and risk remain outside the border.
- **Policy-only derivation.** Policy Engine materialises `effective_finding_*` collections and emits overlays; other services consume but never mutate them.
- **Tenant enforcement.** Authority-issued DPoP scopes flow through Gateway to every service; raw stores and overlays include `tenant` strictly.
---
## 2·Aggregation-Only Contract focus
### 2.1 Responsibilities at the boundary
| Area | Services | Responsibilities under AOC | Forbidden under AOC |
|------|----------|-----------------------------|---------------------|
| **Ingestion (Concelier / Excititor)** | `StellaOps.Concelier.WebService`, `StellaOps.Excititor.WebService` | Fetch upstream advisories/VEX, verify signatures, compute linksets, append immutable documents to `advisory_raw` / `vex_raw`, emit observability signals, expose raw read APIs. | Computing severity, consensus, suppressions, or policy hints; merging upstream sources into a single derived record; mutating existing documents. |
| **Policy & Overlay** | `StellaOps.Policy.Engine`, Scheduler | Join SBOM inventory with raw advisories/VEX, evaluate policies, issue `effective_finding_*` overlays, drive remediation workflows. | Writing to raw collections; bypassing guard scopes; running without recorded provenance. |
| **Experience layers** | Console, CLI, Exporters | Surface raw facts + policy overlays; run `stella aoc verify`; render AOC dashboards and reports. | Accepting ingestion payloads that lack provenance or violate guard results. |
### 2.2 Raw stores
| Collection | Purpose | Key fields | Notes |
|------------|---------|------------|-------|
| `advisory_raw` | Immutable vendor/ecosystem advisory documents. | `_id`, `tenant`, `source.*`, `upstream.*`, `content.raw`, `linkset`, `supersedes`. | Idempotent by `(source.vendor, upstream.upstream_id, upstream.content_hash)`. |
| `vex_raw` | Immutable vendor VEX statements. | Mirrors `advisory_raw`; `identifiers.statements` summarises affected components. | Maintains supersedes chain identical to advisory flow. |
| Change streams (`advisory_raw_stream`, `vex_raw_stream`) | Feed Policy Engine and Scheduler. | `operationType`, `documentKey`, `fullDocument`, `tenant`, `traceId`. | Scope filtered per tenant before delivery. |
### 2.3 Guarded ingestion sequence
```mermaid
sequenceDiagram
participant Upstream as Upstream Source
participant Connector as Concelier/Excititor Connector
participant Guard as AOCWriteGuard
participant Mongo as MongoDB (advisory_raw / vex_raw)
participant Stream as Change Stream
participant Policy as Policy Engine
Upstream-->>Connector: CSAF / OSV / VEX document
Connector->>Connector: Normalize transport, compute content_hash
Connector->>Guard: Candidate raw doc (source + upstream + content + linkset)
Guard-->>Connector: ERR_AOC_00x on violation
Guard->>Mongo: Append immutable document (with tenant & supersedes)
Mongo-->>Stream: Change event (tenant scoped)
Stream->>Policy: Raw delta payload
Policy->>Policy: Evaluate policies, compute effective findings
```
---
### 2.4 Authority scopes & tenancy
| Scope | Holder | Purpose | Notes |
|-------|--------|---------|-------|
| `advisory:ingest` / `vex:ingest` | Concelier / Excititor collectors | Append raw documents through ingestion endpoints. | Paired with tenant claims; requests without tenant are rejected. |
| `advisory:read` / `vex:read` | DevOps verify identity, CLI | Run `stella aoc verify` or call `/aoc/verify`. | Read-only; cannot mutate raw docs. |
| `effective:write` | Policy Engine | Materialise `effective_finding_*` overlays. | Only Policy Engine identity may hold; ingestion contexts receive `ERR_AOC_006` if they attempt. |
| `findings:read` | Console, CLI, exports | Consume derived findings. | Enforced by Gateway and downstream services. |
---
## 3·Data & control flow highlights
1. **Ingestion:** Concelier / Excititor connectors fetch upstream documents, compute linksets, and hand payloads to `AOCWriteGuard`. Guards validate schema, provenance, forbidden fields, supersedes pointers, and append-only rules before writing to Mongo.
2. **Verification:** `stella aoc verify` (CLI/CI) and `/aoc/verify` endpoints replay guard checks against stored documents, mapping `ERR_AOC_00x` codes to exit codes for automation.
3. **Policy evaluation:** Mongo change streams deliver tenant-scoped raw deltas. Policy Engine joins SBOM inventory (via BOM Index), executes deterministic policies, writes overlays, and emits events to Scheduler/Notify.
4. **Experience surfaces:** Console renders an AOC dashboard showing ingestion latency, guard violations, and supersedes depth. CLI exposes raw-document fetch helpers for auditing. Offline Kit bundles raw collections alongside guard configs to keep air-gapped installs verifiable.
5. **Observability:** All services emit `ingestion_write_total`, `aoc_violation_total{code}`, `ingestion_latency_seconds`, and trace spans `ingest.fetch`, `ingest.transform`, `ingest.write`, `aoc.guard`. Logs correlate via `traceId`, `tenant`, `source.vendor`, and `content_hash`.
---
## 4·Offline & disaster readiness
- **Offline Kit:** Packages raw Mongo snapshots (`advisory_raw`, `vex_raw`) plus guard configuration and CLI verifier binaries so air-gapped sites can re-run AOC checks before promotion.
@@ -177,10 +177,10 @@ sequenceDiagram
- [ ] Mongo schema validators deployed for `advisory_raw` and `vex_raw`; change streams scoped per tenant.
- [ ] Authority scopes (`advisory:*`, `vex:*`, `effective:*`) configured in Gateway and validated via integration tests.
- [ ] `stella aoc verify` wired into CI/CD pipelines with seeded violation fixtures.
- [ ] Console AOC dashboard and CLI documentation reference the new ingestion contract.
- [ ] Offline Kit bundles include guard configs, verifier tooling, and documentation updates.
- [ ] Observability dashboards include violation, latency, and supersedes depth metrics with alert thresholds.
---
- [ ] Console AOC dashboard and CLI documentation reference the new ingestion contract.
- [ ] Offline Kit bundles include guard configs, verifier tooling, and documentation updates.
- [ ] Observability dashboards include violation, latency, and supersedes depth metrics with alert thresholds.
---
*Last updated: 2025-11-03 (Replay planning refresh).*

View File

@@ -179,9 +179,10 @@ Determinism guard instrumentation wraps the evaluator, rejecting access to forbi
- **Queue:** Mongo queue with lease; each job assigned `leaseDuration`, `maxAttempts`.
- **Workers:** Lease jobs, execute evaluation pipeline, report status (success/failure/canceled). Failures with recoverable errors requeue with backoff; determinism or schema violations mark job `failed` and raise incident event.
- **Fairness:** Round-robin per `{tenant, policyId}`; emergency jobs (`priority=emergency`) jump queue but limited via circuit breaker.
- **Replay:** On demand, orchestrator rehydrates run via stored cursors and exports sealed bundle for audit/CI determinism checks.
---
- **Replay:** On demand, orchestrator rehydrates run via stored cursors and exports sealed bundle for audit/CI determinism checks.
- **Batch evaluation service (`/api/policy/eval/batch`):** Stateless evaluator powering Findings Ledger and replay/offline workflows. Requests contain canonical ledger events plus optional current projection; responses return status/severity/labels/rationale without mutating state. Policy Engine enforces per-tenant cost budgets, caches results by `(tenantId, policyVersion, eventHash, projectionHash)`, and falls back to inline evaluation when the remote service is disabled.
---
## 7·Security & Tenancy

View File

@@ -112,7 +112,7 @@ Failures throw `SurfaceEnvironmentException` with error codes (`SURFACE_ENV_MISS
## 6. Integration Guidance
- **Scanner Worker**: call `services.AddSurfaceEnvironment()` in `Program.cs` before registering analyzers. Pass `hostContext.Configuration.GetSection("Surface")` for overrides.
- **Scanner WebService**: build environment during startup, then expose selected values via diagnostics (`/internal/surface` when diagnostics enabled).
- **Scanner WebService**: build environment during startup using `AddSurfaceEnvironment`, `AddSurfaceValidation`, `AddSurfaceFileCache`, and `AddSurfaceSecrets`; readiness checks execute the validator runner and scan/report APIs emit Surface CAS pointers derived from the resolved configuration.
- **Zastava Observer/Webhook**: use the same builder; ensure Helm charts set `ZASTAVA_` variables.
- **Scheduler Planner (future)**: treat Surface.Env as read-only input; do not mutate settings.

View File

@@ -66,6 +66,14 @@ Surface.FS exposes a gRPC/HTTP API consumed by .NET clients:
.NET client wraps these calls and handles retries using Polly policies.
### WebService integration (2025-11-05)
- `/api/v1/scans/{id}` and `/api/v1/reports` responses now include a `surface` block containing:
- `manifestUri` `cas://` pointer to the Surface manifest JSON.
- `manifestDigest` canonical SHA-256 over the manifest payload.
- `manifest.artifacts[]` deterministic list with `kind`, `uri`, `digest`, `mediaType`, `format`, and optional `view`. URIs reuse the `ArtifactObjectKeyBuilder` semantics (`cas://{bucket}/{rootPrefix}/images/...`).
- This allows UI/CLI consumers to fetch manifests or artefacts without additional Surface.FS round-trips.
## 4. Library Responsibilities
Surface.FS library for .NET hosts provides:

View File

@@ -7,7 +7,7 @@
- See [`../findings-ledger/schema.md`](../findings-ledger/schema.md) for the canonical SQL schema, hashing strategy, and fixtures underpinning these collections.
- **Collections / tables**
- `finding_records` canonical, policy-derived findings enriched with advisory, VEX, SBOM, runtime context. Includes `policyVersion`, `advisoryRawIds`, `vexRawIds`, `sbomComponentId`, and `explainBundleRef`.
- `finding_records` canonical, policy-derived findings enriched with advisory, VEX, SBOM, runtime context. Includes `policyVersion`, `advisoryRawIds`, `vexRawIds`, `sbomComponentId`, `policyRationale` (array of explain bundle URIs or remediation notes returned by `/api/policy/eval/batch`), and `explainBundleRef`.
- `finding_history` append-only state transitions (`new`, `triaged`, `accepted_risk`, `remediated`, `false_positive`, etc.) with timestamps, actor, and justification.
- `triage_actions` discrete operator actions (comment, assignment, remediation note, ticket link) with immutable provenance.
- `remediation_plans` structured remediation steps (affected assets, deadlines, recommended fixes, auto-generated from SRM/AI hints).
@@ -17,7 +17,7 @@
## 2) Triage workflow
1. **Ingest effective findings** from Policy Engine (stream `policy.finding.delta`). Each delta updates `finding_records`, generates history entries, and triggers notification rules.
1. **Ingest effective findings** from Policy Engine (stream `policy.finding.delta`) and on-demand batch evaluations. Projection workers call `/api/policy/eval/batch` with ledger event payloads to receive status/severity/label/rationale updates before writing `finding_records`. Each delta updates `finding_records`, generates history entries, and triggers notification rules.
2. **Prioritisation** uses contextual heuristics: runtime exposure, VEX status, policy severity, AI hints. Stored as `priorityScore` with provenance from Zastava/AI modules.
3. **Assignment & collaboration** Operators claim findings, add comments, attach evidence, and link tickets. Assignment uses Authority identities and RBAC.
4. **Remediation tracking** Link remediation plans, record progress, and integrate with Scheduler for follow-up scans once fixes deploy.

View File

@@ -1,78 +1,78 @@
# Notifications Overview
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
Notifications Studio turns raw platform events into concise, tenant-scoped alerts that reach the right responders without overwhelming them. The service is sovereign/offline-first, follows the Aggregation-Only Contract (AOC), and produces deterministic outputs so the same configuration yields identical deliveries across environments.
---
## 1. Mission & value
- **Reduce noise.** Only materially new or high-impact changes reach chat, email, or webhooks thanks to rule filters, throttles, and digest windows.
- **Explainable results.** Every delivery is traceable back to a rule, action, and event payload stored in the delivery ledger; operators can audit what fired and why.
- **Safe by default.** Secrets remain in external stores, templates are sandboxed, quiet hours and throttles prevent storms, and idempotency guarantees protect downstream systems.
- **Offline-aligned.** All configuration, templates, and plug-ins ship with Offline Kits; no external SaaS is required to send notifications.
---
## 2. Core capabilities
| Capability | What it does | Key docs |
|------------|--------------|----------|
| Rules engine | Declarative matchers for event kinds, severities, namespaces, VEX context, KEV flags, and more. | [`notifications/rules.md`](rules.md) |
| Channel catalog | Slack, Teams, Email, Webhook connectors loaded via restart-time plug-ins; metadata stored without secrets. | [`notifications/architecture.md`](architecture.md) |
# Notifications Overview
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
Notifications Studio turns raw platform events into concise, tenant-scoped alerts that reach the right responders without overwhelming them. The service is sovereign/offline-first, follows the Aggregation-Only Contract (AOC), and produces deterministic outputs so the same configuration yields identical deliveries across environments.
---
## 1. Mission & value
- **Reduce noise.** Only materially new or high-impact changes reach chat, email, or webhooks thanks to rule filters, throttles, and digest windows.
- **Explainable results.** Every delivery is traceable back to a rule, action, and event payload stored in the delivery ledger; operators can audit what fired and why.
- **Safe by default.** Secrets remain in external stores, templates are sandboxed, quiet hours and throttles prevent storms, and idempotency guarantees protect downstream systems.
- **Offline-aligned.** All configuration, templates, and plug-ins ship with Offline Kits; no external SaaS is required to send notifications.
---
## 2. Core capabilities
| Capability | What it does | Key docs |
|------------|--------------|----------|
| Rules engine | Declarative matchers for event kinds, severities, namespaces, VEX context, KEV flags, and more. | [`notifications/rules.md`](rules.md) |
| Channel catalog | Slack, Teams, Email, Webhook connectors loaded via restart-time plug-ins; metadata stored without secrets. | [`notifications/architecture.md`](architecture.md) |
| Templates | Locale-aware, deterministic rendering via safe helpers; channel defaults plus tenant-specific overrides. | [`notifications/templates.md`](templates.md) |
| Digests | Coalesce bursts into periodic summaries with deterministic IDs and audit trails. | [`notifications/digests.md`](digests.md) |
| Delivery ledger | Tracks rendered payload hashes, attempts, throttles, and outcomes for every action. | [`modules/notify/architecture.md`](../modules/notify/architecture.md#7-data-model-mongo) |
| Ack tokens | DSSE-signed acknowledgement tokens with webhook allowlists and escalation guardrails enforced by Authority. | [`modules/notify/architecture.md`](../modules/notify/architecture.md#81-ack-tokens--escalation-workflows) |
---
## 3. How it fits into StellaOps
1. **Producers emit events.** Scanner, Scheduler, VEX Lens, Attestor, and Zastava publish canonical envelopes (`NotifyEvent`) onto the internal bus.
2. **Notify.Worker evaluates rules.** For each tenant, the worker applies match filters, VEX gates, throttles, and digest policies before rendering the action.
3. **Connectors deliver.** Channel plug-ins send the rendered payload to Slack/Teams/Email/Webhook targets and report back attempts and outcomes.
4. **Consumers investigate.** Operators pivot from message links into Console dashboards, SBOM views, or policy overlays with correlation IDs preserved.
---
## 3. How it fits into StellaOps
1. **Producers emit events.** Scanner, Scheduler, VEX Lens, Attestor, and Zastava publish canonical envelopes (`NotifyEvent`) onto the internal bus.
2. **Notify.Worker evaluates rules.** For each tenant, the worker applies match filters, VEX gates, throttles, and digest policies before rendering the action.
3. **Connectors deliver.** Channel plug-ins send the rendered payload to Slack/Teams/Email/Webhook targets and report back attempts and outcomes.
4. **Consumers investigate.** Operators pivot from message links into Console dashboards, SBOM views, or policy overlays with correlation IDs preserved.
The Notify WebService fronts worker state with REST APIs used by the UI and CLI. Tenants authenticate via StellaOps Authority scopes `notify.viewer`, `notify.operator`, and (for escalated actions) `notify.admin`. All operations require the tenant header (`X-StellaOps-Tenant`) to preserve sovereignty boundaries.
---
## 4. Operating model
| Area | Guidance |
|------|----------|
---
## 4. Operating model
| Area | Guidance |
|------|----------|
| **Tenancy** | Each rule, channel, template, and delivery belongs to exactly one tenant. Cross-tenant sharing is intentionally unsupported. |
| **Determinism** | Configuration persistence normalises strings and sorts collections. Template rendering produces identical `bodyHash` values when inputs match. |
| **Scaling** | Workers scale horizontally; per-tenant rule snapshots are cached and refreshed from Mongo change streams. Redis (or equivalent) guards throttles and locks. |
| **Offline** | Offline Kits include plug-ins, default templates, and seed rules. Operators can edit YAML/JSON manifests before air-gapped deployment. |
| **Security** | Channel secrets use indirection (`secretRef`), Authority-protected OAuth clients secure API access, and delivery payloads are redacted before storage where required. |
| **Module boundaries** | 2025-11-02 decision: keep `src/Notify/` as the shared notification toolkit and `src/Notifier/` as the Notifications Studio runtime host until a packaging RFC covers the implications of merging. |
---
## 5. Getting started (first 30 minutes)
| Step | Goal | Reference |
|------|------|-----------|
| 1 | Deploy Notify WebService + Worker with Mongo and Redis | [`modules/notify/architecture.md`](../modules/notify/architecture.md#1-runtime-shape--projects) |
---
## 5. Getting started (first 30 minutes)
| Step | Goal | Reference |
|------|------|-----------|
| 1 | Deploy Notify WebService + Worker with Mongo and Redis | [`modules/notify/architecture.md`](../modules/notify/architecture.md#1-runtime-shape--projects) |
| 2 | Register OAuth clients/scopes in Authority | [`etc/authority.yaml.sample`](../../etc/authority.yaml.sample) |
| 3 | Install channel plug-ins and capture secret references | [`plugins/notify`](../../plugins) |
| 4 | Create a tenant rule and test preview | [`POST /channels/{id}/test`](../modules/notify/architecture.md#8-external-apis-webservice) |
| 5 | Inspect deliveries and digests | `/api/v1/notify/deliveries`, `/api/v1/notify/digests` |
---
## 6. Alignment with implementation work
| Backlog item | Impact on docs | Status |
|--------------|----------------|--------|
| `NOTIFY-SVC-38-001..004` | Foundational correlation, throttling, simulation hooks. | **In progress** align behaviour once services publish beta APIs. |
| `NOTIFY-SVC-39-001..004` | Adds correlation engine, digest generator, simulation API, quiet hours. | **Pending** revisit rule/digest sections when these tasks merge. |
Action: coordinate with the Notifications Service Guild when `NOTIFY-SVC-39-001..004` land to validate payload fields, quiet-hours semantics, and any new connector metadata that should be documented here and in the channel-specific guides.
---
> **Imposed rule reminder:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
| 3 | Install channel plug-ins and capture secret references | [`plugins/notify`](../../plugins) |
| 4 | Create a tenant rule and test preview | [`POST /channels/{id}/test`](../modules/notify/architecture.md#8-external-apis-webservice) |
| 5 | Inspect deliveries and digests | `/api/v1/notify/deliveries`, `/api/v1/notify/digests` |
---
## 6. Alignment with implementation work
| Backlog item | Impact on docs | Status |
|--------------|----------------|--------|
| `NOTIFY-SVC-38-001..004` | Foundational correlation, throttling, simulation hooks. | **In progress** align behaviour once services publish beta APIs. |
| `NOTIFY-SVC-39-001..004` | Adds correlation engine, digest generator, simulation API, quiet hours. | **Pending** revisit rule/digest sections when these tasks merge. |
Action: coordinate with the Notifications Service Guild when `NOTIFY-SVC-39-001..004` land to validate payload fields, quiet-hours semantics, and any new connector metadata that should be documented here and in the channel-specific guides.
---
> **Imposed rule reminder:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

View File

@@ -1,423 +1,423 @@
# Stella Ops — Deterministic Replay Specification
Version: 1.0
Status: Draft / Internal Technical Reference
Audience: Core developers, module maintainers, audit engineers.
---
## 1. Purpose
Deterministic Replay allows any completed Stella Ops scan to be **reproduced byte-for-byte** with full cryptographic validation.
It guarantees that SBOMs, Findings, and VEX evaluations can be re-executed later to:
- prove historical compliance decisions,
- attribute changes precisely to feeds, rules, or tools,
- support dual-signing (FIPS + regional crypto),
- and anchor cryptographic evidence in offline or public ledgers.
Replay requires that all inputs and environmental conditions are **captured, hashed, and sealed** at scan time.
---
## 2. Architecture Overview
```mermaid
graph TD
A[Scanner.WebService] --> B[Replay Manifest]
A --> C[InputBundle]
A --> D[OutputBundle]
B --> E[DSSE Envelope]
C --> F[Feedser Snapshot Export]
C --> G[Policy/Lattice Bundle]
D --> H[DSSE Outputs (SBOM, Findings, VEX)]
E --> I[MongoDB: replay_runs]
C --> J[Blob Store: Input/Output Bundles]
````
### Core Artifacts
| Artifact | Description | Format |
| ------------------- | ------------------------------------------------------ | -------------------------- |
| **Replay Manifest** | Immutable JSON describing all scan inputs and outputs. | JSON (canonicalized) |
| **InputBundle** | Feeds, rules, policies, tool binaries (hashed). | `.tar.zst` |
| **OutputBundle** | SBOM, Findings, VEX, logs. | `.tar.zst` |
| **DSSE Envelope** | Signed metadata for each artifact. | JSON / JWS |
| **Merkle Map** | Layer and feed chunk trees. | JSON (embedded or sidecar) |
---
## 3. Replay Manifest Schema (v1)
### 3.1 Top-level Layout
```jsonc
{
"schemaVersion": "1.0",
"scan": {
"id": "uuid",
"time": "2025-10-29T13:05:33Z",
"mode": "record",
"scannerVersion": "10.1.3",
"cryptoProfile": "FIPS-140-3+GOST-R-34.10-2012"
},
"subject": {
"ociDigest": "sha256:abcd...",
"layers": [
{ "layerDigest": "...", "merkleRoot": "...", "leafCount": 144 }
]
},
"inputs": {
"feeds": [
{
"name": "nvd",
"snapshotHash": "sha256:...",
"snapshotTime": "2025-10-29T12:00:00Z",
"merkleRoot": "..."
}
],
"rulesBundleHash": "sha256:...",
"tools": [
{ "name": "sbomer", "version": "10.1.3", "sha256": "..." },
{ "name": "scanner", "version": "10.1.3", "sha256": "..." },
{ "name": "vexer", "version": "10.1.3", "sha256": "..." }
],
"env": {
"os": "linux",
"arch": "x64",
"locale": "en_US.UTF-8",
"tz": "UTC",
"seed": "H(scan.id||merkleRootAllLayers)",
"flags": ["offline"]
}
},
"policy": {
"latticeHash": "sha256:...",
"mutes": [
{ "id": "MUTE-1234", "reason": "vendor ack", "approvedBy": "authority@example.com", "approvedAt": "2025-10-29T12:55Z" }
],
"trustProfile": "sha256:..."
},
"outputs": {
"sbomHash": "sha256:...",
"findingsHash": "sha256:...",
"vexHash": "sha256:...",
"logHash": "sha256:..."
},
"provenance": {
"signer": "scanner.authority",
"dsseEnvelopeHash": "sha256:...",
"rekorEntry": "optional"
}
}
```
---
## 4. Deterministic Execution Rules
### 4.1 Environment Normalization
* **Clock:** frozen to `scan.time` unless a rule explicitly requires “now”.
* **Random seed:** derived as `H(scan.id || MerkleRootAllLayers)`.
* **Locale/TZ:** enforced per manifest; deviations cause validation error.
* **Filesystem normalization:**
* Normalize perms to 0644/0755.
* Path separators = `/`.
* Newlines = LF.
* JSON key order = lexical.
### 4.2 Concurrency & I/O
* File traversal: stable lexicographic order.
* Parallel jobs: ordered reduction by subject path.
* Temporary directories: ephemeral but deterministic hash seeds.
### 4.3 Feeds & Policies
* All network I/O disabled; feeds must be read from snapshot bundles.
* Policies and suppressions must resolve by hash, not name.
---
## 5. DSSE and Signing
### 5.1 Envelope Structure
```jsonc
{
"payloadType": "application/vnd.stella.replay.manifest+json",
"payload": "<base64-encoded canonical JSON>",
"signatures": [
{ "keyid": "authority-root-fips", "sig": "..." },
{ "keyid": "authority-root-gost", "sig": "..." }
]
}
```
### 5.2 Verification Steps
1. Decode payload → verify canonical form.
2. Verify each signature chain against RootPack (offline trust anchors).
3. Recompute hash and compare to `dsseEnvelopeHash` in manifest.
4. Optionally verify Rekor inclusion proof.
---
## 6. CLI Interface
### 6.1 Recording a Scan
```bash
stella scan image:tag --record ./out/
```
Produces:
```
out/
├─ manifest.json
├─ manifest.dsse.json
├─ inputbundle.tar.zst
├─ outputbundle.tar.zst
└─ signatures/
```
### 6.2 Verifying
```bash
stella verify manifest.json
```
* Checks all hashes and DSSE envelopes.
* Prints summary:
```
✅ Verified: SBOM, Findings, VEX, Tools, Feeds, Policy
```
### 6.3 Replaying
```bash
stella replay manifest.json --strict
stella replay manifest.json --what-if --vary=feeds
```
* `--strict`: all inputs locked; identical result expected.
* `--what-if`: varies only specified dimension(s).
### 6.4 Diffing
```bash
stella diff manifestA.json manifestB.json
```
Shows field-level differences (feed snapshot, tool, or policy hash).
---
## 7. MongoDB Schema
### 7.1 `replay_runs`
```jsonc
{
"_id": "uuid",
"manifestHash": "sha256:...",
"status": "verified|failed|replayed",
"createdAt": "...",
"updatedAt": "...",
"signatures": [{ "profile": "FIPS", "verified": true }],
"outputs": {
"sbom": "sha256:...",
"findings": "sha256:..."
}
}
```
### 7.2 `bundles`
```jsonc
{
"_id": "sha256:...",
"type": "input|output|rootpack",
"size": 4123123,
"location": "/var/lib/stella/bundles/<sha>.tar.zst"
}
```
### 7.3 `subjects`
```jsonc
{
"ociDigest": "sha256:abcd...",
"layers": [
{ "layerDigest": "...", "merkleRoot": "...", "leafCount": 120 }
]
}
```
---
## 8. Layer Merkle Implementation
### 8.1 Algorithm
```csharp
static string ComputeMerkleRoot(string layerTarPath)
{
const int ChunkSize = 4 * 1024 * 1024;
var hashes = new List<byte[]>();
using var fs = File.OpenRead(layerTarPath);
var buffer = new byte[ChunkSize];
int read;
using var sha = SHA256.Create();
while ((read = fs.Read(buffer, 0, buffer.Length)) > 0)
hashes.Add(sha.ComputeHash(buffer, 0, read));
while (hashes.Count > 1)
hashes = hashes
.Select((h, i) => (h, i))
.GroupBy(x => x.i / 2)
.Select(g => sha.ComputeHash(g.SelectMany(x => x.h).ToArray()))
.ToList();
return Convert.ToHexString(hashes.Single());
}
```
### 8.2 Stored Values
```json
{
"layerDigest": "sha256:...",
"merkleRoot": "b81f...",
"leafCount": 240,
"leavesHash": "sha256:..."
}
```
---
## 9. Replay Engine Implementation Notes (.NET 10)
### 9.1 Manifest Parsing
Use `System.Text.Json` with deterministic ordering:
```csharp
var options = new JsonSerializerOptions {
WriteIndented = false,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
TypeInfoResolverChain = { new OrderedResolver() }
};
```
### 9.2 Stable Output
Normalize SBOM/Findings/VEX JSON:
```csharp
string Canonicalize(string json) =>
JsonSerializer.Serialize(
JsonSerializer.Deserialize<JsonDocument>(json),
options);
```
### 9.3 Verification Flow
```csharp
var manifest = Manifest.Load("manifest.json");
VerifySignatures(manifest);
VerifyHashes(manifest);
if (mode == Strict) RunPipeline(manifest);
else RunPipelineWithVariation(manifest, vary);
```
### 9.4 Failure Modes
| Condition | Action |
| -------------------------------- | ----------------------------- |
| Missing snapshot or bundle | Error: `InputBundleMissing` |
| Feed hash mismatch | Error: `FeedSnapshotDrift` |
| Tool binary hash mismatch | Reject replay |
| Output hash drift in strict mode | Mark as failed, emit diff log |
| Invalid signature | Reject manifest |
---
## 10. Crypto Profiles and RootPack
### 10.1 Example Profiles
| Profile | Algorithms | Notes |
| -------------- | ------------------------------------- | ----------------------- |
| **FIPS-140-3** | ECDSA-P256 / SHA-256 / AES-GCM | Default for US/EU |
| **GOST** | GOST R 34.10-2012 / GOST R 34.11-2012 | Russia |
| **SM** | SM2 / SM3 / SM4 | China |
| **eIDAS** | RSA-PSS / SHA-256 | EU qualified signatures |
### 10.2 Dual-Signing Example
```bash
stella sign manifest.json --profiles=FIPS,GOST
```
Produces:
```
signatures/
├─ manifest.dsse.fips.json
└─ manifest.dsse.gost.json
```
---
## 11. Test Strategy
| Test | Description | Expected Result |
| ---------------------- | ------------------------------------ | --------------------------- |
| **Golden Replay** | Repeat identical scan → same outputs | ✅ identical hashes |
| **Feed Drift Test** | Replay with updated feeds | Only `inputs.feeds` changes |
| **Tool Upgrade Test** | Replay with new scanner version | Reject or diff by `tools` |
| **Policy Change Test** | Different lattice/mutes | Diff by `policy` section |
| **Cross-Arch Test** | x64 vs arm64 | Identical outputs |
| **Corrupted Bundle** | Tamper bundle | Verification fails |
---
## 12. Example Verification Output
```
$ stella verify manifest.json
[✓] Manifest integrity: OK
[✓] DSSE signatures (FIPS,GOST): OK
[✓] Feeds snapshot hash: OK
[✓] Policy + mutes hash: OK
[✓] Toolchain hash: OK
[✓] SBOM/VEX outputs: OK
Result: VERIFIED
```
---
## 13. Future Extensions
* Support **SPDX 3.0.1** alongside CycloneDX 1.6.
* Add **per-file Merkle proofs** for local scans.
* Ledger anchoring (Rekor, distributed Proof-Market).
* Post-quantum signatures (Dilithium/Falcon).
* Replay orchestration API (`/api/replay/:id`).
---
## 14. Summary
Deterministic Replay freezes every element of a scan:
> *image → feeds → policy → toolchain → environment → outputs → signatures.*
By enforcing canonical input/output states and verifiable cryptographic bindings, Stella Ops achieves **regulatory-grade replayability**, **regional crypto compliance**, and **immutable provenance** across all scans.
---
# Stella Ops — Deterministic Replay Specification
Version: 1.0
Status: Draft / Internal Technical Reference
Audience: Core developers, module maintainers, audit engineers.
---
## 1. Purpose
Deterministic Replay allows any completed Stella Ops scan to be **reproduced byte-for-byte** with full cryptographic validation.
It guarantees that SBOMs, Findings, and VEX evaluations can be re-executed later to:
- prove historical compliance decisions,
- attribute changes precisely to feeds, rules, or tools,
- support dual-signing (FIPS + regional crypto),
- and anchor cryptographic evidence in offline or public ledgers.
Replay requires that all inputs and environmental conditions are **captured, hashed, and sealed** at scan time.
---
## 2. Architecture Overview
```mermaid
graph TD
A[Scanner.WebService] --> B[Replay Manifest]
A --> C[InputBundle]
A --> D[OutputBundle]
B --> E[DSSE Envelope]
C --> F[Feedser Snapshot Export]
C --> G[Policy/Lattice Bundle]
D --> H[DSSE Outputs (SBOM, Findings, VEX)]
E --> I[MongoDB: replay_runs]
C --> J[Blob Store: Input/Output Bundles]
````
### Core Artifacts
| Artifact | Description | Format |
| ------------------- | ------------------------------------------------------ | -------------------------- |
| **Replay Manifest** | Immutable JSON describing all scan inputs and outputs. | JSON (canonicalized) |
| **InputBundle** | Feeds, rules, policies, tool binaries (hashed). | `.tar.zst` |
| **OutputBundle** | SBOM, Findings, VEX, logs. | `.tar.zst` |
| **DSSE Envelope** | Signed metadata for each artifact. | JSON / JWS |
| **Merkle Map** | Layer and feed chunk trees. | JSON (embedded or sidecar) |
---
## 3. Replay Manifest Schema (v1)
### 3.1 Top-level Layout
```jsonc
{
"schemaVersion": "1.0",
"scan": {
"id": "uuid",
"time": "2025-10-29T13:05:33Z",
"mode": "record",
"scannerVersion": "10.1.3",
"cryptoProfile": "FIPS-140-3+GOST-R-34.10-2012"
},
"subject": {
"ociDigest": "sha256:abcd...",
"layers": [
{ "layerDigest": "...", "merkleRoot": "...", "leafCount": 144 }
]
},
"inputs": {
"feeds": [
{
"name": "nvd",
"snapshotHash": "sha256:...",
"snapshotTime": "2025-10-29T12:00:00Z",
"merkleRoot": "..."
}
],
"rulesBundleHash": "sha256:...",
"tools": [
{ "name": "sbomer", "version": "10.1.3", "sha256": "..." },
{ "name": "scanner", "version": "10.1.3", "sha256": "..." },
{ "name": "vexer", "version": "10.1.3", "sha256": "..." }
],
"env": {
"os": "linux",
"arch": "x64",
"locale": "en_US.UTF-8",
"tz": "UTC",
"seed": "H(scan.id||merkleRootAllLayers)",
"flags": ["offline"]
}
},
"policy": {
"latticeHash": "sha256:...",
"mutes": [
{ "id": "MUTE-1234", "reason": "vendor ack", "approvedBy": "authority@example.com", "approvedAt": "2025-10-29T12:55Z" }
],
"trustProfile": "sha256:..."
},
"outputs": {
"sbomHash": "sha256:...",
"findingsHash": "sha256:...",
"vexHash": "sha256:...",
"logHash": "sha256:..."
},
"provenance": {
"signer": "scanner.authority",
"dsseEnvelopeHash": "sha256:...",
"rekorEntry": "optional"
}
}
```
---
## 4. Deterministic Execution Rules
### 4.1 Environment Normalization
* **Clock:** frozen to `scan.time` unless a rule explicitly requires “now”.
* **Random seed:** derived as `H(scan.id || MerkleRootAllLayers)`.
* **Locale/TZ:** enforced per manifest; deviations cause validation error.
* **Filesystem normalization:**
* Normalize perms to 0644/0755.
* Path separators = `/`.
* Newlines = LF.
* JSON key order = lexical.
### 4.2 Concurrency & I/O
* File traversal: stable lexicographic order.
* Parallel jobs: ordered reduction by subject path.
* Temporary directories: ephemeral but deterministic hash seeds.
### 4.3 Feeds & Policies
* All network I/O disabled; feeds must be read from snapshot bundles.
* Policies and suppressions must resolve by hash, not name.
---
## 5. DSSE and Signing
### 5.1 Envelope Structure
```jsonc
{
"payloadType": "application/vnd.stella.replay.manifest+json",
"payload": "<base64-encoded canonical JSON>",
"signatures": [
{ "keyid": "authority-root-fips", "sig": "..." },
{ "keyid": "authority-root-gost", "sig": "..." }
]
}
```
### 5.2 Verification Steps
1. Decode payload → verify canonical form.
2. Verify each signature chain against RootPack (offline trust anchors).
3. Recompute hash and compare to `dsseEnvelopeHash` in manifest.
4. Optionally verify Rekor inclusion proof.
---
## 6. CLI Interface
### 6.1 Recording a Scan
```bash
stella scan image:tag --record ./out/
```
Produces:
```
out/
├─ manifest.json
├─ manifest.dsse.json
├─ inputbundle.tar.zst
├─ outputbundle.tar.zst
└─ signatures/
```
### 6.2 Verifying
```bash
stella verify manifest.json
```
* Checks all hashes and DSSE envelopes.
* Prints summary:
```
✅ Verified: SBOM, Findings, VEX, Tools, Feeds, Policy
```
### 6.3 Replaying
```bash
stella replay manifest.json --strict
stella replay manifest.json --what-if --vary=feeds
```
* `--strict`: all inputs locked; identical result expected.
* `--what-if`: varies only specified dimension(s).
### 6.4 Diffing
```bash
stella diff manifestA.json manifestB.json
```
Shows field-level differences (feed snapshot, tool, or policy hash).
---
## 7. MongoDB Schema
### 7.1 `replay_runs`
```jsonc
{
"_id": "uuid",
"manifestHash": "sha256:...",
"status": "verified|failed|replayed",
"createdAt": "...",
"updatedAt": "...",
"signatures": [{ "profile": "FIPS", "verified": true }],
"outputs": {
"sbom": "sha256:...",
"findings": "sha256:..."
}
}
```
### 7.2 `bundles`
```jsonc
{
"_id": "sha256:...",
"type": "input|output|rootpack",
"size": 4123123,
"location": "/var/lib/stella/bundles/<sha>.tar.zst"
}
```
### 7.3 `subjects`
```jsonc
{
"ociDigest": "sha256:abcd...",
"layers": [
{ "layerDigest": "...", "merkleRoot": "...", "leafCount": 120 }
]
}
```
---
## 8. Layer Merkle Implementation
### 8.1 Algorithm
```csharp
static string ComputeMerkleRoot(string layerTarPath)
{
const int ChunkSize = 4 * 1024 * 1024;
var hashes = new List<byte[]>();
using var fs = File.OpenRead(layerTarPath);
var buffer = new byte[ChunkSize];
int read;
using var sha = SHA256.Create();
while ((read = fs.Read(buffer, 0, buffer.Length)) > 0)
hashes.Add(sha.ComputeHash(buffer, 0, read));
while (hashes.Count > 1)
hashes = hashes
.Select((h, i) => (h, i))
.GroupBy(x => x.i / 2)
.Select(g => sha.ComputeHash(g.SelectMany(x => x.h).ToArray()))
.ToList();
return Convert.ToHexString(hashes.Single());
}
```
### 8.2 Stored Values
```json
{
"layerDigest": "sha256:...",
"merkleRoot": "b81f...",
"leafCount": 240,
"leavesHash": "sha256:..."
}
```
---
## 9. Replay Engine Implementation Notes (.NET 10)
### 9.1 Manifest Parsing
Use `System.Text.Json` with deterministic ordering:
```csharp
var options = new JsonSerializerOptions {
WriteIndented = false,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
TypeInfoResolverChain = { new OrderedResolver() }
};
```
### 9.2 Stable Output
Normalize SBOM/Findings/VEX JSON:
```csharp
string Canonicalize(string json) =>
JsonSerializer.Serialize(
JsonSerializer.Deserialize<JsonDocument>(json),
options);
```
### 9.3 Verification Flow
```csharp
var manifest = Manifest.Load("manifest.json");
VerifySignatures(manifest);
VerifyHashes(manifest);
if (mode == Strict) RunPipeline(manifest);
else RunPipelineWithVariation(manifest, vary);
```
### 9.4 Failure Modes
| Condition | Action |
| -------------------------------- | ----------------------------- |
| Missing snapshot or bundle | Error: `InputBundleMissing` |
| Feed hash mismatch | Error: `FeedSnapshotDrift` |
| Tool binary hash mismatch | Reject replay |
| Output hash drift in strict mode | Mark as failed, emit diff log |
| Invalid signature | Reject manifest |
---
## 10. Crypto Profiles and RootPack
### 10.1 Example Profiles
| Profile | Algorithms | Notes |
| -------------- | ------------------------------------- | ----------------------- |
| **FIPS-140-3** | ECDSA-P256 / SHA-256 / AES-GCM | Default for US/EU |
| **GOST** | GOST R 34.10-2012 / GOST R 34.11-2012 | Russia |
| **SM** | SM2 / SM3 / SM4 | China |
| **eIDAS** | RSA-PSS / SHA-256 | EU qualified signatures |
### 10.2 Dual-Signing Example
```bash
stella sign manifest.json --profiles=FIPS,GOST
```
Produces:
```
signatures/
├─ manifest.dsse.fips.json
└─ manifest.dsse.gost.json
```
---
## 11. Test Strategy
| Test | Description | Expected Result |
| ---------------------- | ------------------------------------ | --------------------------- |
| **Golden Replay** | Repeat identical scan → same outputs | ✅ identical hashes |
| **Feed Drift Test** | Replay with updated feeds | Only `inputs.feeds` changes |
| **Tool Upgrade Test** | Replay with new scanner version | Reject or diff by `tools` |
| **Policy Change Test** | Different lattice/mutes | Diff by `policy` section |
| **Cross-Arch Test** | x64 vs arm64 | Identical outputs |
| **Corrupted Bundle** | Tamper bundle | Verification fails |
---
## 12. Example Verification Output
```
$ stella verify manifest.json
[✓] Manifest integrity: OK
[✓] DSSE signatures (FIPS,GOST): OK
[✓] Feeds snapshot hash: OK
[✓] Policy + mutes hash: OK
[✓] Toolchain hash: OK
[✓] SBOM/VEX outputs: OK
Result: VERIFIED
```
---
## 13. Future Extensions
* Support **SPDX 3.0.1** alongside CycloneDX 1.6.
* Add **per-file Merkle proofs** for local scans.
* Ledger anchoring (Rekor, distributed Proof-Market).
* Post-quantum signatures (Dilithium/Falcon).
* Replay orchestration API (`/api/replay/:id`).
---
## 14. Summary
Deterministic Replay freezes every element of a scan:
> *image → feeds → policy → toolchain → environment → outputs → signatures.*
By enforcing canonical input/output states and verifiable cryptographic bindings, Stella Ops achieves **regulatory-grade replayability**, **regional crypto compliance**, and **immutable provenance** across all scans.
---

View File

@@ -1,113 +1,113 @@
# Stella Ops — Developer Guide: Deterministic Replay
## Purpose
Deterministic Replay ensures any past scan can be re-executed byte-for-byte, producing identical SBOM, Findings, and VEX results, cryptographically verifiable for audits or compliance.
Replay is the foundation for:
- **Audit proofs** (exact past state reproduction)
- **Diff analysis** (feeds, policies, tool versions)
- **Cross-region verification** (same outputs on different hosts)
- **Long-term cryptographic trust** (re-sign with new crypto profiles)
---
## Core Concepts
| Term | Description |
|------|--------------|
| **Replay Manifest** | Immutable JSON describing all inputs, tools, env, and outputs of a scan. |
| **InputBundle** | Snapshot of feeds, rules, policies, and toolchain binaries used. |
| **OutputBundle** | SBOM, Findings, VEX, and logs from a completed scan. |
| **Layer Merkle** | Per-layer hash tree for precise deduplication and drift detection. |
| **DSSE Envelope** | Digital signature wrapper for each attestation (SBOM, Findings, Manifest, etc.). |
---
## What to Freeze
| Category | Example Contents | Required in Manifest |
|-----------|------------------|----------------------|
| **Subject** | OCI image digest, per-layer Merkle roots | ✅ |
| **Outputs** | SBOM, Findings, VEX, logs (content hashes) | ✅ |
| **Toolchain** | Sbomer, Scanner, Vexer binaries + versions + SHA256 | ✅ |
| **Feeds/VEX sources** | Full or pruned snapshot with Merkle proofs | ✅ |
| **Policy Bundle** | Lattice rules, mutes, trust profiles, thresholds | ✅ |
| **Environment** | OS, arch, locale, TZ, deterministic seed, runtime flags | ✅ |
| **Crypto Profile** | Algorithm suites (FIPS, GOST, SM, eIDAS) | ✅ |
---
## Replay Modes
| Mode | Purpose | Input Variation | Expected Output |
|------|----------|-----------------|-----------------|
| **Strict Replay** | Audit proof | None | Bit-for-bit identical |
| **What-If Replay** | Change impact analysis | One dimension (feeds/tools/policy) | Deterministic diff |
Example:
```
stella replay manifest.json --strict
stella replay manifest.json --what-if --vary=feeds
```
---
## Developer Responsibilities
| Module | Role |
|---------|------|
| **Scanner.WebService** | Capture full input set and produce Replay Manifest + DSSE sigs. |
| **Sbomer** | Generate deterministic SBOM; normalize ordering and JSON formatting. |
| **Vexer/Excititor** | Apply lattice and mutes from policy bundle; record gating logic. |
| **Feedser/Concelier** | Freeze and export feed snapshots or Merkle proofs. |
| **Authority** | Manage signer keys and crypto profiles; issue DSSE envelopes. |
| **CLI** | Provide `scan --record`, `replay`, `verify`, `diff` commands. |
---
## Workflow
1. `stella scan image:tag --record out/`
- Generates Replay Manifest, InputBundle, OutputBundle, DSSE sigs.
2. `stella verify manifest.json`
- Validates hashes, signatures, and completeness.
3. `stella replay manifest.json --strict`
- Re-executes in sealed mode; expect byte-identical results.
4. `stella replay manifest.json --what-if --vary=feeds`
- Runs with new feeds; diff is attributed to feeds only.
5. `stella diff manifestA manifestB`
- Attribute differences by hash comparison.
---
## Storage
- **Mongo collections**
- `replay_runs`: manifest + DSSE envelopes + status
- `bundles`: content-addressed (input/output/rootpack)
- `subjects`: OCI digests, Merkle roots per layer
- **File store**
- Bundles stored as `<sha256>.tar.zst`
---
## Developer Checklist
- [ ] All inputs (feeds, policies, tools, env) hashed and recorded.
- [ ] JSON normalization: key order, number format, newline mode.
- [ ] Random seed = `H(scan.id || MerkleRootAllLayers)`.
- [ ] Clock fixed to `scan.time` unless policy requires “now”.
- [ ] DSSE multi-sig supported (FIPS + regional).
- [ ] Manifest signed + optionally anchored to Rekor ledger.
- [ ] Replay comparison mode tested across x64/arm64.
---
## References
See also:
- `DETERMINISTIC_REPLAY.md` — detailed manifest schema & CLI examples.
- `../docs/CRYPTO_SOVEREIGN_READY.md` — RootPack and dual-signature handling.
---
# Stella Ops — Developer Guide: Deterministic Replay
## Purpose
Deterministic Replay ensures any past scan can be re-executed byte-for-byte, producing identical SBOM, Findings, and VEX results, cryptographically verifiable for audits or compliance.
Replay is the foundation for:
- **Audit proofs** (exact past state reproduction)
- **Diff analysis** (feeds, policies, tool versions)
- **Cross-region verification** (same outputs on different hosts)
- **Long-term cryptographic trust** (re-sign with new crypto profiles)
---
## Core Concepts
| Term | Description |
|------|--------------|
| **Replay Manifest** | Immutable JSON describing all inputs, tools, env, and outputs of a scan. |
| **InputBundle** | Snapshot of feeds, rules, policies, and toolchain binaries used. |
| **OutputBundle** | SBOM, Findings, VEX, and logs from a completed scan. |
| **Layer Merkle** | Per-layer hash tree for precise deduplication and drift detection. |
| **DSSE Envelope** | Digital signature wrapper for each attestation (SBOM, Findings, Manifest, etc.). |
---
## What to Freeze
| Category | Example Contents | Required in Manifest |
|-----------|------------------|----------------------|
| **Subject** | OCI image digest, per-layer Merkle roots | ✅ |
| **Outputs** | SBOM, Findings, VEX, logs (content hashes) | ✅ |
| **Toolchain** | Sbomer, Scanner, Vexer binaries + versions + SHA256 | ✅ |
| **Feeds/VEX sources** | Full or pruned snapshot with Merkle proofs | ✅ |
| **Policy Bundle** | Lattice rules, mutes, trust profiles, thresholds | ✅ |
| **Environment** | OS, arch, locale, TZ, deterministic seed, runtime flags | ✅ |
| **Crypto Profile** | Algorithm suites (FIPS, GOST, SM, eIDAS) | ✅ |
---
## Replay Modes
| Mode | Purpose | Input Variation | Expected Output |
|------|----------|-----------------|-----------------|
| **Strict Replay** | Audit proof | None | Bit-for-bit identical |
| **What-If Replay** | Change impact analysis | One dimension (feeds/tools/policy) | Deterministic diff |
Example:
```
stella replay manifest.json --strict
stella replay manifest.json --what-if --vary=feeds
```
---
## Developer Responsibilities
| Module | Role |
|---------|------|
| **Scanner.WebService** | Capture full input set and produce Replay Manifest + DSSE sigs. |
| **Sbomer** | Generate deterministic SBOM; normalize ordering and JSON formatting. |
| **Vexer/Excititor** | Apply lattice and mutes from policy bundle; record gating logic. |
| **Feedser/Concelier** | Freeze and export feed snapshots or Merkle proofs. |
| **Authority** | Manage signer keys and crypto profiles; issue DSSE envelopes. |
| **CLI** | Provide `scan --record`, `replay`, `verify`, `diff` commands. |
---
## Workflow
1. `stella scan image:tag --record out/`
- Generates Replay Manifest, InputBundle, OutputBundle, DSSE sigs.
2. `stella verify manifest.json`
- Validates hashes, signatures, and completeness.
3. `stella replay manifest.json --strict`
- Re-executes in sealed mode; expect byte-identical results.
4. `stella replay manifest.json --what-if --vary=feeds`
- Runs with new feeds; diff is attributed to feeds only.
5. `stella diff manifestA manifestB`
- Attribute differences by hash comparison.
---
## Storage
- **Mongo collections**
- `replay_runs`: manifest + DSSE envelopes + status
- `bundles`: content-addressed (input/output/rootpack)
- `subjects`: OCI digests, Merkle roots per layer
- **File store**
- Bundles stored as `<sha256>.tar.zst`
---
## Developer Checklist
- [ ] All inputs (feeds, policies, tools, env) hashed and recorded.
- [ ] JSON normalization: key order, number format, newline mode.
- [ ] Random seed = `H(scan.id || MerkleRootAllLayers)`.
- [ ] Clock fixed to `scan.time` unless policy requires “now”.
- [ ] DSSE multi-sig supported (FIPS + regional).
- [ ] Manifest signed + optionally anchored to Rekor ledger.
- [ ] Replay comparison mode tested across x64/arm64.
---
## References
See also:
- `DETERMINISTIC_REPLAY.md` — detailed manifest schema & CLI examples.
- `../docs/CRYPTO_SOVEREIGN_READY.md` — RootPack and dual-signature handling.
---