61f963fd52
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Added `LedgerMetrics` class to record write latency and total events for ledger operations. - Created comprehensive tests for Ruby packages endpoints, covering scenarios for missing inventory, successful retrieval, and identifier handling. - Introduced `TestSurfaceSecretsScope` for managing environment variables during tests. - Developed `ProvenanceMongoExtensions` for attaching DSSE provenance and trust information to event documents. - Implemented `EventProvenanceWriter` and `EventWriter` classes for managing event provenance in MongoDB. - Established MongoDB indexes for efficient querying of events based on provenance and trust. - Added models and JSON parsing logic for DSSE provenance and trust information.
234 lines
7.7 KiB
Markdown
234 lines
7.7 KiB
Markdown
# Inline DSSE Provenance
|
||
|
||
> **Status:** Draft – aligns with the November 2025 advisory “store DSSE attestation refs inline on every SBOM/VEX event node.”
|
||
> **Owners:** Authority Guild · Feedser Guild · Platform Guild · Docs Guild.
|
||
|
||
This document defines how Stella Ops records provenance for SBOM, VEX, scan, and derived events: every event node in the Mongo event graph includes DSSE + Rekor references and verification metadata so audits and replay become first-class queries.
|
||
|
||
---
|
||
|
||
## 1. Event patch (Mongo schema)
|
||
|
||
```jsonc
|
||
{
|
||
"_id": "evt_...",
|
||
"kind": "SBOM|VEX|SCAN|INGEST|DERIVED",
|
||
"subject": {
|
||
"purl": "pkg:nuget/example@1.2.3",
|
||
"digest": { "sha256": "..." },
|
||
"version": "1.2.3"
|
||
},
|
||
"provenance": {
|
||
"dsse": {
|
||
"envelopeDigest": "sha256:...",
|
||
"payloadType": "application/vnd.in-toto+json",
|
||
"key": {
|
||
"keyId": "cosign:SHA256-PKIX:ABC...",
|
||
"issuer": "fulcio",
|
||
"algo": "ECDSA"
|
||
},
|
||
"rekor": {
|
||
"logIndex": 1234567,
|
||
"uuid": "b3f0...",
|
||
"integratedTime": 1731081600,
|
||
"mirrorSeq": 987654 // optional
|
||
},
|
||
"chain": [
|
||
{ "type": "build", "id": "att:build#...", "digest": "sha256:..." },
|
||
{ "type": "sbom", "id": "att:sbom#...", "digest": "sha256:..." }
|
||
]
|
||
}
|
||
},
|
||
"trust": {
|
||
"verified": true,
|
||
"verifier": "Authority@stella",
|
||
"witnesses": 1,
|
||
"policyScore": 0.92
|
||
},
|
||
"ts": "2025-11-11T12:00:00Z"
|
||
}
|
||
```
|
||
|
||
### Key fields
|
||
|
||
| Field | Description |
|
||
|-------|-------------|
|
||
| `provenance.dsse.envelopeDigest` | SHA-256 of the DSSE envelope (not payload). |
|
||
| `provenance.dsse.payloadType` | Usually `application/vnd.in-toto+json`. |
|
||
| `provenance.dsse.key` | Key fingerprint / issuer / algorithm. |
|
||
| `provenance.dsse.rekor` | Rekor transparency log metadata (index, UUID, integrated time). |
|
||
| `provenance.dsse.chain` | Optional chain of dependent attestations (build → sbom → scan). |
|
||
| `trust.*` | Result of local verification (DSSE signature, Rekor proof, policy). |
|
||
|
||
---
|
||
|
||
## 2. Write path (ingest flow)
|
||
|
||
1. **Obtain provenance metadata** for each attested artifact (build, SBOM, VEX, scan). The CI script (`scripts/publish_attestation_with_provenance.sh`) captures `envelopeDigest`, Rekor `logIndex`/`uuid`, and key info.
|
||
2. **Authority/Feedser** verify the DSSE + Rekor proof (local cosign/rekor libs or the Signer service) and set `trust.verified = true`, `trust.verifier = "Authority@stella"`, `trust.witnesses = 1`.
|
||
3. **Attach** the provenance block before appending the event to Mongo, using `StellaOps.Provenance.Mongo` helpers.
|
||
4. **Backfill** historical events by resolving known subjects → attestation digests and running an update script.
|
||
|
||
### 2.1 Supplying metadata from Concelier statements
|
||
|
||
Concelier ingestion jobs can now inline provenance when they create advisory statements. Add an `AdvisoryProvenance` entry with `kind = "dsse"` (or `dsse-metadata` / `attestation-dsse`) and set `value` to the same JSON emitted by the CI snippet. `AdvisoryEventLog` and `AdvisoryMergeService` automatically parse that entry, hydrate `AdvisoryStatementInput.Provenance/Trust`, and persist the metadata alongside the statement.
|
||
|
||
```json
|
||
{
|
||
"source": "attestor",
|
||
"kind": "dsse",
|
||
"value": "{ \"dsse\": { \"envelopeDigest\": \"sha256:…\", \"payloadType\": \"application/vnd.in-toto+json\" }, \"trust\": { \"verified\": true, \"verifier\": \"Authority@stella\" } }",
|
||
"recordedAt": "2025-11-10T00:00:00Z"
|
||
}
|
||
```
|
||
|
||
Providing the metadata during ingestion keeps new statements self-contained and reduces the surface that the `/events/statements/{statementId}/provenance` endpoint needs to backfill later.
|
||
|
||
Reference helper: `src/__Libraries/StellaOps.Provenance.Mongo/ProvenanceMongoExtensions.cs`.
|
||
|
||
---
|
||
|
||
## 3. CI/CD snippet
|
||
|
||
See `scripts/publish_attestation_with_provenance.sh`:
|
||
|
||
```bash
|
||
rekor-cli upload --rekor_server "$REKOR_URL" \
|
||
--artifact "$ATTEST_PATH" --type dsse --format json > rekor-upload.json
|
||
LOG_INDEX=$(jq '.LogIndex' rekor-upload.json)
|
||
UUID=$(jq -r '.UUID' rekor-upload.json)
|
||
ENVELOPE_SHA256=$(sha256sum "$ATTEST_PATH" | awk '{print $1}')
|
||
cat > provenance-meta.json <<EOF
|
||
{
|
||
"subject": { "imageRef": "$IMAGE_REF", "digest": { "sha256": "$IMAGE_DIGEST" } },
|
||
"dsse": {
|
||
"envelopeDigest": "sha256:$ENVELOPE_SHA256",
|
||
"payloadType": "application/vnd.in-toto+json",
|
||
"key": { "keyId": "$KEY_ID", "issuer": "$KEY_ISSUER", "algo": "$KEY_ALGO" },
|
||
"rekor": { "logIndex": $LOG_INDEX, "uuid": "$UUID", "integratedTime": $(jq '.IntegratedTime' rekor-upload.json) }
|
||
}
|
||
}
|
||
EOF
|
||
```
|
||
|
||
Feedser ingests this JSON and maps it to `DsseProvenance` + `TrustInfo`.
|
||
|
||
---
|
||
|
||
## 4. Mongo indexes
|
||
|
||
Create indexes to keep provenance queries fast (`mongosh`):
|
||
|
||
```javascript
|
||
db.events.createIndex(
|
||
{ "subject.digest.sha256": 1, "kind": 1, "provenance.dsse.rekor.logIndex": 1 },
|
||
{ name: "events_by_subject_kind_provenance" }
|
||
);
|
||
|
||
db.events.createIndex(
|
||
{ "kind": 1, "trust.verified": 1, "provenance.dsse.rekor.logIndex": 1 },
|
||
{ name: "events_unproven_by_kind" }
|
||
);
|
||
|
||
db.events.createIndex(
|
||
{ "provenance.dsse.rekor.logIndex": 1 },
|
||
{ name: "events_by_rekor_logindex" }
|
||
);
|
||
```
|
||
|
||
Corresponding C# helper: `MongoIndexes.EnsureEventIndexesAsync`.
|
||
|
||
---
|
||
|
||
## 5. Query recipes
|
||
|
||
* **All proven VEX for an image digest:**
|
||
|
||
```javascript
|
||
db.events.find({
|
||
kind: "VEX",
|
||
"subject.digest.sha256": "<digest>",
|
||
"provenance.dsse.rekor.logIndex": { $exists: true },
|
||
"trust.verified": true
|
||
})
|
||
```
|
||
|
||
* **Compliance gap (unverified data used for decisions):**
|
||
|
||
```javascript
|
||
db.events.aggregate([
|
||
{ $match: { kind: { $in: ["VEX","SBOM","SCAN"] } } },
|
||
{ $match: {
|
||
$or: [
|
||
{ "trust.verified": { $ne: true } },
|
||
{ "provenance.dsse.rekor.logIndex": { $exists: false } }
|
||
]
|
||
}
|
||
},
|
||
{ $group: { _id: "$kind", count: { $sum: 1 } } }
|
||
])
|
||
```
|
||
|
||
* **Replay slice:** filter for events where `provenance.dsse.chain` covers build → sbom → scan and export referenced attestation digests.
|
||
|
||
---
|
||
|
||
## 6. Policy gates
|
||
|
||
Examples:
|
||
|
||
```yaml
|
||
rules:
|
||
- id: GATE-PROVEN-VEX
|
||
when:
|
||
all:
|
||
- kind: "VEX"
|
||
- trust.verified: true
|
||
- key.keyId in VendorAllowlist
|
||
- rekor.integratedTime <= releaseFreeze
|
||
then:
|
||
decision: ALLOW
|
||
|
||
- id: BLOCK-UNPROVEN
|
||
when:
|
||
any:
|
||
- trust.verified != true
|
||
- provenance.dsse.rekor.logIndex missing
|
||
then:
|
||
decision: FAIL
|
||
reason: "Unproven evidence influences decision; require Rekor-backed attestation."
|
||
```
|
||
|
||
---
|
||
|
||
## 7. UI nudges
|
||
|
||
* **Provenance chip** on findings/events: `Verified • Rekor#1234567 • KeyID:cosign:...` (click → inclusion proof & DSSE preview).
|
||
* Facet filter: `Provenance = Verified / Missing / Key-Policy-Mismatch`.
|
||
|
||
---
|
||
|
||
## 8. Implementation tasks
|
||
|
||
| Task ID | Scope |
|
||
|---------|-------|
|
||
| `PROV-INLINE-401-028` | Extend Authority/Feedser write-paths to attach `provenance.dsse` + `trust` blocks using `StellaOps.Provenance.Mongo`. |
|
||
| `PROV-BACKFILL-401-029` | Backfill historical events with DSSE/Rekor refs based on existing attestation digests. |
|
||
| `PROV-INDEX-401-030` | Create Mongo indexes and expose helper queries for audits. |
|
||
|
||
Keep this document updated when new attestation types or mirror/witness policies land.
|
||
|
||
---
|
||
|
||
## 9. Feedser API for provenance updates
|
||
|
||
Feedser exposes a lightweight endpoint for attaching provenance after an event is recorded:
|
||
|
||
```
|
||
POST /events/statements/{statementId}/provenance
|
||
Headers: X-Stella-Tenant, Authorization (if Authority is enabled)
|
||
Body: { "dsse": { ... }, "trust": { ... } }
|
||
```
|
||
|
||
The body matches the JSON emitted by `publish_attestation_with_provenance.sh`. Feedser validates the payload, ensures `trust.verified = true`, and then calls `AttachStatementProvenanceAsync` so the DSSE metadata lands inline on the target statement. Clients receive HTTP 202 on success, 400 on malformed input, and 404 if the statement id is unknown.
|