consolidation of some of the modules, localization fixes, product advisories work, qa work
This commit is contained in:
master
@@ -1,51 +1,20 @@
|
||||
# Provenance
|
||||
# StellaOps Provenance (Relocated)
|
||||
|
||||
> Provenance attestation library for SLSA/DSSE compliance.
|
||||
> **Sprint 204 (2026-03-04):** The Provenance module source has been consolidated under the Attestor trust domain.
|
||||
> Source code is now at `src/Attestor/StellaOps.Provenance.Attestation/` and `src/Attestor/StellaOps.Provenance.Attestation.Tool/`.
|
||||
> Architecture documentation is now in the [Attestor architecture dossier](../attestor/architecture.md#trust-domain-model-sprint-204----2026-03-04).
|
||||
> Archived standalone docs are in `docs-archived/modules/provenance/`.
|
||||
|
||||
## Purpose
|
||||
## Purpose (unchanged)
|
||||
|
||||
Provenance provides deterministic, verifiable provenance attestations for all StellaOps artifacts. It enables SLSA compliance through DSSE statement generation, Merkle tree construction, and cryptographic verification.
|
||||
Provenance is a **library** (not a standalone service) that provides deterministic, verifiable provenance attestations for all StellaOps artifacts. It enables SLSA compliance through DSSE statement generation, Merkle tree construction, and cryptographic verification.
|
||||
|
||||
## Quick Links
|
||||
## Note on StellaOps.Provenance (shared library)
|
||||
|
||||
- [Architecture](./architecture.md) - Technical design and implementation details
|
||||
- [Guides](./guides/) - Attestation generation guides
|
||||
The `src/__Libraries/StellaOps.Provenance/` library is a separate, lower-level provenance data model used by Concelier and other consumers. It was NOT moved by Sprint 204 and remains at its original location.
|
||||
|
||||
## Status
|
||||
## Why the move
|
||||
|
||||
| Attribute | Value |
|
||||
|-----------|-------|
|
||||
| **Maturity** | Production |
|
||||
| **Last Reviewed** | 2025-12-29 |
|
||||
| **Maintainer** | Security Guild |
|
||||
Provenance attestation libraries are consumed primarily by the Attestor trust domain (proofchain, evidence packs, verification). Consolidating source ownership under `src/Attestor/` clarifies trust-boundary responsibilities.
|
||||
|
||||
## Key Features
|
||||
|
||||
- **DSSE Statement Generation**: Build provenance attestations per DSSE spec
|
||||
- **SLSA Compliance**: Support for SLSA build predicates
|
||||
- **Merkle Tree Construction**: Content-addressed integrity verification
|
||||
- **Promotion Attestations**: Track artifact promotions across environments
|
||||
- **Verification Harness**: Validate attestation chains
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Upstream (this module depends on)
|
||||
- **Signer/KMS** - Key management for signing (delegated)
|
||||
|
||||
### Downstream (modules that depend on this)
|
||||
- **Attestor** - Stores generated attestations
|
||||
- **EvidenceLocker** - Evidence bundle attestations
|
||||
- **ExportCenter** - Export attestations
|
||||
|
||||
## Notes
|
||||
|
||||
Provenance is a **library**, not a standalone service. It does not:
|
||||
- Store attestations (handled by Attestor and EvidenceLocker)
|
||||
- Hold signing keys (delegated to Signer/KMS)
|
||||
|
||||
All attestation outputs are deterministic with canonical JSON serialization.
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Attestor Architecture](../attestor/architecture.md)
|
||||
- [DSSE Specification](../../security/trust-and-signing.md)
|
||||
See the [Trust Domain Model](../attestor/architecture.md#trust-domain-model-sprint-204----2026-03-04) for details.
|
||||
|
||||
@@ -1,316 +0,0 @@
|
||||
# component_architecture_provenance.md - **Stella Ops Provenance** (2025Q4)
|
||||
|
||||
> Provenance attestation library for SLSA/DSSE compliance.
|
||||
|
||||
> **Scope.** Library architecture for **Provenance**: shared libraries and tooling for generating, signing, and verifying provenance attestations (DSSE/SLSA). Used by evidence bundles, exports, and timeline verification flows.
|
||||
|
||||
---
|
||||
|
||||
## 0) Mission & boundaries
|
||||
|
||||
**Mission.** Provide **deterministic, verifiable provenance attestations** for all StellaOps artifacts. Enable SLSA compliance through DSSE statement generation, Merkle tree construction, and cryptographic verification.
|
||||
|
||||
**Boundaries.**
|
||||
|
||||
* Provenance is a **library**, not a standalone service.
|
||||
* Provenance **does not** store attestations. Storage is handled by Attestor and EvidenceLocker.
|
||||
* Provenance **does not** hold signing keys. Key management is delegated to Signer/KMS.
|
||||
* All attestation outputs are **deterministic** with canonical JSON serialization.
|
||||
|
||||
---
|
||||
|
||||
## 1) Solution & project layout
|
||||
|
||||
```
|
||||
src/Provenance/
|
||||
├─ StellaOps.Provenance.Attestation/ # Core attestation library
|
||||
│ ├─ AGENTS.md # Guild charter
|
||||
│ ├─ PromotionAttestation.cs # Promotion statement builder
|
||||
│ ├─ BuildModels.cs # SLSA build predicate models
|
||||
│ ├─ Signers.cs # Signer abstractions
|
||||
│ ├─ Verification.cs # Verification harness
|
||||
│ └─ Hex.cs # Hex encoding utilities
|
||||
│
|
||||
├─ StellaOps.Provenance.Attestation.Tool/ # CLI tool for attestation
|
||||
│ ├─ Program.cs
|
||||
│ └─ README.md
|
||||
│
|
||||
└─ __Tests/
|
||||
└─ StellaOps.Provenance.Attestation.Tests/
|
||||
├─ PromotionAttestationBuilderTests.cs
|
||||
├─ VerificationTests.cs
|
||||
├─ SignersTests.cs
|
||||
├─ MerkleTreeTests.cs
|
||||
└─ RotatingSignerTests.cs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2) External dependencies
|
||||
|
||||
* **Signer** - Cryptographic signing operations
|
||||
* **Cryptography** - Hash computation, signature algorithms
|
||||
* **EvidenceLocker** - Consumes attestations for storage
|
||||
* **ExportCenter** - Attaches attestations to export bundles
|
||||
* **.NET 10** - Runtime target
|
||||
|
||||
---
|
||||
|
||||
## 3) Contracts & data model
|
||||
|
||||
### 3.1 DSSE Statement
|
||||
|
||||
Dead Simple Signing Envelope (DSSE) format:
|
||||
|
||||
```json
|
||||
{
|
||||
"payloadType": "application/vnd.in-toto+json",
|
||||
"payload": "<base64-encoded-statement>",
|
||||
"signatures": [
|
||||
{
|
||||
"keyid": "sha256:abc123...",
|
||||
"sig": "<base64-signature>"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 3.2 SLSA Provenance Predicate
|
||||
|
||||
```json
|
||||
{
|
||||
"_type": "https://in-toto.io/Statement/v1",
|
||||
"subject": [
|
||||
{
|
||||
"name": "pkg:oci/scanner@sha256:abc123",
|
||||
"digest": { "sha256": "abc123..." }
|
||||
}
|
||||
],
|
||||
"predicateType": "https://slsa.dev/provenance/v1",
|
||||
"predicate": {
|
||||
"buildDefinition": {
|
||||
"buildType": "https://stellaops.dev/build/v1",
|
||||
"externalParameters": {},
|
||||
"internalParameters": {},
|
||||
"resolvedDependencies": []
|
||||
},
|
||||
"runDetails": {
|
||||
"builder": {
|
||||
"id": "https://stellaops.dev/builders/scanner"
|
||||
},
|
||||
"metadata": {
|
||||
"invocationId": "build-2025-01-15-abc123",
|
||||
"startedOn": "2025-01-15T10:30:00Z",
|
||||
"finishedOn": "2025-01-15T10:35:00Z"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3.3 Promotion Attestation
|
||||
|
||||
For artifact promotion across environments:
|
||||
|
||||
```csharp
|
||||
public sealed class PromotionAttestation
|
||||
{
|
||||
public required string ArtifactDigest { get; init; }
|
||||
public required string SourceEnvironment { get; init; }
|
||||
public required string TargetEnvironment { get; init; }
|
||||
public required DateTimeOffset PromotedAt { get; init; }
|
||||
public required string ApprovedBy { get; init; }
|
||||
public required IReadOnlyList<string> PolicyDigests { get; init; }
|
||||
public string? MerkleRoot { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4) Core Components
|
||||
|
||||
### 4.1 Signer Abstractions
|
||||
|
||||
```csharp
|
||||
public interface IAttestationSigner
|
||||
{
|
||||
string KeyId { get; }
|
||||
string Algorithm { get; }
|
||||
|
||||
Task<byte[]> SignAsync(
|
||||
ReadOnlyMemory<byte> payload,
|
||||
CancellationToken ct);
|
||||
}
|
||||
|
||||
public interface IRotatingSigner : IAttestationSigner
|
||||
{
|
||||
DateTimeOffset KeyNotBefore { get; }
|
||||
DateTimeOffset KeyNotAfter { get; }
|
||||
Task<IAttestationSigner> GetCurrentSignerAsync(CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
### 4.2 Verification Harness
|
||||
|
||||
```csharp
|
||||
public interface IAttestationVerifier
|
||||
{
|
||||
Task<VerificationResult> VerifyAsync(
|
||||
DsseEnvelope envelope,
|
||||
VerificationOptions options,
|
||||
CancellationToken ct);
|
||||
}
|
||||
|
||||
public sealed record VerificationResult
|
||||
{
|
||||
public required bool IsValid { get; init; }
|
||||
public required string KeyId { get; init; }
|
||||
public DateTimeOffset? SignedAt { get; init; }
|
||||
public IReadOnlyList<string>? Warnings { get; init; }
|
||||
public string? ErrorMessage { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### 4.3 Merkle Tree Utilities
|
||||
|
||||
For evidence chain verification:
|
||||
|
||||
```csharp
|
||||
public static class MerkleTree
|
||||
{
|
||||
public static string ComputeRoot(IEnumerable<string> leaves);
|
||||
public static IReadOnlyList<string> ComputePath(
|
||||
IReadOnlyList<string> leaves,
|
||||
int leafIndex);
|
||||
public static bool VerifyPath(
|
||||
string leaf,
|
||||
IReadOnlyList<string> path,
|
||||
string root);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5) CLI Tool
|
||||
|
||||
`StellaOps.Provenance.Attestation.Tool` provides CLI commands:
|
||||
|
||||
```bash
|
||||
# Generate provenance attestation
|
||||
provenance-tool generate \
|
||||
--subject "pkg:oci/scanner@sha256:abc123" \
|
||||
--builder "stellaops/ci" \
|
||||
--output attestation.json
|
||||
|
||||
# Sign attestation
|
||||
provenance-tool sign \
|
||||
--input attestation.json \
|
||||
--key-id "kms://keys/signing-key" \
|
||||
--output attestation.dsse.json
|
||||
|
||||
# Verify attestation
|
||||
provenance-tool verify \
|
||||
--input attestation.dsse.json \
|
||||
--trust-root trust-bundle.json
|
||||
|
||||
# Generate promotion attestation
|
||||
provenance-tool promote \
|
||||
--artifact "sha256:abc123" \
|
||||
--from staging \
|
||||
--to production \
|
||||
--approver "user@example.com"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6) Security & compliance
|
||||
|
||||
* **SLSA L3 compliance**: Build provenance with hermetic builds
|
||||
* **Key rotation**: RotatingSigner supports key rotation with overlap
|
||||
* **Determinism**: Canonical JSON ensures reproducible digests
|
||||
* **Offline verification**: Trust bundles for air-gapped verification
|
||||
* **Threat model**: Reviewed before each release
|
||||
|
||||
---
|
||||
|
||||
## 7) Performance targets
|
||||
|
||||
* **Statement generation**: < 10ms for typical attestation
|
||||
* **Signing**: Depends on KMS (target < 100ms for HSM)
|
||||
* **Verification**: < 50ms for single signature
|
||||
* **Merkle root**: < 100ms for 10,000 leaves
|
||||
|
||||
---
|
||||
|
||||
## 8) Testing matrix
|
||||
|
||||
* **Serialization tests**: Deterministic JSON output across runs
|
||||
* **Signing tests**: Round-trip sign/verify
|
||||
* **Merkle tests**: Path generation and verification
|
||||
* **Rotation tests**: Key rotation with overlap handling
|
||||
* **Integration tests**: Full attestation flow with mock KMS
|
||||
|
||||
---
|
||||
|
||||
## 9) Sample Artifacts
|
||||
|
||||
Samples committed under `samples/provenance/`:
|
||||
|
||||
```
|
||||
samples/provenance/
|
||||
├─ slsa-provenance-v1.json # Sample SLSA statement
|
||||
├─ promotion-attestation.json # Sample promotion
|
||||
├─ trust-bundle.json # Sample trust root
|
||||
└─ verify-example.sh # Verification script
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10) Integration Points
|
||||
|
||||
### 10.1 EvidenceLocker
|
||||
|
||||
Evidence bundles include attestations:
|
||||
|
||||
```json
|
||||
{
|
||||
"bundleId": "eb-2025-01-15-abc123",
|
||||
"attestations": [
|
||||
{
|
||||
"type": "slsa-provenance",
|
||||
"dsse": { /* DSSE envelope */ }
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 10.2 ExportCenter
|
||||
|
||||
Exports attach attestations to manifests:
|
||||
|
||||
```json
|
||||
{
|
||||
"exportId": "export-abc123",
|
||||
"manifest": { /* export manifest */ },
|
||||
"attestation": { /* DSSE envelope */ }
|
||||
}
|
||||
```
|
||||
|
||||
### 10.3 CLI
|
||||
|
||||
Scanner and other tools generate attestations:
|
||||
|
||||
```bash
|
||||
stella scan image:tag --attest --output sbom.cdx.json
|
||||
# Produces sbom.cdx.json + sbom.cdx.json.dsse
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
* Attestor architecture: `../attestor/architecture.md`
|
||||
* Signer architecture: `../signer/architecture.md`
|
||||
* EvidenceLocker: `../evidence-locker/architecture.md`
|
||||
* SLSA specification: https://slsa.dev/provenance/v1
|
||||
* DSSE specification: https://github.com/secure-systems-lab/dsse
|
||||
@@ -1,355 +0,0 @@
|
||||
# Inline DSSE Provenance
|
||||
|
||||
> **Status:** Draft – aligns with the November 2025 advisory “store DSSE attestation refs inline on every SBOM/VEX event node.”
|
||||
> **Owners:** Authority Guild · Feedser Guild · Platform Guild · Docs Guild.
|
||||
|
||||
This document defines how Stella Ops records provenance for SBOM, VEX, scan, and derived events: every event node in the PostgreSQL event store includes DSSE + Rekor references and verification metadata so audits and replay become first-class queries.
|
||||
|
||||
---
|
||||
|
||||
## 1. Event patch (PostgreSQL schema)
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"_id": "evt_...",
|
||||
"kind": "SBOM|VEX|SCAN|INGEST|DERIVED",
|
||||
"subject": {
|
||||
"purl": "pkg:nuget/example@1.2.3",
|
||||
"digest": { "sha256": "..." },
|
||||
"version": "1.2.3"
|
||||
},
|
||||
"provenance": {
|
||||
"dsse": {
|
||||
"envelopeDigest": "sha256:...",
|
||||
"payloadType": "application/vnd.in-toto+json",
|
||||
"key": {
|
||||
"keyId": "cosign:SHA256-PKIX:ABC...",
|
||||
"issuer": "fulcio",
|
||||
"algo": "ECDSA"
|
||||
},
|
||||
"rekor": {
|
||||
"logIndex": 1234567,
|
||||
"uuid": "b3f0...",
|
||||
"integratedTime": 1731081600,
|
||||
"mirrorSeq": 987654 // optional
|
||||
},
|
||||
"chain": [
|
||||
{ "type": "build", "id": "att:build#...", "digest": "sha256:..." },
|
||||
{ "type": "sbom", "id": "att:sbom#...", "digest": "sha256:..." }
|
||||
]
|
||||
}
|
||||
},
|
||||
"trust": {
|
||||
"verified": true,
|
||||
"verifier": "Authority@stella",
|
||||
"witnesses": 1,
|
||||
"policyScore": 0.92
|
||||
},
|
||||
"ts": "2025-11-11T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### Key fields
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `provenance.dsse.envelopeDigest` | SHA-256 of the DSSE envelope (not payload). |
|
||||
| `provenance.dsse.payloadType` | Usually `application/vnd.in-toto+json`. |
|
||||
| `provenance.dsse.key` | Key fingerprint / issuer / algorithm. |
|
||||
| `provenance.dsse.rekor` | Rekor transparency log metadata (index, UUID, integrated time). |
|
||||
| `provenance.dsse.chain` | Optional chain of dependent attestations (build → sbom → scan). |
|
||||
| `trust.*` | Result of local verification (DSSE signature, Rekor proof, policy). |
|
||||
|
||||
---
|
||||
|
||||
## 2. Write path (ingest flow)
|
||||
|
||||
1. **Obtain provenance metadata** for each attested artifact (build, SBOM, VEX, scan). The CI script (`scripts/publish_attestation_with_provenance.sh`) captures `envelopeDigest`, Rekor `logIndex`/`uuid`, and key info.
|
||||
2. **Authority/Feedser** verify the DSSE + Rekor proof (local cosign/rekor libs or the Signer service) and set `trust.verified = true`, `trust.verifier = "Authority@stella"`, `trust.witnesses = 1`.
|
||||
3. **Attach** the provenance block before appending the event to PostgreSQL, using `StellaOps.Provenance.Postgres` helpers.
|
||||
4. **Backfill** historical events by resolving known subjects → attestation digests and running an update script.
|
||||
|
||||
### 2.1 Supplying metadata from Concelier statements
|
||||
|
||||
Concelier ingestion jobs can now inline provenance when they create advisory statements. Add an `AdvisoryProvenance` entry with `kind = "dsse"` (or `dsse-metadata` / `attestation-dsse`) and set `value` to the same JSON emitted by the CI snippet. `AdvisoryEventLog` and `AdvisoryMergeService` automatically parse that entry, hydrate `AdvisoryStatementInput.Provenance/Trust`, and persist the metadata alongside the statement.
|
||||
|
||||
```json
|
||||
{
|
||||
"source": "attestor",
|
||||
"kind": "dsse",
|
||||
"value": "{ \"dsse\": { \"envelopeDigest\": \"sha256:…\", \"payloadType\": \"application/vnd.in-toto+json\" }, \"trust\": { \"verified\": true, \"verifier\": \"Authority@stella\" } }",
|
||||
"recordedAt": "2025-11-10T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
Providing the metadata during ingestion keeps new statements self-contained and reduces the surface that the `/events/statements/{statementId}/provenance` endpoint needs to backfill later.
|
||||
|
||||
Reference helper: `src/__Libraries/StellaOps.Provenance.Postgres/ProvenancePostgresExtensions.cs`.
|
||||
|
||||
---
|
||||
|
||||
### 2.2 Advisory AI structured chunk schema (GHSA/Cisco parity)
|
||||
|
||||
Advisory AI consumes the canonical `Advisory` aggregate and emits structured chunks that mirror GHSA GraphQL and Cisco PSIRT provenance anchors. The response contract is:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"advisoryKey": "CVE-2025-0001",
|
||||
"fingerprint": "<sha256 of canonical advisory>",
|
||||
"total": 3,
|
||||
"truncated": false,
|
||||
"entries": [
|
||||
{
|
||||
"type": "workaround", // sorted by (type, observationPath, documentId)
|
||||
"chunkId": "c0ffee12", // sha256(advisory.observationId + observationPath)[:16]
|
||||
"content": { /* structured field */ },
|
||||
"provenance": {
|
||||
"documentId": "tenant-a:chunk:newest", // PostgreSQL id of backing observation
|
||||
"observationPath": "/references/0", // JSON Pointer into the observation
|
||||
"source": "nvd",
|
||||
"kind": "workaround",
|
||||
"value": "tenant-a:chunk:newest",
|
||||
"recordedAt": "2025-01-07T00:00:00Z",
|
||||
"fieldMask": ["/references/0"]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Determinism requirements:
|
||||
|
||||
- Order entries by `(type, observationPath, documentId)` to keep cache keys stable across nodes.
|
||||
- Always include the advisory `fingerprint` in cache keys and responses.
|
||||
- Preserve observation-level provenance by emitting both `documentId` and `observationPath` under `provenance`.
|
||||
|
||||
These anchors let Attestor/Console deep-link evidence and allow offline mirrors to prove origin without merging transforms.
|
||||
|
||||
---
|
||||
|
||||
## 3. CI/CD snippet
|
||||
|
||||
See `scripts/publish_attestation_with_provenance.sh`:
|
||||
|
||||
```bash
|
||||
rekor-cli upload --rekor_server "$REKOR_URL" \
|
||||
--artifact "$ATTEST_PATH" --type dsse --format json > rekor-upload.json
|
||||
LOG_INDEX=$(jq '.LogIndex' rekor-upload.json)
|
||||
UUID=$(jq -r '.UUID' rekor-upload.json)
|
||||
ENVELOPE_SHA256=$(sha256sum "$ATTEST_PATH" | awk '{print $1}')
|
||||
cat > provenance-meta.json <<EOF
|
||||
{
|
||||
"subject": { "imageRef": "$IMAGE_REF", "digest": { "sha256": "$IMAGE_DIGEST" } },
|
||||
"dsse": {
|
||||
"envelopeDigest": "sha256:$ENVELOPE_SHA256",
|
||||
"payloadType": "application/vnd.in-toto+json",
|
||||
"key": { "keyId": "$KEY_ID", "issuer": "$KEY_ISSUER", "algo": "$KEY_ALGO" },
|
||||
"rekor": { "logIndex": $LOG_INDEX, "uuid": "$UUID", "integratedTime": $(jq '.IntegratedTime' rekor-upload.json) }
|
||||
}
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
Feedser ingests this JSON and maps it to `DsseProvenance` + `TrustInfo`.
|
||||
|
||||
---
|
||||
|
||||
## 4. PostgreSQL indexes
|
||||
|
||||
Create indexes to keep provenance queries fast (PostgreSQL DDL):
|
||||
|
||||
```sql
|
||||
-- events_by_subject_kind_provenance
|
||||
CREATE INDEX events_by_subject_kind_provenance
|
||||
ON events (subject_digest_sha256, kind, provenance_dsse_rekor_log_index);
|
||||
|
||||
-- events_unproven_by_kind
|
||||
CREATE INDEX events_unproven_by_kind
|
||||
ON events (kind, trust_verified, provenance_dsse_rekor_log_index);
|
||||
|
||||
-- events_by_rekor_logindex
|
||||
CREATE INDEX events_by_rekor_logindex
|
||||
ON events (provenance_dsse_rekor_log_index);
|
||||
|
||||
-- events_by_envelope_digest (partial index for non-null values)
|
||||
CREATE INDEX events_by_envelope_digest
|
||||
ON events (provenance_dsse_envelope_digest)
|
||||
WHERE provenance_dsse_envelope_digest IS NOT NULL;
|
||||
|
||||
-- events_by_ts_kind_verified
|
||||
CREATE INDEX events_by_ts_kind_verified
|
||||
ON events (ts DESC, kind, trust_verified);
|
||||
```
|
||||
|
||||
Deployment options:
|
||||
- **Ops script:** `psql -d stellaops_db -f ops/postgres/indices/events_provenance_indices.sql`
|
||||
- **C# helper:** `PostgresIndexes.EnsureEventIndexesAsync(connection, ct)`
|
||||
|
||||
This section was updated as part of `PROV-INDEX-401-030` (completed 2025-11-27).
|
||||
|
||||
---
|
||||
|
||||
## 5. Query recipes
|
||||
|
||||
* **All proven VEX for an image digest:**
|
||||
|
||||
```sql
|
||||
SELECT * FROM events
|
||||
WHERE kind = 'VEX'
|
||||
AND subject_digest_sha256 = '<digest>'
|
||||
AND provenance_dsse_rekor_log_index IS NOT NULL
|
||||
AND trust_verified = true;
|
||||
```
|
||||
|
||||
* **Compliance gap (unverified data used for decisions):**
|
||||
|
||||
```sql
|
||||
SELECT kind, COUNT(*) as count
|
||||
FROM events
|
||||
WHERE kind IN ('VEX', 'SBOM', 'SCAN')
|
||||
AND (trust_verified IS NOT TRUE
|
||||
OR provenance_dsse_rekor_log_index IS NULL)
|
||||
GROUP BY kind;
|
||||
```
|
||||
|
||||
* **Replay slice:** filter for events where `provenance.dsse.chain` covers build → sbom → scan and export referenced attestation digests.
|
||||
|
||||
---
|
||||
|
||||
## 6. Policy gates
|
||||
|
||||
Examples:
|
||||
|
||||
```yaml
|
||||
rules:
|
||||
- id: GATE-PROVEN-VEX
|
||||
when:
|
||||
all:
|
||||
- kind: "VEX"
|
||||
- trust.verified: true
|
||||
- key.keyId in VendorAllowlist
|
||||
- rekor.integratedTime <= releaseFreeze
|
||||
then:
|
||||
decision: ALLOW
|
||||
|
||||
- id: BLOCK-UNPROVEN
|
||||
when:
|
||||
any:
|
||||
- trust.verified != true
|
||||
- provenance.dsse.rekor.logIndex missing
|
||||
then:
|
||||
decision: FAIL
|
||||
reason: "Unproven evidence influences decision; require Rekor-backed attestation."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. UI nudges
|
||||
|
||||
* **Provenance chip** on findings/events: `Verified • Rekor#1234567 • KeyID:cosign:...` (click → inclusion proof & DSSE preview).
|
||||
* Facet filter: `Provenance = Verified / Missing / Key-Policy-Mismatch`.
|
||||
|
||||
---
|
||||
|
||||
## 8. Implementation tasks
|
||||
|
||||
| Task ID | Scope |
|
||||
|---------|-------|
|
||||
| `PROV-INLINE-401-028` | Extend Authority/Feedser write-paths to attach `provenance.dsse` + `trust` blocks using `StellaOps.Provenance.Postgres`. |
|
||||
| `PROV-BACKFILL-401-029` | Backfill historical events with DSSE/Rekor refs based on existing attestation digests. |
|
||||
| `PROV-INDEX-401-030` | Create PostgreSQL indexes and expose helper queries for audits. |
|
||||
|
||||
Keep this document updated when new attestation types or mirror/witness policies land.
|
||||
|
||||
---
|
||||
|
||||
## 9. Feedser API for provenance updates
|
||||
|
||||
Feedser exposes a lightweight endpoint for attaching provenance after an event is recorded:
|
||||
|
||||
```
|
||||
POST /events/statements/{statementId}/provenance
|
||||
Headers: X-Stella-Tenant, Authorization (if Authority is enabled)
|
||||
Body: { "dsse": { ... }, "trust": { ... } }
|
||||
```
|
||||
|
||||
The body matches the JSON emitted by `publish_attestation_with_provenance.sh`. Feedser validates the payload, ensures `trust.verified = true`, and then calls `AttachStatementProvenanceAsync` so the DSSE metadata lands inline on the target statement. Clients receive HTTP 202 on success, 400 on malformed input, and 404 if the statement id is unknown.
|
||||
|
||||
---
|
||||
|
||||
## 10. Backfill service
|
||||
|
||||
`EventProvenanceBackfillService` (`src/StellaOps.Events.Postgres/EventProvenanceBackfillService.cs`) orchestrates backfilling historical events with DSSE provenance metadata.
|
||||
|
||||
### 10.1 Components
|
||||
|
||||
| Class | Purpose |
|
||||
|-------|---------|
|
||||
| `IAttestationResolver` | Interface for resolving attestation metadata by subject digest. |
|
||||
| `EventProvenanceBackfillService` | Queries unproven events, resolves attestations, updates events. |
|
||||
| `StubAttestationResolver` | Test/development stub implementation. |
|
||||
|
||||
### 10.2 Usage
|
||||
|
||||
```csharp
|
||||
var resolver = new MyAttestationResolver(rekorClient, attestationRepo);
|
||||
var backfillService = new EventProvenanceBackfillService(postgresConnection, resolver);
|
||||
|
||||
// Count unproven events
|
||||
var count = await backfillService.CountUnprovenEventsAsync(
|
||||
new[] { "SBOM", "VEX", "SCAN" });
|
||||
|
||||
// Backfill with progress reporting
|
||||
var progress = new Progress<BackfillResult>(r =>
|
||||
Console.WriteLine($"{r.EventId}: {r.Status}"));
|
||||
|
||||
var summary = await backfillService.BackfillAllAsync(
|
||||
kinds: new[] { "SBOM", "VEX", "SCAN" },
|
||||
limit: 1000,
|
||||
progress: progress);
|
||||
|
||||
Console.WriteLine($"Processed: {summary.TotalProcessed}");
|
||||
Console.WriteLine($"Success: {summary.SuccessCount}");
|
||||
Console.WriteLine($"Not found: {summary.NotFoundCount}");
|
||||
Console.WriteLine($"Errors: {summary.ErrorCount}");
|
||||
```
|
||||
|
||||
### 10.3 Implementing IAttestationResolver
|
||||
|
||||
Implementations should query the attestation store (Rekor, CAS, or local PostgreSQL) by subject digest:
|
||||
|
||||
```csharp
|
||||
public class RekorAttestationResolver : IAttestationResolver
|
||||
{
|
||||
private readonly IRekorClient _rekor;
|
||||
private readonly IAttestationRepository _attestations;
|
||||
|
||||
public async Task<AttestationResolution?> ResolveAsync(
|
||||
string subjectDigestSha256,
|
||||
string eventKind,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
// Look up attestation by subject digest
|
||||
var record = await _attestations.GetAsync(subjectDigestSha256, eventKind, cancellationToken);
|
||||
if (record is null) return null;
|
||||
|
||||
// Fetch Rekor proof if available
|
||||
var proof = await _rekor.GetProofAsync(record.RekorUuid, RekorBackend.Sigstore, cancellationToken);
|
||||
|
||||
return new AttestationResolution
|
||||
{
|
||||
Dsse = new DsseProvenance { /* ... */ },
|
||||
Trust = new TrustInfo { Verified = true, Verifier = "Authority@stella" },
|
||||
AttestationId = record.Id
|
||||
};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 10.4 Reference files
|
||||
|
||||
- `src/StellaOps.Events.Postgres/IAttestationResolver.cs`
|
||||
- `src/StellaOps.Events.Postgres/EventProvenanceBackfillService.cs`
|
||||
- `src/StellaOps.Events.Postgres/StubAttestationResolver.cs`
|
||||
|
||||
This section was added as part of `PROV-BACKFILL-401-029` (completed 2025-11-27).
|
||||
@@ -1,16 +0,0 @@
|
||||
# Provenance Backfill Plan (Sprint 401)
|
||||
|
||||
Artifacts available
|
||||
- Attestation inventory: `docs/modules/provenance/guides/attestation-inventory-2025-11-18.ndjson`
|
||||
- Subject→Rekor map: `docs/modules/provenance/guides/subject-rekor-map-2025-11-18.json`
|
||||
|
||||
Procedure (deterministic)
|
||||
1) Load inventory NDJSON; validate UUID/ULID and digest formats.
|
||||
2) For each record, resolve Rekor entry via the subject→Rekor map; if missing, record gap and skip write.
|
||||
3) Emit backfilled events to the provenance store using `scripts/publish_attestation_with_provenance.sh --mode backfill` (add `--subject` and `--rekor` arguments) with sorted input to guarantee stable ordering.
|
||||
4) Log every backfilled subject + Rekor digest pair to `logs/provenance-backfill-2025-11-18.ndjson` (UTC timestamps, ISO-8601).
|
||||
5) Rerun until gaps are zero; then mark PROV-BACKFILL-401-029 DONE.
|
||||
|
||||
Determinism
|
||||
- Sort by subject, then rekorEntry before processing.
|
||||
- Use canonical JSON writer for outputs; timestamps in UTC `O` format.
|
||||
@@ -1,76 +0,0 @@
|
||||
# Provenance & Attestation Reference
|
||||
|
||||
> **Imposed rule:** All exported evidence must ship with DSSE + transparency proof bundles; unsigned or proof-less artifacts are rejected at ingress and may not be stored in the Evidence Locker.
|
||||
|
||||
This guide explains how StellaOps generates, signs, verifies, and distributes DSSE attestations for SBOMs, policy evaluations, and runtime evidence.
|
||||
|
||||
## 1. Attestation Workflow (online and offline)
|
||||
|
||||
1. **Producer** (Scanner, Policy Engine, runtime probes) emits a payload and a request to sign.
|
||||
2. **Signer** authenticates the caller, validates supply-chain policy (release integrity, image pinning), then signs using keyless or tenant KMS keys.
|
||||
3. **Attestor** wraps the payload in DSSE, records it in Rekor v2 (when online), persists the bundle plus inclusion proof, and exposes a verification package API.
|
||||
4. **Export Center** and **Evidence Locker** embed the bundle and proof into export artifacts for offline replay; CLI retrieves the same package via `stella attest fetch`.
|
||||
5. **Verifiers** (CLI, Policy Engine, auditors) validate signature roots, Rekor proof, and optional transparency witness endorsements.
|
||||
|
||||
## 2. DSSE Payload Types & Schemas
|
||||
|
||||
Supported payload types (all versioned and protobuf/JSON dual-encoded):
|
||||
|
||||
- `StellaOps.BuildProvenance@1`
|
||||
- `StellaOps.SBOMAttestation@1`
|
||||
- `StellaOps.ScanResults@1`
|
||||
- `StellaOps.PolicyEvaluation@1`
|
||||
- `StellaOps.VEXAttestation@1`
|
||||
- `StellaOps.RiskProfileEvidence@1`
|
||||
- `StellaOps.PromotionAttestation@1` (predicate `stella.ops/promotion@v1`, see `docs/release/promotion-attestations.md`)
|
||||
|
||||
Schema sources: `src/Attestor/StellaOps.Attestor.Types` and module dossiers. All payloads include:
|
||||
|
||||
- `subject` (digest + PURL/NEVRA coordinates)
|
||||
- `timestamp` (UTC, ISO-8601)
|
||||
- `producer` (service + version)
|
||||
- `critical` block (policy version, scanner defs, reachability context)
|
||||
- `materials` (SBOM/VEX references) and optional `auxiliary_proofs`
|
||||
|
||||
## 3. Signing & storage controls
|
||||
|
||||
- **Key policy:** Short-lived OIDC keyless by default; tenant KMS allowed; Ed25519 and ECDSA P-256 supported.
|
||||
- **Inclusion:** Rekor v2 UUID + log index cached; when offline, the Attestor stamps a `transparency_pending` marker to be replayed later.
|
||||
- **WORM:** Evidence Locker keeps immutable copies; retention and legal hold are enforced per tenant and surfaced in `docs/modules/evidence-locker/guides/evidence-locker.md`.
|
||||
- **Redaction:** Sensitive fields (secrets, PII) must be excluded at payload creation; the signer refuses payloads marked `pii=true` without a redaction ticket.
|
||||
|
||||
## 4. Verification workflow
|
||||
|
||||
Command-line (online or offline bundle):
|
||||
|
||||
```sh
|
||||
stella attest verify \
|
||||
--bundle path/to/bundle.dsse.json \
|
||||
--rekor-root pubkeys/rekor.pub \
|
||||
--fulcio-root pubkeys/fulcio.pub \
|
||||
--certificate-chain pubkeys/issuer-chain.pem
|
||||
```
|
||||
|
||||
Verification steps performed by services and CLI:
|
||||
|
||||
- Validate DSSE signature against Fulcio/tenant roots and certificate policies.
|
||||
- Confirm subject digest matches expected container/image/SBOM digest.
|
||||
- Check Rekor inclusion proof and (if present) transparency witness signatures.
|
||||
- Enforce freshness: reject bundles older than `attestation.max_age_days` (tenant policy).
|
||||
- Record verification result into Timeline events for auditability.
|
||||
|
||||
## 5. Offline / air-gap posture
|
||||
|
||||
- Export Center emits self-contained bundles (`*.dsse.json`, `rekor-proof.json`, `cert-chain.pem`) plus a verification manifest for deterministic replay.
|
||||
- CLI `stella attest verify --bundle bundle.dsse.json --offline` skips Rekor lookups and relies on embedded proofs.
|
||||
- When connectivity returns, the Attestor replays pending `transparency_pending` entries and updates Evidence Locker indexes; Timeline events capture the replay.
|
||||
|
||||
## 6. References
|
||||
|
||||
- `docs/modules/signer/architecture.md`
|
||||
- `docs/modules/attestor/architecture.md`
|
||||
- `docs/modules/export-center/architecture.md`
|
||||
- `docs/modules/policy/architecture.md`
|
||||
- `docs/modules/telemetry/architecture.md`
|
||||
- `docs/modules/evidence-locker/guides/evidence-locker.md`
|
||||
- `src/Provenance/StellaOps.Provenance.Attestation`
|
||||
Reference in New Issue
Block a user