stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

docs consoliation work

Browse Source
This commit is contained in:
StellaOps Bot
2025-12-24 14:19:46 +02:00
parent 40362de568
commit 5540ce9430
58 changed files with 2671 additions and 1751 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
docs/modules/attestor/README.md
View File

@@ -1,5 +1,5 @@
# StellaOps Attestor
# StellaOps Attestor
Attestor converts signed DSSE evidence from the Signer into transparency-log proofs and verifiable reports for every downstream surface (Policy Engine, Export Center, CLI, Console, Scheduler). It is the trust backbone that proves SBOM, scan, VEX, and policy artefacts were signed, witnessed, and preserved without tampering.
## Latest updates (2025-11-30)
@@ -11,19 +11,19 @@ Attestor converts signed DSSE evidence from the Signer into transparency-log pro
- **Evidence first:** organisations need portable, verifiable attestations that prove build provenance, SBOM availability, policy verdicts, and VEX statements.
- **Policy enforcement:** verification policies ensure only approved issuers, key types, witnesses, and freshness windows are accepted.
- **Sovereign/offline-ready:** Attestor archives envelopes, signatures, and proofs so air-gapped deployments can replay verification without contacting external services.
## Roles & surfaces
- **Subjects:** immutable digests for container images, SBOMs, reports, and policy bundles.
- **Issuers:** builders, scanners, policy engines, or operators signing DSSE envelopes using keyless (Fulcio), KMS/HSM, or FIDO2 keys.
- **Consumers:** CLI/SDK, Console, Export Center, Scanner, Policy Engine, and Notify retrieving verification bundles or triggering policy checks.
- **Scopes:** Authority issues `attestor.write`, `attestor.verify`, `attestor.read`, and administrative scopes for issuer/key management; every call is bound with mTLS + DPoP.
## Supported payloads
- `StellaOps.BuildProvenance@1`, `StellaOps.SBOMAttestation@1`
- `StellaOps.ScanResults@1`, `StellaOps.VEXAttestation@1`
- `StellaOps.PolicyEvaluation@1`, `StellaOps.RiskProfileEvidence@1`
All predicates capture subjects, issuer metadata, policy context, materials, optional witnesses, and versioned schemas. Unsupported predicates return `422 predicate_unsupported`.
## Roles & surfaces
- **Subjects:** immutable digests for container images, SBOMs, reports, and policy bundles.
- **Issuers:** builders, scanners, policy engines, or operators signing DSSE envelopes using keyless (Fulcio), KMS/HSM, or FIDO2 keys.
- **Consumers:** CLI/SDK, Console, Export Center, Scanner, Policy Engine, and Notify retrieving verification bundles or triggering policy checks.
- **Scopes:** Authority issues `attestor.write`, `attestor.verify`, `attestor.read`, and administrative scopes for issuer/key management; every call is bound with mTLS + DPoP.
## Supported payloads
- `StellaOps.BuildProvenance@1`, `StellaOps.SBOMAttestation@1`
- `StellaOps.ScanResults@1`, `StellaOps.VEXAttestation@1`
- `StellaOps.PolicyEvaluation@1`, `StellaOps.RiskProfileEvidence@1`
All predicates capture subjects, issuer metadata, policy context, materials, optional witnesses, and versioned schemas. Unsupported predicates return `422 predicate_unsupported`.
## Trust & envelope model
- DSSE envelopes are canonicalised, hashed, and stored alongside the Rekor UUID, index, and proof.
- Signature modes span keyless (Fulcio), keyful (KMS/HSM), and hardware-backed (FIDO2). Multiple signatures are supported per envelope.
@@ -40,27 +40,27 @@ All predicates capture subjects, issuer metadata, policy context, materials, opt
- **Console:** Evidence browser, verification reports, chain-of-custody graph, issuer/key management, attestation workbench, and bulk verification flows.
- **CLI / SDK:** `stella attest sign|verify|list|fetch|key` commands plus language SDKs to integrate build pipelines and offline verification scripts.
- **Policy Studio:** Verification policies author required predicate types, issuers, witness requirements, and freshness windows; simulations show enforcement impact.
## Storage, offline & air-gap posture
- MongoDB stores entry metadata, dedupe keys, and audit events; object storage optionally archives DSSE bundles.
- Export Center packages attestation bundles (`stella export attestation-bundle`) for Offline Kit delivery.
- Transparency logs can be mirrored; offline mode records gaps and provides compensating controls.
## Storage, offline & air-gap posture
- PostgreSQL stores entry metadata, dedupe keys, and audit events; object storage optionally archives DSSE bundles.
- Export Center packages attestation bundles (`stella export attestation-bundle`) for Offline Kit delivery.
- Transparency logs can be mirrored; offline mode records gaps and provides compensating controls.
## Observability & performance
- Metrics: `attestor_submission_total`, `attestor_verify_seconds`, `attestor_cache_hit_ratio`, `attestor_rekor_latency_seconds`.
- Logs capture tenant, issuer, subject digests, Rekor UUID, proof status, and policy verdict.
- Performance target: ≥1 000 envelopes/minute per worker with cached verification, batched operations, and concurrency controls.
- Observability assets: `operations/observability.md` and `operations/dashboards/attestor-observability.json` (offline import).
## Key integrations
- Signer (DSSE source), Authority (scopes & tenancy), Export Center (attestation bundles), Policy Engine (verification policies), Scanner/Excititor (subject evidence), Notify (key rotation & verification alerts), Observability stack (dashboards/alerts).
## Backlog references
- DOCS-ATTEST-73-001 … DOCS-ATTEST-75-002 (Attestor console, key management, air-gap bundles) in ../../TASKS.md.
- EXPORT-ATTEST-75-002 (Export Center attestation packaging) in ../export-center/TASKS.md.
## Epic alignment
- **Epic 19 – Attestor Console:** console experience, verification APIs, issuer/key governance, transparency integration, and offline bundles.
- **Epic 10 – Export Center:** provenance alignment so exports carry signed manifests and attestation bundles.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
## Key integrations
- Signer (DSSE source), Authority (scopes & tenancy), Export Center (attestation bundles), Policy Engine (verification policies), Scanner/Excititor (subject evidence), Notify (key rotation & verification alerts), Observability stack (dashboards/alerts).
## Backlog references
- DOCS-ATTEST-73-001 … DOCS-ATTEST-75-002 (Attestor console, key management, air-gap bundles) in ../../TASKS.md.
- EXPORT-ATTEST-75-002 (Export Center attestation packaging) in ../export-center/TASKS.md.
## Epic alignment
- **Epic 19 – Attestor Console:** console experience, verification APIs, issuer/key governance, transparency integration, and offline bundles.
- **Epic 10 – Export Center:** provenance alignment so exports carry signed manifests and attestation bundles.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

2
docs/modules/attestor/architecture.md
View File

@@ -482,7 +482,7 @@ The worker honours `bulkVerification.itemDelayMilliseconds` for throttling and r
* **Proof acquisition**:
* In synchronous mode, poll the log for inclusion up to `proofTimeoutMs`.
* In asynchronous mode, return `pending` and schedule a **proof fetcher** job (Mongo job doc + backoff).
* In asynchronous mode, return `pending` and schedule a **proof fetcher** job (PostgreSQL job record + backoff).
* **Mirrors/dual logs**:
* When `logPreference="both"`, submit to primary and mirror; store **both** UUIDs (primary canonical).

365
docs/modules/attestor/reg-evidence-architecture.md Normal file
View File

@@ -0,0 +1,365 @@
# Resolved Execution Graph (REG) Evidence Architecture
> **Status:** Proposed
> **Sprint Series:** 8100.0012.*
> **Last Updated:** 2025-12-24
## Overview
This document describes the architectural enhancements to StellaOps' evidence and attestation subsystems based on the **Merkle-Hash REG** (Resolved Execution Graph) pattern. The core insight: when every node in an execution graph is identified by a **Merkle hash of its normalized content**, evidence can be attached to *content itself* rather than brittle positional indices.
## Motivation
### Problem Statement
StellaOps currently has robust foundations for content-addressed identifiers, Merkle trees, and attestations. However, three gaps limit the system's verifiability:
1. **Canonicalizer versioning:** Hash collisions possible if canonicalization logic changes
2. **Fragmented evidence models:** Different modules use different evidence structures
3. **Implicit graph roots:** Merkle roots are computed but not independently attested
### Target Benefits
| Benefit | Description |
|---------|-------------|
| **Reproducible proofs** | Verifiers re-hash node content and check against stored hashes—no fragile indices |
| **Dedup & reuse** | Identical content across pipelines collapses to one ID; one piece of evidence justifies many occurrences |
| **Audits + time travel** | Snapshot the graph's Merkle root; any replay matching the root guarantees identical nodes & evidence |
| **Offline verification** | Attestations are self-contained; no network required to verify |
| **Exception stability** | Exceptions bind to content hashes, not "row 42"; stable across rebuilds |
## Architecture
### Component Diagram
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ StellaOps REG Architecture │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ StellaOps. │ │ StellaOps. │ │ StellaOps. │ │
│ │ Canonical.Json │────▶│ Evidence.Core │◀────│ Attestor. │ │
│ │ │ │ │ │ GraphRoot │ │
│ │ • CanonVersion │ │ • IEvidence │ │ │ │
│ │ • CanonJson │ │ • EvidenceRecord│ │ • IGraphRoot- │ │
│ │ • Versioned │ │ • IEvidenceStore│ │ Attestor │ │
│ │ Hashing │ │ • Adapters │ │ • Verification │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │
│ │ │ │ │
│ └────────────────────────┼────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ Content-Addressed Store │ │
│ │ │ │
│ │ Evidence by subject_node_id │ Graph roots by root_hash │ │
│ │ ──────────────────────────── │ ────────────────────────── │ │
│ │ sha256:abc... → [evidence] │ sha256:xyz... → attestation │ │
│ │ sha256:def... → [evidence] │ sha256:uvw... → attestation │ │
│ └──────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Scanner │ │ Policy │ │ Excititor │ │
│ │ ────────────── │ │ ────────────── │ │ ────────────── │ │
│ │ • EvidenceBundle│────▶│ • Exception │────▶│ • VexObservation│ │
│ │ • ProofSpine │ │ Application │ │ • VexLinkset │ │
│ │ • RichGraph │ │ • PolicyVerdict │ │ │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │
│ │ │ │ │
│ └────────────────────────┼────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ Unified IEvidence│ │
│ │ via Adapters │ │
│ └──────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Data Flow
```
1. Content Creation
─────────────────
Component/Node → Canonicalize(content, version) → SHA-256 → node_id
2. Evidence Attachment
────────────────────
Analysis Result → IEvidence { subject_node_id, type, payload, provenance }
→ EvidenceId = hash(subject + type + payload + provenance)
→ Store(evidence)
3. Graph Construction
───────────────────
Nodes + Edges → Sort(node_ids) + Sort(edge_ids) → Merkle Tree → root_hash
4. Graph Attestation
──────────────────
GraphRootAttestationRequest → GraphRootAttestor
→ In-Toto Statement { subject: [root, artifact] }
→ DSSE Sign
→ Optional: Rekor Publish
5. Verification
─────────────
Download attestation → Verify signature
→ Fetch nodes/edges by ID
→ Recompute Merkle root
→ Compare with attested root
```
## Implementation Sprints
| Sprint | Title | Dependency | Key Deliverables |
|--------|-------|------------|------------------|
| 8100.0012.0001 | Canonicalizer Versioning | None | `CanonVersion`, `CanonicalizeVersioned()`, backward compatibility |
| 8100.0012.0002 | Unified Evidence Model | 0001 | `IEvidence`, `EvidenceRecord`, `IEvidenceStore`, adapters |
| 8100.0012.0003 | Graph Root Attestation | 0001, 0002 | `IGraphRootAttestor`, in-toto statements, Rekor integration |
### Sprint Sequence Diagram
```
Week 1-2 Week 3-4 Week 5-6 Week 7-8
──────── ──────── ──────── ────────
│ │ │ │
│ 0001: Canon │ │ │
│ Versioning │ │ │
│ ┌─────────┐ │ │ │
│ │Wave 0-1 │────┼─▶ 0002: Unified│ │
│ │Wave 2-3 │ │ Evidence │ │
│ │Wave 4 │ │ ┌─────────┐ │ │
│ └─────────┘ │ │Wave 0-1 │──┼─▶ 0003: Graph │
│ │ │Wave 2-3 │ │ Root Attest │
│ │ │Wave 4 │ │ ┌─────────┐ │
│ │ └─────────┘ │ │Wave 0-1 │ │
│ │ │ │Wave 2-4 │ │
│ │ │ │Wave 5 │ │
│ │ │ └─────────┘ │
▼ ▼ ▼ ▼
```
## Technical Specifications
### Canonicalization Version Marker
```json
{
"_canonVersion": "stella:canon:v1",
"evidenceSource": "stellaops/scanner/reachability",
"sbomEntryId": "sha256:abc123...:pkg:npm/lodash@4.17.21",
...
}
```
The `_canonVersion` field:
- Uses underscore prefix to sort first lexicographically
- Identifies the exact canonicalization algorithm
- Enables graceful version migration
- Is included in all content-addressed hash computations
### Unified Evidence Record
```typescript
interface IEvidence {
// Content-addressed binding
subjectNodeId: string; // "sha256:{hex}" - what this evidence is about
evidenceId: string; // "sha256:{hex}" - ID of this evidence record
// Type and payload
evidenceType: EvidenceType; // reachability, scan, policy, vex, etc.
payload: Uint8Array; // Canonical JSON bytes
payloadSchemaVersion: string;
// Attestation
signatures: EvidenceSignature[];
// Provenance
provenance: {
generatorId: string;
generatorVersion: string;
generatedAt: DateTimeOffset;
inputsDigest?: string;
correlationId?: string;
};
// External storage
externalPayloadCid?: string;
}
```
### Graph Root Attestation (In-Toto)
```json
{
"_type": "https://in-toto.io/Statement/v1",
"subject": [
{
"name": "sha256:abc123...",
"digest": { "sha256": "abc123..." }
},
{
"name": "sha256:def456...",
"digest": { "sha256": "def456..." }
}
],
"predicateType": "https://stella-ops.org/attestation/graph-root/v1",
"predicate": {
"graphType": "ResolvedExecutionGraph",
"rootHash": "sha256:abc123...",
"nodeCount": 1247,
"edgeCount": 3891,
"nodeIds": ["sha256:...", ...],
"edgeIds": ["sha256:...", ...],
"inputs": {
"policyDigest": "sha256:...",
"feedsDigest": "sha256:...",
"toolchainDigest": "sha256:...",
"paramsDigest": "sha256:..."
},
"evidenceIds": ["sha256:...", ...],
"canonVersion": "stella:canon:v1",
"computedAt": "2025-12-24T10:30:00Z",
"computedBy": "stellaops/attestor/graph-root",
"computedByVersion": "1.0.0"
}
}
```
## Verification Guarantees
### What Can Be Verified
| Claim | Verification Method |
|-------|---------------------|
| "This graph contains exactly these nodes" | Recompute Merkle root from node IDs |
| "This evidence applies to node X" | Check evidence.subjectNodeId == node.id |
| "This attestation was signed by key K" | Verify DSSE envelope signature |
| "This graph was published to transparency log" | Check Rekor inclusion proof |
| "These inputs produced this graph" | Check inputs.* digests match expectations |
### What Cannot Be Verified (Without Additional Trust)
| Claim | Reason | Mitigation |
|-------|--------|------------|
| "The analysis was performed correctly" | Semantic, not structural | Trust the generator; audit toolchain |
| "No evidence was omitted" | Attester controls content | Require known evidence types; policy enforcement |
| "The inputs are authentic" | External dependency | Chain attestations; verify feed signatures |
## Migration Path
### Phase 1: Parallel Operation (Sprints 0001-0003)
- New versioned hashing alongside legacy
- New evidence model with adapters for existing types
- Graph root attestations optional
### Phase 2: Gradual Adoption (Post-Sprints)
- Emit migration warnings for legacy hashes
- Prefer IEvidence in new code
- Enable graph root attestations by default
### Phase 3: Deprecation (Future)
- Remove legacy hash acceptance
- Require IEvidence for all evidence storage
- Require graph root attestations for verification
## Compatibility Considerations
### Existing Attestations
Attestations generated before canonicalizer versioning remain valid:
- Verification detects legacy format (no `_canonVersion` field)
- Falls back to legacy canonicalization
- Logs warning for migration tracking
### Existing Evidence
Existing evidence types (`EvidenceBundle`, `EvidenceStatement`, etc.) continue working:
- Adapters convert to `IEvidence` on demand
- Original types remain in storage
- Gradual migration via write-through
### API Stability
Public APIs remain backward compatible:
- New methods added, not changed
- Optional parameters for new features
- Explicit opt-in for graph root attestations
## Performance Considerations
| Operation | Impact | Mitigation |
|-----------|--------|------------|
| Version field injection | ~100 bytes per hash | Negligible |
| Merkle tree computation | O(n log n) for n nodes | Existing algorithm; no change |
| Graph root attestation | +1 DSSE sign per graph | Batching; async |
| Evidence store queries | Index on subject_node_id | Composite index; pagination |
| Rekor publishing | Network latency | Optional; async; batching |
## Security Considerations
### Threat Model
| Threat | Mitigation |
|--------|------------|
| Hash collision attacks | SHA-256 with 256-bit security; version namespacing |
| Signature forgery | DSSE with ECDSA/EdDSA; key rotation |
| Evidence tampering | Content-addressed storage; Merkle verification |
| Replay attacks | Timestamp in provenance; Rekor log index |
| Canonicalization attacks | RFC 8785 compliance; explicit versioning |
### Key Management
Graph root attestations use the existing Signer module:
- Keys managed via Authority plugin
- Rotation policy applies
- Certificate chains optional for external verification
## References
- [RFC 8785 - JSON Canonicalization Scheme (JCS)](https://datatracker.ietf.org/doc/html/rfc8785)
- [in-toto Attestation Framework](https://github.com/in-toto/attestation)
- [DSSE - Dead Simple Signing Envelope](https://github.com/secure-systems-lab/dsse)
- [Sigstore Rekor](https://docs.sigstore.dev/rekor/overview/)
- [Merkle Tree - Wikipedia](https://en.wikipedia.org/wiki/Merkle_tree)
## Appendix A: File Locations
| Component | Path |
|-----------|------|
| CanonVersion | `src/__Libraries/StellaOps.Canonical.Json/CanonVersion.cs` |
| CanonJson (versioned) | `src/__Libraries/StellaOps.Canonical.Json/CanonJson.cs` |
| IEvidence | `src/__Libraries/StellaOps.Evidence.Core/IEvidence.cs` |
| EvidenceRecord | `src/__Libraries/StellaOps.Evidence.Core/EvidenceRecord.cs` |
| IEvidenceStore | `src/__Libraries/StellaOps.Evidence.Core/IEvidenceStore.cs` |
| Adapters | `src/__Libraries/StellaOps.Evidence.Core/Adapters/` |
| IGraphRootAttestor | `src/Attestor/__Libraries/StellaOps.Attestor.GraphRoot/IGraphRootAttestor.cs` |
| GraphRootAttestation | `src/Attestor/__Libraries/StellaOps.Attestor.GraphRoot/Models/` |
## Appendix B: Example Evidence Queries
```csharp
// Get all reachability evidence for a package
var evidence = await evidenceStore.GetBySubjectAsync(
subjectNodeId: "sha256:abc123...pkg:npm/lodash@4.17.21",
typeFilter: EvidenceType.Reachability);
// Verify a graph root attestation
var result = await graphRootAttestor.VerifyAsync(
envelope: downloadedEnvelope,
nodes: fetchedNodes,
edges: fetchedEdges);
if (!result.IsValid)
throw new VerificationException(result.FailureReason);
// Check if evidence exists before creating duplicate
if (!await evidenceStore.ExistsAsync(subjectNodeId, EvidenceType.Scan))
{
await evidenceStore.StoreAsync(newEvidence);
}
```

2
docs/modules/attestor/ttl-validation.md
View File

@@ -1,5 +1,7 @@
# Attestor TTL Validation Runbook
> **DEPRECATED:** This runbook tests MongoDB TTL indexes, which are no longer used. Attestor now uses PostgreSQL for persistence (Sprint 4400). See `docs/db/SPECIFICATION.md` for current database schema.
> **Purpose:** confirm MongoDB TTL indexes and Redis expirations for the attestation dedupe store behave as expected on a production-like stack.
## Prerequisites