docs consolidation

This commit is contained in:
StellaOps Bot
2025-12-24 21:45:46 +02:00
parent 4231305fec
commit 43e2af88f6
76 changed files with 2887 additions and 796 deletions

View File

@@ -28,7 +28,7 @@
* **Rekor v2** — tilebacked transparency log endpoint(s).
* **MinIO (S3)** — optional archive store for DSSE envelopes & verification bundles.
* **PostgreSQL** — local cache of `{uuid, index, proof, artifactSha256, bundleSha256}`; job state; audit.
* **Redis** — dedupe/idempotency keys and shortlived ratelimit buckets.
* **Valkey** — dedupe/idempotency keys and shortlived ratelimit buckets.
* **Licensing Service (optional)** — “endorse” call for crosslog publishing when customer optsin.
Trust boundary: **Only the Signer** is allowed to call submission endpoints; enforced by **mTLS peer cert allowlist** + `aud=attestor` OpTok.
@@ -619,8 +619,8 @@ attestor:
bucket: "stellaops"
prefix: "attest/"
objectLock: "governance"
redis:
url: "redis://redis:6379/2"
valkey:
url: "valkey://valkey:6379/2"
quotas:
perCaller:
qps: 50

View File

@@ -0,0 +1,238 @@
# Graph Root Attestation
## Overview
Graph root attestation is a mechanism for creating cryptographically signed, content-addressed proofs of graph state. It enables offline verification that replayed graphs match the original attested state by computing a Merkle root from sorted node/edge IDs and input digests, then wrapping it in a DSSE envelope with an in-toto statement.
## Purpose
Graph root attestations solve the problem of proving graph authenticity without reconstructing the entire proof chain. They enable:
- **Offline Verification**: Download an attestation, recompute the root from stored nodes/edges, compare
- **Audit Snapshots**: Point-in-time proof of graph state for compliance
- **Evidence Linking**: Reference attested roots (not transient IDs) in evidence chains
- **Transparency**: Optional Rekor publication for public auditability
## Architecture
### Components
```
┌─────────────────────────────────────────────────────────────────┐
│ GraphRootAttestor │
├─────────────────────────────────────────────────────────────────┤
│ AttestAsync(request) → GraphRootAttestationResult │
│ VerifyAsync(envelope, nodes, edges) → VerificationResult │
└─────────────────┬───────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ IMerkleRootComputer │
├─────────────────────────────────────────────────────────────────┤
│ ComputeRoot(leaves) → byte[] │
│ Algorithm → "sha256" │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ EnvelopeSignatureService │
├─────────────────────────────────────────────────────────────────┤
│ Sign(payload, key) → EnvelopeSignature │
└─────────────────────────────────────────────────────────────────┘
```
### Data Flow
1. **Request** → Sorted node/edge IDs + input digests
2. **Merkle Tree** → Compute SHA-256 root from leaves
3. **In-Toto Statement** → Build attestation with predicate
4. **Canonicalize** → JCS (RFC 8785) with version marker
5. **Sign** → DSSE envelope with Ed25519/ECDSA
6. **Store/Publish** → CAS storage + optional Rekor
## Graph Types
The `GraphType` enum identifies what kind of graph is being attested:
| GraphType | Description |
|-----------|-------------|
| `Unknown` | Unspecified graph type |
| `CallGraph` | Function/method call relationships |
| `DependencyGraph` | Package/library dependencies (SBOM) |
| `SbomGraph` | SBOM component graph |
| `EvidenceGraph` | Linked evidence records |
| `PolicyGraph` | Policy decision trees |
| `ProofSpine` | Proof chain spine segments |
| `ReachabilityGraph` | Code reachability analysis |
| `VexLinkageGraph` | VEX statement linkages |
| `Custom` | Application-specific graph |
## Models
### GraphRootAttestationRequest
Input to the attestation service:
```csharp
public sealed record GraphRootAttestationRequest
{
public required GraphType GraphType { get; init; }
public required IReadOnlyList<string> NodeIds { get; init; }
public required IReadOnlyList<string> EdgeIds { get; init; }
public required string PolicyDigest { get; init; }
public required string FeedsDigest { get; init; }
public required string ToolchainDigest { get; init; }
public required string ParamsDigest { get; init; }
public required string ArtifactDigest { get; init; }
public IReadOnlyList<string> EvidenceIds { get; init; } = [];
public bool PublishToRekor { get; init; } = false;
public string? SigningKeyId { get; init; }
}
```
### GraphRootAttestation (In-Toto Statement)
The attestation follows the in-toto Statement/v1 format:
```json
{
"_type": "https://in-toto.io/Statement/v1",
"subject": [
{
"name": "sha256:abc123...",
"digest": { "sha256": "abc123..." }
},
{
"name": "sha256:artifact...",
"digest": { "sha256": "artifact..." }
}
],
"predicateType": "https://stella-ops.org/attestation/graph-root/v1",
"predicate": {
"graphType": "DependencyGraph",
"rootHash": "sha256:abc123...",
"rootAlgorithm": "sha256",
"nodeCount": 1247,
"edgeCount": 3891,
"nodeIds": ["node-a", "node-b", ...],
"edgeIds": ["edge-1", "edge-2", ...],
"inputs": {
"policyDigest": "sha256:policy...",
"feedsDigest": "sha256:feeds...",
"toolchainDigest": "sha256:tools...",
"paramsDigest": "sha256:params..."
},
"evidenceIds": ["ev-1", "ev-2"],
"canonVersion": "stella:canon:v1",
"computedAt": "2025-12-26T10:30:00Z",
"computedBy": "stellaops/attestor/graph-root",
"computedByVersion": "1.0.0"
}
}
```
## Merkle Root Computation
The root is computed from leaves in this deterministic order:
1. **Sorted node IDs** (lexicographic, ordinal)
2. **Sorted edge IDs** (lexicographic, ordinal)
3. **Policy digest**
4. **Feeds digest**
5. **Toolchain digest**
6. **Params digest**
Each leaf is SHA-256 hashed, then combined pairwise until a single root remains.
```
ROOT
/ \
H(L12) H(R12)
/ \ / \
H(n1) H(n2) H(e1) H(policy)
```
## Usage
### Creating an Attestation
```csharp
var services = new ServiceCollection();
services.AddGraphRootAttestation(sp => keyId => GetSigningKey(keyId));
var provider = services.BuildServiceProvider();
var attestor = provider.GetRequiredService<IGraphRootAttestor>();
var request = new GraphRootAttestationRequest
{
GraphType = GraphType.DependencyGraph,
NodeIds = graph.Nodes.Select(n => n.Id).ToList(),
EdgeIds = graph.Edges.Select(e => e.Id).ToList(),
PolicyDigest = "sha256:abc...",
FeedsDigest = "sha256:def...",
ToolchainDigest = "sha256:ghi...",
ParamsDigest = "sha256:jkl...",
ArtifactDigest = imageDigest
};
var result = await attestor.AttestAsync(request);
Console.WriteLine($"Root: {result.RootHash}");
Console.WriteLine($"Nodes: {result.NodeCount}");
Console.WriteLine($"Edges: {result.EdgeCount}");
```
### Verifying an Attestation
```csharp
var envelope = LoadEnvelope("attestation.dsse.json");
var nodes = LoadNodes("nodes.ndjson");
var edges = LoadEdges("edges.ndjson");
var result = await attestor.VerifyAsync(envelope, nodes, edges);
if (result.IsValid)
{
Console.WriteLine($"✓ Verified: {result.ComputedRoot}");
Console.WriteLine($" Nodes: {result.NodeCount}");
Console.WriteLine($" Edges: {result.EdgeCount}");
}
else
{
Console.WriteLine($"✗ Failed: {result.FailureReason}");
Console.WriteLine($" Expected: {result.ExpectedRoot}");
Console.WriteLine($" Computed: {result.ComputedRoot}");
}
```
## Offline Verification Workflow
1. **Obtain attestation**: Download DSSE envelope from storage or transparency log
2. **Verify signature**: Check envelope signature against trusted public keys
3. **Extract predicate**: Parse `GraphRootPredicate` from the payload
4. **Fetch graph data**: Download nodes and edges by ID from CAS
5. **Recompute root**: Apply Merkle tree algorithm to node/edge IDs + input digests
6. **Compare**: Computed root must match `predicate.RootHash`
## Determinism Guarantees
Graph root attestations are fully deterministic:
- **Sorting**: All IDs sorted lexicographically (ordinal comparison)
- **Canonicalization**: RFC 8785 JCS with `stella:canon:v1` version marker
- **Hashing**: SHA-256 only
- **Timestamps**: UTC ISO-8601 (not included in root computation)
Same inputs always produce the same root hash, enabling replay verification.
## Security Considerations
- **Signature verification**: Always verify DSSE envelope signatures before trusting attestations
- **Key management**: Use short-lived signing keys; rotate regularly
- **Transparency**: Publish to Rekor for tamper-evident audit trail
- **Input validation**: Validate all digests are properly formatted before attestation
## Related Documentation
- [DSSE Envelopes](./dsse-envelopes.md) - Envelope format and signing
- [Proof Chain](./proof-chain.md) - Overall proof chain architecture
- [Canonical JSON](../../modules/platform/canonical-json.md) - Canonicalization scheme

View File

@@ -1,6 +0,0 @@
# Keys and Issuers (DOCS-ATTEST-74-001)
- Maintain issuer registry (KMS IDs, key IDs, allowed predicates).
- Rotate keys with overlap; publish fingerprints and validity in registry file.
- Offline operation: bundle registry with bootstrap; no remote fetch.
- Each attestation must include issuer ID and key ID; verify against registry.

View File

@@ -1,9 +0,0 @@
# Attestor Overview (DOCS-ATTEST-73-001)
High-level description of the Attestor service and its contracts.
- Purpose: verify DSSE/attestations, supply transparency info, and expose attestation APIs without deriving verdicts.
- Components: WebService, Worker, KMS integration, Transparency log (optional), Evidence links.
- Rule banner: aggregation-only; no policy decisions.
- Tenancy: all attestations scoped per tenant; cross-tenant reads forbidden.
- Offline posture: allow offline verification using bundled trust roots and Rekor checkpoints when available.

View File

@@ -1,12 +0,0 @@
# Attestor Policies (DOCS-ATTEST-73-003)
Guidance on verification policies applied by Attestor.
- Scope: DSSE envelope validation, subject hash matching, optional transparency checks.
- Policy fields:
- allowed issuers / key IDs
- required predicates (e.g., `stella.ops/vexObservation@v1`)
- transparency requirements (allow/require/skip)
- freshness window for attestations
- Determinism: policies must be pure; no external lookups in sealed mode.
- Versioning: include `policyVersion` and hash; store alongside attestation records.

View File

@@ -90,6 +90,68 @@ This specification defines the implementation of a cryptographically verifiable
5. **Numbers in shortest form**
6. **Deterministic array ordering** (by semantic key: bom-ref, purl)
### Canonicalization Versioning
Content-addressed identifiers embed a canonicalization version marker to prevent hash collisions when the canonicalization algorithm evolves. This ensures that:
- **Forward compatibility**: Future algorithm changes won't invalidate existing hashes.
- **Verifier clarity**: Verifiers know exactly which algorithm to use.
- **Auditability**: Hash provenance is cryptographically bound to algorithm version.
**Version Marker Format:**
```json
{
"_canonVersion": "stella:canon:v1",
"sbomEntryId": "...",
"vulnerabilityId": "..."
}
```
| Field | Description |
|-------|-------------|
| `_canonVersion` | Underscore prefix ensures lexicographic first position after sorting |
| Value format | `stella:canon:v<N>` where N is the version number |
| Current version | `stella:canon:v1` (RFC 8785 JSON canonicalization) |
**V1 Algorithm Specification:**
| Property | Behavior |
|----------|----------|
| Standard | RFC 8785 (JSON Canonicalization Scheme) |
| Key sorting | Ordinal string comparison |
| Whitespace | None (compact JSON) |
| Encoding | UTF-8 without BOM |
| Numbers | IEEE 754, shortest representation |
| Escaping | Minimal (only required characters) |
**Version Detection:**
```csharp
// Detect if canonical JSON includes version marker
public static bool IsVersioned(ReadOnlySpan<byte> canonicalJson)
{
return canonicalJson.Length > 20 &&
canonicalJson.StartsWith("{\"_canonVersion\":"u8);
}
// Extract version from versioned canonical JSON
public static string? ExtractVersion(ReadOnlySpan<byte> canonicalJson)
{
// Parse and return the _canonVersion value, or null if not versioned
}
```
**Migration Strategy:**
| Phase | Behavior | Timeline |
|-------|----------|----------|
| Phase 1 (Current) | Generate v1 hashes; accept both legacy and v1 for verification | Now |
| Phase 2 | Log deprecation warnings for legacy hashes | +6 months |
| Phase 3 | Reject legacy hashes; require v1 | +12 months |
See also: [Canonicalization Migration Guide](../../operations/canon-version-migration.md)
## DSSE Predicate Types
### 1. Evidence Statement (`evidence.stella/v1`)
@@ -194,6 +256,101 @@ This specification defines the implementation of a cryptographically verifiable
```
**Signer**: Generator key
### 7. Graph Root Statement (`graph-root.stella/v1`)
The Graph Root attestation provides tamper-evident commitment to graph analysis results (dependency graphs, call graphs, reachability graphs) by computing a Merkle root over canonicalized node and edge identifiers.
```json
{
"_type": "https://in-toto.io/Statement/v1",
"subject": [
{
"name": "graph-root://<graphType>/<merkleRoot>",
"digest": {
"sha256": "<merkle-root-hex>"
}
}
],
"predicateType": "https://stella-ops.org/predicates/graph-root/v1",
"predicate": {
"graphType": "DependencyGraph|CallGraph|ReachabilityGraph|...",
"merkleRoot": "sha256:<hex>",
"nodeCount": 1234,
"edgeCount": 5678,
"canonVersion": "stella:canon:v1",
"inputs": {
"sbomDigest": "sha256:<hex>",
"analyzerDigest": "sha256:<hex>",
"configDigest": "sha256:<hex>"
},
"createdAt": "2025-01-12T10:30:00Z"
}
}
```
**Signer**: Graph Analyzer key
#### Supported Graph Types
| Graph Type | Use Case |
|------------|----------|
| `DependencyGraph` | Package/library dependency analysis |
| `CallGraph` | Function-level call relationships |
| `ReachabilityGraph` | Vulnerability reachability analysis |
| `DataFlowGraph` | Data flow and taint tracking |
| `ControlFlowGraph` | Code execution paths |
| `InheritanceGraph` | OOP class hierarchies |
| `ModuleGraph` | Module/namespace dependencies |
| `BuildGraph` | Build system dependencies |
| `ContainerLayerGraph` | Container layer relationships |
#### Merkle Root Computation
The Merkle root is computed deterministically:
1. **Canonicalize Node IDs**: Sort all node identifiers lexicographically
2. **Canonicalize Edge IDs**: Sort all edge identifiers (format: `{source}->{target}`)
3. **Combine**: Concatenate sorted nodes + sorted edges
4. **Binary Tree**: Build SHA-256 Merkle tree with odd-node duplication
5. **Root**: Extract 32-byte root as `sha256:<hex>`
```
Merkle Tree Structure:
[root]
/ \
[h01] [h23]
/ \ / \
[n0] [n1] [n2] [n3]
```
#### Integration with Proof Spine
Graph root attestations can be referenced in proof spines:
```json
{
"predicateType": "proofspine.stella/v1",
"predicate": {
"sbomEntryId": "<SBOMEntryID>",
"evidenceIds": ["<ID1>", "<ID2>"],
"reasoningId": "<ID>",
"vexVerdictId": "<ID>",
"graphRootIds": ["<GraphRootID1>"],
"policyVersion": "v2.3.1",
"proofBundleId": "<ProofBundleID>"
}
}
```
#### Verification Steps
1. Parse DSSE envelope and verify signature against allowed keys
2. Extract predicate and Merkle root
3. Re-canonicalize provided node/edge IDs using `stella:canon:v1`
4. Recompute Merkle root from canonicalized inputs
5. Compare computed root to claimed root
6. If Rekor entry exists, verify transparency log inclusion
## Database Schema
### Tables
@@ -205,6 +362,7 @@ This specification defines the implementation of a cryptographically verifiable
| `proofchain.spines` | Proof spine aggregations linking evidence to verdicts |
| `proofchain.trust_anchors` | Trust anchor configurations for verification |
| `proofchain.rekor_entries` | Rekor transparency log entries |
| `proofchain.graph_roots` | Graph root attestations with Merkle roots |
| `proofchain.key_history` | Key lifecycle history for rotation |
| `proofchain.key_audit_log` | Audit log for key operations |
@@ -282,6 +440,7 @@ The 13-step verification algorithm:
- [Database Schema Sprint](../../implplan/SPRINT_0501_0006_0001_proof_chain_database_schema.md)
- [CLI Integration Sprint](../../implplan/SPRINT_0501_0007_0001_proof_chain_cli_integration.md)
- [Key Rotation Sprint](../../implplan/SPRINT_0501_0008_0001_proof_chain_key_rotation.md)
- [Graph Root Attestation](./graph-root-attestation.md)
- [Attestor Architecture](./architecture.md)
- [Signer Architecture](../signer/architecture.md)
- [Database Specification](../../db/SPECIFICATION.md)

View File

@@ -1,49 +0,0 @@
# Attestor TTL Validation Runbook
> **DEPRECATED:** This runbook tests MongoDB TTL indexes, which are no longer used. Attestor now uses PostgreSQL for persistence (Sprint 4400). See `docs/db/SPECIFICATION.md` for current database schema.
> **Purpose:** confirm MongoDB TTL indexes and Redis expirations for the attestation dedupe store behave as expected on a production-like stack.
## Prerequisites
- Docker Desktop or compatible daemon with the Compose plugin enabled.
- Local ports `27017` and `6379` free.
- `dotnet` SDK 10.0 preview (same as repo toolchain).
- Network access to pull `mongo:7` and `redis:7` images.
## Quickstart
1. From the repo root export any required proxy settings, then run
```bash
scripts/run-attestor-ttl-validation.sh
```
The helper script:
- Spins up `mongo:7` and `redis:7` containers.
- Sets `ATTESTOR_LIVE_MONGO_URI` / `ATTESTOR_LIVE_REDIS_URI`.
- Executes the live TTL test suite (`Category=LiveTTL`) in `StellaOps.Attestor.Tests`.
- Tears the stack down automatically.
2. Capture the test output (`ttl-validation-<timestamp>.log`) and attach it to the sprint evidence folder (`docs/modules/attestor/evidence/`).
## Result handling
- **Success:** Tests complete in ~34 minutes with `Total tests: 2, Passed: 2`. Store the log and note the run in `docs/implplan/archived/SPRINT_0100_0001_0001_identity_signing.md` under ATTESTOR-72-003.
- **Failure:** Preserve:
- `docker compose logs` for both services.
- `mongosh` output of `db.dedupe.getIndexes()` and sample documents.
- `redis-cli --raw ttl attestor:ttl:live:bundle:<id>`.
File an incident in the Attestor Guild channel and link the captured artifacts.
## Manual verification (optional)
If the helper script cannot be used:
1. Start MongoDB and Redis manually with equivalent configuration.
2. Set `ATTESTOR_LIVE_MONGO_URI` and `ATTESTOR_LIVE_REDIS_URI`.
3. Run `dotnet test src/Attestor/StellaOps.Attestor.sln --no-build --filter "Category=LiveTTL"`.
4. Follow the evidence handling steps above.
## Ownership
- Primary: Attestor Service Guild.
- Partner: QA Guild (observes TTL metrics, confirms evidence archiving).
## 2025-11-03 validation summary
- **Stack:** `mongod` 7.0.5 (tarball) + `mongosh` 2.0.2, `redis-server` 7.2.4 (source build) running on localhost without Docker.
- **Mongo results:** `dedupe` TTL index (`ttlAt`, `expireAfterSeconds: 0`) confirmed; document inserted with 20s TTL expired automatically after ~80s (expected allocator sweep). Evidence: `docs/modules/attestor/evidence/2025-11-03-mongo-ttl-validation.txt`.
- **Redis results:** Key `attestor:ttl:live:bundle:validation` set with 45s TTL reached `TTL=-2` after ~47s confirming expiry propagation. Evidence: `docs/modules/attestor/evidence/2025-11-03-redis-ttl-validation.txt`.
- **Notes:** Local binaries built/run to accommodate sandbox without Docker; services shut down after validation.

View File

@@ -1,9 +0,0 @@
# Attestor Workflows (DOCS-ATTEST-73-004)
Sequence of ingest, verify, and bulk operations.
1. **Ingest**: receive DSSE, validate schema, hash subjects, store envelope + metadata.
2. **Verify**: run policy checks (issuer, predicate, transparency optional), compute verification record.
3. **Persist**: store verification result with `verificationId`, `attestationId`, `policyVersion`, timestamps.
4. **Bulk ops**: batch verify envelopes; export results to timeline/audit logs.
5. **Audit**: expose read API for verification records; include determinism hash of inputs.