Fix build and code structure improvements. New but essential UI functionality. CI improvements. Documentation improvements. AI module improvements.
This commit is contained in:
StellaOps Bot
278
docs/modules/evidence-locker/architecture.md
Normal file
278
docs/modules/evidence-locker/architecture.md
Normal file
@@ -0,0 +1,278 @@
|
||||
# component_architecture_evidence_locker.md - **Stella Ops EvidenceLocker** (2025Q4)
|
||||
|
||||
> Sealed, immutable storage for vulnerability scan evidence and audit logs.
|
||||
|
||||
> **Scope.** Implementation-ready architecture for **EvidenceLocker**: tamper-proof evidence chains for compliance and forensic analysis with content-addressable storage and cryptographic sealing.
|
||||
|
||||
---
|
||||
|
||||
## 0) Mission & boundaries
|
||||
|
||||
**Mission.** Provide **immutable, sealed storage** for vulnerability scan evidence, audit logs, and compliance artifacts. Ensure tamper-proof evidence chains with cryptographic verification for forensic analysis and regulatory compliance.
|
||||
|
||||
**Boundaries.**
|
||||
|
||||
* EvidenceLocker **stores** evidence; it does not generate verdicts.
|
||||
* EvidenceLocker **seals** bundles; signing is delegated to Signer.
|
||||
* Evidence is **immutable** once sealed; no modifications or deletions.
|
||||
* Supports **offline export** for air-gapped compliance audits.
|
||||
|
||||
---
|
||||
|
||||
## 1) Solution & project layout
|
||||
|
||||
```
|
||||
src/EvidenceLocker/StellaOps.EvidenceLocker/
|
||||
├─ StellaOps.EvidenceLocker.Core/ # Sealing, verification, chain validation
|
||||
│ ├─ Services/
|
||||
│ │ ├─ ISealingService.cs # Sealing interface
|
||||
│ │ ├─ SealingService.cs # Cryptographic sealing
|
||||
│ │ ├─ IVerificationService.cs # Verification interface
|
||||
│ │ └─ ChainValidator.cs # Evidence chain validation
|
||||
│ └─ Models/
|
||||
│ ├─ EvidenceBundle.cs # Bundle model
|
||||
│ ├─ EvidenceItem.cs # Individual evidence item
|
||||
│ └─ SealManifest.cs # Seal metadata
|
||||
│
|
||||
├─ StellaOps.EvidenceLocker.Infrastructure/ # Storage adapters, bundle management
|
||||
│ ├─ Storage/
|
||||
│ │ ├─ IEvidenceStore.cs # Storage interface
|
||||
│ │ ├─ FileSystemStore.cs # Local filesystem
|
||||
│ │ └─ ObjectStore.cs # S3/RustFS storage
|
||||
│ └─ Persistence/
|
||||
│ └─ PostgresRepository.cs # Metadata persistence
|
||||
│
|
||||
├─ StellaOps.EvidenceLocker.WebService/ # HTTP API for submission/retrieval
|
||||
│ └─ Program.cs
|
||||
│
|
||||
├─ StellaOps.EvidenceLocker.Worker/ # Background sealing and archival
|
||||
│ └─ SealingWorker.cs
|
||||
│
|
||||
└─ StellaOps.EvidenceLocker.Tests/ # Unit and integration tests
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2) External dependencies
|
||||
|
||||
* **PostgreSQL** - Metadata storage (schema: `evidence_locker`)
|
||||
* **RustFS/S3** - Object storage for evidence bundles
|
||||
* **Signer** - Cryptographic sealing operations
|
||||
* **Authority** - Authentication and authorization
|
||||
* **ExportCenter** - Evidence bundle export
|
||||
|
||||
---
|
||||
|
||||
## 3) Contracts & data model
|
||||
|
||||
### 3.1 EvidenceBundle
|
||||
|
||||
```json
|
||||
{
|
||||
"bundleId": "eb-2025-01-15-abc123",
|
||||
"tenantId": "tenant-xyz",
|
||||
"createdAt": "2025-01-15T10:30:00.000000Z",
|
||||
"sealedAt": "2025-01-15T10:30:05.000000Z",
|
||||
"status": "sealed",
|
||||
"items": [
|
||||
{
|
||||
"itemId": "item-001",
|
||||
"type": "sbom",
|
||||
"format": "cyclonedx-json",
|
||||
"digest": "sha256:abc123...",
|
||||
"size": 45678,
|
||||
"casUri": "cas://evidence/items/abc123"
|
||||
},
|
||||
{
|
||||
"itemId": "item-002",
|
||||
"type": "scan-result",
|
||||
"format": "stellaops-findings-v1",
|
||||
"digest": "sha256:def456...",
|
||||
"size": 12345,
|
||||
"casUri": "cas://evidence/items/def456"
|
||||
}
|
||||
],
|
||||
"seal": {
|
||||
"algorithm": "sha256",
|
||||
"rootHash": "sha256:fedcba...",
|
||||
"signature": "base64...",
|
||||
"keyId": "sha256:keyabc..."
|
||||
},
|
||||
"chain": {
|
||||
"previousBundleId": "eb-2025-01-14-xyz789",
|
||||
"previousRootHash": "sha256:prevhash...",
|
||||
"sequenceNumber": 42
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3.2 Evidence Item Types
|
||||
|
||||
| Type | Format | Description |
|
||||
|------|--------|-------------|
|
||||
| `sbom` | cyclonedx-json, spdx-json | Software Bill of Materials |
|
||||
| `scan-result` | stellaops-findings-v1 | Vulnerability scan findings |
|
||||
| `policy-verdict` | stellaops-verdict-v1 | Policy evaluation result |
|
||||
| `vex-statement` | openvex-v1 | VEX statement |
|
||||
| `audit-log` | ndjson | Audit trail events |
|
||||
| `attestation` | dsse-v1 | DSSE attestation envelope |
|
||||
|
||||
### 3.3 Seal Manifest
|
||||
|
||||
```csharp
|
||||
public sealed record SealManifest
|
||||
{
|
||||
public required string BundleId { get; init; }
|
||||
public required string RootHash { get; init; }
|
||||
public required string Algorithm { get; init; }
|
||||
public required DateTimeOffset SealedAt { get; init; }
|
||||
public required string KeyId { get; init; }
|
||||
public required byte[] Signature { get; init; }
|
||||
public required IReadOnlyList<string> ItemDigests { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4) REST API (EvidenceLocker.WebService)
|
||||
|
||||
All under `/api/v1/evidence`. Auth: **OpTok**.
|
||||
|
||||
```
|
||||
POST /bundles { items: EvidenceItem[] } → { bundleId, status: "pending" }
|
||||
POST /bundles/{id}/items { item: EvidenceItem } → { itemId }
|
||||
POST /bundles/{id}/seal → { status: "sealed", seal: SealManifest }
|
||||
|
||||
GET /bundles/{id} → { bundle: EvidenceBundle }
|
||||
GET /bundles/{id}/items/{itemId} → binary content
|
||||
GET /bundles/{id}/verify → { valid: bool, details }
|
||||
|
||||
GET /bundles?tenant={id}&from={date}&to={date} → { bundles: BundleSummary[] }
|
||||
|
||||
POST /export { bundleIds: string[], format: "zip"|"tar" } → { exportId }
|
||||
GET /export/{id} → binary archive
|
||||
GET /export/{id}/status → { status, progress }
|
||||
|
||||
GET /healthz | /readyz | /metrics
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5) Configuration (YAML)
|
||||
|
||||
```yaml
|
||||
EvidenceLocker:
|
||||
Postgres:
|
||||
ConnectionString: "Host=postgres;Database=evidence_locker;..."
|
||||
|
||||
Storage:
|
||||
Provider: "rustfs" # or "filesystem", "s3"
|
||||
RustFs:
|
||||
Endpoint: "http://rustfs:8080"
|
||||
Bucket: "stellaops-evidence"
|
||||
Filesystem:
|
||||
BasePath: "/data/evidence"
|
||||
|
||||
Sealing:
|
||||
Policy: "immediate" # or "batch"
|
||||
BatchSize: 100
|
||||
BatchIntervalSeconds: 60
|
||||
Algorithm: "sha256"
|
||||
|
||||
Retention:
|
||||
DefaultDays: 2555 # 7 years
|
||||
ComplianceDays: 3650 # 10 years for regulated
|
||||
|
||||
Export:
|
||||
MaxBundlesPerExport: 1000
|
||||
CompressionLevel: 6
|
||||
|
||||
Authority:
|
||||
Issuer: "https://authority.stellaops.local"
|
||||
RequiredScopes: ["evidence:read", "evidence:write"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6) Sealing Process
|
||||
|
||||
### 6.1 Sealing Flow
|
||||
|
||||
```
|
||||
1. Bundle created (status: "pending")
|
||||
└─ Items added with content digests
|
||||
|
||||
2. Sealing triggered (immediate or batch)
|
||||
├─ Compute Merkle root from item digests
|
||||
├─ Include chain pointer (previous bundle hash)
|
||||
└─ Request signature from Signer
|
||||
|
||||
3. Bundle sealed (status: "sealed")
|
||||
└─ Immutable; no further modifications
|
||||
```
|
||||
|
||||
### 6.2 Chain Integrity
|
||||
|
||||
Evidence chains are linked via Merkle roots:
|
||||
|
||||
```
|
||||
Bundle N-1 Bundle N Bundle N+1
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│ rootHash: H1│◄────────────│ prevHash: H1│◄─────────────│ prevHash: H2│
|
||||
│ seq: 41 │ │ rootHash: H2│ │ rootHash: H3│
|
||||
└─────────────┘ │ seq: 42 │ │ seq: 43 │
|
||||
└─────────────┘ └─────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7) Security & compliance
|
||||
|
||||
* **Immutability**: Sealed bundles cannot be modified
|
||||
* **Tamper detection**: Merkle tree verification for all items
|
||||
* **Chain validation**: Linked bundle verification
|
||||
* **Encryption at rest**: Optional bundle encryption
|
||||
* **Access control**: Tenant-scoped with Authority tokens
|
||||
* **Audit trail**: All access logged
|
||||
|
||||
---
|
||||
|
||||
## 8) Performance targets
|
||||
|
||||
* **Item ingestion**: < 100ms P95 per item
|
||||
* **Sealing**: < 500ms P95 per bundle (up to 100 items)
|
||||
* **Verification**: < 200ms P95 per bundle
|
||||
* **Export**: < 5s P95 for 100 bundles
|
||||
|
||||
---
|
||||
|
||||
## 9) Observability
|
||||
|
||||
**Metrics:**
|
||||
* `evidence.bundles.created_total{tenant}`
|
||||
* `evidence.bundles.sealed_total{tenant}`
|
||||
* `evidence.items.ingested_total{type}`
|
||||
* `evidence.verification.duration_seconds`
|
||||
* `evidence.storage.bytes_total`
|
||||
|
||||
**Tracing:** Spans for ingestion, sealing, verification, export.
|
||||
|
||||
---
|
||||
|
||||
## 10) Testing matrix
|
||||
|
||||
* **Sealing tests**: Correct Merkle root computation
|
||||
* **Chain tests**: Linked bundle verification
|
||||
* **Tamper tests**: Detection of modified items
|
||||
* **Export tests**: Archive integrity verification
|
||||
* **Retention tests**: Policy enforcement
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
* Bundle packaging: `./bundle-packaging.md`
|
||||
* Attestation contract: `./attestation-contract.md`
|
||||
* Evidence bundle spec: `./evidence-bundle-v1.md`
|
||||
* ExportCenter: `../export-center/architecture.md`
|
||||
* Attestor: `../attestor/architecture.md`
|
||||
Reference in New Issue
Block a user