05597616d6
- Created expected JSON files for Go modules and workspaces. - Added go.mod and go.sum files for example projects. - Implemented private module structure with expected JSON output. - Introduced vendored dependencies with corresponding expected JSON. - Developed PostgresGraphJobStore for managing graph jobs. - Established SQL migration scripts for graph jobs schema. - Implemented GraphJobRepository for CRUD operations on graph jobs. - Created IGraphJobRepository interface for repository abstraction. - Added unit tests for GraphJobRepository to ensure functionality.
158 lines
8.1 KiB
Markdown
158 lines
8.1 KiB
Markdown
# CAS (Content Addressable Storage) Infrastructure Contract
|
|
|
|
> **Status:** APPROVED
|
|
> **Version:** 1.0.0
|
|
> **Last Updated:** 2025-12-06
|
|
> **Owner:** Platform Storage Guild
|
|
|
|
## Overview
|
|
|
|
This contract defines the Content Addressable Storage (CAS) infrastructure for StellaOps, using RustFS as the S3-compatible storage backend. The design provides:
|
|
|
|
- **Content-addressed storage** — Objects addressed by SHA-256 hash
|
|
- **Immutable evidence storage** — Write-once, never-delete for audit trails
|
|
- **Lifecycle management** — Automated retention policy enforcement
|
|
- **Service account isolation** — Fine-grained access control per service
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ CAS Infrastructure │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
|
│ │ rustfs-cas │ │ rustfs-evidence │ │rustfs-attestation│ │
|
|
│ │ (mutable) │ │ (immutable) │ │ (immutable) │ │
|
|
│ │ │ │ │ │ │ │
|
|
│ │ • scanner- │ │ • evidence- │ │ • attestations │ │
|
|
│ │ artifacts │ │ bundles │ │ • dsse-envelopes│ │
|
|
│ │ • surface-cache │ │ • merkle-roots │ │ • rekor-receipts│ │
|
|
│ │ • runtime-facts │ │ • hash-chains │ │ │ │
|
|
│ │ • signals-data │ │ │ │ │ │
|
|
│ │ • provenance- │ │ │ │ │ │
|
|
│ │ feed │ │ │ │ │ │
|
|
│ │ • replay- │ │ │ │ │ │
|
|
│ │ bundles │ │ │ │ │ │
|
|
│ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ │
|
|
│ │ │ │ │
|
|
│ └────────────────────┼────────────────────┘ │
|
|
│ │ │
|
|
│ ┌───────────┴───────────┐ │
|
|
│ │ cas-lifecycle │ │
|
|
│ │ (retention manager) │ │
|
|
│ └───────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Retention Policies
|
|
|
|
Aligned with best-in-class on-premise vulnerability scanners:
|
|
|
|
| Data Type | Retention | Rationale | Scanner Comparison |
|
|
|-----------|-----------|-----------|-------------------|
|
|
| Vulnerability DB | 7 days | Fresh advisories required | Trivy: 7d, Grype: 5d |
|
|
| SBOM artifacts | 365 days | Audit compliance (SOC2, ISO27001) | Anchore: 365d |
|
|
| Scan results | 90 days | Common compliance window | Snyk: 90d enterprise |
|
|
| Evidence bundles | Indefinite | Immutable audit trail | N/A (StellaOps unique) |
|
|
| Attestations | Indefinite | Signed, verifiable | N/A (StellaOps unique) |
|
|
| Temp artifacts | 1 day | Work-in-progress cleanup | Standard practice |
|
|
|
|
## Access Control Matrix
|
|
|
|
### Service Accounts
|
|
|
|
| Service | Buckets | Permissions | Purpose |
|
|
|---------|---------|-------------|---------|
|
|
| `scanner` | scanner-artifacts, surface-cache, runtime-facts | read, write | Scan job artifacts, cache |
|
|
| `signals` | runtime-facts, signals-data, provenance-feed | read, write | Runtime signal ingestion |
|
|
| `replay` | replay-bundles, inputs-lock | read, write | Deterministic replay |
|
|
| `ledger` | evidence-bundles, merkle-roots, hash-chains | read, write | Evidence ledger writes |
|
|
| `exporter` | evidence-bundles | read | Export center reads |
|
|
| `attestor` | attestations, dsse-envelopes, rekor-receipts | read, write | Attestation storage |
|
|
| `verifier` | attestations, dsse-envelopes, rekor-receipts | read | Verification reads |
|
|
| `readonly` | * | read | Global audit access |
|
|
|
|
### Bucket Classification
|
|
|
|
| Bucket | Storage Type | Lifecycle | Access Pattern |
|
|
|--------|--------------|-----------|----------------|
|
|
| scanner-artifacts | rustfs-cas | 90 days | Write-heavy |
|
|
| surface-cache | rustfs-cas | 7 days | Read-heavy, cache |
|
|
| runtime-facts | rustfs-cas | 90 days | Write-heavy |
|
|
| signals-data | rustfs-cas | 90 days | Write-heavy |
|
|
| provenance-feed | rustfs-cas | 90 days | Append-only |
|
|
| replay-bundles | rustfs-cas | 365 days | Read-heavy |
|
|
| inputs-lock | rustfs-cas | 365 days | Write-once |
|
|
| evidence-bundles | rustfs-evidence | Indefinite | Write-once |
|
|
| merkle-roots | rustfs-evidence | Indefinite | Append-only |
|
|
| hash-chains | rustfs-evidence | Indefinite | Append-only |
|
|
| attestations | rustfs-attestation | Indefinite | Write-once |
|
|
| dsse-envelopes | rustfs-attestation | Indefinite | Write-once |
|
|
| rekor-receipts | rustfs-attestation | Indefinite | Write-once |
|
|
|
|
## Docker Compose Integration
|
|
|
|
```yaml
|
|
# Use with existing compose files
|
|
docker compose -f docker-compose.cas.yaml -f docker-compose.dev.yaml up -d
|
|
|
|
# Standalone CAS
|
|
docker compose -f docker-compose.cas.yaml up -d
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
See `deploy/compose/env/cas.env.example` for full configuration.
|
|
|
|
Key variables:
|
|
- `RUSTFS_*_API_KEY` — Admin API keys (CHANGE IN PRODUCTION)
|
|
- `RUSTFS_*_KEY` — Service account keys (GENERATE UNIQUE)
|
|
- `CAS_*_PATH` — Data directory paths
|
|
- `CAS_RETENTION_*_DAYS` — Retention policy overrides
|
|
|
|
## Endpoints
|
|
|
|
| Service | Port | Path | Purpose |
|
|
|---------|------|------|---------|
|
|
| rustfs-cas | 8180 | /api/v1 | Mutable CAS storage |
|
|
| rustfs-evidence | 8181 | /api/v1 | Immutable evidence |
|
|
| rustfs-attestation | 8182 | /api/v1 | Immutable attestations |
|
|
|
|
## Health Checks
|
|
|
|
All RustFS instances expose `/health` endpoint:
|
|
|
|
```bash
|
|
curl http://localhost:8180/health # CAS
|
|
curl http://localhost:8181/health # Evidence
|
|
curl http://localhost:8182/health # Attestations
|
|
```
|
|
|
|
## Migration from MinIO
|
|
|
|
For existing deployments using MinIO:
|
|
|
|
1. Deploy CAS infrastructure alongside MinIO
|
|
2. Configure scanner/signals services with `RUSTFS_*` endpoints
|
|
3. Migrate data using `stella cas migrate --source minio --target rustfs`
|
|
4. Verify data integrity with `stella cas verify --bucket <name>`
|
|
5. Update service configurations to use RustFS
|
|
6. Decommission MinIO after validation
|
|
|
|
## Tasks Unblocked
|
|
|
|
This contract unblocks the CAS approval gate (PREP-SIGNALS-24-002):
|
|
|
|
- **24-002:** Surface cache availability → UNBLOCKED
|
|
- **24-003:** Runtime facts ingestion → UNBLOCKED
|
|
- **24-004:** Authority scopes → UNBLOCKED
|
|
- **24-005:** Scoring outputs → UNBLOCKED
|
|
- **GRAPH-INDEX-28-007 through 28-010** → UNBLOCKED
|
|
|
|
## Changelog
|
|
|
|
| Date | Version | Change |
|
|
|------|---------|--------|
|
|
| 2025-12-06 | 1.0.0 | Initial contract with RustFS, retention policies, access controls |
|