feat: Add Go module and workspace test fixtures
- Created expected JSON files for Go modules and workspaces. - Added go.mod and go.sum files for example projects. - Implemented private module structure with expected JSON output. - Introduced vendored dependencies with corresponding expected JSON. - Developed PostgresGraphJobStore for managing graph jobs. - Established SQL migration scripts for graph jobs schema. - Implemented GraphJobRepository for CRUD operations on graph jobs. - Created IGraphJobRepository interface for repository abstraction. - Added unit tests for GraphJobRepository to ensure functionality.
This commit is contained in:
StellaOps Bot
157
docs/contracts/cas-infrastructure.md
Normal file
157
docs/contracts/cas-infrastructure.md
Normal file
@@ -0,0 +1,157 @@
|
||||
# CAS (Content Addressable Storage) Infrastructure Contract
|
||||
|
||||
> **Status:** APPROVED
|
||||
> **Version:** 1.0.0
|
||||
> **Last Updated:** 2025-12-06
|
||||
> **Owner:** Platform Storage Guild
|
||||
|
||||
## Overview
|
||||
|
||||
This contract defines the Content Addressable Storage (CAS) infrastructure for StellaOps, using RustFS as the S3-compatible storage backend. The design provides:
|
||||
|
||||
- **Content-addressed storage** — Objects addressed by SHA-256 hash
|
||||
- **Immutable evidence storage** — Write-once, never-delete for audit trails
|
||||
- **Lifecycle management** — Automated retention policy enforcement
|
||||
- **Service account isolation** — Fine-grained access control per service
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ CAS Infrastructure │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ rustfs-cas │ │ rustfs-evidence │ │rustfs-attestation│ │
|
||||
│ │ (mutable) │ │ (immutable) │ │ (immutable) │ │
|
||||
│ │ │ │ │ │ │ │
|
||||
│ │ • scanner- │ │ • evidence- │ │ • attestations │ │
|
||||
│ │ artifacts │ │ bundles │ │ • dsse-envelopes│ │
|
||||
│ │ • surface-cache │ │ • merkle-roots │ │ • rekor-receipts│ │
|
||||
│ │ • runtime-facts │ │ • hash-chains │ │ │ │
|
||||
│ │ • signals-data │ │ │ │ │ │
|
||||
│ │ • provenance- │ │ │ │ │ │
|
||||
│ │ feed │ │ │ │ │ │
|
||||
│ │ • replay- │ │ │ │ │ │
|
||||
│ │ bundles │ │ │ │ │ │
|
||||
│ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ │
|
||||
│ │ │ │ │
|
||||
│ └────────────────────┼────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────────┴───────────┐ │
|
||||
│ │ cas-lifecycle │ │
|
||||
│ │ (retention manager) │ │
|
||||
│ └───────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Retention Policies
|
||||
|
||||
Aligned with best-in-class on-premise vulnerability scanners:
|
||||
|
||||
| Data Type | Retention | Rationale | Scanner Comparison |
|
||||
|-----------|-----------|-----------|-------------------|
|
||||
| Vulnerability DB | 7 days | Fresh advisories required | Trivy: 7d, Grype: 5d |
|
||||
| SBOM artifacts | 365 days | Audit compliance (SOC2, ISO27001) | Anchore: 365d |
|
||||
| Scan results | 90 days | Common compliance window | Snyk: 90d enterprise |
|
||||
| Evidence bundles | Indefinite | Immutable audit trail | N/A (StellaOps unique) |
|
||||
| Attestations | Indefinite | Signed, verifiable | N/A (StellaOps unique) |
|
||||
| Temp artifacts | 1 day | Work-in-progress cleanup | Standard practice |
|
||||
|
||||
## Access Control Matrix
|
||||
|
||||
### Service Accounts
|
||||
|
||||
| Service | Buckets | Permissions | Purpose |
|
||||
|---------|---------|-------------|---------|
|
||||
| `scanner` | scanner-artifacts, surface-cache, runtime-facts | read, write | Scan job artifacts, cache |
|
||||
| `signals` | runtime-facts, signals-data, provenance-feed | read, write | Runtime signal ingestion |
|
||||
| `replay` | replay-bundles, inputs-lock | read, write | Deterministic replay |
|
||||
| `ledger` | evidence-bundles, merkle-roots, hash-chains | read, write | Evidence ledger writes |
|
||||
| `exporter` | evidence-bundles | read | Export center reads |
|
||||
| `attestor` | attestations, dsse-envelopes, rekor-receipts | read, write | Attestation storage |
|
||||
| `verifier` | attestations, dsse-envelopes, rekor-receipts | read | Verification reads |
|
||||
| `readonly` | * | read | Global audit access |
|
||||
|
||||
### Bucket Classification
|
||||
|
||||
| Bucket | Storage Type | Lifecycle | Access Pattern |
|
||||
|--------|--------------|-----------|----------------|
|
||||
| scanner-artifacts | rustfs-cas | 90 days | Write-heavy |
|
||||
| surface-cache | rustfs-cas | 7 days | Read-heavy, cache |
|
||||
| runtime-facts | rustfs-cas | 90 days | Write-heavy |
|
||||
| signals-data | rustfs-cas | 90 days | Write-heavy |
|
||||
| provenance-feed | rustfs-cas | 90 days | Append-only |
|
||||
| replay-bundles | rustfs-cas | 365 days | Read-heavy |
|
||||
| inputs-lock | rustfs-cas | 365 days | Write-once |
|
||||
| evidence-bundles | rustfs-evidence | Indefinite | Write-once |
|
||||
| merkle-roots | rustfs-evidence | Indefinite | Append-only |
|
||||
| hash-chains | rustfs-evidence | Indefinite | Append-only |
|
||||
| attestations | rustfs-attestation | Indefinite | Write-once |
|
||||
| dsse-envelopes | rustfs-attestation | Indefinite | Write-once |
|
||||
| rekor-receipts | rustfs-attestation | Indefinite | Write-once |
|
||||
|
||||
## Docker Compose Integration
|
||||
|
||||
```yaml
|
||||
# Use with existing compose files
|
||||
docker compose -f docker-compose.cas.yaml -f docker-compose.dev.yaml up -d
|
||||
|
||||
# Standalone CAS
|
||||
docker compose -f docker-compose.cas.yaml up -d
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
See `deploy/compose/env/cas.env.example` for full configuration.
|
||||
|
||||
Key variables:
|
||||
- `RUSTFS_*_API_KEY` — Admin API keys (CHANGE IN PRODUCTION)
|
||||
- `RUSTFS_*_KEY` — Service account keys (GENERATE UNIQUE)
|
||||
- `CAS_*_PATH` — Data directory paths
|
||||
- `CAS_RETENTION_*_DAYS` — Retention policy overrides
|
||||
|
||||
## Endpoints
|
||||
|
||||
| Service | Port | Path | Purpose |
|
||||
|---------|------|------|---------|
|
||||
| rustfs-cas | 8180 | /api/v1 | Mutable CAS storage |
|
||||
| rustfs-evidence | 8181 | /api/v1 | Immutable evidence |
|
||||
| rustfs-attestation | 8182 | /api/v1 | Immutable attestations |
|
||||
|
||||
## Health Checks
|
||||
|
||||
All RustFS instances expose `/health` endpoint:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8180/health # CAS
|
||||
curl http://localhost:8181/health # Evidence
|
||||
curl http://localhost:8182/health # Attestations
|
||||
```
|
||||
|
||||
## Migration from MinIO
|
||||
|
||||
For existing deployments using MinIO:
|
||||
|
||||
1. Deploy CAS infrastructure alongside MinIO
|
||||
2. Configure scanner/signals services with `RUSTFS_*` endpoints
|
||||
3. Migrate data using `stella cas migrate --source minio --target rustfs`
|
||||
4. Verify data integrity with `stella cas verify --bucket <name>`
|
||||
5. Update service configurations to use RustFS
|
||||
6. Decommission MinIO after validation
|
||||
|
||||
## Tasks Unblocked
|
||||
|
||||
This contract unblocks the CAS approval gate (PREP-SIGNALS-24-002):
|
||||
|
||||
- **24-002:** Surface cache availability → UNBLOCKED
|
||||
- **24-003:** Runtime facts ingestion → UNBLOCKED
|
||||
- **24-004:** Authority scopes → UNBLOCKED
|
||||
- **24-005:** Scoring outputs → UNBLOCKED
|
||||
- **GRAPH-INDEX-28-007 through 28-010** → UNBLOCKED
|
||||
|
||||
## Changelog
|
||||
|
||||
| Date | Version | Change |
|
||||
|------|---------|--------|
|
||||
| 2025-12-06 | 1.0.0 | Initial contract with RustFS, retention policies, access controls |
|
||||
Reference in New Issue
Block a user