feat: Add RustFS artifact object store and migration tool
	
		
			
	
		
	
	
		
	
		
			Some checks failed
		
		
	
	
		
			
				
	
				Docs CI / lint-and-preview (push) Has been cancelled
				
			
		
		
	
	
				
					
				
			
		
			Some checks failed
		
		
	
	Docs CI / lint-and-preview (push) Has been cancelled
				
			- Implemented RustFsArtifactObjectStore for managing artifacts in RustFS. - Added unit tests for RustFsArtifactObjectStore functionality. - Created a RustFS migrator tool to transfer objects from S3 to RustFS. - Introduced policy preview and report models for API integration. - Added fixtures and tests for policy preview and report functionality. - Included necessary metadata and scripts for cache_pkg package.
This commit is contained in:
		| @@ -1,6 +1,6 @@ | ||||
| # component_architecture_devops.md — **Stella Ops Release & Operations** (2025Q4) | ||||
|  | ||||
| > **Scope.** Implementation‑ready blueprint for **how Stella Ops is built, versioned, signed, distributed, upgraded, licensed (PoE)**, and operated in customer environments (online and air‑gapped). Covers reproducible builds, supply‑chain attestations, registries, offline kits, migration/rollback, artifact lifecycle (MinIO/Mongo), monitoring SLOs, and customer activation. | ||||
| > **Scope.** Implementation‑ready blueprint for **how Stella Ops is built, versioned, signed, distributed, upgraded, licensed (PoE)**, and operated in customer environments (online and air‑gapped). Covers reproducible builds, supply‑chain attestations, registries, offline kits, migration/rollback, artifact lifecycle (RustFS default + Mongo, S3 fallback), monitoring SLOs, and customer activation. | ||||
|  | ||||
| --- | ||||
|  | ||||
| @@ -257,12 +257,12 @@ Signer validates **scanner** image’s cosign identity + calendar tag for **rele | ||||
|  | ||||
| --- | ||||
|  | ||||
| ## 7) Artifact lifecycle & storage (MinIO/Mongo) | ||||
| ## 7) Artifact lifecycle & storage (RustFS/Mongo) | ||||
|  | ||||
| ### 7.1 Buckets & prefixes (MinIO) | ||||
| ### 7.1 Buckets & prefixes (RustFS) | ||||
|  | ||||
| ``` | ||||
| s3://stellaops/ | ||||
| rustfs://stellaops/ | ||||
|   scanner/ | ||||
|     layers/<sha256>/sbom.cdx.json.zst | ||||
|     images/<imgDigest>/inventory.cdx.pb | ||||
| @@ -283,7 +283,7 @@ s3://stellaops/ | ||||
|  | ||||
| * **`short`**: working artifacts (diffs, queues) — TTL 7–14 days. | ||||
| * **`default`**: SBOMs & indexes — TTL 90–180 days (configurable). | ||||
| * **`compliance`**: signed reports & attested exports — **Object Lock** (governance/compliance) 1–7 years. | ||||
| * **`compliance`**: signed reports & attested exports — retention enforced via RustFS hold or S3 Object Lock (governance/compliance) 1–7 years. | ||||
|  | ||||
| ### 7.3 Artifact Lifecycle Controller (ALC) | ||||
|  | ||||
| @@ -292,6 +292,9 @@ s3://stellaops/ | ||||
|   * Artifacts referenced by **reports** or **tickets** are pinned. | ||||
|   * ILM actions logged; UI shows per‑class usage & upcoming purges. | ||||
|  | ||||
| > **Migration note.** Follow `docs/ops/scanner-rustfs-migration.md` when transitioning existing | ||||
| > MinIO buckets to RustFS. The provided migrator is idempotent and safe to rerun per prefix. | ||||
|  | ||||
| ### 7.4 Mongo retention | ||||
|  | ||||
| * **Scanner**: `runtime.events` use TTL (e.g., 30–90 days); **catalog** permanent. | ||||
| @@ -313,7 +316,7 @@ s3://stellaops/ | ||||
| * **Golden signals**: | ||||
|  | ||||
|   * **Latency**: token issuance, sign→attest round‑trip, scan enqueue→emit, export build. | ||||
|   * **Saturation**: queue depth, Mongo write IOPS, MinIO net throughput. | ||||
|   * **Saturation**: queue depth, Mongo write IOPS, RustFS throughput / queue depth (or S3 metrics when in fallback mode). | ||||
|   * **Traffic**: scans/min, attestations/min, webhook admits/min. | ||||
|   * **Errors**: 5xx rates, cosign verification failures, Rekor timeouts. | ||||
|  | ||||
| @@ -460,7 +463,7 @@ services: | ||||
| * `attestor.submit_latency_seconds{quantile=0.95}` < 0.3. | ||||
| * `scanner.scan_latency_seconds{quantile=0.95}` < target per image size. | ||||
| * `concelier.export.duration_seconds` stable; `excititor.consensus.conflicts_total` not exploding after policy changes. | ||||
| * MinIO `s3_requests_errors_total` near zero; Mongo `opcounters` hit expected baseline. | ||||
| * RustFS request error rate near zero (or `s3_requests_errors_total` when operating against S3); Mongo `opcounters` hit expected baseline. | ||||
|  | ||||
| ### Appendix B — Upgrade safety checklist | ||||
|  | ||||
|   | ||||
		Reference in New Issue
	
	Block a user