feat: Add RustFS artifact object store and migration tool
- Implemented RustFsArtifactObjectStore for managing artifacts in RustFS. - Added unit tests for RustFsArtifactObjectStore functionality. - Created a RustFS migrator tool to transfer objects from S3 to RustFS. - Introduced policy preview and report models for API integration. - Added fixtures and tests for policy preview and report functionality. - Included necessary metadata and scripts for cache_pkg package.
This commit is contained in:
@@ -49,7 +49,7 @@
|
||||
|
||||
* **Fulcio** (Sigstore CA) — issues short‑lived signing certs (keyless).
|
||||
* **Rekor v2** (tile‑backed transparency log).
|
||||
* **MinIO** — S3‑compatible object store with lifecycle & Object Lock.
|
||||
* **RustFS** — offline-first object store with deterministic REST API (S3/MinIO fallback available for legacy installs).
|
||||
* **MongoDB** — catalog, advisories, VEX, scheduler, notify.
|
||||
* **Queue** — Redis Streams / NATS / RabbitMQ (pluggable).
|
||||
* **OCI Registry** — must support **Referrers API** (discover SBOMs/signatures).
|
||||
@@ -81,7 +81,7 @@ flowchart LR
|
||||
ATT[Attestor\n(Rekor v2 submit/verify)]
|
||||
UI[Web UI (Angular)]
|
||||
Z[Zastava\n(Runtime Inspector/Enforcer)]
|
||||
MIN[(MinIO S3)]
|
||||
RFS[(RustFS object store)]
|
||||
MGO[(MongoDB)]
|
||||
QUE[(Queue/Streams)]
|
||||
end
|
||||
@@ -94,7 +94,7 @@ flowchart LR
|
||||
CLI -->|scan/build| SW
|
||||
SW -->|jobs| QUE
|
||||
QUE --> WK
|
||||
WK --> MIN
|
||||
WK --> RFS
|
||||
SW --> MGO
|
||||
CONC --> MGO
|
||||
EXC --> MGO
|
||||
@@ -225,13 +225,13 @@ LS --> IA: PoE (mTLS client cert or JWT with cnf=K_inst), CRL/OCSP/introspect
|
||||
|
||||
---
|
||||
|
||||
## 6) Storage & catalogs (MinIO/Mongo)
|
||||
## 6) Storage & catalogs (RustFS/Mongo)
|
||||
|
||||
**RustFS layout (default)**
|
||||
|
||||
**MinIO layout**
|
||||
|
||||
```
|
||||
s3://stellaops/
|
||||
layers/<sha256>/sbom.cdx.json.zst
|
||||
```
|
||||
rustfs://stellaops/
|
||||
layers/<sha256>/sbom.cdx.json.zst
|
||||
layers/<sha256>/sbom.spdx.json.zst
|
||||
images/<imgDigest>/inventory.cdx.pb
|
||||
images/<imgDigest>/usage.cdx.pb
|
||||
@@ -248,7 +248,7 @@ s3://stellaops/
|
||||
|
||||
**Retention**
|
||||
|
||||
* MinIO **ILM** for coarse TTL; Scanner.WebService GC decrements `refCount` and deletes unreferenced metadata; **Object Lock** for immutable classes (auditable artifacts).
|
||||
* RustFS applies retention via `X-RustFS-Retain-Seconds`; Scanner.WebService GC decrements `refCount` and deletes unreferenced metadata; S3/MinIO fallback retains native Object Lock when enabled.
|
||||
|
||||
---
|
||||
|
||||
@@ -395,7 +395,7 @@ services:
|
||||
ui: { image: stellaops/ui, depends_on: [scanner-web, concelier, excititor, scheduler-web, notify-web] }
|
||||
```
|
||||
|
||||
* **Backups:** Mongo dumps; MinIO versioned buckets & replication; Rekor v2 DB snapshots; JWKS/Fulcio/KMS key rotation.
|
||||
* **Backups:** Mongo dumps; RustFS snapshots (or S3 versioning when fallback driver is used); Rekor v2 DB snapshots; JWKS/Fulcio/KMS key rotation.
|
||||
* **Ops runbooks:** Scheduler catch‑up after Concelier/Excititor recovery; connector key rotation (Slack/Teams/SMTP).
|
||||
* **SLOs & alerts:** lag between Concelier/Excititor export and first rescan verdict; delivery failure rates by channel.
|
||||
|
||||
@@ -408,7 +408,7 @@ services:
|
||||
* **Notify metrics:** `notify.sent_total{channel}`, `notify.dropped_total{reason}`, `notify.digest_coalesced_total`, `notify.latency_ms`.
|
||||
* **Tracing:** per‑stage spans; correlation IDs across Scanner→Signer→Attestor and Concelier/Excititor→Scheduler→Scanner→Notify.
|
||||
* **Audit logs:** every signing records `license_id`, `image_digest`, `policy_digest`, and Rekor UUID; Scheduler records who scheduled what; Notify records where, when, and why messages were sent or deduped.
|
||||
* **Compliance:** MinIO **Object Lock** for immutable artifacts; reproducible outputs via policy digest + SBOM digest in predicate.
|
||||
* **Compliance:** RustFS retention headers (or MinIO Object Lock when operating in S3 mode) keep immutable artifacts tamper‑resistant; reproducible outputs via policy digest + SBOM digest in predicate.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user