feat: Add RustFS artifact object store and migration tool
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Implemented RustFsArtifactObjectStore for managing artifacts in RustFS.
- Added unit tests for RustFsArtifactObjectStore functionality.
- Created a RustFS migrator tool to transfer objects from S3 to RustFS.
- Introduced policy preview and report models for API integration.
- Added fixtures and tests for policy preview and report functionality.
- Included necessary metadata and scripts for cache_pkg package.
This commit is contained in:
Vladimir Moushkov
2025-10-23 18:53:18 +03:00
parent aaa5fbfb78
commit f4d7a15a00
117 changed files with 4849 additions and 725 deletions

View File

@@ -17,11 +17,11 @@ completely isolated network:
| **Provenance** | Cosign signature, SPDX 2.3 SBOM, intoto SLSA attestation |
| **Attested manifest** | `offline-manifest.json` + detached JWS covering bundle metadata, signed during export. |
| **Delta patches** | Daily diff bundles keep size \<350MB |
| **Scanner plug-ins** | OS analyzers plus the Node.js, Go, and .NET language analyzers packaged under `plugins/scanner/analyzers/**` with manifests so Workers load deterministically offline. |
| **Scanner plug-ins** | OS analyzers plus the Node.js, Go, .NET, and Python language analyzers packaged under `plugins/scanner/analyzers/**` with manifests so Workers load deterministically offline. |
**RU BDU note:** ship the official Russian Trusted Root/Sub CA bundle (`certificates/russian_trusted_bundle.pem`) inside the kit so `concelier:httpClients:source.bdu:trustedRootPaths` can resolve it when the service runs in an airgapped network. Drop the most recent `vulxml.zip` alongside the kit if operators need a cold-start cache.
**Language analyzers:** the kit now carries the restart-only Node.js, Go, and .NET analyzer plug-ins (`plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Node/`, `plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/`, `plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/`). Drop the directories alongside Worker binaries so the unified plug-in catalog can load them without outbound fetches; upcoming Python/Rust plug-ins will follow the same layout.
**Language analyzers:** the kit now carries the restart-only Node.js, Go, .NET, and Python analyzer plug-ins (`plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Node/`, `...Lang.Go/`, `...Lang.DotNet/`, `...Lang.Python/`). Drop the directories alongside Worker binaries so the unified plug-in catalog can load them without outbound fetches; Rust remains on the Wave4 roadmap.
*Scanner core:* C# 12 on **.NET{{ dotnet }}**.
*Imports are idempotent and atomic — no service downtime.*
@@ -99,6 +99,24 @@ Example excerpt (2025-10-23 kit) showing the Go and .NET analyzer plug-in payloa
"size": 647,
"capturedAt": "2025-10-23T00:00:00Z"
}
{
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.dll",
"sha256": "28b6e06c7cabf3b78f13f801cbb14962093f3d42c4ae9ec01babbcd14cda4644",
"size": 53760,
"capturedAt": "2025-10-23T00:00:00Z"
}
{
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.pdb",
"sha256": "be4e34b4dc9a790fe1299e84213343b7c8ea90a2d22e5d7d1aa7585b8fedc946",
"size": 34516,
"capturedAt": "2025-10-23T00:00:00Z"
}
{
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/manifest.json",
"sha256": "bceea1e7542aae860b0ec5ba7b8b3aa960b21edc4d1efe60afc98ce289341ac3",
"size": 671,
"capturedAt": "2025-10-23T00:00:00Z"
}
```
---
@@ -130,7 +148,7 @@ The CLI validates recorded digests (when `.metadata.json` is present) before str
**Quick smoke test:** before import, verify the tarball carries the Go analyzer plug-in:
```bash
tar -tzf stella-ops-offline-kit-<DATE>.tgz 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/*'
tar -tzf stella-ops-offline-kit-<DATE>.tgz 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*'
```
The manifest lookup above and this `tar` listing should both surface the Go analyzer DLL, PDB, and manifest entries before the kit is promoted.