feat(docs): Add comprehensive documentation for Vexer, Vulnerability Explorer, and Zastava modules
- Introduced AGENTS.md, README.md, TASKS.md, and implementation_plan.md for Vexer, detailing mission, responsibilities, key components, and operational notes. - Established similar documentation structure for Vulnerability Explorer and Zastava modules, including their respective workflows, integrations, and observability notes. - Created risk scoring profiles documentation outlining the core workflow, factor model, governance, and deliverables. - Ensured all modules adhere to the Aggregation-Only Contract and maintain determinism and provenance in outputs.
This commit is contained in:
		
							
								
								
									
										88
									
								
								docs/modules/scanner/operations/rustfs-migration.md
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										88
									
								
								docs/modules/scanner/operations/rustfs-migration.md
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,88 @@ | ||||
| # Scanner Artifact Store Migration (MinIO → RustFS) | ||||
|  | ||||
| ## Overview | ||||
|  | ||||
| Sprint 11 introduces **RustFS** as the default artifact store for the Scanner plane. Existing | ||||
| deployments running MinIO (or any S3-compatible backend) must migrate stored SBOM artefacts to RustFS | ||||
| before switching the Scanner hosts to `scanner.artifactStore.driver = "rustfs"`. | ||||
|  | ||||
| This runbook covers the recommended migration workflow and validation steps. | ||||
|  | ||||
| ## Prerequisites | ||||
|  | ||||
| - RustFS service deployed and reachable from the Scanner control plane (`http(s)://rustfs:8080`). | ||||
| - Existing MinIO/S3 credentials with read access to the current bucket. | ||||
| - CLI environment with the StellaOps source tree (for the migration tool) and `dotnet 10` SDK. | ||||
| - Maintenance window sized to copy all artefacts (migration is read-only on the source bucket). | ||||
|  | ||||
| ## 1. Snapshot source bucket (optional but recommended) | ||||
|  | ||||
| If the MinIO deployment offers versioning or snapshots, take one before migrating. For non-versioned | ||||
| deployments, capture an external backup (e.g., `mc mirror` to offline storage). | ||||
|  | ||||
| ## 2. Dry-run the migrator | ||||
|  | ||||
| ``` | ||||
| dotnet run --project src/Tools/RustFsMigrator -- \ | ||||
|   --s3-bucket scanner-artifacts \ | ||||
|   --s3-endpoint http://stellaops-minio:9000 \ | ||||
|   --s3-access-key stellaops \ | ||||
|   --s3-secret-key dev-minio-secret \ | ||||
|   --rustfs-endpoint http://stellaops-rustfs:8080 \ | ||||
|   --rustfs-bucket scanner-artifacts \ | ||||
|   --prefix scanner/ \ | ||||
|   --dry-run | ||||
| ``` | ||||
|  | ||||
| The dry-run enumerates keys and reports the object count without writing to RustFS. Use this to | ||||
| estimate migration time. | ||||
|  | ||||
| ## 3. Execute migration | ||||
|  | ||||
| Remove the `--dry-run` flag to copy data. Optional flags: | ||||
|  | ||||
| - `--immutable` – mark all migrated objects as immutable (`X-RustFS-Immutable`). | ||||
| - `--retain-days 365` – request retention (in days) via `X-RustFS-Retain-Seconds`. | ||||
| - `--rustfs-api-key-header` / `--rustfs-api-key` – provide auth headers when RustFS is protected. | ||||
|  | ||||
| The tool streams each object from S3 and performs an idempotent `PUT` to RustFS preserving the key | ||||
| structure (e.g., `scanner/layers/<sha256>/sbom.cdx.json.zst`). | ||||
|  | ||||
| ## 4. Verify sample objects | ||||
|  | ||||
| Pick a handful of SBOM digests and confirm: | ||||
|  | ||||
| 1. `GET /api/v1/buckets/<bucket>/objects/<key>` returns the expected payload (size + SHA-256). | ||||
| 2. Scanner WebService configured with `scanner.artifactStore.driver = "rustfs"` can fetch the same | ||||
|    artefacts (Smoke test: `GET /api/v1/scanner/sboms/<digest>?format=cdx-json`). | ||||
|  | ||||
| ## 5. Switch Scanner hosts | ||||
|  | ||||
| Update configuration (Helm/Compose/environment) to set: | ||||
|  | ||||
| ``` | ||||
| scanner: | ||||
|   artifactStore: | ||||
|     driver: rustfs | ||||
|     endpoint: http://stellaops-rustfs:8080 | ||||
|     bucket: scanner-artifacts | ||||
|     timeoutSeconds: 30 | ||||
| ``` | ||||
|  | ||||
| Redeploy Scanner WebService and Worker. Monitor logs for `RustFS` upload/download messages and | ||||
| Prometheus scrape (`rustfs_requests_total`). | ||||
|  | ||||
| ## 6. Cleanup legacy MinIO (optional) | ||||
|  | ||||
| After a complete migration and validation period, decommission the MinIO bucket or repurpose it for | ||||
| other components (Concelier still supports S3). Ensure backups reference RustFS snapshots going | ||||
| forward. | ||||
|  | ||||
| ## Troubleshooting | ||||
|  | ||||
| - **Uploads fail (HTTP 4xx/5xx):** Check RustFS logs and confirm API key headers. Re-run the migrator | ||||
|   for the affected keys. | ||||
| - **Missing objects post-cutover:** Re-run the migrator with the specific `--prefix`. The tool is | ||||
|   idempotent and safely overwrites existing objects. | ||||
| - **Performance tuning:** Run multiple instances of the migrator with disjoint prefixes if needed; the | ||||
|   RustFS API is stateless and supports parallel PUTs. | ||||
		Reference in New Issue
	
	Block a user