Add unit tests for SBOM ingestion and transformation
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Implement `SbomIngestServiceCollectionExtensionsTests` to verify the SBOM ingestion pipeline exports snapshots correctly. - Create `SbomIngestTransformerTests` to ensure the transformation produces expected nodes and edges, including deduplication of license nodes and normalization of timestamps. - Add `SbomSnapshotExporterTests` to test the export functionality for manifest, adjacency, nodes, and edges. - Introduce `VexOverlayTransformerTests` to validate the transformation of VEX nodes and edges. - Set up project file for the test project with necessary dependencies and configurations. - Include JSON fixture files for testing purposes.
This commit is contained in:
master
75
docs/modules/evidence-locker/bundle-packaging.md
Normal file
75
docs/modules/evidence-locker/bundle-packaging.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Evidence Locker Bundle Packaging
|
||||
|
||||
> Sprint 160 / Task EVID-OBS-54-002 — deterministic tarball packaging for download/export.
|
||||
|
||||
The Evidence Locker emits a **single `bundle.tgz` artifact** for every sealed bundle. The artifact is deterministic so that operators can re-run packaging and obtain identical bytes when the manifest and signature are unchanged.
|
||||
|
||||
## Layout
|
||||
|
||||
The tar stream is written with **POSIX/PAX entries** and wrapped in a gzip layer:
|
||||
|
||||
```
|
||||
bundle.tgz
|
||||
├── manifest.json # Re-emitted DSSE payload (pretty JSON, canonical ordering)
|
||||
├── signature.json # DSSE signature + key metadata + RFC3161 timestamp (if present)
|
||||
├── bundle.json # Locker metadata (ids, status, root hash, storage key, timestamps)
|
||||
├── checksums.txt # SHA-256 root hash + per-entry hashes from the manifest
|
||||
└── instructions.txt # Offline verification steps and retention guidance
|
||||
```
|
||||
|
||||
### Determinism traits
|
||||
|
||||
- **Gzip header timestamp** is pinned to `2025-01-01T00:00:00Z` so CI fixtures remain stable.
|
||||
- All tar entries use the same fixed mtime/atime/ctime, `0644` permissions, and UTF-8 encoding.
|
||||
- JSON files are serialized with `JsonSerializerDefaults.Web` + indentation to stabilise ordering.
|
||||
- `checksums.txt` sorts manifest entries by `canonicalPath` and prefixes the Merkle root (`root <hash>`).
|
||||
- `instructions.txt` conditionally adds timestamp verification steps when an RFC3161 token exists.
|
||||
|
||||
## Download endpoint
|
||||
|
||||
`GET /evidence/{bundleId}/download`
|
||||
|
||||
- Requires scopes: `evidence:read`.
|
||||
- Streams `application/gzip` content with `Content-Disposition: attachment; filename="bundle.tgz"`.
|
||||
- Emits quota headers (`X-Stella-Quota-*`) and audit events mirroring snapshot fetches.
|
||||
- Returns `404` when the bundle is not sealed or the package has not been materialised.
|
||||
|
||||
The endpoint reuses `EvidenceBundlePackagingService` and caches the packaged object in the configured object store (`tenants/{tenant}/bundles/{bundle}/bundle.tgz`). If the underlying storage key changes (for example, during migration from filesystem to S3), the repository is updated atomically.
|
||||
|
||||
## Verification guidance
|
||||
|
||||
1. Download `bundle.tgz` and read `instructions.txt`; the first section lists bundle id, root hash, and creation/timestamp information.
|
||||
2. Verify `checksums.txt` against the transferred archive to detect transit corruption.
|
||||
3. Use the StellaOps CLI (`stella evidence verify bundle.tgz`) or the provenance verifier library to validate `signature.json`.
|
||||
4. When present, validate the RFC3161 timestamp token with the configured TSA endpoint.
|
||||
|
||||
These steps match the offline procedure described in `docs/forensics/evidence-locker.md` (Portable Evidence section). Update that guide whenever packaging fields change.
|
||||
|
||||
## Portable bundle (`portable-bundle-v1.tgz`)
|
||||
|
||||
When sealed or air-gapped environments need a redacted evidence artifact, request:
|
||||
|
||||
`GET /evidence/{bundleId}/portable`
|
||||
|
||||
The portable archive is deterministic and contains only non-sensitive metadata:
|
||||
|
||||
```
|
||||
portable-bundle-v1.tgz
|
||||
├── manifest.json # Canonical manifest (identical to sealed bundle)
|
||||
├── signature.json # DSSE signature + optional RFC3161 token
|
||||
├── bundle.json # Redacted metadata (no tenant/storage identifiers)
|
||||
├── checksums.txt # SHA-256 root + entry checksums
|
||||
├── instructions-portable.txt # Sealed-mode transfer + verification guidance
|
||||
└── verify-offline.sh # Offline verification helper script (POSIX shell)
|
||||
```
|
||||
|
||||
Portable packaging traits:
|
||||
|
||||
- `bundle.json` excludes tenant identifiers, storage keys, and free-form descriptions. It adds `portableGeneratedAt` along with entry counts and totals for audit purposes.
|
||||
- `incidentMetadata` is preserved only when incident mode injects `incident.*` keys into the manifest metadata.
|
||||
- `verify-offline.sh` extracts the bundle, validates checksums (using `sha256sum`/`shasum`), surfaces the Merkle root hash, and reminds operators to run `stella evidence verify --bundle <archive>`.
|
||||
- `instructions-portable.txt` mirrors the sealed documentation but calls out the offline script and redaction constraints.
|
||||
|
||||
Portable bundles reuse the same DSSE payload and timestamp, so downstream verifiers can validate signatures without additional configuration. The Evidence Locker tracks the portable storage key separately to honour write-once semantics for both sealed and portable artifacts.
|
||||
|
||||
For step-by-step sealed-mode guidance see `docs/airgap/portable-evidence.md`.
|
||||
13
docs/modules/evidence-locker/compliance-checklist.md
Normal file
13
docs/modules/evidence-locker/compliance-checklist.md
Normal file
@@ -0,0 +1,13 @@
|
||||
# Evidence Locker Compliance Checklist (Sprint 160)
|
||||
|
||||
- [x] Postgres schema created via deterministic SQL migrations (`evidence_locker.*` tables, schema version tracking).
|
||||
- [x] Row-level security enforced per tenant via `app.current_tenant` guard function.
|
||||
- [x] Evidence bundle storage keys are content-addressed (sha256) and unique per tenant/bundle.
|
||||
- [x] Object-store abstraction provides local filesystem and Amazon S3 drivers with optional WORM enforcement.
|
||||
- [x] Startup migrations wired via hosted service with opt-out flag (`ApplyMigrationsAtStartup`).
|
||||
- [x] Integration tests cover schema bootstrap, RLS behaviour, and storage drivers (filesystem, S3 fake client).
|
||||
- [x] Temporary artifacts cleaned deterministically; filesystem targets validated in tests.
|
||||
- [x] Timeline publisher emits bundle sealed and hold events with DSSE metadata when enabled; offline deployments fall back to null publisher.
|
||||
- [x] Bundle packaging outputs deterministic `bundle.tgz` (fixed gzip mtime, sorted checksums, timestamp-aware instructions) and `/evidence/{id}/download` streams the cached object with audit logging.
|
||||
- [x] Incident mode extends bundle retention, captures incident request snapshots, and emits activation/deactivation events to Timeline Indexer and Notifier stubs with unit + web integration coverage.
|
||||
- [x] Portable bundle flow (`/evidence/{id}/portable`) emits `portable-bundle-v1.tgz` with redacted metadata, deterministic offline script, and write-once storage tracking.
|
||||
24
docs/modules/evidence-locker/incident-mode.md
Normal file
24
docs/modules/evidence-locker/incident-mode.md
Normal file
@@ -0,0 +1,24 @@
|
||||
# Evidence Locker Incident Mode
|
||||
|
||||
> Sprint 55 / Task EVID-OBS-55-001 – retention & debug hooks
|
||||
|
||||
Incident mode is a service-wide switch that increases forensic fidelity when StellaOps enters a suspected compromise or SLO breach. The Evidence Locker reacts to the flag in four ways:
|
||||
|
||||
1. **Extended retention.** Every newly sealed bundle receives an `ExpiresAt` timestamp of `CreatedAt + Incident.RetentionExtensionDays` so downstream TTL jobs keep artefacts long enough for investigation.
|
||||
2. **Debug artefacts.** Snapshot requests emit an `incident/request-*.json` payload into the object store. The payload captures the normalized request metadata/materials plus the incident stamp so offline replay tooling has everything it needs. The manifest surfaces the artefact under the `incident/` section and packaging streams it alongside the canonical bundle files.
|
||||
3. **Manifest metadata.** Bundles carry `incident.mode`, `incident.changedAt`, and `incident.retentionExtensionDays` metadata so verifiers and auditors can see exactly when the mode toggled and how long retention was extended.
|
||||
4. **Operational signals.** Activation/deactivation events are published to the Timeline Indexer (and, via the notifier stub, to the future Notify integration). The `IEvidenceTimelinePublisher` now emits `evidence.incident.mode` with `state` and retention attributes, giving Ops a canonical audit trail.
|
||||
|
||||
Configuration lives under `EvidenceLocker:Incident`:
|
||||
|
||||
```jsonc
|
||||
"EvidenceLocker": {
|
||||
"Incident": {
|
||||
"Enabled": true,
|
||||
"RetentionExtensionDays": 60,
|
||||
"CaptureRequestSnapshot": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`IncidentModeManager` watches the options and raises events whenever the state flips. Tests cover retention math, timeline/notifier fan-out, and the new debug artefact path.
|
||||
Reference in New Issue
Block a user