600f3a7a3c
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Console CI / console-ci (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
- Added graph.inspect.v1 documentation outlining payload structure and determinism rules. - Created JSON schema for graph.inspect.v1 to enforce payload validation. - Defined mapping rules for graph relationships, advisories, and VEX statements. feat(notifications): establish remediation blueprint for gaps NR1-NR10 - Documented requirements, evidence, and tests for Notifier runtime. - Specified deliverables and next steps for addressing identified gaps. docs(notifications): organize operations and schemas documentation - Created README files for operations, schemas, and security notes to clarify deliverables and policies. feat(advisory): implement PostgreSQL caching for Link-Not-Merge linksets - Created database schema for advisory linkset cache. - Developed repository for managing advisory linkset cache operations. - Added tests to ensure correct functionality of the AdvisoryLinksetCacheRepository.
99 lines
6.5 KiB
Markdown
99 lines
6.5 KiB
Markdown
# Evidence Locker Bundle Packaging
|
||
|
||
> Sprint 160 / Task EVID-OBS-54-002 — deterministic tarball packaging for download/export.
|
||
|
||
The Evidence Locker emits a **single `bundle.tgz` artifact** for every sealed bundle. The artifact is deterministic so that operators can re-run packaging and obtain identical bytes when the manifest and signature are unchanged.
|
||
|
||
## Layout
|
||
|
||
The tar stream is written with **POSIX/PAX entries** and wrapped in a gzip layer:
|
||
|
||
```
|
||
bundle.tgz
|
||
├── manifest.json # Re-emitted DSSE payload (pretty JSON, canonical ordering)
|
||
├── signature.json # DSSE signature + key metadata + RFC3161 timestamp (if present)
|
||
├── bundle.json # Locker metadata (ids, status, root hash, storage key, timestamps)
|
||
├── checksums.txt # SHA-256 root hash + per-entry hashes from the manifest
|
||
└── instructions.txt # Offline verification steps and retention guidance
|
||
```
|
||
|
||
### Determinism traits
|
||
|
||
- **Gzip header timestamp** is pinned to `2025-01-01T00:00:00Z` so CI fixtures remain stable.
|
||
- All tar entries use the same fixed mtime/atime/ctime, `0644` permissions, and UTF-8 encoding.
|
||
- JSON files are serialized with `JsonSerializerDefaults.Web` + indentation to stabilise ordering.
|
||
- `checksums.txt` sorts manifest entries by `canonicalPath` and prefixes the Merkle root (`root <hash>`).
|
||
- `instructions.txt` conditionally adds timestamp verification steps when an RFC3161 token exists.
|
||
|
||
## Download endpoint
|
||
|
||
`GET /evidence/{bundleId}/download`
|
||
|
||
- Requires scopes: `evidence:read`.
|
||
- Streams `application/gzip` content with `Content-Disposition: attachment; filename="bundle.tgz"`.
|
||
- Emits quota headers (`X-Stella-Quota-*`) and audit events mirroring snapshot fetches.
|
||
- Returns `404` when the bundle is not sealed or the package has not been materialised.
|
||
|
||
The endpoint reuses `EvidenceBundlePackagingService` and caches the packaged object in the configured object store (`tenants/{tenant}/bundles/{bundle}/bundle.tgz`). If the underlying storage key changes (for example, during migration from filesystem to S3), the repository is updated atomically.
|
||
|
||
## Verification guidance
|
||
|
||
Upcoming EB1–EB10 remediation (Sprint 0161; advisory `docs/product-advisories/archived/27-Nov-2025-superseded/28-Nov-2025 - Evidence Bundle and Replay Contracts.md`):
|
||
- Publish `bundle.manifest.schema.json` and `checksums.schema.json` with canonical JSON rules and signatures.
|
||
- Document the Merkle hash recipe and DSSE predicate/log policy.
|
||
- Ship an offline verifier script and golden bundles/replay fixtures to prove determinism.
|
||
- Add incident-mode activation/exit records and redaction/tenant isolation guidance for portable bundles.
|
||
|
||
Canonical schemas now live in `docs/modules/evidence-locker/schemas/` (EB1, EB2). Offline verification steps and the embeddable script are documented in `docs/modules/evidence-locker/verify-offline.md` (EB9); use the computed Merkle root as the DSSE subject for sealed and portable bundles.
|
||
|
||
### Merkle recipe (example)
|
||
```bash
|
||
cd bundle
|
||
find . -type f ! -name checksums.txt -print0 | sort -z | xargs -0 sha256sum > checksums.txt
|
||
sha256sum checksums.txt | awk '{print $1}' > merkle-root.txt
|
||
```
|
||
Use the resulting root as the DSSE subject and store `checksums.txt` inside the bundle.
|
||
|
||
1. Download `bundle.tgz` and read `instructions.txt`; the first section lists bundle id, root hash, and creation/timestamp information.
|
||
2. Verify `checksums.txt` against the transferred archive to detect transit corruption.
|
||
3. Use the StellaOps CLI (`stella evidence verify bundle.tgz`) or the provenance verifier library to validate `signature.json`.
|
||
4. When present, validate the RFC3161 timestamp token with the configured TSA endpoint.
|
||
|
||
These steps match the offline procedure described in `docs/forensics/evidence-locker.md` (Portable Evidence section). Update that guide whenever packaging fields change.
|
||
|
||
### Merkle + CAS rules (EB3/EB6)
|
||
- **Canonical inventory:** `checksums.txt` MUST be generated from the manifest entries sorted lexicographically by `canonicalPath`.
|
||
- **Subject binding:** DSSE subject is `sha256(checksums.txt)` (Merkle root); OCI digest of `bundle.tgz` is secondary metadata only.
|
||
- **Chunking strategy:** Default `strategy=none`. When chunked CAS storage is enabled, record `chunking.strategy`, `chunkSizeBytes`, and `casDigestAlgorithm` in `checksums.schema.json`; chunk hashes are folded deterministically (fixed-size or buzhash) before the per-entry sha256 is calculated.
|
||
- **Compression invariants:** tar → gzip with pinned mtime (`2025-01-01T00:00:00Z`), `0644` perms, uid/gid `0:0`, UTF-8 headers; failing these invalidates fixtures.
|
||
- **Stable tooling:** CI must pin `tar`, `gzip`, and hashing tool versions; regenerate golden fixtures only when these pins change and record the versions in `tests/EvidenceLocker/Bundles/Golden/expected.json`.
|
||
|
||
## Portable bundle (`portable-bundle-v1.tgz`)
|
||
|
||
When sealed or air-gapped environments need a redacted evidence artifact, request:
|
||
|
||
`GET /evidence/{bundleId}/portable`
|
||
|
||
The portable archive is deterministic and contains only non-sensitive metadata:
|
||
|
||
```
|
||
portable-bundle-v1.tgz
|
||
├── manifest.json # Canonical manifest (identical to sealed bundle)
|
||
├── signature.json # DSSE signature + optional RFC3161 token
|
||
├── bundle.json # Redacted metadata (no tenant/storage identifiers)
|
||
├── checksums.txt # SHA-256 root + entry checksums
|
||
├── instructions-portable.txt # Sealed-mode transfer + verification guidance
|
||
└── verify-offline.sh # Offline verification helper script (POSIX shell)
|
||
```
|
||
|
||
Portable packaging traits:
|
||
|
||
- `bundle.json` excludes tenant identifiers, storage keys, and free-form descriptions. It adds `portableGeneratedAt` along with entry counts and totals for audit purposes.
|
||
- `incidentMetadata` is preserved only when incident mode injects `incident.*` keys into the manifest metadata.
|
||
- `verify-offline.sh` extracts the bundle, validates checksums (using `sha256sum`/`shasum`), surfaces the Merkle root hash, and reminds operators to run `stella evidence verify --bundle <archive>`.
|
||
- `instructions-portable.txt` mirrors the sealed documentation but calls out the offline script and redaction constraints.
|
||
|
||
Portable bundles reuse the same DSSE payload and timestamp, so downstream verifiers can validate signatures without additional configuration. The Evidence Locker tracks the portable storage key separately to honour write-once semantics for both sealed and portable artifacts.
|
||
|
||
For step-by-step sealed-mode guidance see `docs/airgap/portable-evidence.md`.
|