feat: Implement Policy Engine Evaluation Service and Cache with unit tests
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Temp commit to debug
This commit is contained in:
master
@@ -1,113 +1,113 @@
|
||||
# Stella Ops — Developer Guide: Deterministic Replay
|
||||
|
||||
## Purpose
|
||||
Deterministic Replay ensures any past scan can be re-executed byte-for-byte, producing identical SBOM, Findings, and VEX results, cryptographically verifiable for audits or compliance.
|
||||
|
||||
Replay is the foundation for:
|
||||
- **Audit proofs** (exact past state reproduction)
|
||||
- **Diff analysis** (feeds, policies, tool versions)
|
||||
- **Cross-region verification** (same outputs on different hosts)
|
||||
- **Long-term cryptographic trust** (re-sign with new crypto profiles)
|
||||
|
||||
---
|
||||
|
||||
## Core Concepts
|
||||
|
||||
| Term | Description |
|
||||
|------|--------------|
|
||||
| **Replay Manifest** | Immutable JSON describing all inputs, tools, env, and outputs of a scan. |
|
||||
| **InputBundle** | Snapshot of feeds, rules, policies, and toolchain binaries used. |
|
||||
| **OutputBundle** | SBOM, Findings, VEX, and logs from a completed scan. |
|
||||
| **Layer Merkle** | Per-layer hash tree for precise deduplication and drift detection. |
|
||||
| **DSSE Envelope** | Digital signature wrapper for each attestation (SBOM, Findings, Manifest, etc.). |
|
||||
|
||||
---
|
||||
|
||||
## What to Freeze
|
||||
|
||||
| Category | Example Contents | Required in Manifest |
|
||||
|-----------|------------------|----------------------|
|
||||
| **Subject** | OCI image digest, per-layer Merkle roots | ✅ |
|
||||
| **Outputs** | SBOM, Findings, VEX, logs (content hashes) | ✅ |
|
||||
| **Toolchain** | Sbomer, Scanner, Vexer binaries + versions + SHA256 | ✅ |
|
||||
| **Feeds/VEX sources** | Full or pruned snapshot with Merkle proofs | ✅ |
|
||||
| **Policy Bundle** | Lattice rules, mutes, trust profiles, thresholds | ✅ |
|
||||
| **Environment** | OS, arch, locale, TZ, deterministic seed, runtime flags | ✅ |
|
||||
| **Crypto Profile** | Algorithm suites (FIPS, GOST, SM, eIDAS) | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## Replay Modes
|
||||
|
||||
| Mode | Purpose | Input Variation | Expected Output |
|
||||
|------|----------|-----------------|-----------------|
|
||||
| **Strict Replay** | Audit proof | None | Bit-for-bit identical |
|
||||
| **What-If Replay** | Change impact analysis | One dimension (feeds/tools/policy) | Deterministic diff |
|
||||
|
||||
Example:
|
||||
```
|
||||
|
||||
stella replay manifest.json --strict
|
||||
stella replay manifest.json --what-if --vary=feeds
|
||||
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Developer Responsibilities
|
||||
|
||||
| Module | Role |
|
||||
|---------|------|
|
||||
| **Scanner.WebService** | Capture full input set and produce Replay Manifest + DSSE sigs. |
|
||||
| **Sbomer** | Generate deterministic SBOM; normalize ordering and JSON formatting. |
|
||||
| **Vexer/Excititor** | Apply lattice and mutes from policy bundle; record gating logic. |
|
||||
| **Feedser/Concelier** | Freeze and export feed snapshots or Merkle proofs. |
|
||||
| **Authority** | Manage signer keys and crypto profiles; issue DSSE envelopes. |
|
||||
| **CLI** | Provide `scan --record`, `replay`, `verify`, `diff` commands. |
|
||||
|
||||
---
|
||||
|
||||
## Workflow
|
||||
|
||||
1. `stella scan image:tag --record out/`
|
||||
- Generates Replay Manifest, InputBundle, OutputBundle, DSSE sigs.
|
||||
2. `stella verify manifest.json`
|
||||
- Validates hashes, signatures, and completeness.
|
||||
3. `stella replay manifest.json --strict`
|
||||
- Re-executes in sealed mode; expect byte-identical results.
|
||||
4. `stella replay manifest.json --what-if --vary=feeds`
|
||||
- Runs with new feeds; diff is attributed to feeds only.
|
||||
5. `stella diff manifestA manifestB`
|
||||
- Attribute differences by hash comparison.
|
||||
|
||||
---
|
||||
|
||||
## Storage
|
||||
|
||||
- **Mongo collections**
|
||||
- `replay_runs`: manifest + DSSE envelopes + status
|
||||
- `bundles`: content-addressed (input/output/rootpack)
|
||||
- `subjects`: OCI digests, Merkle roots per layer
|
||||
- **File store**
|
||||
- Bundles stored as `<sha256>.tar.zst`
|
||||
|
||||
---
|
||||
|
||||
## Developer Checklist
|
||||
|
||||
- [ ] All inputs (feeds, policies, tools, env) hashed and recorded.
|
||||
- [ ] JSON normalization: key order, number format, newline mode.
|
||||
- [ ] Random seed = `H(scan.id || MerkleRootAllLayers)`.
|
||||
- [ ] Clock fixed to `scan.time` unless policy requires “now”.
|
||||
- [ ] DSSE multi-sig supported (FIPS + regional).
|
||||
- [ ] Manifest signed + optionally anchored to Rekor ledger.
|
||||
- [ ] Replay comparison mode tested across x64/arm64.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
See also:
|
||||
- `DETERMINISTIC_REPLAY.md` — detailed manifest schema & CLI examples.
|
||||
- `../docs/CRYPTO_SOVEREIGN_READY.md` — RootPack and dual-signature handling.
|
||||
|
||||
---
|
||||
# Stella Ops — Developer Guide: Deterministic Replay
|
||||
|
||||
## Purpose
|
||||
Deterministic Replay ensures any past scan can be re-executed byte-for-byte, producing identical SBOM, Findings, and VEX results, cryptographically verifiable for audits or compliance.
|
||||
|
||||
Replay is the foundation for:
|
||||
- **Audit proofs** (exact past state reproduction)
|
||||
- **Diff analysis** (feeds, policies, tool versions)
|
||||
- **Cross-region verification** (same outputs on different hosts)
|
||||
- **Long-term cryptographic trust** (re-sign with new crypto profiles)
|
||||
|
||||
---
|
||||
|
||||
## Core Concepts
|
||||
|
||||
| Term | Description |
|
||||
|------|--------------|
|
||||
| **Replay Manifest** | Immutable JSON describing all inputs, tools, env, and outputs of a scan. |
|
||||
| **InputBundle** | Snapshot of feeds, rules, policies, and toolchain binaries used. |
|
||||
| **OutputBundle** | SBOM, Findings, VEX, and logs from a completed scan. |
|
||||
| **Layer Merkle** | Per-layer hash tree for precise deduplication and drift detection. |
|
||||
| **DSSE Envelope** | Digital signature wrapper for each attestation (SBOM, Findings, Manifest, etc.). |
|
||||
|
||||
---
|
||||
|
||||
## What to Freeze
|
||||
|
||||
| Category | Example Contents | Required in Manifest |
|
||||
|-----------|------------------|----------------------|
|
||||
| **Subject** | OCI image digest, per-layer Merkle roots | ✅ |
|
||||
| **Outputs** | SBOM, Findings, VEX, logs (content hashes) | ✅ |
|
||||
| **Toolchain** | Sbomer, Scanner, Vexer binaries + versions + SHA256 | ✅ |
|
||||
| **Feeds/VEX sources** | Full or pruned snapshot with Merkle proofs | ✅ |
|
||||
| **Policy Bundle** | Lattice rules, mutes, trust profiles, thresholds | ✅ |
|
||||
| **Environment** | OS, arch, locale, TZ, deterministic seed, runtime flags | ✅ |
|
||||
| **Crypto Profile** | Algorithm suites (FIPS, GOST, SM, eIDAS) | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## Replay Modes
|
||||
|
||||
| Mode | Purpose | Input Variation | Expected Output |
|
||||
|------|----------|-----------------|-----------------|
|
||||
| **Strict Replay** | Audit proof | None | Bit-for-bit identical |
|
||||
| **What-If Replay** | Change impact analysis | One dimension (feeds/tools/policy) | Deterministic diff |
|
||||
|
||||
Example:
|
||||
```
|
||||
|
||||
stella replay manifest.json --strict
|
||||
stella replay manifest.json --what-if --vary=feeds
|
||||
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Developer Responsibilities
|
||||
|
||||
| Module | Role |
|
||||
|---------|------|
|
||||
| **Scanner.WebService** | Capture full input set and produce Replay Manifest + DSSE sigs. |
|
||||
| **Sbomer** | Generate deterministic SBOM; normalize ordering and JSON formatting. |
|
||||
| **Vexer/Excititor** | Apply lattice and mutes from policy bundle; record gating logic. |
|
||||
| **Feedser/Concelier** | Freeze and export feed snapshots or Merkle proofs. |
|
||||
| **Authority** | Manage signer keys and crypto profiles; issue DSSE envelopes. |
|
||||
| **CLI** | Provide `scan --record`, `replay`, `verify`, `diff` commands. |
|
||||
|
||||
---
|
||||
|
||||
## Workflow
|
||||
|
||||
1. `stella scan image:tag --record out/`
|
||||
- Generates Replay Manifest, InputBundle, OutputBundle, DSSE sigs.
|
||||
2. `stella verify manifest.json`
|
||||
- Validates hashes, signatures, and completeness.
|
||||
3. `stella replay manifest.json --strict`
|
||||
- Re-executes in sealed mode; expect byte-identical results.
|
||||
4. `stella replay manifest.json --what-if --vary=feeds`
|
||||
- Runs with new feeds; diff is attributed to feeds only.
|
||||
5. `stella diff manifestA manifestB`
|
||||
- Attribute differences by hash comparison.
|
||||
|
||||
---
|
||||
|
||||
## Storage
|
||||
|
||||
- **Mongo collections**
|
||||
- `replay_runs`: manifest + DSSE envelopes + status
|
||||
- `bundles`: content-addressed (input/output/rootpack)
|
||||
- `subjects`: OCI digests, Merkle roots per layer
|
||||
- **File store**
|
||||
- Bundles stored as `<sha256>.tar.zst`
|
||||
|
||||
---
|
||||
|
||||
## Developer Checklist
|
||||
|
||||
- [ ] All inputs (feeds, policies, tools, env) hashed and recorded.
|
||||
- [ ] JSON normalization: key order, number format, newline mode.
|
||||
- [ ] Random seed = `H(scan.id || MerkleRootAllLayers)`.
|
||||
- [ ] Clock fixed to `scan.time` unless policy requires “now”.
|
||||
- [ ] DSSE multi-sig supported (FIPS + regional).
|
||||
- [ ] Manifest signed + optionally anchored to Rekor ledger.
|
||||
- [ ] Replay comparison mode tested across x64/arm64.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
See also:
|
||||
- `DETERMINISTIC_REPLAY.md` — detailed manifest schema & CLI examples.
|
||||
- `../docs/CRYPTO_SOVEREIGN_READY.md` — RootPack and dual-signature handling.
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user