chore(sprints): archive 20260226 advisories and expand deterministic tests
This commit is contained in:
81
docs/modules/tools/supply-chain-hardening-suite.md
Normal file
81
docs/modules/tools/supply-chain-hardening-suite.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# Supply-Chain Hardening Suite
|
||||
|
||||
## Purpose
|
||||
|
||||
The supply-chain hardening suite provides deterministic negative-path and mutation testing for scanner/attestor/symbols evidence workflows without requiring external network calls.
|
||||
|
||||
Working location:
|
||||
- `tests/supply-chain/`
|
||||
|
||||
## Lanes
|
||||
|
||||
1. `01-jcs-property`
|
||||
- Verifies canonicalization idempotence.
|
||||
- Verifies key-order permutation invariance.
|
||||
- Verifies duplicate-key rejection.
|
||||
|
||||
2. `02-schema-fuzz`
|
||||
- Runs deterministic schema-aware mutation lane.
|
||||
- Emits crash diagnostics and replay seed on unexpected exceptions.
|
||||
- Enforces zero-crash gate in CI.
|
||||
|
||||
3. `03-rekor-neg`
|
||||
- Simulates Rekor negative paths (413/424/504/unsupported/202).
|
||||
- Verifies deterministic error classification.
|
||||
- Emits per-case `diagnostic_blob.json` and bundle archive.
|
||||
|
||||
4. `04-big-dsse-referrers`
|
||||
- Validates oversized DSSE and malformed referrer rejection behavior.
|
||||
- Requires deterministic `unknown_state` and `reprocessToken` outputs.
|
||||
|
||||
5. `05-corpus`
|
||||
- Stores deterministic fixture corpus.
|
||||
- Provides deterministic archive manifest builder for corpus updates.
|
||||
|
||||
## Execution Profiles
|
||||
|
||||
1. PR / push gate profile (`smoke`)
|
||||
- Seed: `20260226`
|
||||
- Fuzz lane bounds: `limit=1000`, `time=60s`
|
||||
- Artifact retention: 14 days
|
||||
|
||||
2. Nightly profile (`nightly`)
|
||||
- Seed: `20260226`
|
||||
- Fuzz lane bounds: `limit=5000`, `time=300s`
|
||||
- Artifact retention: 30 days
|
||||
|
||||
## Commands
|
||||
|
||||
1. Run smoke profile:
|
||||
- `python tests/supply-chain/run_suite.py --profile smoke --seed 20260226`
|
||||
|
||||
2. Run nightly profile:
|
||||
- `python tests/supply-chain/run_suite.py --profile nightly --seed 20260226`
|
||||
|
||||
3. Rebuild corpus archive metadata:
|
||||
- `python tests/supply-chain/05-corpus/build_corpus_archive.py --output out/supply-chain/05-corpus`
|
||||
|
||||
## CI Integration
|
||||
|
||||
Workflow:
|
||||
- `.gitea/workflows/supply-chain-hardening.yml`
|
||||
|
||||
Outputs:
|
||||
- `out/supply-chain/summary.json`
|
||||
- lane-level `junit.xml` files
|
||||
- lane-level `report.json` files
|
||||
- `03-rekor-neg/rekor_negative_cases.tar.gz`
|
||||
- `04-big-dsse-referrers/big_dsse_payloads.tar.gz`
|
||||
|
||||
## Failure Replay
|
||||
|
||||
1. Download CI artifact `supply-chain-hardening-<run-id>`.
|
||||
2. Read failing lane diagnostics under `failures/<case-id>/`.
|
||||
3. Re-run locally with the same seed:
|
||||
- `python tests/supply-chain/run_suite.py --profile smoke --seed 20260226 --output out/supply-chain-replay`
|
||||
|
||||
## Advisory Traceability
|
||||
|
||||
| Advisory | Sprint | Coverage |
|
||||
| --- | --- | --- |
|
||||
| `docs-archived/product/advisories/20260222 - Fuzz & mutation hardening suite.md` | `docs-archived/implplan/2026-03-03-completed-sprints/SPRINT_20260226_228_Tools_supply_chain_fuzz_mutation_hardening_suite.md` | Lanes `01` through `05` + CI gate |
|
||||
Reference in New Issue
Block a user