Add integration tests for Proof Chain and Reachability workflows

- Implement ProofChainTestFixture for PostgreSQL-backed integration tests.
- Create StellaOps.Integration.ProofChain project with necessary dependencies.
- Add ReachabilityIntegrationTests to validate call graph extraction and reachability analysis.
- Introduce ReachabilityTestFixture for managing corpus and fixture paths.
- Establish StellaOps.Integration.Reachability project with required references.
- Develop UnknownsWorkflowTests to cover the unknowns lifecycle: detection, ranking, escalation, and resolution.
- Create StellaOps.Integration.Unknowns project with dependencies for unknowns workflow.
This commit is contained in:
StellaOps Bot
2025-12-20 22:19:26 +02:00
parent 3c6e14fca5
commit efe9bd8cfe
86 changed files with 9616 additions and 323 deletions

View File

@@ -0,0 +1,107 @@
# Golden Test Corpus
This directory contains the golden test corpus for StellaOps scoring validation.
Each test case is a complete, reproducible scenario with known-good inputs and expected outputs.
## Schema Version
**Corpus Version**: `1.0.0`
**Scoring Algorithm**: `v2.0` (See `docs/modules/scanner/scoring-algorithm.md`)
**OpenVEX Schema**: `0.2.0`
**SPDX Version**: `3.0.1`
**CycloneDX Version**: `1.6`
## Directory Structure
```
golden-corpus/
├── README.md # This file
├── corpus-manifest.json # Index of all test cases with hashes
├── corpus-version.json # Versioning metadata
├── severity-levels/ # CVE severity coverage
│ ├── critical/
│ ├── high/
│ ├── medium/
│ └── low/
├── vex-scenarios/ # VEX override scenarios
│ ├── not-affected/
│ ├── affected/
│ ├── fixed/
│ └── under-investigation/
├── reachability/ # Reachability analysis scenarios
│ ├── reachable/
│ ├── unreachable/
│ └── unknown/
└── composite/ # Complex multi-factor scenarios
├── reachable-with-vex/
└── unreachable-high-severity/
```
## Test Case Format
Each test case directory contains:
| File | Description |
|------|-------------|
| `case.json` | Scenario metadata and description |
| `sbom.spdx.json` | SPDX 3.0.1 SBOM |
| `sbom.cdx.json` | CycloneDX 1.6 SBOM (optional) |
| `manifest.json` | Scan manifest with digest bindings |
| `vex.openvex.json` | OpenVEX document (if applicable) |
| `callgraph.json` | Static call graph (if reachability applies) |
| `proof-bundle.json` | Expected proof bundle structure |
| `expected-score.json` | Expected scoring output |
## Expected Score Format
```json
{
"schema_version": "stellaops.golden.expected/v1",
"score_hash": "sha256:...",
"stella_score": 7.5,
"base_cvss": 9.8,
"temporal_cvss": 8.5,
"environmental_cvss": 7.5,
"vex_impact": -1.0,
"reachability_impact": -1.3,
"kev_flag": false,
"exploit_maturity": "proof-of-concept",
"determinism_salt": "frozen-2025-01-15T00:00:00Z"
}
```
## Running Golden Tests
```bash
# Run all golden tests
dotnet test tests/integration/StellaOps.Integration.Determinism \
--filter "Category=GoldenCorpus"
# Regenerate expected outputs (after algorithm changes)
dotnet run --project bench/tools/corpus-regenerate -- \
--corpus-path bench/golden-corpus \
--algorithm-version v2.0
```
## Adding New Cases
1. Create directory under appropriate category
2. Add all required files (see Test Case Format)
3. Run corpus validation: `dotnet run --project bench/tools/corpus-validate`
4. Update `corpus-manifest.json` hash entries
5. Commit with message: `corpus: add <case-id> for <scenario>`
## Versioning Policy
- **Patch** (1.0.x): Add new cases, fix existing case data
- **Minor** (1.x.0): Algorithm tuning that preserves relative ordering
- **Major** (x.0.0): Algorithm changes that alter expected scores
When scoring algorithm changes:
1. Increment corpus version
2. Regenerate all expected scores
3. Document changes in CHANGELOG.md