117 lines
6.9 KiB
Markdown
117 lines
6.9 KiB
Markdown
# Fixture Harvester Tool (Deterministic Fixture Rewriter)
|
|
|
|
## Module
|
|
Tools
|
|
|
|
## Status
|
|
VERIFIED
|
|
|
|
## Description
|
|
CLI tool for deterministic test fixture management. Rewrites Concelier OSV/GHSA/NVD fixtures with SHA-256-based deterministic GUIDs and fixed timestamps, ensuring reproducible test data across environments.
|
|
|
|
## Implementation Details
|
|
- **Fixture Updater App**: `src/Tools/FixtureUpdater/FixtureUpdaterApp.cs` (96 lines) -- CLI entry point using `System.CommandLine`. Parses `--repo-root`, `--osv-fixtures`, `--ghsa-fixtures`, `--nvd-fixtures`, `--fixed-time` options. Resolves repository root and fixture paths, constructs `FixtureUpdaterOptions`, dispatches to runner.
|
|
- **Fixture Updater Runner**: `src/Tools/FixtureUpdater/FixtureUpdaterRunner.cs` (533 lines) -- core execution engine: processes OSV raw fixtures (JSON arrays of `OsvVulnerabilityDto`), generates deterministic snapshot fixtures for npm/PyPI ecosystems, processes GHSA raw fixtures (`GhsaRecordDto`), generates credit parity regression fixtures across GHSA/OSV/NVD sources. Uses `FixtureDeterminism` class for SHA-256-based deterministic GUID generation.
|
|
- **Program.cs**: `src/Tools/FixtureUpdater/Program.cs` (3 lines) -- delegates to `FixtureUpdaterApp.RunAsync(args)`.
|
|
|
|
## E2E Test Plan
|
|
- [x] Run the fixture updater tool twice with the same inputs and verify outputs are bit-for-bit identical (determinism check)
|
|
- [x] Verify error reporting includes context about which fixture source caused the failure
|
|
|
|
## Verification
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 1 code review + Tier 2d test verification
|
|
- **Build**: Passes (0 errors, 0 warnings)
|
|
- **Tests**: 2 tests pass (determinism verification, error reporting with context)
|
|
- **Caveat**: Original feature description overstated capabilities. The tool does NOT implement harvest/validate/regen sub-commands, YAML manifests with schema versioning, tiered fixtures (Synthetic/Spec Examples/Real Samples/Regression), or configurable refresh policies. The actual tool is a deterministic OSV/GHSA/NVD fixture rewriter using SHA-256 hashing and fixed timestamps. Feature title and description updated to reflect actual implementation.
|
|
|
|
## Recheck (Run-002)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay.
|
|
- **Tests**: PASS (`src/Tools/__Tests/FixtureUpdater.Tests`: 2/2).
|
|
- **Tier 2 Evidence**: `docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-002/tier2-integration-check.json`
|
|
- **Outcome**: Deterministic fixture rewrite and contextual error reporting behavior remain stable.
|
|
|
|
|
|
## Recheck (Run-003)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay.
|
|
- **Tests**: PASS (`src/Tools/__Tests/FixtureUpdater.Tests`: 2/2).
|
|
- **Tier 2 Evidence**: `docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-003/tier2-integration-check.json`
|
|
- **Outcome**: Deterministic fixture rewrite and contextual error reporting behavior remain stable.
|
|
|
|
## Recheck (Run-004)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay.
|
|
- **Tests**: PASS (FixtureUpdater 2/2).
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-004/tier2-integration-check.json
|
|
- **Outcome**: Checked Tools behavior remains healthy in follow-up replay.
|
|
|
|
## Recheck (Run-005)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay.
|
|
- **Tests**: PASS (FixtureUpdater 2/2).
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-005/tier2-integration-check.json
|
|
- **Outcome**: Checked Tools behavior remains healthy in continued replay.
|
|
|
|
## Recheck (Run-006)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay.
|
|
- **Tests**: PASS (FixtureUpdater 2/2).
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-006/tier2-integration-check.json
|
|
- **Outcome**: Checked Tools behavior remains healthy in continued replay.
|
|
|
|
## Recheck (Run-007)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay.
|
|
- **Tests**: PASS (FixtureUpdater 2/2).
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-007/tier2-integration-check.json
|
|
- **Outcome**: Checked Tools behavior remains healthy in continued replay.
|
|
|
|
## Recheck (Run-008)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay.
|
|
- **Tests**: PASS (FixtureUpdater 2/2).
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-008/tier2-integration-check.json
|
|
- **Outcome**: Checked Tools behavior remains healthy in continued replay.
|
|
|
|
|
|
## Recheck (Run-009)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay.
|
|
- **Tests**: PASS (WorkflowGenerator 76/76, FixtureUpdater 2/2, GoldenPairs 9/9; total 87/87).
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-009/tier2-integration-check.json
|
|
- **Outcome**: Checked tools behavior remains healthy in continued replay.
|
|
## Recheck (Run-010)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay.
|
|
- **Tests**: PASS (WorkflowGenerator 76/76, FixtureUpdater 2/2, GoldenPairs 9/9; total 87/87).
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-010/tier2-integration-check.json
|
|
- **Outcome**: Checked tools behavior remains healthy in continued replay.
|
|
## Recheck (Run-011)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay.
|
|
- **Tests**: PASS (WorkflowGenerator 76/76, FixtureUpdater 2/2, GoldenPairs 9/9; total 87/87).
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-011/tier2-integration-check.json
|
|
- **Outcome**: Checked tools behavior remains healthy in continued replay.
|
|
## Recheck (Run-012)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic tools suite replay.
|
|
- **Tests**: PASS (WorkflowGenerator 76/76, FixtureUpdater 2/2, GoldenPairs 9/9; total 87/87).
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-012/tier2-integration-check.json
|
|
- **Outcome**: Checked tools behavior remains healthy in continued replay.
|
|
|
|
## Recheck (Run-013)
|
|
- **Verified**: 2026-02-10
|
|
- **Method**: Tier 2d deterministic integration replay with fresh command-output evidence.
|
|
- **Tests**: PASS (2/2; Tools matrix 87/87: WorkflowGenerator 76, FixtureUpdater 2, GoldenPairs 9.)
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-013/tier2-integration-check.json
|
|
- **Outcome**: Checked Tools behavior remains healthy in continued replay.
|
|
|
|
## Recheck (Run-014)
|
|
- **Verified**: 2026-02-11
|
|
- **Method**: Strict Tier 2 CLI replay with positive and negative command paths.
|
|
- **Tests**: PASS (`src/Tools/__Tests/FixtureUpdater.Tests`: 4/4).
|
|
- **Tier 2 Evidence**: docs/qa/feature-checks/runs/tools/fixture-harvester-tool/run-014/tier2-cli-check.json
|
|
- **Outcome**: Deterministic fixture rewrite behavior and invalid-argument error handling verified via end-user CLI commands.
|