536f6249a6
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Created CycloneDX and SPDX SBOM files for both reachable and unreachable images. - Added symbols.json detailing function entry and sink points in the WordPress code. - Included runtime traces for function calls in both reachable and unreachable scenarios. - Developed OpenVEX files indicating vulnerability status and justification for both cases. - Updated README for evaluator harness to guide integration with scanner output.
54 lines
3.6 KiB
Markdown
54 lines
3.6 KiB
Markdown
# Link-Not-Merge Determinism Test Plan
|
||
|
||
**Task:** MERGE-LNM-21-003 — replace legacy merge determinism suites with observation/linkset regressions now that `NoMergeEnabled` is defaulted to `true`.
|
||
|
||
## Objectives
|
||
- Validate raw advisory documents remain byte-stable through observation/linkset materialisation.
|
||
- Ensure conflicts detected during linkset building surface in telemetry and persisted artifacts without merge-side mutation.
|
||
- Keep canonical hash output stable for exports/evidence bundles after repeated runs.
|
||
|
||
## Test Coverage Outline
|
||
1. **Raw → Observation determinism**
|
||
- Feed canonical advisory raw fixtures containing mixed casing, duplicate aliases, and provenance metadata.
|
||
- Assert repeated runs of `AdvisoryObservationFactory` emit identical observations (structural equality + canonical JSON hash).
|
||
- Verify raw linkset payload retains original ordering/whitespace while canonical linkset stays normalised.
|
||
- Initial coverage implemented via `AdvisoryObservationFactoryTests.Create_IsDeterministicAcrossRuns` (core tests).
|
||
|
||
2. **Linkset conflict surfacing**
|
||
- Build linksets from conflicting advisory observations (e.g., differing severity or status flags).
|
||
- Confirm conflict markers propagate to `AdvisoryLinkset` outputs and associated metrics/log records.
|
||
- Capture deterministic ordering of conflict explanations for evidence exports.
|
||
- Coverage landed via `AdvisoryObservationFactoryTests.Create_PreservesRawReferencesForConflictAudits` (raw linkset + attribute parity) and `AdvisoryEventLogTests.AppendAsync_SortsConflictStatementIds` (canonical conflict JSON + stable hashes).
|
||
|
||
3. **Evidence/export parity**
|
||
- Re-run observation/linkset pipelines against identical fixtures and assert resulting evidence manifests hash-identically.
|
||
- Track monotonic `supersedes` chains and ensure canonical link records include `PRIMARY` schemes.
|
||
- `JsonExportSnapshotBuilderTests.WriteAsync_DifferentInputOrderProducesSameDigest` now proves export bundles remain byte-identical regardless of advisory enumeration order; digest sampling extends `ProducesIdenticalBytesAcrossRuns`.
|
||
|
||
## Mongo2Go/OpenSSL toolchain
|
||
|
||
Concelier solution tests (and most connector suites) depend on Mongo2Go’s embedded `mongod`, which is linked against OpenSSL 1.1. The repo already ships the required libraries in `tests/native/openssl-1.1/linux-x64/{libcrypto.so.1.1,libssl.so.1.1}`; use them instead of installing global packages so offline runners stay deterministic.
|
||
|
||
1. Add the shim to your shell before executing any Mongo-backed suite:
|
||
|
||
```bash
|
||
export LD_LIBRARY_PATH="$(git rev-parse --show-toplevel)/tests/native/openssl-1.1/linux-x64:${LD_LIBRARY_PATH:-}"
|
||
```
|
||
|
||
2. For single commands you can prefix the invocation (handy for CI copy/paste):
|
||
|
||
```bash
|
||
LD_LIBRARY_PATH="$(pwd)/tests/native/openssl-1.1/linux-x64" \
|
||
dotnet test src/Concelier/StellaOps.Concelier.sln --nologo
|
||
```
|
||
|
||
3. The shim’s provenance and troubleshooting notes live in `tests/native/openssl-1.1/README.md`; reference it when mirroring the toolchain into air-gapped runners.
|
||
|
||
## Migration Steps
|
||
- [x] Retire `StellaOps.Concelier.Merge.Tests` determinism suites once observation/linkset equivalents land.
|
||
- [x] Introduce new regression fixtures under `StellaOps.Concelier.Core.Tests` (shared via `StellaOps.Concelier.Testing`).
|
||
- [ ] Wire test helpers to Mongo in-memory harness for end-to-end parity runs.
|
||
- [ ] Update documentation (`docs/migration/no-merge.md`) with validation checklist once new tests are green.
|
||
|
||
_Pending_: execute suites on a workstation with the .NET 10 preview SDK; local environment lacks a functioning CLI, so validation runs must happen downstream.
|