git.stella-ops.org/docs/dev/lnm-determinism-tests.md

Link-Not-Merge Determinism Test Plan

Task: MERGE-LNM-21-003 — replace legacy merge determinism suites with observation/linkset regressions now that NoMergeEnabled is defaulted to true.

Objectives

• Validate raw advisory documents remain byte-stable through observation/linkset materialisation.
• Ensure conflicts detected during linkset building surface in telemetry and persisted artifacts without merge-side mutation.
• Keep canonical hash output stable for exports/evidence bundles after repeated runs.

Test Coverage Outline

1. Raw → Observation determinism

   • Feed canonical advisory raw fixtures containing mixed casing, duplicate aliases, and provenance metadata.
   • Assert repeated runs of AdvisoryObservationFactory emit identical observations (structural equality + canonical JSON hash).
   • Verify raw linkset payload retains original ordering/whitespace while canonical linkset stays normalised.
   • Initial coverage implemented via AdvisoryObservationFactoryTests.Create_IsDeterministicAcrossRuns (core tests).

2. Linkset conflict surfacing

   • Build linksets from conflicting advisory observations (e.g., differing severity or status flags).
   • Confirm conflict markers propagate to AdvisoryLinkset outputs and associated metrics/log records.
   • Capture deterministic ordering of conflict explanations for evidence exports.
   • Coverage landed via AdvisoryObservationFactoryTests.Create_PreservesRawReferencesForConflictAudits (raw linkset + attribute parity) and AdvisoryEventLogTests.AppendAsync_SortsConflictStatementIds (canonical conflict JSON + stable hashes).

3. Evidence/export parity

   • Re-run observation/linkset pipelines against identical fixtures and assert resulting evidence manifests hash-identically.
   • Track monotonic supersedes chains and ensure canonical link records include PRIMARY schemes.
   • JsonExportSnapshotBuilderTests.WriteAsync_DifferentInputOrderProducesSameDigest now proves export bundles remain byte-identical regardless of advisory enumeration order; digest sampling extends ProducesIdenticalBytesAcrossRuns.

Mongo2Go/OpenSSL toolchain

Concelier solution tests (and most connector suites) depend on Mongo2Go’s embedded mongod, which is linked against OpenSSL 1.1. The repo already ships the required libraries in tests/native/openssl-1.1/linux-x64/{libcrypto.so.1.1,libssl.so.1.1}; use them instead of installing global packages so offline runners stay deterministic.

1. Add the shim to your shell before executing any Mongo-backed suite:

   export LD_LIBRARY_PATH="$(git rev-parse --show-toplevel)/tests/native/openssl-1.1/linux-x64:${LD_LIBRARY_PATH:-}"
   

2. For single commands you can prefix the invocation (handy for CI copy/paste):

   LD_LIBRARY_PATH="$(pwd)/tests/native/openssl-1.1/linux-x64" \
     dotnet test src/Concelier/StellaOps.Concelier.sln --nologo
   

3. The shim’s provenance and troubleshooting notes live in tests/native/openssl-1.1/README.md; reference it when mirroring the toolchain into air-gapped runners.

Migration Steps

• Retire StellaOps.Concelier.Merge.Tests determinism suites once observation/linkset equivalents land.
• Introduce new regression fixtures under StellaOps.Concelier.Core.Tests (shared via StellaOps.Concelier.Testing).
• Wire test helpers to Mongo in-memory harness for end-to-end parity runs.
• Update documentation (docs/migration/no-merge.md) with validation checklist once new tests are green.

Pending: execute suites on a workstation with the .NET 10 preview SDK; local environment lacks a functioning CLI, so validation runs must happen downstream.