stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 16 Releases Wiki Activity
Files
49922dff5ad4435420dac177e832ba4290ceaea9
git.stella-ops.org/docs/dev/lnm-determinism-tests.md
master 536f6249a6
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Details
Add SBOM, symbols, traces, and VEX files for CVE-2022-21661 SQLi case 
- Created CycloneDX and SPDX SBOM files for both reachable and unreachable images.
- Added symbols.json detailing function entry and sink points in the WordPress code.
- Included runtime traces for function calls in both reachable and unreachable scenarios.
- Developed OpenVEX files indicating vulnerability status and justification for both cases.
- Updated README for evaluator harness to guide integration with scanner output.
2025-11-08 20:53:45 +02:00

3.6 KiB
Raw Blame History

Link-Not-Merge Determinism Test Plan

Task: MERGE-LNM-21-003 — replace legacy merge determinism suites with observation/linkset regressions now that NoMergeEnabled is defaulted to true.

Objectives

  • Validate raw advisory documents remain byte-stable through observation/linkset materialisation.
  • Ensure conflicts detected during linkset building surface in telemetry and persisted artifacts without merge-side mutation.
  • Keep canonical hash output stable for exports/evidence bundles after repeated runs.

Test Coverage Outline

  1. Raw → Observation determinism

    • Feed canonical advisory raw fixtures containing mixed casing, duplicate aliases, and provenance metadata.
    • Assert repeated runs of AdvisoryObservationFactory emit identical observations (structural equality + canonical JSON hash).
    • Verify raw linkset payload retains original ordering/whitespace while canonical linkset stays normalised.
    • Initial coverage implemented via AdvisoryObservationFactoryTests.Create_IsDeterministicAcrossRuns (core tests).

  2. Linkset conflict surfacing

    • Build linksets from conflicting advisory observations (e.g., differing severity or status flags).
    • Confirm conflict markers propagate to AdvisoryLinkset outputs and associated metrics/log records.
    • Capture deterministic ordering of conflict explanations for evidence exports.
    • Coverage landed via AdvisoryObservationFactoryTests.Create_PreservesRawReferencesForConflictAudits (raw linkset + attribute parity) and AdvisoryEventLogTests.AppendAsync_SortsConflictStatementIds (canonical conflict JSON + stable hashes).

  3. Evidence/export parity

    • Re-run observation/linkset pipelines against identical fixtures and assert resulting evidence manifests hash-identically.
    • Track monotonic supersedes chains and ensure canonical link records include PRIMARY schemes.
    • JsonExportSnapshotBuilderTests.WriteAsync_DifferentInputOrderProducesSameDigest now proves export bundles remain byte-identical regardless of advisory enumeration order; digest sampling extends ProducesIdenticalBytesAcrossRuns.

Mongo2Go/OpenSSL toolchain

Concelier solution tests (and most connector suites) depend on Mongo2Go’s embedded mongod, which is linked against OpenSSL 1.1. The repo already ships the required libraries in tests/native/openssl-1.1/linux-x64/{libcrypto.so.1.1,libssl.so.1.1}; use them instead of installing global packages so offline runners stay deterministic.

  1. Add the shim to your shell before executing any Mongo-backed suite:

    export LD_LIBRARY_PATH="$(git rev-parse --show-toplevel)/tests/native/openssl-1.1/linux-x64:${LD_LIBRARY_PATH:-}"

  2. For single commands you can prefix the invocation (handy for CI copy/paste):

    LD_LIBRARY_PATH="$(pwd)/tests/native/openssl-1.1/linux-x64" \
  dotnet test src/Concelier/StellaOps.Concelier.sln --nologo

  3. The shim’s provenance and troubleshooting notes live in tests/native/openssl-1.1/README.md; reference it when mirroring the toolchain into air-gapped runners.

Migration Steps

  • Retire StellaOps.Concelier.Merge.Tests determinism suites once observation/linkset equivalents land.
  • Introduce new regression fixtures under StellaOps.Concelier.Core.Tests (shared via StellaOps.Concelier.Testing).
  • Wire test helpers to Mongo in-memory harness for end-to-end parity runs.
  • Update documentation (docs/migration/no-merge.md) with validation checklist once new tests are green.

Pending: execute suites on a workstation with the .NET 10 preview SDK; local environment lacks a functioning CLI, so validation runs must happen downstream.