diff --git a/docs/implplan/SPRINT_20260419_028_Tools_targeted_xunit_runner_workflow.md b/docs/implplan/SPRINT_20260419_028_Tools_targeted_xunit_runner_workflow.md new file mode 100644 index 000000000..04db33f3d --- /dev/null +++ b/docs/implplan/SPRINT_20260419_028_Tools_targeted_xunit_runner_workflow.md @@ -0,0 +1,50 @@ +# Sprint 20260419-028 - Tools Targeted xUnit Runner Workflow + +## Topic & Scope +- Harden the repo-side targeted test workflow for xUnit v3 projects that run under Microsoft Testing Platform. +- Add a small helper so targeted verification uses one deterministic command instead of ad hoc build plus `dotnet exec` steps. +- Update QA and testing guidance so maintainers stop relying on `dotnet test --filter` where it is ignored. +- Working directory: `scripts/`. +- Cross-module touchpoints explicitly allowed for this sprint: `docs/qa/feature-checks/`, `docs/code-of-conduct/`, `docs/implplan/`. +- Expected evidence: runnable helper script, synced QA/testing docs, and a focused verification run against real Platform and Concelier test assemblies. + +## Dependencies & Concurrency +- Follows `docs/implplan/SPRINT_20260418_001_Platform_advisory_setup_truthfulness_hardening.md`, which exposed the current xUnit v3 targeted-test mismatch while closing backend setup verification. +- Safe to execute in parallel with unrelated module work because the write scope is limited to repo-level scripts plus shared QA/testing docs. + +## Documentation Prerequisites +- `docs/qa/feature-checks/FLOW.md` +- `docs/code-of-conduct/TESTING_PRACTICES.md` +- `scripts/test-stabilization/run-tests-batch.ps1` +- `docs/implplan/SPRINT_20260418_001_Platform_advisory_setup_truthfulness_hardening.md` + +## Delivery Tracker + +### TEST-RUNNER-001 - Add deterministic targeted xUnit helper and guidance +Status: DOING +Dependency: none +Owners: Developer, Test Automation, Documentation author +Task description: +- Some repo test projects expose the xUnit v3 in-process runner through Microsoft Testing Platform. In that configuration, `dotnet test --filter` can be ignored even when the caller expects a narrow subset, which makes targeted QA evidence misleading and wastes shell/process budget during investigation. +- Add a repo-level helper that rebuilds a specific test project when needed, resolves the produced DLL, and executes the xUnit runner directly with method/class/namespace/trait filters. Update shared QA/testing docs to make that workflow the default for xUnit v3 targeted verification and explicitly call out that unified exec saturation is an external agent-host concern, not a Stella Ops runtime defect. + +Completion criteria: +- [ ] A repo-level PowerShell helper runs targeted xUnit v3 tests from a `.csproj` using direct DLL execution. +- [ ] QA flow guidance documents when `dotnet test --filter` is valid and when direct xUnit DLL execution is required. +- [ ] Testing practices document the targeted xUnit v3 workflow and expected evidence capture. +- [ ] Focused verification proves the helper works against the Platform and Concelier setup/advisory test projects. + +## Execution Log +| Date (UTC) | Update | Owner | +| --- | --- | --- | +| 2026-04-19 | Sprint created after the setup/advisory hardening verification exposed that several repo test projects use xUnit v3 under Microsoft Testing Platform, where `dotnet test --filter` does not provide trustworthy targeted execution. | Codex | + +## Decisions & Risks +- Decision: fix the repo-side verification workflow with a helper script plus shared QA/testing guidance rather than per-module notes, because the runner behavior is infrastructure-wide and not specific to Platform or Concelier. +- Risk: a helper that assumes a single target framework may mis-handle multi-target test projects. The first version must either support explicit framework selection or fail clearly when the `.csproj` declares multiple target frameworks. +- External-note boundary: the repeated unified-exec session warnings come from the agent host process budget, not from Stella Ops application code. This sprint can reduce command churn by consolidating targeted test execution into one helper, but it cannot change the external host limit itself. + +## Next Checkpoints +- Add the helper under `scripts/`. +- Update shared QA/testing docs with the exact invocation pattern. +- Verify the helper against the targeted Platform and Concelier setup/advisory methods. diff --git a/scripts/test-targeted-xunit.ps1 b/scripts/test-targeted-xunit.ps1 new file mode 100644 index 000000000..250ab8d3e --- /dev/null +++ b/scripts/test-targeted-xunit.ps1 @@ -0,0 +1,254 @@ +#!/usr/bin/env pwsh +<# +.SYNOPSIS + Runs targeted xUnit v3 tests by executing the compiled test DLL directly. + +.DESCRIPTION + Some Stella Ops test projects run under Microsoft Testing Platform with the + xUnit v3 in-process runner. In that mode, `dotnet test --filter` may be + ignored, which makes targeted verification misleading. This helper rebuilds + the requested test project if needed, resolves the output test assembly, and + invokes `dotnet exec ` with xUnit's native filter arguments. + +.PARAMETER Project + Path to the test `.csproj`. + +.PARAMETER Method + Fully qualified test method to run. May be specified more than once. + +.PARAMETER Class + Fully qualified test class to run. May be specified more than once. + +.PARAMETER Namespace + Namespace filter to run. May be specified more than once. + +.PARAMETER Trait + xUnit trait filter in `name=value` form. May be specified more than once. + +.PARAMETER QueryFilter + xUnit query filter string. Do not combine with `Method`, `Class`, `Namespace`, + or `Trait`. + +.PARAMETER Framework + Target framework. Required when the project uses `TargetFrameworks`. + +.PARAMETER Configuration + Build configuration. Defaults to `Debug`. + +.PARAMETER Reporter + xUnit reporter to use. Defaults to `verbose`. + +.PARAMETER SkipBuild + Skip the project build and run the existing test DLL. + +.PARAMETER BuildProjectReferences + Whether to build project references. Defaults to `$false` to keep targeted + verification fast and avoid broad graph rebuilds when outputs are already + available. + +.PARAMETER AdditionalRunnerArgument + Additional raw xUnit runner arguments appended after the generated filters. +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] + [string]$Project, + + [string[]]$Method = @(), + [string[]]$Class = @(), + [string[]]$Namespace = @(), + [string[]]$Trait = @(), + [string]$QueryFilter, + [string]$Framework, + [string]$Configuration = 'Debug', + [string]$Reporter = 'verbose', + [switch]$SkipBuild, + [bool]$BuildProjectReferences = $false, + [string[]]$AdditionalRunnerArgument = @() +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' + +function Resolve-RepoRoot { + return Split-Path -Parent $PSScriptRoot +} + +function Resolve-ProjectPath { + param( + [Parameter(Mandatory)] + [string]$InputPath, + [Parameter(Mandatory)] + [string]$RepoRoot + ) + + if ([System.IO.Path]::IsPathRooted($InputPath)) { + return (Resolve-Path -LiteralPath $InputPath).ProviderPath + } + + return (Resolve-Path -LiteralPath (Join-Path $RepoRoot $InputPath)).ProviderPath +} + +function Get-ProjectMetadata { + param( + [Parameter(Mandatory)] + [string]$ProjectPath + ) + + [xml]$projectXml = Get-Content -LiteralPath $ProjectPath -Raw + $propertyGroups = @($projectXml.Project.PropertyGroup) + + $assemblyName = $null + $targetFramework = $null + $targetFrameworks = $null + + foreach ($group in $propertyGroups) { + if (-not $assemblyName -and $group.AssemblyName) { + $assemblyName = $group.AssemblyName.Trim() + } + + if (-not $targetFramework -and $group.TargetFramework) { + $targetFramework = $group.TargetFramework.Trim() + } + + if (-not $targetFrameworks -and $group.TargetFrameworks) { + $targetFrameworks = $group.TargetFrameworks.Trim() + } + } + + if (-not $assemblyName) { + $assemblyName = [System.IO.Path]::GetFileNameWithoutExtension($ProjectPath) + } + + return [pscustomobject]@{ + AssemblyName = $assemblyName + TargetFramework = $targetFramework + TargetFrameworks = $targetFrameworks + } +} + +function Resolve-Framework { + param( + [Parameter(Mandatory)] + $Metadata, + [string]$RequestedFramework + ) + + if ($RequestedFramework) { + return $RequestedFramework + } + + if ($Metadata.TargetFramework) { + return $Metadata.TargetFramework + } + + if ($Metadata.TargetFrameworks) { + $frameworks = @($Metadata.TargetFrameworks.Split(';', [System.StringSplitOptions]::RemoveEmptyEntries)) + if ($frameworks.Count -eq 1) { + return $frameworks[0].Trim() + } + + throw "Project declares multiple target frameworks ($($Metadata.TargetFrameworks)). Specify -Framework explicitly." + } + + throw "Could not determine a target framework from '$ProjectPath'. Specify -Framework explicitly." +} + +function Assert-FilterShape { + param( + [string[]]$MethodFilters, + [string[]]$ClassFilters, + [string[]]$NamespaceFilters, + [string[]]$TraitFilters, + [string]$Query + ) + + $hasSimpleFilters = $MethodFilters.Count -gt 0 -or + $ClassFilters.Count -gt 0 -or + $NamespaceFilters.Count -gt 0 -or + $TraitFilters.Count -gt 0 + + if (-not $hasSimpleFilters -and [string]::IsNullOrWhiteSpace($Query)) { + throw 'Specify at least one filter via -Method, -Class, -Namespace, -Trait, or -QueryFilter.' + } + + if ($hasSimpleFilters -and -not [string]::IsNullOrWhiteSpace($Query)) { + throw 'Do not mix -QueryFilter with -Method/-Class/-Namespace/-Trait.' + } +} + +$repoRoot = Resolve-RepoRoot +$projectPath = Resolve-ProjectPath -InputPath $Project -RepoRoot $repoRoot +$metadata = Get-ProjectMetadata -ProjectPath $projectPath +$resolvedFramework = Resolve-Framework -Metadata $metadata -RequestedFramework $Framework + +Assert-FilterShape -MethodFilters $Method -ClassFilters $Class -NamespaceFilters $Namespace -TraitFilters $Trait -Query $QueryFilter + +$projectDirectory = Split-Path -Parent $projectPath +$assemblyPath = Join-Path $projectDirectory "bin\$Configuration\$resolvedFramework\$($metadata.AssemblyName).dll" + +if (-not $SkipBuild) { + $buildArguments = @( + 'build', + $projectPath, + '--no-restore', + '--nologo', + '--disable-build-servers', + '-v', 'minimal', + '-c', $Configuration, + '-p:BuildProjectReferences=' + ($BuildProjectReferences ? 'true' : 'false'), + '/m:1' + ) + + if ($resolvedFramework) { + $buildArguments += @('-f', $resolvedFramework) + } + + Write-Host "Building $projectPath ($resolvedFramework, $Configuration)..." + & dotnet @buildArguments + if ($LASTEXITCODE -ne 0) { + exit $LASTEXITCODE + } +} + +if (-not (Test-Path -LiteralPath $assemblyPath)) { + throw "Compiled test assembly not found: $assemblyPath" +} + +$runnerArguments = @( + 'exec', + $assemblyPath, + '-noLogo' +) + +if (-not [string]::IsNullOrWhiteSpace($Reporter)) { + $runnerArguments += @('-reporter', $Reporter) +} + +if (-not [string]::IsNullOrWhiteSpace($QueryFilter)) { + $runnerArguments += @('-filter', $QueryFilter) +} + +foreach ($value in $Method) { + $runnerArguments += @('-method', $value) +} + +foreach ($value in $Class) { + $runnerArguments += @('-class', $value) +} + +foreach ($value in $Namespace) { + $runnerArguments += @('-namespace', $value) +} + +foreach ($value in $Trait) { + $runnerArguments += @('-trait', $value) +} + +foreach ($value in $AdditionalRunnerArgument) { + $runnerArguments += $value +} + +Write-Host "Running targeted xUnit assembly $assemblyPath" +& dotnet @runnerArguments +exit $LASTEXITCODE