9075bad2d9
- Implemented `MongoIndexModelTests` to verify index models for various stores. - Created `OpenApiMetadataFactory` with methods to generate OpenAPI metadata. - Added tests for `OpenApiMetadataFactory` to ensure expected defaults and URL overrides. - Introduced `ObserverSurfaceSecrets` and `WebhookSurfaceSecrets` for managing secrets. - Developed `RuntimeSurfaceFsClient` and `WebhookSurfaceFsClient` for manifest retrieval. - Added dependency injection tests for `SurfaceEnvironmentRegistration` in both Observer and Webhook contexts. - Implemented tests for secret resolution in `ObserverSurfaceSecretsTests` and `WebhookSurfaceSecretsTests`. - Created `EnsureLinkNotMergeCollectionsMigrationTests` to validate MongoDB migration logic. - Added project files for MongoDB tests and NuGet package mirroring.
41 lines
2.6 KiB
Markdown
41 lines
2.6 KiB
Markdown
# EntryTrace Heuristic Review Cadence
|
|
|
|
EntryTrace heuristics must stay aligned with competitor techniques and new runtime behaviours. This cadence makes updates predictable and deterministic.
|
|
|
|
## Objectives
|
|
- Refresh shell/launcher heuristics quarterly using the latest gap analysis in `docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md`.
|
|
- Re-run explain-trace fixtures to confirm deterministic outputs and document any newly unsupported constructs.
|
|
- Ensure operator-facing explainability stays in sync with emitted diagnostics and metrics.
|
|
|
|
## Cadence
|
|
- **Frequency:** Quarterly (Jan, Apr, Jul, Oct) or sooner when critical regressions are discovered.
|
|
- **Owners:** EntryTrace Guild with QA Guild pairing.
|
|
- **Inputs:** Gap benchmark doc, new runtime samples from support channels, and anonymised customer repros (when permitted).
|
|
- **Outputs:**
|
|
- Updated heuristics/diagnostics in `StellaOps.Scanner.EntryTrace` with deterministic fixtures.
|
|
- Changelog entry in `src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/TASKS.md`.
|
|
- Sprint log updates under the active `SPRINT_0138_0000_0001_scanner_ruby_parity.md` when cadence items land.
|
|
|
|
## Workflow
|
|
1) **Collect & triage signals**
|
|
- Parse new gaps from the benchmark doc; map each to an EntryTrace detector area (shell parser, interpreter tracer, PATH resolver).
|
|
- Classify as _coverage gap_, _precision issue_, or _observability gap_.
|
|
2) **Fixture-first update**
|
|
- Add/extend fixtures in `StellaOps.Scanner.EntryTrace.Tests/Fixtures` before modifying code.
|
|
- Use deterministic serializers to keep fixture outputs byte-stable.
|
|
3) **Implement & validate**
|
|
- Update analyzers/diagnostics; run `dotnet test src/Scanner/__Tests/StellaOps.Scanner.EntryTrace.Tests/StellaOps.Scanner.EntryTrace.Tests.csproj --nologo --verbosity minimal`.
|
|
- Confirm metrics counters (`entrytrace_*`) and explain-trace text stay consistent.
|
|
4) **Record explainability**
|
|
- Update explain-trace catalog (diagnostic enum descriptions) when new reasons are introduced.
|
|
- Add operator notes to sprint log if remediation guidance changes.
|
|
5) **Publish**
|
|
- Attach a brief summary to the sprint Execution Log and to `TASKS.md` with date + scope.
|
|
|
|
## Fail-safe & rollback
|
|
- Keep previous fixture baselines; if a heuristic widens too far, revert to prior fixture sets to restore determinism.
|
|
- Prefer additive diagnostics over behavioural regressions; when behaviour must change, document it in the sprint log and `TASKS.md`.
|
|
|
|
## Ownership transitions
|
|
- If the cadence cannot run on schedule, mark the relevant sprint task `BLOCKED` with the reason and hand off to the Project Manager to re-staff before the next window.
|