|
|
|
|
@@ -2,7 +2,7 @@
|
|
|
|
|
|
|
|
|
|
## Goals
|
|
|
|
|
- Track schema revisions for `Schedule` and `Run` documents so storage upgrades are deterministic across air-gapped installs.
|
|
|
|
|
- Provide reusable upgrade helpers that normalize Mongo snapshots (raw BSON → JSON) into the latest DTOs without mutating inputs.
|
|
|
|
|
- Provide reusable upgrade helpers that normalize PostgreSQL snapshots (raw JSONB → JSON) into the latest DTOs without mutating inputs.
|
|
|
|
|
- Formalize the allowed `RunState` graph and surface guard-rail helpers (timestamps, stats monotonicity) for planners/runners.
|
|
|
|
|
|
|
|
|
|
## Non-goals
|
|
|
|
|
@@ -17,7 +17,7 @@
|
|
|
|
|
- `scheduler.impact-set@1` (shared envelope used by planners).
|
|
|
|
|
- Expose `EnsureSchedule`, `EnsureRun`, `EnsureImpactSet` helpers mirroring the Notify model pattern to normalize missing/whitespace values.
|
|
|
|
|
- Extend `Schedule`, `Run`, and `ImpactSet` records with an optional `schemaVersion` constructor parameter defaulting through the `Ensure*` helpers. The canonical JSON serializer will list `schemaVersion` first so documents round-trip deterministically.
|
|
|
|
|
- Persisted Mongo documents will now always include `schemaVersion`; exporters/backups can rely on this when bundling Offline Kit snapshots.
|
|
|
|
|
- Persisted PostgreSQL documents will now always include `schemaVersion`; exporters/backups can rely on this when bundling Offline Kit snapshots.
|
|
|
|
|
|
|
|
|
|
## Migration Helper Shape
|
|
|
|
|
- Add `SchedulerSchemaMigration` static class with:
|
|
|
|
|
@@ -55,8 +55,8 @@
|
|
|
|
|
- Expose small helper to tag `RunReason.ImpactWindowFrom/To` automatically when set by planners (using normalized ISO-8601).
|
|
|
|
|
|
|
|
|
|
## Interaction Points
|
|
|
|
|
- **WebService**: call `SchedulerSchemaMigration.UpgradeSchedule` when returning schedules from Mongo, so clients always see the newest DTO regardless of stored version.
|
|
|
|
|
- **Storage.Mongo**: wrap DTO round-trips; the migration helper acts during read, and the state machine ensures updates respect transition rules before writing.
|
|
|
|
|
- **WebService**: call `SchedulerSchemaMigration.UpgradeSchedule` when returning schedules from PostgreSQL, so clients always see the newest DTO regardless of stored version.
|
|
|
|
|
- **Storage.Postgres**: wrap DTO round-trips; the migration helper acts during read, and the state machine ensures updates respect transition rules before writing.
|
|
|
|
|
- **Queue/Worker**: use `RunStateMachine.EnsureTransition` to guard planner/runner state updates (replace ad-hoc `with run` clones).
|
|
|
|
|
- **Offline Kit**: embed `schemaVersion` in exported JSON/Trivy artifacts; migrations ensure air-gapped upgrades flow without manual scripts.
|
|
|
|
|
|
|
|
|
|
@@ -67,20 +67,20 @@
|
|
|
|
|
4. Update modules (Storage, WebService, Worker) to use new helpers; add logging around migrations/transitions.
|
|
|
|
|
|
|
|
|
|
## Test Strategy
|
|
|
|
|
- **Migration happy-path**: load sample Mongo fixtures for `schedule@1` and `run@1`, assert `schemaVersion` normalization, deduplicated subscribers, limits defaults. Include snapshots without the version field to exercise defaulting logic.
|
|
|
|
|
- **Migration happy-path**: load sample PostgreSQL fixtures for `schedule@1` and `run@1`, assert `schemaVersion` normalization, deduplicated subscribers, limits defaults. Include snapshots without the version field to exercise defaulting logic.
|
|
|
|
|
- **Legacy upgrade cases**: craft synthetic `schedule@0` / `run@0` JSON fragments (missing new fields, using old enum names) and verify version-specific fixups produce the latest DTO while populating `MigrationResult.Warnings`.
|
|
|
|
|
- **Strict mode behavior**: attempt to upgrade documents with unexpected properties and ensure warnings/throws align with configuration.
|
|
|
|
|
- **Run state transitions**: unit-test `RunStateMachine` for every allowed edge, invalid transitions, and timestamp/error invariants (e.g., `FinishedAt` only set on terminal states). Provide parameterized tests to confirm stats monotonicity enforcement.
|
|
|
|
|
- **Serialization determinism**: round-trip upgraded DTOs via `CanonicalJsonSerializer` to confirm property order includes `schemaVersion` first and produces stable hashes.
|
|
|
|
|
- **Documentation snippets**: extend module README or API docs with example migrations/run-state usage; verify via doc samples test (if available) or include as part of CI doc linting.
|
|
|
|
|
|
|
|
|
|
## Open Questions
|
|
|
|
|
- Do we need downgrade (`ToVersion`) helpers for Offline Kit exports? (Assumed no for now. Add backlog item if required.)
|
|
|
|
|
- Should `ImpactSet` migrations live here or in ImpactIndex module? (Lean towards here because DTO defined in Models; coordinate with ImpactIndex guild if they need specialized upgrades.)
|
|
|
|
|
- How do we surface migration warnings to telemetry? Proposal: caller logs `warning` with `MigrationResult.Warnings` immediately after calling helper.
|
|
|
|
|
|
|
|
|
|
## Status — 2025-10-20
|
|
|
|
|
|
|
|
|
|
- `SchedulerSchemaMigration` now upgrades legacy `@0` schedule/run/impact-set documents to the `@1` schema, defaulting missing counters/arrays and normalizing booleans & severities. Each backfill emits a warning so storage/web callers can log the mutation.
|
|
|
|
|
- `RunStateMachine.EnsureTransition` guards timestamp ordering and stats monotonicity; builders and extension helpers are wired into the scheduler worker/web service plans.
|
|
|
|
|
- Tests exercising legacy upgrades live in `StellaOps.Scheduler.Models.Tests/SchedulerSchemaMigrationTests.cs`; add new fixtures there when introducing additional schema versions.
|
|
|
|
|
## Open Questions
|
|
|
|
|
- Do we need downgrade (`ToVersion`) helpers for Offline Kit exports? (Assumed no for now. Add backlog item if required.)
|
|
|
|
|
- Should `ImpactSet` migrations live here or in ImpactIndex module? (Lean towards here because DTO defined in Models; coordinate with ImpactIndex guild if they need specialized upgrades.)
|
|
|
|
|
- How do we surface migration warnings to telemetry? Proposal: caller logs `warning` with `MigrationResult.Warnings` immediately after calling helper.
|
|
|
|
|
|
|
|
|
|
## Status — 2025-10-20
|
|
|
|
|
|
|
|
|
|
- `SchedulerSchemaMigration` now upgrades legacy `@0` schedule/run/impact-set documents to the `@1` schema, defaulting missing counters/arrays and normalizing booleans & severities. Each backfill emits a warning so storage/web callers can log the mutation.
|
|
|
|
|
- `RunStateMachine.EnsureTransition` guards timestamp ordering and stats monotonicity; builders and extension helpers are wired into the scheduler worker/web service plans.
|
|
|
|
|
- Tests exercising legacy upgrades live in `StellaOps.Scheduler.Models.Tests/SchedulerSchemaMigrationTests.cs`; add new fixtures there when introducing additional schema versions.
|
|
|
|
|
|