Files
git.stella-ops.org/docs/modules/excititor/operations/consensus-removal-runbook.md
master d519782a8f
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
prep docs and service updates
2025-11-21 06:56:36 +00:00

1.9 KiB

Excititor Consensus Removal Runbook (AOC-19-004)

  • Date: 2025-11-21
  • Scope: EXCITITOR-CORE-AOC-19-004
  • Goal: Eliminate legacy consensus/merged severity fields so Excititor remains aggregation-only.

Cutover steps

  1. Freeze consensus refreshDisableConsensus=true (default) forces refresh loop off. Keep this enabled during migration.
  2. Schema cleanup — migrate collections to remove or null legacy fields:
    • vex_consensus / vex_consensus_holds: drop/ignore fields consensusDigest, policyVersion, policyRevisionId, policyDigest, summary, signals, status (merged) once Policy takes over.
    • vex_observations / materialized exports: ensure no merged severity/status fields are written.
    • vex_mirror exports: stop emitting consensus JSON; retain raw observations only.
  3. Telemetry: emit counter excititor.ingest.consensus.disabled (tags tenant, source, connectorId) once per batch to prove cutover.
  4. Guards: AOC guards reject any incoming/derived field in {mergedSeverity, consensusScore, computedStatus}.
  5. Backfill: run one-off job to set consensusDisabled=true on legacy records and remove merged fields without touching raw observations.
  6. Verification: regression checklist (per tenant):
    • No writes to vex_consensus* collections after cutover.
    • Ingest + export fixtures show only raw observations/linksets; snapshots deterministic.
    • Telemetry counter present; absence of consensus refresh logs.

Config

Excititor:Worker:
  DisableConsensus: true   # keep true post-cutover

Test plan (after disk space is restored)

  • Unit: AOC guard rejects merged fields.
  • Integration (Mongo2Go): ingest batch containing merged fields → rejected; telemetry counter increments.
  • Worker: start with DisableConsensus=true → consensus refresh loop does not schedule; log once at startup.