# Sprint 503 - Ops & Offline · 190.B) Ops Devops.I

Active items only. Completed/historic work now resides in docs/implplan/archived/tasks.md (updated 2025-11-08).

[Ops & Offline] 190.B) Ops Devops.I
Depends on: Sprint 100.A - Attestor, Sprint 110.A - AdvisoryAI, Sprint 120.A - AirGap, Sprint 130.A - Scanner, Sprint 140.A - Graph, Sprint 150.A - Orchestrator, Sprint 160.A - EvidenceLocker, Sprint 170.A - Notifier, Sprint 180.A - Cli

## Topic & Scope
- Stand up CI, signing, and offline pipelines that unblock module sprints without embedding DevOps work in dev backlogs.
- Provide sealed/airgap bootstrap artefacts and mirrors required by downstream airgap/attestation tasks.
- Ensure AOC/guard rails are enforced in CI across ingestion-heavy modules.

## Dependencies & Concurrency
- Upstream artefacts: mirror bundle automation (DEVOPS-AIRGAP-57-001), AOC analyzers, module-specific prep notes referenced per task.
- Runs in parallel with module sprints; deliverables are CI/pipeline assets, not code changes inside module working dirs.

## Documentation Prerequisites
- docs/modules/devops/architecture.md
- docs/modules/ci/architecture.md
- docs/airgap/** (for sealed-mode tasks)

## Delivery Tracker
| Task ID | State | Task description | Owners (Source) |
| --- | --- | --- | --- |
| DEVOPS-AIAI-31-001 | TODO | Stand up CI pipelines, inference monitoring, privacy logging review, and perf dashboards for Advisory AI (summaries/conflicts/remediation). | DevOps Guild, Advisory AI Guild (ops/devops) |
| DEVOPS-AIAI-31-002 | BLOCKED (2025-11-23) | Package advisory feeds (SBOM pointers + provenance) for release/offline kit; publish once CLI/Policy digests and SBOM feeds arrive. | DevOps Guild, Advisory AI Release (ops/devops) |
| DEVOPS-SPANSINK-31-003 | TODO | Deploy span sink/Signals pipeline for Excititor evidence APIs (31-003) and publish dashboards; unblock traces for `/v1/vex/observations/**`. | DevOps Guild · Observability Guild (ops/devops) |
| DEVOPS-AIRGAP-56-001 | TODO | Ship deny-all egress policies for Kubernetes (NetworkPolicy/eBPF) and docker-compose firewall rules; provide verification script for sealed mode. | DevOps Guild (ops/devops) |
| DEVOPS-AIRGAP-56-002 | TODO | Provide import tooling for bundle staging: checksum validation, offline object-store loader scripts, removable media guidance. Dependencies: DEVOPS-AIRGAP-56-001. | DevOps Guild, AirGap Importer Guild (ops/devops) |
| DEVOPS-AIRGAP-56-003 | TODO | Build Bootstrap Pack pipeline bundling images/charts, generating checksums, and publishing manifest for offline transfer. Dependencies: DEVOPS-AIRGAP-56-002. | DevOps Guild, Container Distribution Guild (ops/devops) |
| DEVOPS-AIRGAP-57-001 | TODO | Automate Mirror Bundle creation jobs with dual-control approvals, artifact signing, and checksum publication. Dependencies: DEVOPS-AIRGAP-56-003. | DevOps Guild, Mirror Creator Guild (ops/devops) |
| DEVOPS-AIRGAP-57-002 | BLOCKED (2025-11-18) | Waiting on upstream DEVOPS-AIRGAP-57-001 (mirror bundle automation) to provide artifacts/endpoints for sealed-mode CI; no sealed fixtures available to exercise tests. | DevOps Guild, Authority Guild (ops/devops) |
| DEVOPS-AIRGAP-58-001 | TODO | Provide local SMTP/syslog container templates and health checks for sealed environments; integrate into Bootstrap Pack. Dependencies: DEVOPS-AIRGAP-57-002. | DevOps Guild, Notifications Guild (ops/devops) |
| DEVOPS-AIRGAP-58-002 | TODO | Ship sealed-mode observability stack (Prometheus/Grafana/Tempo/Loki) pre-configured with offline dashboards and no remote exporters. Dependencies: DEVOPS-AIRGAP-58-001. | DevOps Guild, Observability Guild (ops/devops) |
| DEVOPS-AOC-19-001 | BLOCKED (2025-10-26) | Integrate the AOC Roslyn analyzer and guard tests into CI, failing builds when ingestion projects attempt banned writes. | DevOps Guild, Platform Guild (ops/devops) |
| DEVOPS-AOC-19-002 | BLOCKED (2025-10-26) | Add pipeline stage executing `stella aoc verify --since` against seeded Mongo snapshots for Concelier + Excititor, publishing violation report artefacts. Dependencies: DEVOPS-AOC-19-001. | DevOps Guild (ops/devops) |
| DEVOPS-AOC-19-003 | BLOCKED (2025-10-26) | Enforce unit test coverage thresholds for AOC guard suites and ensure coverage exported to dashboards. Dependencies: DEVOPS-AOC-19-002. | DevOps Guild, QA Guild (ops/devops) |
| DEVOPS-AOC-19-101 | TODO (2025-10-28) | Draft supersedes backfill rollout (freeze window, dry-run steps, rollback) once advisory_raw idempotency index passes staging verification. Dependencies: DEVOPS-AOC-19-003. | DevOps Guild, Concelier Storage Guild (ops/devops) |
| DEVOPS-ATTEST-73-001 | TODO | Provision CI pipelines for attestor service (lint/test/security scan, seed data) and manage secrets for KMS drivers. | DevOps Guild, Attestor Service Guild (ops/devops) |
| DEVOPS-ATTEST-73-002 | TODO | Establish secure storage for signing keys (vault integration, rotation schedule) and audit logging. Dependencies: DEVOPS-ATTEST-73-001. | DevOps Guild, KMS Guild (ops/devops) |
| DEVOPS-ATTEST-74-001 | TODO | Deploy transparency log witness infrastructure and monitoring. Dependencies: DEVOPS-ATTEST-73-002. | DevOps Guild, Transparency Guild (ops/devops) |
| DEVOPS-GRAPH-INDEX-28-010-REL | TODO | Publish signed Helm/Compose/offline bundles for Graph Indexer; depends on GRAPH-INDEX-28-010 dev artefacts. | DevOps Guild, Graph Indexer Guild (ops/devops) |
| DEVOPS-LNM-21-101-REL | TODO | Run/apply shard/index migrations (Concelier LNM) in release pipelines; capture artefacts and rollback scripts. | DevOps Guild, Concelier Storage Guild (ops/devops) |
| DEVOPS-LNM-21-102-REL | TODO | Package/publish LNM backfill/rollback bundles for release/offline kit; depends on 21-102 dev outputs. | DevOps Guild, Concelier Storage Guild (ops/devops) |
| DEVOPS-LNM-21-103-REL | TODO | Publish/rotate object-store seeds and offline bootstraps with provenance hashes; depends on 21-103 dev outputs. | DevOps Guild, Concelier Storage Guild (ops/devops) |
| DEVOPS-STORE-AOC-19-005-REL | BLOCKED | Release/offline-kit packaging for Concelier backfill; waiting on dataset hash + dev rehearsal. | DevOps Guild, Concelier Storage Guild (ops/devops) |
| DEVOPS-CONCELIER-CI-24-101 | DONE (2025-11-25) | Provide clean CI runner + warmed NuGet cache + vstest harness for Concelier WebService & Storage; deliver TRX/binlogs and unblock CONCELIER-GRAPH-24-101/28-102 and LNM-21-004..203. | DevOps Guild, Concelier Core Guild (ops/devops) |
| DEVOPS-SCANNER-CI-11-001 | TODO | Supply warmed cache/diag runner for Scanner analyzers (LANG-11-001, JAVA 21-005/008) with binlogs + TRX; unblock restore/test hangs. | DevOps Guild, Scanner EPDR Guild (ops/devops) |
| DEVOPS-SCANNER-JAVA-21-011-REL | TODO | Package/sign Java analyzer plug-in once dev task 21-011 delivers; publish to Offline Kit/CLI release pipelines with provenance. | DevOps Guild, Scanner Release Guild (ops/devops) |
| DEVOPS-SBOM-23-001 | TODO | Publish vetted offline NuGet feed + CI recipe for SbomService; prove with `dotnet test` run and share cache hashes; unblock SBOM-CONSOLE-23-001/002. | DevOps Guild, SBOM Service Guild (ops/devops) |
| FEED-REMEDIATION-1001 | BLOCKED (2025-11-24) | Define remediation scope and runbook for overdue feeds (CCCS/CERTBUND); schedule refresh; depends on PREP-FEEDCONN-ICS-KISA-PLAN. | Concelier Feed Owners (ops/devops) |
| FEEDCONN-ICSCISA-02-012 / FEEDCONN-KISA-02-008 | BLOCKED (2025-11-24) | Publish provenance refresh/connector schedule for ICSCISA/KISA feeds; execute remediation per runbook once owners provide plan. | Concelier Feed Owners (ops/devops) |

## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-25 | Delivered Concelier CI runner harness (`ops/devops/concelier-ci-runner/run-concelier-ci.sh`) with warmed NuGet cache + TRX/binlogs; artefacts land under `ops/devops/artifacts/concelier-ci/<ts>`. | DevOps |
| 2025-11-25 | Local execution of the runner still hits MSBuild worker shutdown on this host (MSB4242); script is ready, but a clean CI agent should be used to produce TRX/binlogs. | DevOps |
| 2025-11-23 | Normalised sprint toward template (sections added); added DEVOPS-CONCELIER-CI-24-101, DEVOPS-SCANNER-CI-11-001, DEVOPS-SBOM-23-001 to absorb CI/restore blockers from module sprints. | Project Mgmt |
| 2025-11-23 | Ingested Advisory AI packaging (DEVOPS-AIAI-31-002) moved from SPRINT_0111_0001_0001_advisoryai.md to keep ops work out of dev sprint. | Project Mgmt |
| 2025-11-24 | Added DEVOPS-SCANNER-JAVA-21-011-REL (moved from SPRINT_0131_0001_0001_scanner_surface.md) to keep DevOps release packaging in ops track. | Project Mgmt |
| 2025-11-24 | Added DEVOPS-SPANSINK-31-003 (Excititor span sink for 31-003 traces) moved from SPRINT_0119_0001_0001_excititor_i per ops-only directive. | Project Mgmt |
| 2025-11-24 | Imported Concelier feed ops items FEED-REMEDIATION-1001 and FEEDCONN-ICSCISA/KISA from Sprint 110; keeping feed remediation in ops track. | Project Mgmt |

## Decisions & Risks
- Mirror bundle automation (DEVOPS-AIRGAP-57-001) and AOC guardrails remain gating risks; several downstream tasks inherit these.
- New CI-runner tasks must produce reproducible binlogs/TRX and cache hashes to keep offline posture intact.

## Next Checkpoints
- 2025-11-25: CI runner provisioning check for Concelier/Scanner/SBOM cache jobs.
- 2025-11-27: Sealed-mode fixture availability review (DEVOPS-AIRGAP-57-002).