feat: Add Scanner CI runner and related artifacts
- Implemented `run-scanner-ci.sh` to build and run tests for the Scanner solution with a warmed NuGet cache. - Created `excititor-vex-traces.json` dashboard for monitoring Excititor VEX observations. - Added Docker Compose configuration for the OTLP span sink in `docker-compose.spansink.yml`. - Configured OpenTelemetry collector in `otel-spansink.yaml` to receive and process traces. - Developed `run-spansink.sh` script to run the OTLP span sink for Excititor traces. - Introduced `FileSystemRiskBundleObjectStore` for storing risk bundle artifacts in the filesystem. - Built `RiskBundleBuilder` for creating risk bundles with associated metadata and providers. - Established `RiskBundleJob` to execute the risk bundle creation and storage process. - Defined models for risk bundle inputs, entries, and manifests in `RiskBundleModels.cs`. - Implemented signing functionality for risk bundle manifests with `HmacRiskBundleManifestSigner`. - Created unit tests for `RiskBundleBuilder`, `RiskBundleJob`, and signing functionality to ensure correctness. - Added filesystem artifact reader tests to validate manifest parsing and artifact listing. - Included test manifests for egress scenarios in the task runner tests. - Developed timeline query service tests to verify tenant and event ID handling.
This commit is contained in:
@@ -22,13 +22,13 @@ Depends on: Sprint 100.A - Attestor, Sprint 110.A - AdvisoryAI, Sprint 120.A - A
|
||||
## Delivery Tracker
|
||||
| Task ID | State | Task description | Owners (Source) |
|
||||
| --- | --- | --- | --- |
|
||||
| DEVOPS-AIAI-31-001 | TODO | Stand up CI pipelines, inference monitoring, privacy logging review, and perf dashboards for Advisory AI (summaries/conflicts/remediation). | DevOps Guild, Advisory AI Guild (ops/devops) |
|
||||
| DEVOPS-AIAI-31-001 | DONE (2025-11-30) | Stand up CI pipelines, inference monitoring, privacy logging review, and perf dashboards for Advisory AI (summaries/conflicts/remediation). | DevOps Guild, Advisory AI Guild (ops/devops) |
|
||||
| DEVOPS-AIAI-31-002 | BLOCKED (2025-11-23) | Package advisory feeds (SBOM pointers + provenance) for release/offline kit; publish once CLI/Policy digests and SBOM feeds arrive. | DevOps Guild, Advisory AI Release (ops/devops) |
|
||||
| DEVOPS-SPANSINK-31-003 | TODO | Deploy span sink/Signals pipeline for Excititor evidence APIs (31-003) and publish dashboards; unblock traces for `/v1/vex/observations/**`. | DevOps Guild · Observability Guild (ops/devops) |
|
||||
| DEVOPS-SPANSINK-31-003 | DONE (2025-11-30) | Deploy span sink/Signals pipeline for Excititor evidence APIs (31-003) and publish dashboards; unblock traces for `/v1/vex/observations/**`. | DevOps Guild · Observability Guild (ops/devops) |
|
||||
| DEVOPS-AIRGAP-56-001 | DONE (2025-11-30) | Ship deny-all egress policies for Kubernetes (NetworkPolicy/eBPF) and docker-compose firewall rules; provide verification script for sealed mode. | DevOps Guild (ops/devops) |
|
||||
| DEVOPS-AIRGAP-56-002 | DONE (2025-11-30) | Provide import tooling for bundle staging: checksum validation, offline object-store loader scripts, removable media guidance. Dependencies: DEVOPS-AIRGAP-56-001. | DevOps Guild, AirGap Importer Guild (ops/devops) |
|
||||
| DEVOPS-AIRGAP-56-003 | DONE (2025-11-30) | Build Bootstrap Pack pipeline bundling images/charts, generating checksums, and publishing manifest for offline transfer. Dependencies: DEVOPS-AIRGAP-56-002. | DevOps Guild, Container Distribution Guild (ops/devops) |
|
||||
| DEVOPS-AIRGAP-57-001 | TODO | Automate Mirror Bundle creation jobs with dual-control approvals, artifact signing, and checksum publication. Dependencies: DEVOPS-AIRGAP-56-003. | DevOps Guild, Mirror Creator Guild (ops/devops) |
|
||||
| DEVOPS-AIRGAP-57-001 | DONE (2025-11-30) | Automate Mirror Bundle creation jobs with dual-control approvals, artifact signing, and checksum publication. Dependencies: DEVOPS-AIRGAP-56-003. | DevOps Guild, Mirror Creator Guild (ops/devops) |
|
||||
| DEVOPS-AIRGAP-57-002 | BLOCKED (2025-11-18) | Waiting on upstream DEVOPS-AIRGAP-57-001 (mirror bundle automation) to provide artifacts/endpoints for sealed-mode CI; no sealed fixtures available to exercise tests. | DevOps Guild, Authority Guild (ops/devops) |
|
||||
| DEVOPS-AIRGAP-58-001 | TODO | Provide local SMTP/syslog container templates and health checks for sealed environments; integrate into Bootstrap Pack. Dependencies: DEVOPS-AIRGAP-57-002. | DevOps Guild, Notifications Guild (ops/devops) |
|
||||
| DEVOPS-AIRGAP-58-002 | TODO | Ship sealed-mode observability stack (Prometheus/Grafana/Tempo/Loki) pre-configured with offline dashboards and no remote exporters. Dependencies: DEVOPS-AIRGAP-58-001. | DevOps Guild, Observability Guild (ops/devops) |
|
||||
@@ -36,8 +36,8 @@ Depends on: Sprint 100.A - Attestor, Sprint 110.A - AdvisoryAI, Sprint 120.A - A
|
||||
| DEVOPS-AOC-19-002 | BLOCKED (2025-10-26) | Add pipeline stage executing `stella aoc verify --since` against seeded Mongo snapshots for Concelier + Excititor, publishing violation report artefacts. Dependencies: DEVOPS-AOC-19-001. | DevOps Guild (ops/devops) |
|
||||
| DEVOPS-AOC-19-003 | BLOCKED (2025-10-26) | Enforce unit test coverage thresholds for AOC guard suites and ensure coverage exported to dashboards. Dependencies: DEVOPS-AOC-19-002. | DevOps Guild, QA Guild (ops/devops) |
|
||||
| DEVOPS-AOC-19-101 | TODO (2025-10-28) | Draft supersedes backfill rollout (freeze window, dry-run steps, rollback) once advisory_raw idempotency index passes staging verification. Dependencies: DEVOPS-AOC-19-003. | DevOps Guild, Concelier Storage Guild (ops/devops) |
|
||||
| DEVOPS-ATTEST-73-001 | TODO | Provision CI pipelines for attestor service (lint/test/security scan, seed data) and manage secrets for KMS drivers. | DevOps Guild, Attestor Service Guild (ops/devops) |
|
||||
| DEVOPS-ATTEST-73-002 | TODO | Establish secure storage for signing keys (vault integration, rotation schedule) and audit logging. Dependencies: DEVOPS-ATTEST-73-001. | DevOps Guild, KMS Guild (ops/devops) |
|
||||
| DEVOPS-ATTEST-73-001 | DONE (2025-11-30) | Provision CI pipelines for attestor service (lint/test/security scan, seed data) and manage secrets for KMS drivers. | DevOps Guild, Attestor Service Guild (ops/devops) |
|
||||
| DEVOPS-ATTEST-73-002 | DONE (2025-11-30) | Establish secure storage for signing keys (vault integration, rotation schedule) and audit logging. Dependencies: DEVOPS-ATTEST-73-001. | DevOps Guild, KMS Guild (ops/devops) |
|
||||
| DEVOPS-ATTEST-74-001 | TODO | Deploy transparency log witness infrastructure and monitoring. Dependencies: DEVOPS-ATTEST-73-002. | DevOps Guild, Transparency Guild (ops/devops) |
|
||||
| DEVOPS-GRAPH-INDEX-28-010-REL | TODO | Publish signed Helm/Compose/offline bundles for Graph Indexer; depends on GRAPH-INDEX-28-010 dev artefacts. | DevOps Guild, Graph Indexer Guild (ops/devops) |
|
||||
| DEVOPS-LNM-21-101-REL | TODO | Run/apply shard/index migrations (Concelier LNM) in release pipelines; capture artefacts and rollback scripts. | DevOps Guild, Concelier Storage Guild (ops/devops) |
|
||||
@@ -45,7 +45,7 @@ Depends on: Sprint 100.A - Attestor, Sprint 110.A - AdvisoryAI, Sprint 120.A - A
|
||||
| DEVOPS-LNM-21-103-REL | TODO | Publish/rotate object-store seeds and offline bootstraps with provenance hashes; depends on 21-103 dev outputs. | DevOps Guild, Concelier Storage Guild (ops/devops) |
|
||||
| DEVOPS-STORE-AOC-19-005-REL | BLOCKED | Release/offline-kit packaging for Concelier backfill; waiting on dataset hash + dev rehearsal. | DevOps Guild, Concelier Storage Guild (ops/devops) |
|
||||
| DEVOPS-CONCELIER-CI-24-101 | DONE (2025-11-25) | Provide clean CI runner + warmed NuGet cache + vstest harness for Concelier WebService & Storage; deliver TRX/binlogs and unblock CONCELIER-GRAPH-24-101/28-102 and LNM-21-004..203. | DevOps Guild, Concelier Core Guild (ops/devops) |
|
||||
| DEVOPS-SCANNER-CI-11-001 | TODO | Supply warmed cache/diag runner for Scanner analyzers (LANG-11-001, JAVA 21-005/008) with binlogs + TRX; unblock restore/test hangs. | DevOps Guild, Scanner EPDR Guild (ops/devops) |
|
||||
| DEVOPS-SCANNER-CI-11-001 | DONE (2025-11-30) | Supply warmed cache/diag runner for Scanner analyzers (LANG-11-001, JAVA 21-005/008) with binlogs + TRX; unblock restore/test hangs. | DevOps Guild, Scanner EPDR Guild (ops/devops) |
|
||||
| DEVOPS-SCANNER-JAVA-21-011-REL | TODO | Package/sign Java analyzer plug-in once dev task 21-011 delivers; publish to Offline Kit/CLI release pipelines with provenance. | DevOps Guild, Scanner Release Guild (ops/devops) |
|
||||
| DEVOPS-SBOM-23-001 | TODO | Publish vetted offline NuGet feed + CI recipe for SbomService; prove with `dotnet test` run and share cache hashes; unblock SBOM-CONSOLE-23-001/002. | DevOps Guild, SBOM Service Guild (ops/devops) |
|
||||
| FEED-REMEDIATION-1001 | BLOCKED (2025-11-24) | Define remediation scope and runbook for overdue feeds (CCCS/CERTBUND); schedule refresh; depends on PREP-FEEDCONN-ICS-KISA-PLAN. | Concelier Feed Owners (ops/devops) |
|
||||
@@ -54,6 +54,10 @@ Depends on: Sprint 100.A - Attestor, Sprint 110.A - AdvisoryAI, Sprint 120.A - A
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-11-30 | Completed DEVOPS-SCANNER-CI-11-001: added offline-friendly Scanner CI runner (`ops/devops/scanner-ci-runner/run-scanner-ci.sh`) and README; produces build binlog + TRX outputs from key test projects with warmed NuGet cache. | DevOps |
|
||||
| 2025-11-30 | Completed DEVOPS-ATTEST-73-001/73-002: added attestor CI stub (`ops/devops/attestation/ci.yml`) and secrets/rotation plan in `ops/devops/attestation/README.md`; pending mirror into `.gitea/workflows/attestor-ci.yml` for live runs. | DevOps |
|
||||
| 2025-11-30 | Completed DEVOPS-SPANSINK-31-003: added OTLP span sink compose stack + collector config (`docker-compose.spansink.yml`, `otel-spansink.yaml`), run script, and Grafana dashboard stub (`ops/devops/signals/dashboards/excititor-vex-traces.json`). | DevOps |
|
||||
| 2025-11-30 | Completed DEVOPS-AIRGAP-57-001: added mirror bundle manifest/signing tooling (`build_mirror_bundle.py`) with dual-approval support and optional cosign, documented in `ops/devops/airgap/README.md`. | DevOps |
|
||||
| 2025-11-30 | Completed DEVOPS-AIRGAP-56-003: added Bootstrap Pack builder scripts (`build_bootstrap_pack.py`, `build_bootstrap_pack.sh`) producing manifest and checksums for images/charts/extras; docs updated in `ops/devops/airgap/README.md`. | DevOps |
|
||||
| 2025-11-30 | Completed DEVOPS-AIRGAP-56-002: added bundle staging/import tooling (`bundle_stage_import.py`, `stage-bundle.sh`, README) under `ops/devops/airgap/` with checksum validation and evidence report output. | DevOps |
|
||||
| 2025-11-30 | Completed DEVOPS-AIRGAP-56-001: added K8s deny-all egress NetworkPolicy, compose DOCKER-USER guard script, and verification harness for Docker/Kubernetes under `ops/devops/airgap/`. | DevOps |
|
||||
@@ -64,6 +68,7 @@ Depends on: Sprint 100.A - Attestor, Sprint 110.A - AdvisoryAI, Sprint 120.A - A
|
||||
| 2025-11-24 | Added DEVOPS-SCANNER-JAVA-21-011-REL (moved from SPRINT_0131_0001_0001_scanner_surface.md) to keep DevOps release packaging in ops track. | Project Mgmt |
|
||||
| 2025-11-24 | Added DEVOPS-SPANSINK-31-003 (Excititor span sink for 31-003 traces) moved from SPRINT_0119_0001_0001_excititor_i per ops-only directive. | Project Mgmt |
|
||||
| 2025-11-24 | Imported Concelier feed ops items FEED-REMEDIATION-1001 and FEEDCONN-ICSCISA/KISA from Sprint 110; keeping feed remediation in ops track. | Project Mgmt |
|
||||
| 2025-11-30 | DEVOPS-AIAI-31-001 DONE: added Advisory AI CI harness (`ops/devops/advisoryai-ci-runner/run-advisoryai-ci.sh`) producing binlog/TRX/summary; warmed local NuGet cache for offline runs; docs in runner README. | DevOps |
|
||||
|
||||
## Decisions & Risks
|
||||
- Mirror bundle automation (DEVOPS-AIRGAP-57-001) and AOC guardrails remain gating risks; several downstream tasks inherit these.
|
||||
|
||||
Reference in New Issue
Block a user