Files
git.stella-ops.org/docs/implplan/SPRINT_507_ops_devops_v.md
StellaOps Bot 35c8f9216f Add tests and implement timeline ingestion options with NATS and Redis subscribers
- Introduced `BinaryReachabilityLifterTests` to validate binary lifting functionality.
- Created `PackRunWorkerOptions` for configuring worker paths and execution persistence.
- Added `TimelineIngestionOptions` for configuring NATS and Redis ingestion transports.
- Implemented `NatsTimelineEventSubscriber` for subscribing to NATS events.
- Developed `RedisTimelineEventSubscriber` for reading from Redis Streams.
- Added `TimelineEnvelopeParser` to normalize incoming event envelopes.
- Created unit tests for `TimelineEnvelopeParser` to ensure correct field mapping.
- Implemented `TimelineAuthorizationAuditSink` for logging authorization outcomes.
2025-12-03 09:46:48 +02:00

6.1 KiB

Sprint 507 · Ops DevOps V (Ops & Offline 190.B)

Topic & Scope

  • Ops & Offline phase V: tenant audit/chaos, VEX Lens/Vuln Explorer CI+observability, hardened Docker images, SBOM/attestations, and Surface.Env/Surface.Secrets rollout.
  • Working directory: ops/devops (plus service-specific Docker/ops assets under ops/devops/*).

Dependencies & Concurrency

  • Depends on Sprint 506 (Ops DevOps IV) outputs and TEN-48 harness for tenant tests.
  • Docker hardening (DOCKER-44-001) underpins SBOM/health endpoints tasks.

Documentation Prerequisites

  • docs/modules/devops/architecture.md
  • ops/devops/README.md
  • ops/devops/docker/base-image-guidelines.md

Delivery Tracker

# Task ID Status Key dependency / next step Owners Task Definition
1 DEVOPS-TEN-49-001 DONE (2025-12-03) Depends on DEVOPS-TEN-48-001 DevOps Guild Deploy audit pipeline, usage metrics, JWKS outage chaos tests, tenant load/perf benchmarks.
2 DEVOPS-VEX-30-001 DONE (2025-12-02) None DevOps Guild · VEX Lens Guild CI/load tests/dashboards/alerts for VEX Lens and Issuer Directory.
3 DEVOPS-VULN-29-001 DONE (2025-12-02) None DevOps Guild · Findings Ledger Guild Provision CI jobs for ledger projector; backups, Merkle anchoring, verification.
4 DEVOPS-VULN-29-002 DONE (2025-12-02) Depends on 29-001 DevOps Guild · Vuln Explorer API Guild Load/perf tests (5M findings/tenant), budget enforcement, SLO dashboards, alerts.
5 DEVOPS-VULN-29-003 DONE (2025-12-02) Depends on 29-002 DevOps Guild · Console Guild Instrument analytics pipeline with query-hash metrics and PII guardrails.
6 DOCKER-44-001 DONE (2025-12-03) None DevOps Guild · Service Owners Multi-stage Dockerfiles with non-root user, RO FS, health scripts for core services.
7 DOCKER-44-002 DONE (2025-12-02) Depends on 44-001 DevOps Guild SBOMs + cosign attestations; integrate verification into CI.
8 DOCKER-44-003 DONE (2025-12-02) Depends on 44-002 DevOps Guild Implement health/version/metrics endpoints; ensure capability merge=false for Concelier/Excitior.
9 OPS-ENV-01 DONE (2025-12-02) None DevOps Guild · Scanner Guild Update manifests/config docs to include Surface.Env vars for Scanner and Zastava.
10 OPS-SECRETS-01 DONE (2025-12-02) None DevOps Guild · Security Guild Secret provisioning workflow for Surface.Secrets (Kubernetes, Compose, Offline Kit).
11 OPS-SECRETS-02 DONE (2025-12-02) Depends on 01 DevOps Guild · Offline Kit Guild Embed Surface.Secrets material into offline kit packaging scripts.

Execution Log

Date (UTC) Update Owner
2025-12-03 Completed DEVOPS-TEN-49-001: added tenant recording/alert rules, k6 load harness, chaos runbook/script, and deploy README import steps. DevOps
2025-12-03 Completed DOCKER-44-001: service build matrix + build-all helper, console Dockerfile/healthcheck, APP_BINARY-ready hardened template. DevOps
2025-12-03 Normalised sprint file to standard template; no status changes. Planning
2025-12-02 Completed OPS-ENV-01: added ZASTAVA_* Surface.Env seeds to Helm ConfigMap + Compose env examples and documented rollout in deploy/README. DevOps
2025-12-02 Completed OPS-SECRETS-01/02: provisioning playbook (ops/devops/secrets/surface-secrets-provisioning.md) covering Kubernetes/Compose/Offline Kit; offline kit bundling covers Surface.Secrets payloads. DevOps
2025-12-02 Started DEVOPS-VULN-29-001: added CI/backup/replay/merkle plan and projection hash verifier script. DevOps
2025-12-02 Completed DEVOPS-VULN-29-001: deterministic replay fixture, snapshot/hash, verifier script, CI/ops plan. DevOps
2025-12-02 Added tenant audit assets for DEVOPS-TEN-49-001: dashboard, alerts, chaos script. DevOps
2025-12-02 Completed DEVOPS-VULN-29-002: k6 load/observability assets and thresholds defined. DevOps
2025-12-02 Started DEVOPS-TEN-49-001: drafted audit/usage/chaos plan covering metrics, JWKS fault drill, load benchmarks. DevOps
2025-12-02 Started DEVOPS-VULN-29-002: added k6 load script, Grafana dashboard stub, alert rules. DevOps
2025-12-02 Completed DEVOPS-VEX-30-001: VEX Lens CI/load/obs plan with k6 scenario, dashboards, alerts, offline posture. DevOps
2025-12-02 Completed DOCKER-44-003: documented endpoint contract/snippet and provided CI verification helper; services guidance for health/version/metrics and capabilities merge=false. DevOps
2025-12-02 Added health endpoint contract + ASP.NET 10 snippet to guide DOCKER-44-003 adoption. DevOps
2025-12-02 Started DOCKER-44-003: added health endpoint verification helper and documented CI usage in base-image guidelines. DevOps
2025-12-02 Completed DOCKER-44-002: SBOM + cosign attestation helper added and documented. DevOps
2025-12-02 Extended DOCKER-44-001: hardened multi-stage template with non-root user/RO FS and shared healthcheck helper. DevOps
2025-12-01 Started DOCKER-44-001: hardened base image blueprint and SDK publish guidance documented. DevOps
2025-11-08 Archived completed/historic work to docs/implplan/archived/tasks.md (updated 2025-11-08). Planning

Decisions & Risks

  • Tenant chaos drills require TEN-48 harness orchestration or manual k6 + jwks-chaos.sh; run on isolated agents with sudo/iptables access to avoid collateral outages.
  • Docker hardening template + service matrix are ready; service owners must adopt the template before enabling readOnlyRootFilesystem in Helm/Compose and before SBOM/attest jobs (44-002) are enforced.
  • Surface.Secrets/Surface.Env alignment retained; validate offline kit unpack paths whenever images/paths change.

Next Checkpoints

  • Run TEN-48 harness once available to exercise tenant chaos/load assets end-to-end.
  • Track service owner adoption of hardened Docker template via ops/devops/docker/build-all.sh and verify_health_endpoints.sh.
  • Validate SBOM/attestation verification in CI with production image names/digests after new images are built from the matrix.