No file to print Fine. Shipping containers, but for software. Here’s the serious version you can paste into your docs without the sarcasm. > **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied. --- # Epic 13: Containerized Distribution & Quickstart **Short name:** `Containerized Distribution & Quickstart` **Primary components:** OCI images for all services, Compose Quickstart, Helm chart for production, Air‑gap bundles **Surfaces:** Container registry, `/deploy/*`, `/docs/install/*`, Console onboarding screen **Touches:** Authority (authN/Z), Web Services API, Orchestrator, Task Runner, Policy Engine, Conseiller (Feedser), Excitator (Vexer), Findings Ledger, Export Center, Notifications Studio, Advisory AI Assistant, Object Storage/KMS, Telemetry **AOC ground rule reminder:** Conseiller and Excitator aggregate and link advisories/VEX. They never merge or mutate source records. Containerized deployments must preserve this behavior and expose links to originals. --- ## 1) What it is A complete, reproducible containerized distribution of StellaOps with three delivery modes: 1. **Quickstart (single host)** using Docker Compose: one command to run a full stack suitable for evaluation and local development. Ships with seed data and sane defaults. 2. **Production Helm chart** for Kubernetes: modular, scalable, secure‑by‑default deployment with optional HA and external dependencies. 3. **Air‑gapped bundles**: signed offline packages containing images, seed configs, and installation scripts for disconnected environments. All images are multi‑arch (amd64/arm64), signed, SBOM‑attached, and versioned with consistent tags. A “Download & Install” doc set guides users from zero to a working system in minutes and to a production‑ready posture in hours. --- ## 2) Why (brief) People don’t adopt tools they can’t run quickly or securely. Containers make our deployment reproducible; Quickstart removes friction; Helm unlocks real ops. Air‑gap bundles acknowledge reality in regulated environments. --- ## 3) How it should work (maximum detail) ### 3.1 Image catalog Build and publish OCI images for the following: * `stella-api` (Web Services API) * `stella-console` (Web UI) * `stella-orchestrator` (source/job scheduler) * `stella-task-runner` (executes Task Packs remotely) * `stella-conseiller` (Feedser; advisory aggregator) * `stella-excitator` (Vexer; VEX aggregator) * `stella-policy` (Policy Engine) * `stella-ledger` (Findings Ledger worker; if separated from API) * `stella-export` (Export Center worker; optional if part of API) * `stella-notify` (Notifications Studio worker) * `stella-ai` (Advisory AI Assistant; lightweight service calling configured LLM backends or local models) * Support services (optionally bundled for Quickstart): `postgres`, `redis`, `object-store` (S3‑compatible), `queue` (NATS or RabbitMQ), `otel-collector`. **Image standards** * **Base:** distroless or minimal; non‑root user; read‑only filesystem; writable `/tmp` only if needed. * **Ports:** declare via labels; expose health endpoints `/health/liveness`, `/health/readiness`. * **Env:** explicit, documented, with safe defaults; secrets via env or file mounts only. * **Config:** `STELLA_*` envs or mounted config directory `/etc/stella/`. * **SBOM:** attach SPDX JSON as OCI artifact and include in `/app/sbom.spdx.json` baked at build time. * **Signing:** cosign attestations for image, SBOM, and provenance. * **Labels:** org.opencontainers.image.* (title, version, revision, source, licenses). * **Entrypoint:** PID 1 with reap; graceful shutdown on SIGTERM; configurable termination grace period. * **Logs:** structured JSON by default; stdout/stderr only. **Tagging scheme** * `:vX.Y.Z` (immutable release) * `:vX.Y.Z-rc.N` (release candidate) * `:edge` (latest main) * `:nightly-YYYYMMDD` (optional) * Multi‑arch manifest lists for linux/amd64 and linux/arm64. ### 3.2 Quickstart (Compose) **Goal:** `curl | sh` equivalent that yields a working stack on a laptop/server with defaults and demo data. No internet beyond pulling images, unless configured. **Compose file `deploy/compose/docker-compose.yml`** * Services: * `api`, `console`, `orchestrator`, `task-runner`, `conseiller`, `excitator`, `policy`, `notify`, `export`, `ai` * `postgres`, `redis`, `minio` (S3), `nats` or `rabbitmq`, `otel-collector` * Volumes: * `pgdata`, `minio-data`, `redis-data`, `stella-state` (for local cache, packs registry) * Networks: * `stella-net` bridge * Ports (defaults): * Console `8080`, API `8081`, MinIO `9000`, NATS/RabbitMQ default ports * Env files: * `.env.example` with safe defaults; users copy to `.env`. **Seed data** * Seed admin account and tenant on first run via `stella-api` migration/seed job. * Seed demo SBOMs, advisories, VEX samples, baseline policy, and a task pack. * On first login, Console shows “Welcome” wizard: confirm endpoints, generate API token, run sample scan import, open Vulnerability Explorer. **Security posture** * Default credentials only for Quickstart; randomize secrets on first `up` and store in `.secrets/` file. * All services run as non‑root; bind to localhost by default unless `EXPOSE_PUBLIC=1` set. * TLS optional via `CADDY` or `nginx` sidecar disabled by default. **One‑liner** * `./deploy/compose/quickstart.sh` does: preflight checks, pulls images, writes `.env`, runs `docker compose up -d`, polls readiness, prints URLs and credentials. **Backups & reset** * `./deploy/compose/backup.sh` creates a tarball of volumes and config. * `./deploy/compose/reset.sh` nukes persistent volumes with a big scary prompt unless `--yes`. ### 3.3 Production Helm chart **Chart location:** `deploy/helm/stella/` with subcharts or toggles. **Chart features** * Components enabled via values: `api`, `console`, `orchestrator`, `taskRunner`, `conseiller`, `excitator`, `policy`, `notify`, `export`, `ai`. * External dependencies by default: * PostgreSQL, Redis, S3 bucket, Message queue, OTel endpoint provided via values. * Optional “bundled” mode for lab clusters using StatefulSets. * Security: * PodSecurityContext: runAsNonRoot, readOnlyRootFilesystem, fsGroup when needed. * NetworkPolicy for east‑west traffic; deny‑all then allow specific ports. * Secrets as `Secret` from External Secrets operator or sealed secrets. * HPA per component; PDBs; liveness/readiness probes. * Ingress: * One hostname for Console, one for API; TLS required in production values. * Option to serve Console as static behind CDN while API behind private ingress gateway. * Config: * Values for Authority provider, token TTLs, policy cache TTL, pack registry endpoint, notifications sinks, export locations. * Feature flags per epic enablement. * Migrations: * `stella-migrator` Job runs before rollouts; idempotent migrations. * Optional “break glass” manual job. * Observability: * `/metrics` endpoints scraped by Prometheus; exemplars via OTel; logs structured. * OpenTelemetry auto‑config via env if collector provided. * Upgrades: * Blue/green or rolling; readiness gates based on background indexers catching up. * Chart hooks to block until Conseiller/Excitator catch up to feed watermarks. ### 3.4 Air‑gapped distribution **Bundle format** * `stella-bundle-vX.Y.Z.tar.zst` containing: * All images as OCI layout (multi‑arch), cosign signatures, SBOMs, SLSA provenance. * `load.sh` to import into a local registry. * `compose/` and `helm/` directories with pinned image digests. * `checksums.txt` and `bundle.sig`. * **Process** * Online build job crafts bundle; signatures produced by CI keys. * Offline install: * Verify `bundle.sig` * `./load.sh --to registry.local:5000` * `helm install stella ./helm -f values-airgap.yaml --set image.registry=registry.local:5000` ### 3.5 Configuration matrix Document every config knob in a single table: * Auth: Authority issuer, JWKS, RBAC cache TTL. * Storage: DB URL, pool sizes, migration flags. * Object store: S3 endpoint, buckets, SSE, IAM. * Queue: URL, prefetch, retention. * Policy engine: rule cache TTL, default policy version. * Conseiller/Excitator: polling intervals, feed sources, retry backoff, max in‑flight; **merge disabled** enforced. * Orchestrator/Task Runner: concurrency, sandbox, network egress policy, artifact retention. * Notifications: sinks, templates path, batch windows. * Export Center: formats enabled, rate limits. * AI Assistant: model endpoint, token limits, guardrails, disable by default. ### 3.6 Health, readiness, and upgrades * **Health endpoints:** `GET /health/liveness` returns 200 if process responsive; `GET /health/readiness` checks dependencies with timeout. * **Graceful shutdown:** SIGTERM starts drain; HTTP returns 503; background workers flush; exit on deadline. * **Upgrade choreography:** migrations run, API becomes ready, workers rolling restart, indexes catch up, AOC evaluation warms caches, then flip traffic. * **Version skew policy:** define supported skew between components; chart validates. ### 3.7 Security & compliance * **Image signing & verification:** cosign attestations; optional admission policy to verify signatures by key. * **SBOM provenance:** attach SPDX and provenance attestations; publish via registry referrers. * **Non‑root & least privilege:** capabilities dropped; only NET_BIND for proxies if needed. * **Secrets handling:** mount from files; avoid putting secrets in args; redacted logs by default. * **Audit:** container labels propagate release metadata to all logs and spans. * **AOC enforcement:** images for Conseiller/Excitator hard‑disable merge code paths via env/defaults. ### 3.8 Quickstart UX polish * Console shows “Connected to Quickstart” banner with a button “View install docs” and “Export pack to production.” * One click to generate a `Task Pack` that exports seed data from Quickstart to a production tenant via Export Center. --- ## 4) Architecture ### 4.1 Repos & layout ``` /deploy /compose docker-compose.yml .env.example quickstart.sh backup.sh reset.sh /helm /stella Chart.yaml values.yaml values-prod.yaml values-airgap.yaml templates/*.yaml /docker stella-api.Dockerfile stella-console.Dockerfile stella-orchestrator.Dockerfile stella-task-runner.Dockerfile stella-conseiller.Dockerfile stella-excitator.Dockerfile stella-policy.Dockerfile stella-notify.Dockerfile stella-export.Dockerfile stella-ai.Dockerfile ``` ### 4.2 CI/CD flow * Build multi‑arch with buildx; run unit/integration tests; embed version metadata and SBOM. * Sign images; push to registry; publish Helm chart with pinned digests. * Generate Air‑gap bundle and signatures. * Smoke test Quickstart on fresh VM; e2e tests exercise Console and CLI parity (Epic 12). --- ## 5) APIs and contracts No new external APIs, but every service must expose: * `GET /health/liveness` and `GET /health/readiness`. * `GET /version` returning `{ version, gitCommit, buildDate }`. * `GET /metrics` when enabled. * Config discovery endpoint for Console with trimmed, safe values (no secrets). * Conseiller/Excitator must expose `GET /capabilities` returning `{"merge": false}` to prove merge is disabled. --- ## 6) Documentation changes Create/update: 1. `/docs/install/overview.md` Supported deployment modes, hardware requirements, network ports, quickstart vs production. 2. `/docs/install/compose-quickstart.md` Preconditions, one‑liner, first‑login wizard, seed data, reset/backup, common pitfalls. 3. `/docs/install/helm-prod.md` Prereqs, external dependencies, values reference, TLS/ingress, HPA, PDB, upgrades, rollbacks. 4. `/docs/install/airgap.md` Bundle verification, loading into private registry, running without internet, patching images. 5. `/docs/install/configuration-reference.md` The full configuration matrix with examples. 6. `/docs/security/supply-chain.md` Image signing, SBOMs, provenance, admission controls, non‑root posture. 7. `/docs/operations/health-and-readiness.md` Endpoints, probes, troubleshooting, expected states during upgrades. 8. `/docs/release/image-catalog.md` All image names, tags, architectures, checksums; mapping between chart version and image digests. 9. `/docs/console/onboarding.md` Quickstart banner, links to install docs, exporting data to production. Add at the top of each page: > **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied. --- ## 7) Implementation plan ### New modules/artifacts * Dockerfiles per service under `/deploy/docker/` with common builder stages. * Helm chart under `/deploy/helm/stella`. * Compose quickstart under `/deploy/compose/`. * Air‑gap bundle generator in CI, script `tools/make-airgap-bundle.sh`. * Seed dataset packaged as container image layer or mounted config. ### Changes to services * Add health/version/metrics endpoints where missing. * Ensure all services read config from env/files with defaults suitable for Quickstart. * Conseiller/Excitator: add hard config flag `DISABLE_MERGE=true` defaulted in images and values. * API: seed job and migration runner; serve `/welcome` state for Console wizard. * Console: onboarding wizard and Quickstart banner. * Task Runner: respect offline mode by failing gracefully if egress blocked. ### Packaging & signing * Embed SBOM in all images; publish as OCI referrers. * Cosign sign images and attest provenance; verify in CI. * Publish checksums and signatures on release page. --- ## 8) Engineering tasks **Images** * [ ] Author multi‑stage Dockerfiles with cache‑efficient builds. * [ ] Add non‑root user, drop capabilities, read‑only FS, healthcheck scripts. * [ ] Generate and attach SBOM for each image. * [ ] Implement `/health/*`, `/version`, optional `/metrics`. **Compose** * [ ] Write `docker-compose.yml` with all core services and deps. * [ ] Create `.env.example`, `quickstart.sh`, `backup.sh`, `reset.sh`. * [ ] Seed job container and sample data ingestion on first run. **Helm** * [ ] Scaffold chart; values for each component; pinned digests. * [ ] Ingress, TLS, HPA, PDB, NetworkPolicy, ServiceAccount/RBAC. * [ ] Migration Job and upgrade hooks; readiness gates for indexers. * [ ] Documentation of values with `helm-docs` generator. **Air‑gap** * [ ] Build script to save images to OCI layout; compress, sign, and checksum. * [ ] `load.sh` to import into private registry and rewrite manifests. * [ ] `values-airgap.yaml` with image registry overrides. **Console & API** * [ ] Onboarding wizard, Quickstart banner, links to docs. * [ ] Seed data endpoints guarded behind `QUICKSTART_MODE`. * [ ] Config discovery endpoint for console. **Security** * [ ] Cosign integration; key management; CI verification step. * [ ] Admission policy example in docs to enforce signatures. * [ ] Secret redaction in logs; env var audit. **Observability** * [ ] OTel config sample; `/metrics` endpoints; compose prom scrape. * [ ] Helm values for tracing and metrics. **Validation** * [ ] Fresh VM smoke test for Compose quickstart. * [ ] Kind cluster e2e for Helm path. * [ ] Air‑gap install test in CI with a local registry. **Docs** * [ ] Write all pages listed in §6 with copy‑pasteable commands and screenshots. * [ ] Include a troubleshooting matrix: symptom → probable cause → fix. * [ ] Add “Imposed rule” header line to each page. --- ## 9) Feature changes required * **Console:** Onboarding wizard, Quickstart banner, and deep links to install docs; “Copy CLI” buttons should prefer the `stella` container image in quickstart if local binary missing. * **API:** Seed job and health endpoints; version reporting; feature flag `QUICKSTART_MODE`. * **Registry/Release tooling:** Publish image catalog and checksums; maintain compatibility matrix per chart version. * **Task Runner:** Offline mode awareness and explicit error when attempting egress in air‑gap. * **Conseiller/Excitator:** enforce non‑merge at runtime and show capability endpoint. --- ## 10) Acceptance criteria * Quickstart: from clean host to working Console in under 5 minutes on a typical laptop; seed data visible; AOC rules active. * Helm: install succeeds with external dependencies; roll forward and roll back with zero data loss; probes green. * Air‑gap: bundle verifies, loads to a private registry, and installs without external network. * All images: signed, SBOM‑attached, non‑root, read‑only FS, health endpoints exposed. * Docs: a new user can complete Quickstart without assistance; a platform team can deploy the chart with only values editing. * Conseiller/Excitator: capability endpoint confirms `merge=false`; tests prove aggregation‑only behavior. --- ## 11) Risks & mitigations * **Config sprawl.** Centralize in `/docs/install/configuration-reference.md` and ship sane defaults. * **Drift between Compose and Helm.** Pin digests; generate manifests from a common values source where possible; CI diff. * **Resource contention in Quickstart.** Limit concurrency; ship low default worker counts; document overrides. * **Air‑gap surprises.** Remove implicit egress; provide offline doc copies in bundle; deterministic artifact paths. * **Security regressions.** Enforce non‑root/read‑only in CI; signature verification gates release. --- ## 12) Philosophy * **First run matters.** Quickstart must be boring, predictable, and immediately useful. * **Prod isn’t a flag.** Helm defaults are safe; “convenience” belongs in Quickstart, not production. * **Prove your supply chain.** Signed images, SBOMs, and provenance are table stakes, not an upsell. > **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.