devops folders consolidate
This commit is contained in:
master
@@ -1,47 +0,0 @@
|
||||
# syntax=docker/dockerfile:1.7-labs
|
||||
|
||||
# StellaOps AdvisoryAI – multi-role container build
|
||||
# Build arg PROJECT selects WebService or Worker; defaults to WebService.
|
||||
# Example builds:
|
||||
# docker build -f ops/advisory-ai/Dockerfile -t stellaops-advisoryai-web \
|
||||
# --build-arg PROJECT=src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj \
|
||||
# --build-arg APP_DLL=StellaOps.AdvisoryAI.WebService.dll .
|
||||
# docker build -f ops/advisory-ai/Dockerfile -t stellaops-advisoryai-worker \
|
||||
# --build-arg PROJECT=src/AdvisoryAI/StellaOps.AdvisoryAI.Worker/StellaOps.AdvisoryAI.Worker.csproj \
|
||||
# --build-arg APP_DLL=StellaOps.AdvisoryAI.Worker.dll .
|
||||
|
||||
ARG SDK_IMAGE=mcr.microsoft.com/dotnet/nightly/sdk:10.0
|
||||
ARG RUNTIME_IMAGE=gcr.io/distroless/dotnet/aspnet:latest
|
||||
ARG PROJECT=src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj
|
||||
ARG APP_DLL=StellaOps.AdvisoryAI.WebService.dll
|
||||
|
||||
FROM ${SDK_IMAGE} AS build
|
||||
WORKDIR /src
|
||||
|
||||
COPY . .
|
||||
|
||||
# Restore only AdvisoryAI graph to keep build smaller.
|
||||
RUN dotnet restore ${PROJECT}
|
||||
|
||||
RUN dotnet publish ${PROJECT} \
|
||||
-c Release \
|
||||
-o /app/publish \
|
||||
/p:UseAppHost=false
|
||||
|
||||
FROM ${RUNTIME_IMAGE} AS runtime
|
||||
WORKDIR /app
|
||||
|
||||
ENV ASPNETCORE_URLS=http://0.0.0.0:8080 \
|
||||
DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true \
|
||||
ADVISORYAI__STORAGE__PLANCACHEDIRECTORY=/app/data/plans \
|
||||
ADVISORYAI__STORAGE__OUTPUTDIRECTORY=/app/data/outputs \
|
||||
ADVISORYAI__QUEUE__DIRECTORYPATH=/app/data/queue \
|
||||
ADVISORYAI__INFERENCE__MODE=Local
|
||||
|
||||
COPY --from=build /app/publish ./
|
||||
|
||||
# Writable mount for queue/cache/output. Guardrail/guardrails can also be mounted under /app/etc.
|
||||
VOLUME ["/app/data", "/app/etc"]
|
||||
|
||||
EXPOSE 8080
|
||||
ENTRYPOINT ["dotnet", "${APP_DLL}"]
|
||||
@@ -1,47 +0,0 @@
|
||||
# AdvisoryAI packaging (AIAI-31-008)
|
||||
|
||||
Artifacts delivered for on-prem / air-gapped deployment:
|
||||
|
||||
- `ops/advisory-ai/Dockerfile` builds WebService and Worker images (multi-role via `PROJECT`/`APP_DLL` args).
|
||||
- `ops/advisory-ai/docker-compose.advisoryai.yaml` runs WebService + Worker with shared data volume; ships remote inference toggle envs.
|
||||
- `ops/advisory-ai/helm/` provides a minimal chart (web + worker) with storage mounts, optional PVC, and remote inference settings.
|
||||
|
||||
## Build images
|
||||
```bash
|
||||
# WebService
|
||||
docker build -f ops/advisory-ai/Dockerfile \
|
||||
-t stellaops-advisoryai-web:dev \
|
||||
--build-arg PROJECT=src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj \
|
||||
--build-arg APP_DLL=StellaOps.AdvisoryAI.WebService.dll .
|
||||
|
||||
# Worker
|
||||
docker build -f ops/advisory-ai/Dockerfile \
|
||||
-t stellaops-advisoryai-worker:dev \
|
||||
--build-arg PROJECT=src/AdvisoryAI/StellaOps.AdvisoryAI.Worker/StellaOps.AdvisoryAI.Worker.csproj \
|
||||
--build-arg APP_DLL=StellaOps.AdvisoryAI.Worker.dll .
|
||||
```
|
||||
|
||||
## Local/offline compose
|
||||
```bash
|
||||
cd ops/advisory-ai
|
||||
docker compose -f docker-compose.advisoryai.yaml up -d --build
|
||||
```
|
||||
- Set `ADVISORYAI__INFERENCE__MODE=Remote` plus `ADVISORYAI__INFERENCE__REMOTE__BASEADDRESS`/`APIKEY` to offload inference.
|
||||
- Default mode is Local (offline-friendly). Queue/cache/output live under `/app/data` (binds to `advisoryai-data` volume).
|
||||
|
||||
## Helm (cluster)
|
||||
```bash
|
||||
helm upgrade --install advisoryai ops/advisory-ai/helm \
|
||||
--set image.repository=stellaops-advisoryai-web \
|
||||
--set image.tag=dev \
|
||||
--set inference.mode=Local
|
||||
```
|
||||
- Enable remote inference: `--set inference.mode=Remote --set inference.remote.baseAddress=https://inference.your.domain --set inference.remote.apiKey=<token>`.
|
||||
- Enable persistence: `--set storage.persistence.enabled=true --set storage.persistence.size=10Gi` or `--set storage.persistence.existingClaim=<pvc>`.
|
||||
- Worker replicas: `--set worker.replicas=2` (or `--set worker.enabled=false` to run WebService only).
|
||||
|
||||
## Operational notes
|
||||
- Data paths (`/app/data/plans`, `/app/data/queue`, `/app/data/outputs`) are configurable via env and pre-created at startup.
|
||||
- Guardrail phrases or policy knobs can be mounted under `/app/etc`; point `ADVISORYAI__GUARDRAILS__PHRASESLIST` to the mounted file.
|
||||
- Observability follows standard ASP.NET JSON logs; add OTEL exporters via `OTEL_EXPORTER_OTLP_ENDPOINT` env when allowed. Keep disabled in sealed/offline deployments.
|
||||
- For air-gapped clusters, publish built images to your registry and reference via `--set image.repository=<registry>/stellaops/advisoryai-web`.
|
||||
@@ -1,55 +0,0 @@
|
||||
version: "3.9"
|
||||
|
||||
# Local/offline deployment for AdvisoryAI WebService + Worker.
|
||||
services:
|
||||
advisoryai-web:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: ops/advisory-ai/Dockerfile
|
||||
args:
|
||||
PROJECT: src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj
|
||||
APP_DLL: StellaOps.AdvisoryAI.WebService.dll
|
||||
image: stellaops-advisoryai-web:dev
|
||||
depends_on:
|
||||
- advisoryai-worker
|
||||
environment:
|
||||
ASPNETCORE_URLS: "http://0.0.0.0:8080"
|
||||
ADVISORYAI__QUEUE__DIRECTORYPATH: "/app/data/queue"
|
||||
ADVISORYAI__STORAGE__PLANCACHEDIRECTORY: "/app/data/plans"
|
||||
ADVISORYAI__STORAGE__OUTPUTDIRECTORY: "/app/data/outputs"
|
||||
ADVISORYAI__INFERENCE__MODE: "Local" # switch to Remote to call an external inference host
|
||||
# ADVISORYAI__INFERENCE__REMOTE__BASEADDRESS: "https://inference.example.com"
|
||||
# ADVISORYAI__INFERENCE__REMOTE__ENDPOINT: "/v1/inference"
|
||||
# ADVISORYAI__INFERENCE__REMOTE__APIKEY: "set-me"
|
||||
# ADVISORYAI__INFERENCE__REMOTE__TIMEOUT: "00:00:30"
|
||||
# Example SBOM context feed; optional.
|
||||
# ADVISORYAI__SBOMBASEADDRESS: "https://sbom.local/v1/sbom/context"
|
||||
# ADVISORYAI__SBOMTENANT: "tenant-a"
|
||||
# ADVISORYAI__GUARDRAILS__PHRASESLIST: "/app/etc/guardrails/phrases.txt"
|
||||
volumes:
|
||||
- advisoryai-data:/app/data
|
||||
- ./etc:/app/etc:ro
|
||||
ports:
|
||||
- "7071:8080"
|
||||
restart: unless-stopped
|
||||
|
||||
advisoryai-worker:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: ops/advisory-ai/Dockerfile
|
||||
args:
|
||||
PROJECT: src/AdvisoryAI/StellaOps.AdvisoryAI.Worker/StellaOps.AdvisoryAI.Worker.csproj
|
||||
APP_DLL: StellaOps.AdvisoryAI.Worker.dll
|
||||
image: stellaops-advisoryai-worker:dev
|
||||
environment:
|
||||
ADVISORYAI__QUEUE__DIRECTORYPATH: "/app/data/queue"
|
||||
ADVISORYAI__STORAGE__PLANCACHEDIRECTORY: "/app/data/plans"
|
||||
ADVISORYAI__STORAGE__OUTPUTDIRECTORY: "/app/data/outputs"
|
||||
ADVISORYAI__INFERENCE__MODE: "Local"
|
||||
volumes:
|
||||
- advisoryai-data:/app/data
|
||||
- ./etc:/app/etc:ro
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
advisoryai-data:
|
||||
@@ -1,6 +0,0 @@
|
||||
apiVersion: v2
|
||||
name: stellaops-advisoryai
|
||||
version: 0.1.0
|
||||
appVersion: "0.1.0"
|
||||
description: AdvisoryAI WebService + Worker packaging for on-prem/air-gapped installs.
|
||||
type: application
|
||||
@@ -1,12 +0,0 @@
|
||||
{{- define "stellaops-advisoryai.name" -}}
|
||||
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "stellaops-advisoryai.fullname" -}}
|
||||
{{- if .Values.fullnameOverride -}}
|
||||
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
|
||||
{{- else -}}
|
||||
{{- $name := default .Chart.Name .Values.nameOverride -}}
|
||||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
@@ -1,71 +0,0 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {{ include "stellaops-advisoryai.fullname" . }}
|
||||
labels:
|
||||
app.kubernetes.io/name: {{ include "stellaops-advisoryai.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
app.kubernetes.io/version: {{ .Chart.AppVersion }}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: {{ include "stellaops-advisoryai.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: {{ include "stellaops-advisoryai.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
spec:
|
||||
containers:
|
||||
- name: web
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
env:
|
||||
- name: ASPNETCORE_URLS
|
||||
value: "http://0.0.0.0:{{ .Values.service.port }}"
|
||||
- name: ADVISORYAI__INFERENCE__MODE
|
||||
value: "{{ .Values.inference.mode }}"
|
||||
- name: ADVISORYAI__INFERENCE__REMOTE__BASEADDRESS
|
||||
value: "{{ .Values.inference.remote.baseAddress }}"
|
||||
- name: ADVISORYAI__INFERENCE__REMOTE__ENDPOINT
|
||||
value: "{{ .Values.inference.remote.endpoint }}"
|
||||
- name: ADVISORYAI__INFERENCE__REMOTE__APIKEY
|
||||
value: "{{ .Values.inference.remote.apiKey }}"
|
||||
- name: ADVISORYAI__INFERENCE__REMOTE__TIMEOUT
|
||||
value: "{{ printf "00:00:%d" .Values.inference.remote.timeoutSeconds }}"
|
||||
- name: ADVISORYAI__STORAGE__PLANCACHEDIRECTORY
|
||||
value: {{ .Values.storage.planCachePath | quote }}
|
||||
- name: ADVISORYAI__STORAGE__OUTPUTDIRECTORY
|
||||
value: {{ .Values.storage.outputPath | quote }}
|
||||
- name: ADVISORYAI__QUEUE__DIRECTORYPATH
|
||||
value: {{ .Values.storage.queuePath | quote }}
|
||||
envFrom:
|
||||
{{- if .Values.extraEnvFrom }}
|
||||
- secretRef:
|
||||
name: {{ .Values.extraEnvFrom | first }}
|
||||
{{- end }}
|
||||
{{- if .Values.extraEnv }}
|
||||
{{- range .Values.extraEnv }}
|
||||
- name: {{ .name }}
|
||||
value: {{ .value | quote }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
ports:
|
||||
- containerPort: {{ .Values.service.port }}
|
||||
volumeMounts:
|
||||
- name: advisoryai-data
|
||||
mountPath: /app/data
|
||||
resources: {{- toYaml .Values.resources | nindent 12 }}
|
||||
volumes:
|
||||
- name: advisoryai-data
|
||||
{{- if .Values.storage.persistence.enabled }}
|
||||
persistentVolumeClaim:
|
||||
claimName: {{ .Values.storage.persistence.existingClaim | default (printf "%s-data" (include "stellaops-advisoryai.fullname" .)) }}
|
||||
{{- else }}
|
||||
emptyDir: {}
|
||||
{{- end }}
|
||||
nodeSelector: {{- toYaml .Values.nodeSelector | nindent 8 }}
|
||||
tolerations: {{- toYaml .Values.tolerations | nindent 8 }}
|
||||
affinity: {{- toYaml .Values.affinity | nindent 8 }}
|
||||
@@ -1,15 +0,0 @@
|
||||
{{- if and .Values.storage.persistence.enabled (not .Values.storage.persistence.existingClaim) }}
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: {{ printf "%s-data" (include "stellaops-advisoryai.fullname" .) }}
|
||||
labels:
|
||||
app.kubernetes.io/name: {{ include "stellaops-advisoryai.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.storage.persistence.size }}
|
||||
{{- end }}
|
||||
@@ -1,17 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "stellaops-advisoryai.fullname" . }}
|
||||
labels:
|
||||
app.kubernetes.io/name: {{ include "stellaops-advisoryai.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
spec:
|
||||
type: {{ .Values.service.type }}
|
||||
selector:
|
||||
app.kubernetes.io/name: {{ include "stellaops-advisoryai.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
ports:
|
||||
- name: http
|
||||
port: {{ .Values.service.port }}
|
||||
targetPort: {{ .Values.service.port }}
|
||||
protocol: TCP
|
||||
@@ -1,66 +0,0 @@
|
||||
{{- if .Values.worker.enabled }}
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {{ include "stellaops-advisoryai.fullname" . }}-worker
|
||||
labels:
|
||||
app.kubernetes.io/name: {{ include "stellaops-advisoryai.name" . }}-worker
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
spec:
|
||||
replicas: {{ .Values.worker.replicas }}
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: {{ include "stellaops-advisoryai.name" . }}-worker
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: {{ include "stellaops-advisoryai.name" . }}-worker
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
spec:
|
||||
containers:
|
||||
- name: worker
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
command: ["dotnet", "StellaOps.AdvisoryAI.Worker.dll"]
|
||||
env:
|
||||
- name: ADVISORYAI__INFERENCE__MODE
|
||||
value: "{{ .Values.inference.mode }}"
|
||||
- name: ADVISORYAI__INFERENCE__REMOTE__BASEADDRESS
|
||||
value: "{{ .Values.inference.remote.baseAddress }}"
|
||||
- name: ADVISORYAI__INFERENCE__REMOTE__ENDPOINT
|
||||
value: "{{ .Values.inference.remote.endpoint }}"
|
||||
- name: ADVISORYAI__INFERENCE__REMOTE__APIKEY
|
||||
value: "{{ .Values.inference.remote.apiKey }}"
|
||||
- name: ADVISORYAI__INFERENCE__REMOTE__TIMEOUT
|
||||
value: "{{ printf "00:00:%d" .Values.inference.remote.timeoutSeconds }}"
|
||||
- name: ADVISORYAI__STORAGE__PLANCACHEDIRECTORY
|
||||
value: {{ .Values.storage.planCachePath | quote }}
|
||||
- name: ADVISORYAI__STORAGE__OUTPUTDIRECTORY
|
||||
value: {{ .Values.storage.outputPath | quote }}
|
||||
- name: ADVISORYAI__QUEUE__DIRECTORYPATH
|
||||
value: {{ .Values.storage.queuePath | quote }}
|
||||
envFrom:
|
||||
{{- if .Values.extraEnvFrom }}
|
||||
- secretRef:
|
||||
name: {{ .Values.extraEnvFrom | first }}
|
||||
{{- end }}
|
||||
{{- if .Values.extraEnv }}
|
||||
{{- range .Values.extraEnv }}
|
||||
- name: {{ .name }}
|
||||
value: {{ .value | quote }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
volumeMounts:
|
||||
- name: advisoryai-data
|
||||
mountPath: /app/data
|
||||
resources: {{- toYaml .Values.worker.resources | nindent 12 }}
|
||||
volumes:
|
||||
- name: advisoryai-data
|
||||
{{- if .Values.storage.persistence.enabled }}
|
||||
persistentVolumeClaim:
|
||||
claimName: {{ .Values.storage.persistence.existingClaim | default (printf "%s-data" (include "stellaops-advisoryai.fullname" .)) }}
|
||||
{{- else }}
|
||||
emptyDir: {}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -1,38 +0,0 @@
|
||||
image:
|
||||
repository: stellaops/advisoryai
|
||||
tag: dev
|
||||
pullPolicy: IfNotPresent
|
||||
|
||||
service:
|
||||
port: 8080
|
||||
type: ClusterIP
|
||||
|
||||
inference:
|
||||
mode: Local # or Remote
|
||||
remote:
|
||||
baseAddress: ""
|
||||
endpoint: "/v1/inference"
|
||||
apiKey: ""
|
||||
timeoutSeconds: 30
|
||||
|
||||
storage:
|
||||
planCachePath: /app/data/plans
|
||||
outputPath: /app/data/outputs
|
||||
queuePath: /app/data/queue
|
||||
persistence:
|
||||
enabled: false
|
||||
existingClaim: ""
|
||||
size: 5Gi
|
||||
|
||||
resources: {}
|
||||
nodeSelector: {}
|
||||
tolerations: []
|
||||
affinity: {}
|
||||
|
||||
worker:
|
||||
enabled: true
|
||||
replicas: 1
|
||||
resources: {}
|
||||
|
||||
extraEnv: [] # list of { name: ..., value: ... }
|
||||
extraEnvFrom: []
|
||||
@@ -1,24 +0,0 @@
|
||||
# Advisory AI CI Runner Harness (DEVOPS-AIAI-31-001)
|
||||
|
||||
Purpose: deterministic, offline-friendly CI harness for Advisory AI service/worker. Produces warmed-cache restore, build binlog, and TRX outputs for the core test suite so downstream sprints can validate without bespoke pipelines.
|
||||
|
||||
Usage
|
||||
- From repo root run: `ops/devops/advisoryai-ci-runner/run-advisoryai-ci.sh`
|
||||
- Outputs land in `ops/devops/artifacts/advisoryai-ci/<UTC timestamp>/`:
|
||||
- `build.binlog` (solution build)
|
||||
- `tests/advisoryai.trx` (VSTest results)
|
||||
- `summary.json` (paths + hashes + durations)
|
||||
|
||||
Environment
|
||||
- Defaults: `DOTNET_CLI_TELEMETRY_OPTOUT=1`, `DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1`, `NUGET_PACKAGES=$REPO/.nuget/packages`.
|
||||
- Sources default to `.nuget/packages`; override via `NUGET_SOURCES` (semicolon-separated).
|
||||
- No external services required; tests are isolated/local.
|
||||
|
||||
What it does
|
||||
1) `dotnet restore` + `dotnet build` on `src/AdvisoryAI/StellaOps.AdvisoryAI.sln` with `/bl`.
|
||||
3) Run the AdvisoryAI test project (`__Tests/StellaOps.AdvisoryAI.Tests`) with TRX output; optional `TEST_FILTER` env narrows scope.
|
||||
4) Emit `summary.json` with artefact paths and SHA256s for reproducibility.
|
||||
|
||||
Notes
|
||||
- Timestamped output folders keep ordering deterministic; consumers should sort lexicographically.
|
||||
- Use `TEST_FILTER="Name~Inference"` to target inference/monitoring-specific tests when iterating.
|
||||
@@ -1,65 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Advisory AI CI runner (DEVOPS-AIAI-31-001)
|
||||
# Builds solution and runs tests with warmed NuGet cache; emits binlog + TRX summary.
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/../../.." && pwd)"
|
||||
ts="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
out_dir="$repo_root/ops/devops/artifacts/advisoryai-ci/$ts"
|
||||
logs_dir="$out_dir/tests"
|
||||
mkdir -p "$logs_dir"
|
||||
|
||||
# Deterministic env
|
||||
export DOTNET_CLI_TELEMETRY_OPTOUT=${DOTNET_CLI_TELEMETRY_OPTOUT:-1}
|
||||
export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=${DOTNET_SKIP_FIRST_TIME_EXPERIENCE:-1}
|
||||
export NUGET_PACKAGES=${NUGET_PACKAGES:-$repo_root/.nuget/packages}
|
||||
export NUGET_SOURCES=${NUGET_SOURCES:-"$repo_root/.nuget/packages"}
|
||||
export TEST_FILTER=${TEST_FILTER:-""}
|
||||
export DOTNET_RESTORE_DISABLE_PARALLEL=${DOTNET_RESTORE_DISABLE_PARALLEL:-1}
|
||||
|
||||
mkdir -p "$NUGET_PACKAGES"
|
||||
|
||||
# Restore sources
|
||||
restore_sources=()
|
||||
IFS=';' read -ra SRC_ARR <<< "$NUGET_SOURCES"
|
||||
for s in "${SRC_ARR[@]}"; do
|
||||
[[ -n "$s" ]] && restore_sources+=(--source "$s")
|
||||
done
|
||||
|
||||
solution="$repo_root/src/AdvisoryAI/StellaOps.AdvisoryAI.sln"
|
||||
dotnet restore "$solution" --ignore-failed-sources "${restore_sources[@]}"
|
||||
|
||||
# Build with binlog (Release for perf parity)
|
||||
build_binlog="$out_dir/build.binlog"
|
||||
dotnet build "$solution" -c Release /p:ContinuousIntegrationBuild=true /bl:"$build_binlog"
|
||||
|
||||
# Tests
|
||||
common_test_args=( -c Release --no-build --results-directory "$logs_dir" )
|
||||
if [[ -n "$TEST_FILTER" ]]; then
|
||||
common_test_args+=( --filter "$TEST_FILTER" )
|
||||
fi
|
||||
|
||||
trx_name="advisoryai.trx"
|
||||
dotnet test "$repo_root/src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" \
|
||||
"${common_test_args[@]}" \
|
||||
--logger "trx;LogFileName=$trx_name"
|
||||
|
||||
# Summarize artefacts
|
||||
summary="$out_dir/summary.json"
|
||||
{
|
||||
printf '{\n'
|
||||
printf ' "timestamp_utc": "%s",\n' "$ts"
|
||||
printf ' "build_binlog": "%s",\n' "${build_binlog#${repo_root}/}"
|
||||
printf ' "tests": [{"project":"AdvisoryAI","trx":"%s"}],\n' "${logs_dir#${repo_root}/}/$trx_name"
|
||||
printf ' "nuget_packages": "%s",\n' "${NUGET_PACKAGES#${repo_root}/}"
|
||||
printf ' "sources": [\n'
|
||||
for i in "${!SRC_ARR[@]}"; do
|
||||
sep=","; [[ $i -eq $((${#SRC_ARR[@]}-1)) ]] && sep=""
|
||||
printf ' "%s"%s\n' "${SRC_ARR[$i]}" "$sep"
|
||||
done
|
||||
printf ' ]\n'
|
||||
printf '}\n'
|
||||
} > "$summary"
|
||||
|
||||
echo "Artifacts written to ${out_dir#${repo_root}/}"
|
||||
@@ -1,25 +0,0 @@
|
||||
# AOC Analyzer CI Contract (DEVOPS-AOC-19-001)
|
||||
|
||||
## Scope
|
||||
Integrate AOC Roslyn analyzer and guard tests into CI to block banned writes in ingestion projects.
|
||||
|
||||
## Steps
|
||||
1) Restore & build analyzers
|
||||
- `dotnet restore src/Aoc/__Analyzers/StellaOps.Aoc.Analyzers/StellaOps.Aoc.Analyzers.csproj`
|
||||
- `dotnet build src/Aoc/__Analyzers/StellaOps.Aoc.Analyzers/StellaOps.Aoc.Analyzers.csproj -c Release`
|
||||
2) Run analyzer on ingestion projects (Authority/Concelier/Excititor ingest paths)
|
||||
- `dotnet build src/Concelier/StellaOps.Concelier.Ingestion/StellaOps.Concelier.Ingestion.csproj -c Release /p:RunAnalyzers=true /p:TreatWarningsAsErrors=true`
|
||||
- `dotnet build src/Authority/StellaOps.Authority.Ingestion/StellaOps.Authority.Ingestion.csproj -c Release /p:RunAnalyzers=true /p:TreatWarningsAsErrors=true`
|
||||
- `dotnet build src/Excititor/StellaOps.Excititor.Ingestion/StellaOps.Excititor.Ingestion.csproj -c Release /p:RunAnalyzers=true /p:TreatWarningsAsErrors=true`
|
||||
3) Guard tests
|
||||
- `dotnet test src/Aoc/__Tests/StellaOps.Aoc.Analyzers.Tests/StellaOps.Aoc.Analyzers.Tests.csproj -c Release`
|
||||
4) Artefacts
|
||||
- Upload `.artifacts/aoc-analyzer.log` and test TRX.
|
||||
|
||||
## Determinism/Offline
|
||||
- Use local feeds (`local-nugets/`); no external fetches post-restore.
|
||||
- Build with `/p:ContinuousIntegrationBuild=true`.
|
||||
|
||||
## Acceptance
|
||||
- CI fails on any analyzer warning in ingestion projects.
|
||||
- Tests pass; artefacts uploaded.
|
||||
@@ -1,22 +0,0 @@
|
||||
# AOC Verify Stage (DEVOPS-AOC-19-002)
|
||||
|
||||
## Purpose
|
||||
Add CI stage to run `stella aoc verify --since <commit>` against seeded Mongo snapshots for Concelier + Excititor, publishing violation reports.
|
||||
|
||||
## Inputs
|
||||
- `STAGING_MONGO_URI` (read-only snapshot).
|
||||
- Optional `AOC_VERIFY_SINCE` (defaults to `HEAD~1`).
|
||||
|
||||
## Steps
|
||||
1) Seed snapshot (if needed)
|
||||
- Restore snapshot into local Mongo or point to read-only staging snapshot.
|
||||
2) Run verify
|
||||
- `dotnet run --project src/Aoc/StellaOps.Aoc.Cli -- verify --since ${AOC_VERIFY_SINCE:-HEAD~1} --mongo $STAGING_MONGO_URI --output .artifacts/aoc-verify.json`
|
||||
3) Fail on violations
|
||||
- Parse `.artifacts/aoc-verify.json`; if `violations > 0`, fail with summary.
|
||||
4) Publish artifacts
|
||||
- Upload `.artifacts/aoc-verify.json` and `.artifacts/aoc-verify.ndjson` (per-violation).
|
||||
|
||||
## Acceptance
|
||||
- Stage fails when violations exist; passes clean otherwise.
|
||||
- Artifacts attached for auditing.
|
||||
@@ -1,73 +0,0 @@
|
||||
# AOC Backfill Release Plan (DEVOPS-STORE-AOC-19-005-REL)
|
||||
|
||||
Scope: Release/offline-kit packaging for Concelier AOC backfill operations.
|
||||
|
||||
## Prerequisites
|
||||
- Dataset hash from dev rehearsal (AOC-19-005 dev outputs)
|
||||
- AOC guard tests passing (DEVOPS-AOC-19-001/002/003 - DONE)
|
||||
- Supersedes rollout plan reviewed (ops/devops/aoc/supersedes-rollout.md)
|
||||
|
||||
## Artefacts
|
||||
- Backfill runner bundle:
|
||||
- `aoc-backfill-runner.tar.gz` - CLI tool + scripts
|
||||
- `aoc-backfill-runner.sbom.json` - SPDX SBOM
|
||||
- `aoc-backfill-runner.dsse.json` - Cosign attestation
|
||||
- Dataset bundle:
|
||||
- `aoc-dataset-{hash}.tar.gz` - Seeded dataset
|
||||
- `aoc-dataset-{hash}.manifest.json` - Manifest with checksums
|
||||
- `aoc-dataset-{hash}.provenance.json` - SLSA provenance
|
||||
- Offline kit slice:
|
||||
- All above + SHA256SUMS + verification scripts
|
||||
|
||||
## Packaging Script
|
||||
|
||||
```bash
|
||||
# Production (CI with secrets)
|
||||
./ops/devops/aoc/package-backfill-release.sh
|
||||
|
||||
# Development (dev key)
|
||||
COSIGN_ALLOW_DEV_KEY=1 COSIGN_PASSWORD=stellaops-dev \
|
||||
DATASET_HASH=dev-rehearsal-placeholder \
|
||||
./ops/devops/aoc/package-backfill-release.sh
|
||||
```
|
||||
|
||||
## Pipeline Outline
|
||||
1) Build backfill runner from `src/Aoc/StellaOps.Aoc.Cli/`
|
||||
2) Generate SBOM with syft
|
||||
3) Sign with cosign (dev key fallback)
|
||||
4) Package dataset (when hash available)
|
||||
5) Create offline bundle with checksums
|
||||
6) Verification:
|
||||
- `stella aoc verify --dry-run`
|
||||
- `cosign verify-blob` for all bundles
|
||||
- `sha256sum --check`
|
||||
7) Publish to release bucket + offline kit
|
||||
|
||||
## Runbook
|
||||
1) Validate AOC guard tests pass in CI
|
||||
2) Run dev rehearsal with test dataset
|
||||
3) Capture dataset hash from rehearsal
|
||||
4) Execute packaging script with production key
|
||||
5) Verify all signatures and checksums
|
||||
6) Upload to release bucket
|
||||
7) Include in offline kit manifest
|
||||
|
||||
## CI Workflow
|
||||
`.gitea/workflows/aoc-backfill-release.yml`
|
||||
|
||||
## Verification
|
||||
```bash
|
||||
# Verify bundle signatures
|
||||
cosign verify-blob \
|
||||
--key tools/cosign/cosign.dev.pub \
|
||||
--bundle out/aoc/aoc-backfill-runner.dsse.json \
|
||||
out/aoc/aoc-backfill-runner.tar.gz
|
||||
|
||||
# Verify checksums
|
||||
cd out/aoc && sha256sum -c SHA256SUMS
|
||||
```
|
||||
|
||||
## Owners
|
||||
- DevOps Guild (pipeline + packaging)
|
||||
- Concelier Storage Guild (dataset + backfill logic)
|
||||
- Platform Security (signing policy)
|
||||
@@ -1,175 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Package AOC backfill release for offline kit
|
||||
# Usage: ./package-backfill-release.sh
|
||||
# Dev mode: COSIGN_ALLOW_DEV_KEY=1 COSIGN_PASSWORD=stellaops-dev DATASET_HASH=dev ./package-backfill-release.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
ROOT=$(cd "$(dirname "$0")/../../.." && pwd)
|
||||
OUT_DIR="${OUT_DIR:-$ROOT/out/aoc}"
|
||||
CREATED="${CREATED:-$(date -u +%Y-%m-%dT%H:%M:%SZ)}"
|
||||
DATASET_HASH="${DATASET_HASH:-}"
|
||||
|
||||
mkdir -p "$OUT_DIR"
|
||||
|
||||
echo "==> AOC Backfill Release Packaging"
|
||||
echo " Output: $OUT_DIR"
|
||||
echo " Dataset hash: ${DATASET_HASH:-<pending>}"
|
||||
|
||||
# Key resolution (same pattern as advisory-ai packaging)
|
||||
resolve_key() {
|
||||
if [[ -n "${COSIGN_KEY_FILE:-}" && -f "$COSIGN_KEY_FILE" ]]; then
|
||||
echo "$COSIGN_KEY_FILE"
|
||||
elif [[ -n "${COSIGN_PRIVATE_KEY_B64:-}" ]]; then
|
||||
local tmp_key="$OUT_DIR/.cosign.key"
|
||||
echo "$COSIGN_PRIVATE_KEY_B64" | base64 -d > "$tmp_key"
|
||||
chmod 600 "$tmp_key"
|
||||
echo "$tmp_key"
|
||||
elif [[ -f "$ROOT/tools/cosign/cosign.key" ]]; then
|
||||
echo "$ROOT/tools/cosign/cosign.key"
|
||||
elif [[ "${COSIGN_ALLOW_DEV_KEY:-0}" == "1" && -f "$ROOT/tools/cosign/cosign.dev.key" ]]; then
|
||||
echo "[info] Using development key (non-production)" >&2
|
||||
echo "$ROOT/tools/cosign/cosign.dev.key"
|
||||
else
|
||||
echo "[error] No signing key available. Set COSIGN_PRIVATE_KEY_B64 or COSIGN_ALLOW_DEV_KEY=1" >&2
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Build AOC CLI if not already built
|
||||
AOC_CLI_PROJECT="$ROOT/src/Aoc/StellaOps.Aoc.Cli/StellaOps.Aoc.Cli.csproj"
|
||||
AOC_CLI_OUT="$OUT_DIR/cli"
|
||||
|
||||
if [[ -f "$AOC_CLI_PROJECT" ]]; then
|
||||
echo "==> Building AOC CLI..."
|
||||
dotnet publish "$AOC_CLI_PROJECT" \
|
||||
-c Release \
|
||||
-o "$AOC_CLI_OUT" \
|
||||
--no-restore 2>/dev/null || echo "[info] Build skipped (may need restore)"
|
||||
else
|
||||
echo "[info] AOC CLI project not found; using placeholder"
|
||||
mkdir -p "$AOC_CLI_OUT"
|
||||
echo "AOC CLI placeholder - build from src/Aoc/StellaOps.Aoc.Cli/" > "$AOC_CLI_OUT/README.txt"
|
||||
fi
|
||||
|
||||
# Create backfill runner bundle
|
||||
echo "==> Creating backfill runner bundle..."
|
||||
RUNNER_TAR="$OUT_DIR/aoc-backfill-runner.tar.gz"
|
||||
tar -czf "$RUNNER_TAR" -C "$AOC_CLI_OUT" .
|
||||
|
||||
# Compute hash
|
||||
sha256() {
|
||||
sha256sum "$1" | awk '{print $1}'
|
||||
}
|
||||
RUNNER_HASH=$(sha256 "$RUNNER_TAR")
|
||||
|
||||
# Generate manifest
|
||||
echo "==> Generating manifest..."
|
||||
MANIFEST="$OUT_DIR/aoc-backfill-runner.manifest.json"
|
||||
cat > "$MANIFEST" <<EOF
|
||||
{
|
||||
"schemaVersion": "1.0.0",
|
||||
"created": "$CREATED",
|
||||
"runner": {
|
||||
"path": "aoc-backfill-runner.tar.gz",
|
||||
"sha256": "$RUNNER_HASH",
|
||||
"size": $(stat -c%s "$RUNNER_TAR" 2>/dev/null || stat -f%z "$RUNNER_TAR")
|
||||
},
|
||||
"dataset": {
|
||||
"hash": "${DATASET_HASH:-pending}",
|
||||
"status": "$( [[ -n "$DATASET_HASH" ]] && echo "available" || echo "pending-dev-rehearsal" )"
|
||||
},
|
||||
"signing": {
|
||||
"mode": "$( [[ "${COSIGN_ALLOW_DEV_KEY:-0}" == "1" ]] && echo "development" || echo "production" )"
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# Sign with cosign if available
|
||||
KEY_FILE=$(resolve_key) || true
|
||||
COSIGN="${COSIGN:-$ROOT/tools/cosign/cosign}"
|
||||
DSSE_OUT="$OUT_DIR/aoc-backfill-runner.dsse.json"
|
||||
|
||||
if [[ -n "${KEY_FILE:-}" ]]; then
|
||||
COSIGN_CMD="${COSIGN:-cosign}"
|
||||
if command -v cosign &>/dev/null; then
|
||||
COSIGN_CMD="cosign"
|
||||
fi
|
||||
|
||||
echo "==> Signing bundle..."
|
||||
COSIGN_PASSWORD="${COSIGN_PASSWORD:-}" "$COSIGN_CMD" sign-blob \
|
||||
--key "$KEY_FILE" \
|
||||
--bundle "$DSSE_OUT" \
|
||||
--tlog-upload=false \
|
||||
--yes \
|
||||
"$RUNNER_TAR" 2>/dev/null || echo "[info] DSSE signing skipped"
|
||||
fi
|
||||
|
||||
# Generate SBOM placeholder
|
||||
echo "==> Generating SBOM..."
|
||||
SBOM="$OUT_DIR/aoc-backfill-runner.sbom.json"
|
||||
cat > "$SBOM" <<EOF
|
||||
{
|
||||
"spdxVersion": "SPDX-2.3",
|
||||
"dataLicense": "CC0-1.0",
|
||||
"SPDXID": "SPDXRef-DOCUMENT",
|
||||
"name": "aoc-backfill-runner",
|
||||
"documentNamespace": "https://stella-ops.org/sbom/aoc-backfill-runner/$CREATED",
|
||||
"creationInfo": {
|
||||
"created": "$CREATED",
|
||||
"creators": ["Tool: stellaops-aoc-packager"]
|
||||
},
|
||||
"packages": [
|
||||
{
|
||||
"name": "StellaOps.Aoc.Cli",
|
||||
"SPDXID": "SPDXRef-Package-aoc-cli",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false
|
||||
}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
|
||||
# Generate provenance
|
||||
echo "==> Generating provenance..."
|
||||
PROVENANCE="$OUT_DIR/aoc-backfill-runner.provenance.json"
|
||||
cat > "$PROVENANCE" <<EOF
|
||||
{
|
||||
"_type": "https://in-toto.io/Statement/v1",
|
||||
"subject": [
|
||||
{
|
||||
"name": "aoc-backfill-runner.tar.gz",
|
||||
"digest": {"sha256": "$RUNNER_HASH"}
|
||||
}
|
||||
],
|
||||
"predicateType": "https://slsa.dev/provenance/v1",
|
||||
"predicate": {
|
||||
"buildDefinition": {
|
||||
"buildType": "https://stella-ops.org/aoc-backfill-release/v1",
|
||||
"internalParameters": {
|
||||
"created": "$CREATED",
|
||||
"datasetHash": "${DATASET_HASH:-pending}"
|
||||
}
|
||||
},
|
||||
"runDetails": {
|
||||
"builder": {"id": "https://stella-ops.org/aoc-backfill-release"}
|
||||
}
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# Generate checksums
|
||||
echo "==> Generating checksums..."
|
||||
cd "$OUT_DIR"
|
||||
sha256sum aoc-backfill-runner.tar.gz aoc-backfill-runner.manifest.json aoc-backfill-runner.sbom.json > SHA256SUMS
|
||||
|
||||
# Cleanup temp key
|
||||
[[ -f "$OUT_DIR/.cosign.key" ]] && rm -f "$OUT_DIR/.cosign.key"
|
||||
|
||||
echo "==> AOC backfill packaging complete"
|
||||
echo " Runner: $RUNNER_TAR"
|
||||
echo " Manifest: $MANIFEST"
|
||||
echo " SBOM: $SBOM"
|
||||
echo " Provenance: $PROVENANCE"
|
||||
echo " Checksums: $OUT_DIR/SHA256SUMS"
|
||||
[[ -f "$DSSE_OUT" ]] && echo " DSSE: $DSSE_OUT"
|
||||
@@ -1,50 +0,0 @@
|
||||
# Supersedes backfill rollout plan (DEVOPS-AOC-19-101)
|
||||
|
||||
Scope: Concelier Link-Not-Merge backfill and supersedes processing once advisory_raw idempotency index is in staging.
|
||||
|
||||
## Preconditions
|
||||
- Idempotency index verified in staging (`advisory_raw` duplicate inserts rejected; log hash recorded).
|
||||
- LNM migrations 21-101/102 applied (shards, TTL, tombstones).
|
||||
- Event transport to NATS/Redis disabled during backfill to avoid noisy downstream replays.
|
||||
- Offline kit mirror includes current hashes for `advisory_raw` and backfill bundle.
|
||||
|
||||
## Rollout steps (staging → prod)
|
||||
1) **Freeze window** (announce 24h prior)
|
||||
- Pause Concelier ingest workers (`CONCELIER_INGEST_ENABLED=false`).
|
||||
- Stop outbox publisher or point to blackhole NATS subject.
|
||||
2) **Dry-run (staging)**
|
||||
- Run backfill job with `--dry-run` to emit counts only.
|
||||
- Verify: new supersedes records count == expected; no write errors; idempotency violations = 0.
|
||||
- Capture logs + SHA256 of generated report.
|
||||
3) **Prod execution**
|
||||
- Run backfill job with `--batch-size=500` and `--stop-on-error`.
|
||||
- Monitor: insert rate, error rate, Mongo oplog lag; target <5% CPU on primary.
|
||||
4) **Validation**
|
||||
- Run consistency check:
|
||||
- `advisory_observations` count stable (no drop).
|
||||
- Supersedes edges present for all prior conflicts.
|
||||
- Idempotency index hit rate <0.1%.
|
||||
- Run API spot check: `/advisories/summary` returns supersedes metadata; `advisory.linkset.updated` events absent during freeze.
|
||||
5) **Unfreeze**
|
||||
- Re-enable ingest + outbox publisher.
|
||||
- Trigger single `advisory.observation.updated@1` replay to confirm event path is healthy.
|
||||
|
||||
## Rollback
|
||||
- If errors >0 or idempotency violations observed:
|
||||
- Stop job, keep ingest paused.
|
||||
- Run rollback script `ops/devops/scripts/rollback-lnm-backfill.js` to remove supersedes/tombstones inserted in current window.
|
||||
- Restore Mongo from last checkpointed snapshot if rollback script fails.
|
||||
|
||||
## Evidence to capture
|
||||
- Job command + arguments.
|
||||
- SHA256 of backfill bundle and report.
|
||||
- Idempotency violation count.
|
||||
- Post-run consistency report (JSON) stored under `ops/devops/artifacts/aoc-supersedes/<timestamp>/`.
|
||||
|
||||
## Monitoring/Alerts
|
||||
- Add temporary Grafana panel for idempotency violations and Mongo ops/sec during job.
|
||||
- Alert if job runtime exceeds 2h or if oplog lag > 60s.
|
||||
|
||||
## Owners
|
||||
- Run: DevOps Guild
|
||||
- Approvals: Concelier Storage Guild + Platform Security
|
||||
@@ -1,20 +0,0 @@
|
||||
# Authority DevOps Crew
|
||||
|
||||
## Mission
|
||||
Operate and harden the StellaOps Authority platform in production and air-gapped environments: container images, deployment assets, observability defaults, backup/restore, and runtime key management.
|
||||
|
||||
## Focus Areas
|
||||
- **Build & Packaging** – Dockerfiles, OCI bundles, offline artefact refresh.
|
||||
- **Deployment Tooling** – Compose/Kubernetes manifests, secrets bootstrap, upgrade paths.
|
||||
- **Observability** – Logging defaults, metrics/trace exporters, dashboards, alert policies.
|
||||
- **Continuity & Security** – Backup/restore guides, key rotation playbooks, revocation propagation.
|
||||
|
||||
## Working Agreements
|
||||
- Track work directly in the relevant `docs/implplan/SPRINT_*.md` rows (TODO → DOING → DONE/BLOCKED); keep entries dated.
|
||||
- Validate container changes with the CI pipeline (`ops/authority` GitHub workflow) before marking DONE.
|
||||
- Update operator documentation in `docs/` together with any behavioural change.
|
||||
- Coordinate with Authority Core and Security Guild before altering sensitive defaults (rate limits, crypto providers, revocation jobs).
|
||||
|
||||
## Required Reading
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
- `docs/modules/airgap/airgap-mode.md`
|
||||
@@ -1,38 +0,0 @@
|
||||
# syntax=docker/dockerfile:1.7-labs
|
||||
|
||||
#
|
||||
# StellaOps Authority – distroless container build
|
||||
# Produces a minimal image containing the Authority host and its plugins.
|
||||
#
|
||||
|
||||
ARG SDK_IMAGE=mcr.microsoft.com/dotnet/nightly/sdk:10.0
|
||||
ARG RUNTIME_IMAGE=gcr.io/distroless/dotnet/aspnet:latest
|
||||
|
||||
FROM ${SDK_IMAGE} AS build
|
||||
|
||||
WORKDIR /src
|
||||
|
||||
# Restore & publish
|
||||
COPY . .
|
||||
RUN dotnet restore src/StellaOps.sln
|
||||
RUN dotnet publish src/Authority/StellaOps.Authority/StellaOps.Authority/StellaOps.Authority.csproj \
|
||||
-c Release \
|
||||
-o /app/publish \
|
||||
/p:UseAppHost=false
|
||||
|
||||
FROM ${RUNTIME_IMAGE} AS runtime
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
ENV ASPNETCORE_URLS=http://0.0.0.0:8080
|
||||
ENV STELLAOPS_AUTHORITY__PLUGINDIRECTORIES__0=/app/plugins
|
||||
ENV STELLAOPS_AUTHORITY__PLUGINS__CONFIGURATIONDIRECTORY=/app/etc/authority.plugins
|
||||
|
||||
COPY --from=build /app/publish ./
|
||||
|
||||
# Provide writable mount points for configs/keys/plugins
|
||||
VOLUME ["/app/etc", "/app/plugins", "/app/keys"]
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
ENTRYPOINT ["dotnet", "StellaOps.Authority.dll"]
|
||||
@@ -1,62 +0,0 @@
|
||||
# StellaOps Authority Container Scaffold
|
||||
|
||||
This directory provides a distroless Dockerfile and `docker-compose` sample for bootstrapping the Authority service alongside MongoDB (required) and Redis (optional).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker Engine 25+ and Compose V2
|
||||
- .NET 10 preview SDK (only required when building locally outside of Compose)
|
||||
- Populated Authority configuration at `etc/authority.yaml` and plugin manifests under `etc/authority.plugins/`
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# 1. Ensure configuration files exist (copied from etc/authority.yaml.sample, etc/authority.plugins/*.yaml)
|
||||
# 2. Build and start the stack
|
||||
docker compose -f ops/authority/docker-compose.authority.yaml up --build
|
||||
```
|
||||
|
||||
`authority.yaml` is mounted read-only at `/etc/authority.yaml` inside the container. Plugin manifests are mounted to `/app/etc/authority.plugins`. Update the issuer URL plus any Mongo credentials in the compose file or via an `.env`.
|
||||
|
||||
To run with pre-built images, replace the `build:` block in the compose file with an `image:` reference.
|
||||
|
||||
## Volumes
|
||||
|
||||
- `mongo-data` – persists MongoDB state.
|
||||
- `redis-data` – optional Redis persistence (enable the service before use).
|
||||
- `authority-keys` – writable volume for Authority signing keys.
|
||||
|
||||
## Environment overrides
|
||||
|
||||
Key environment variables (mirroring `StellaOpsAuthorityOptions`):
|
||||
|
||||
| Variable | Description |
|
||||
| --- | --- |
|
||||
| `STELLAOPS_AUTHORITY__ISSUER` | Public issuer URL advertised by Authority |
|
||||
| `STELLAOPS_AUTHORITY__PLUGINDIRECTORIES__0` | Primary plugin binaries directory inside the container |
|
||||
| `STELLAOPS_AUTHORITY__PLUGINS__CONFIGURATIONDIRECTORY` | Path to plugin manifest directory |
|
||||
|
||||
For additional options, see `etc/authority.yaml.sample`.
|
||||
|
||||
> **Graph Explorer reminder:** When enabling Cartographer or Graph API components, update `etc/authority.yaml` so the `cartographer-service` client includes `properties.serviceIdentity: "cartographer"` and a tenant hint. Authority now rejects `graph:write` tokens that lack this marker, so existing deployments must apply the update before rolling out the new build.
|
||||
|
||||
> **Console endpoint reminder:** The Console UI now calls `/console/tenants`, `/console/profile`, and `/console/token/introspect`. Reverse proxies must forward the `X-Stella-Tenant` header (derived from the access token) so Authority can enforce tenancy; audit events are logged under `authority.console.*`. Admin actions obey a five-minute fresh-auth window reported by `/console/profile`, so keep session timeout prompts aligned with that value.
|
||||
|
||||
## Key rotation automation (OPS3)
|
||||
|
||||
The `key-rotation.sh` helper wraps the `/internal/signing/rotate` endpoint delivered with CORE10. It can run in CI/CD once the new PEM key is staged on the Authority host volume.
|
||||
|
||||
```bash
|
||||
AUTHORITY_BOOTSTRAP_KEY=$(cat ~/.secrets/authority-bootstrap.key) \
|
||||
./key-rotation.sh \
|
||||
--authority-url https://authority.stella-ops.local \
|
||||
--key-id authority-signing-2025 \
|
||||
--key-path ../certificates/authority-signing-2025.pem \
|
||||
--meta rotatedBy=pipeline --meta changeTicket=OPS-1234
|
||||
```
|
||||
|
||||
- `--key-path` should resolve from the Authority content root (same as `docs/11_AUTHORITY.md` SOP).
|
||||
- Provide `--source`/`--provider` if the key loader differs from the default file-based provider.
|
||||
- Pass `--dry-run` during rehearsals to inspect the JSON payload without invoking the API.
|
||||
|
||||
After rotation, export a fresh revocation bundle (`stellaops-cli auth revoke export`) so downstream mirrors consume signatures from the new `kid`. The canonical operational steps live in `docs/11_AUTHORITY.md` – make sure any local automation keeps that guide as source of truth.
|
||||
@@ -1,5 +0,0 @@
|
||||
# Completed Tasks
|
||||
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
| OPS3.KEY-ROTATION | DONE (2025-10-12) | DevOps Crew, Authority Core | CORE10.JWKS | Implement key rotation tooling + pipeline hook once rotating JWKS lands. Document SOP and secret handling. | ✅ CLI/script rotates keys + updates JWKS; ✅ Pipeline job documented; ✅ docs/ops runbook updated. |
|
||||
@@ -1,59 +0,0 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
authority:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: ops/authority/Dockerfile
|
||||
image: stellaops-authority:dev
|
||||
container_name: stellaops-authority
|
||||
depends_on:
|
||||
mongo:
|
||||
condition: service_started
|
||||
environment:
|
||||
# Override issuer to match your deployment URL.
|
||||
STELLAOPS_AUTHORITY__ISSUER: "https://authority.localtest.me"
|
||||
# Point the Authority host at the Mongo instance defined below.
|
||||
STELLAOPS_AUTHORITY__PLUGINDIRECTORIES__0: "/app/plugins"
|
||||
STELLAOPS_AUTHORITY__PLUGINS__CONFIGURATIONDIRECTORY: "/app/etc/authority.plugins"
|
||||
volumes:
|
||||
# Mount Authority configuration + plugins (edit etc/authority.yaml before running).
|
||||
- ../../etc/authority.yaml:/etc/authority.yaml:ro
|
||||
- ../../etc/authority.plugins:/app/etc/authority.plugins:ro
|
||||
# Optional: persist plugin binaries or key material outside the container.
|
||||
- authority-keys:/app/keys
|
||||
ports:
|
||||
- "8080:8080"
|
||||
restart: unless-stopped
|
||||
|
||||
mongo:
|
||||
image: mongo:7
|
||||
container_name: stellaops-authority-mongo
|
||||
command: ["mongod", "--bind_ip_all"]
|
||||
environment:
|
||||
MONGO_INITDB_ROOT_USERNAME: stellaops
|
||||
MONGO_INITDB_ROOT_PASSWORD: stellaops
|
||||
volumes:
|
||||
- mongo-data:/data/db
|
||||
ports:
|
||||
- "27017:27017"
|
||||
restart: unless-stopped
|
||||
|
||||
valkey:
|
||||
image: valkey/valkey:9.0.1-alpine
|
||||
container_name: stellaops-authority-valkey
|
||||
command: ["valkey-server", "--save", "60", "1"]
|
||||
volumes:
|
||||
- valkey-data:/data
|
||||
ports:
|
||||
- "6379:6379"
|
||||
restart: unless-stopped
|
||||
# Uncomment to enable if/when Authority consumes Valkey.
|
||||
# deploy:
|
||||
# replicas: 0
|
||||
|
||||
volumes:
|
||||
mongo-data:
|
||||
valkey-data:
|
||||
authority-keys:
|
||||
|
||||
@@ -1,189 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'USAGE'
|
||||
Usage: key-rotation.sh --authority-url URL --api-key TOKEN --key-id ID --key-path PATH [options]
|
||||
|
||||
Required flags:
|
||||
-u, --authority-url Base Authority URL (e.g. https://authority.example.com)
|
||||
-k, --api-key Bootstrap API key (x-stellaops-bootstrap-key header)
|
||||
-i, --key-id Identifier (kid) for the new signing key
|
||||
-p, --key-path Path (relative to Authority content root or absolute) where the PEM key lives
|
||||
|
||||
Optional flags:
|
||||
-s, --source Key source loader identifier (default: file)
|
||||
-a, --algorithm Signing algorithm (default: ES256)
|
||||
--provider Preferred crypto provider name
|
||||
-m, --meta key=value Additional metadata entries for the rotation record (repeatable)
|
||||
--dry-run Print the JSON payload instead of invoking the API
|
||||
-h, --help Show this help
|
||||
|
||||
Environment fallbacks:
|
||||
AUTHORITY_URL, AUTHORITY_BOOTSTRAP_KEY, AUTHORITY_KEY_SOURCE, AUTHORITY_KEY_PROVIDER
|
||||
|
||||
Example:
|
||||
AUTHORITY_BOOTSTRAP_KEY=$(cat key.txt) \\
|
||||
./key-rotation.sh -u https://authority.local \\
|
||||
-i authority-signing-2025 \\
|
||||
-p ../certificates/authority-signing-2025.pem \\
|
||||
-m rotatedBy=pipeline -m ticket=OPS-1234
|
||||
USAGE
|
||||
}
|
||||
|
||||
require_python() {
|
||||
if command -v python3 >/dev/null 2>&1; then
|
||||
PYTHON_BIN=python3
|
||||
elif command -v python >/dev/null 2>&1; then
|
||||
PYTHON_BIN=python
|
||||
else
|
||||
echo "error: python3 (or python) is required for JSON encoding" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
json_quote() {
|
||||
"$PYTHON_BIN" - "$1" <<'PY'
|
||||
import json, sys
|
||||
print(json.dumps(sys.argv[1]))
|
||||
PY
|
||||
}
|
||||
|
||||
AUTHORITY_URL="${AUTHORITY_URL:-}"
|
||||
API_KEY="${AUTHORITY_BOOTSTRAP_KEY:-}"
|
||||
KEY_ID=""
|
||||
KEY_PATH=""
|
||||
SOURCE="${AUTHORITY_KEY_SOURCE:-file}"
|
||||
ALGORITHM="ES256"
|
||||
PROVIDER="${AUTHORITY_KEY_PROVIDER:-}"
|
||||
DRY_RUN=false
|
||||
declare -a METADATA=()
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
-u|--authority-url)
|
||||
AUTHORITY_URL="$2"
|
||||
shift 2
|
||||
;;
|
||||
-k|--api-key)
|
||||
API_KEY="$2"
|
||||
shift 2
|
||||
;;
|
||||
-i|--key-id)
|
||||
KEY_ID="$2"
|
||||
shift 2
|
||||
;;
|
||||
-p|--key-path)
|
||||
KEY_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
-s|--source)
|
||||
SOURCE="$2"
|
||||
shift 2
|
||||
;;
|
||||
-a|--algorithm)
|
||||
ALGORITHM="$2"
|
||||
shift 2
|
||||
;;
|
||||
--provider)
|
||||
PROVIDER="$2"
|
||||
shift 2
|
||||
;;
|
||||
-m|--meta)
|
||||
METADATA+=("$2")
|
||||
shift 2
|
||||
;;
|
||||
--dry-run)
|
||||
DRY_RUN=true
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option: $1" >&2
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z "$AUTHORITY_URL" || -z "$API_KEY" || -z "$KEY_ID" || -z "$KEY_PATH" ]]; then
|
||||
echo "error: missing required arguments" >&2
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
case "$AUTHORITY_URL" in
|
||||
http://*|https://*) ;;
|
||||
*)
|
||||
echo "error: --authority-url must include scheme (http/https)" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
require_python
|
||||
|
||||
payload="{"
|
||||
payload+="\"keyId\":$(json_quote "$KEY_ID"),"
|
||||
payload+="\"location\":$(json_quote "$KEY_PATH"),"
|
||||
payload+="\"source\":$(json_quote "$SOURCE"),"
|
||||
payload+="\"algorithm\":$(json_quote "$ALGORITHM"),"
|
||||
if [[ -n "$PROVIDER" ]]; then
|
||||
payload+="\"provider\":$(json_quote "$PROVIDER"),"
|
||||
fi
|
||||
|
||||
if [[ ${#METADATA[@]} -gt 0 ]]; then
|
||||
payload+="\"metadata\":{"
|
||||
for entry in "${METADATA[@]}"; do
|
||||
if [[ "$entry" != *=* ]]; then
|
||||
echo "warning: ignoring metadata entry '$entry' (expected key=value)" >&2
|
||||
continue
|
||||
fi
|
||||
key="${entry%%=*}"
|
||||
value="${entry#*=}"
|
||||
payload+="$(json_quote "$key"):$(json_quote "$value"),"
|
||||
done
|
||||
if [[ "${payload: -1}" == "," ]]; then
|
||||
payload="${payload::-1}"
|
||||
fi
|
||||
payload+="},"
|
||||
fi
|
||||
|
||||
if [[ "${payload: -1}" == "," ]]; then
|
||||
payload="${payload::-1}"
|
||||
fi
|
||||
payload+="}"
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
echo "# Dry run payload:"
|
||||
echo "$payload"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
tmp_response="$(mktemp)"
|
||||
cleanup() { rm -f "$tmp_response"; }
|
||||
trap cleanup EXIT
|
||||
|
||||
http_code=$(curl -sS -o "$tmp_response" -w "%{http_code}" \
|
||||
-X POST "${AUTHORITY_URL%/}/internal/signing/rotate" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "x-stellaops-bootstrap-key: $API_KEY" \
|
||||
--data "$payload")
|
||||
|
||||
if [[ "$http_code" != "200" && "$http_code" != "201" ]]; then
|
||||
echo "error: rotation API returned HTTP $http_code" >&2
|
||||
cat "$tmp_response" >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Rotation request accepted (HTTP $http_code). Response:"
|
||||
cat "$tmp_response"
|
||||
|
||||
echo
|
||||
echo "Fetching JWKS to confirm active key..."
|
||||
curl -sS "${AUTHORITY_URL%/}/jwks" || true
|
||||
echo
|
||||
echo "Done. Remember to update authority.yaml with the new key metadata to keep restarts consistent."
|
||||
@@ -1,32 +0,0 @@
|
||||
CI runner for **DEVOPS-CI-110-001** (Concelier + Excititor smoke)
|
||||
==================================================================
|
||||
|
||||
Scope
|
||||
-----
|
||||
- Warm NuGet cache from `local-nugets`, `.nuget/packages`, and (optionally) NuGet.org.
|
||||
- Ensure OpenSSL 1.1 is present (installs `libssl1.1` when available via `apt-get`).
|
||||
- Run lightweight slices:
|
||||
- Concelier WebService: `HealthAndReadyEndpointsRespond`
|
||||
- Excititor WebService: `AirgapImportEndpointTests*`
|
||||
- Emit TRX + logs to `ops/devops/artifacts/ci-110/<timestamp>/`.
|
||||
|
||||
Usage
|
||||
-----
|
||||
```bash
|
||||
export NUGET_SOURCES="/mnt/e/dev/git.stella-ops.org/local-nugets;/mnt/e/dev/git.stella-ops.org/.nuget/packages;https://api.nuget.org/v3/index.json"
|
||||
export TIMESTAMP=$(date -u +%Y%m%dT%H%M%SZ) # optional, for reproducible paths
|
||||
bash ops/devops/ci-110-runner/run-ci-110.sh
|
||||
```
|
||||
|
||||
Artifacts
|
||||
---------
|
||||
- TRX: `ops/devops/artifacts/ci-110/<timestamp>/trx/`
|
||||
- `concelier-health.trx` (1 test)
|
||||
- `excititor-airgapimport.fqn.trx` (2 tests)
|
||||
- Logs + restores under `ops/devops/artifacts/ci-110/<timestamp>/logs/`.
|
||||
|
||||
Notes
|
||||
-----
|
||||
- The runner uses `--no-build` on test slices; prior restores are included in the script.
|
||||
- If OpenSSL 1.1 is not present and `apt-get` cannot install `libssl1.1`, set `LD_LIBRARY_PATH` to a pre-installed OpenSSL 1.1 location before running.
|
||||
- Extend the runner by adding more `run_test_slice` calls for additional suites; keep filters tight to avoid long hangs on constrained CI.
|
||||
@@ -1,92 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# CI helper for DEVOPS-CI-110-001
|
||||
# - Warms NuGet cache from local sources
|
||||
# - Ensures OpenSSL 1.1 compatibility if available
|
||||
# - Runs targeted Concelier and Excititor test slices with TRX output
|
||||
# - Writes artefacts under ops/devops/artifacts/ci-110/<timestamp>/
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="${ROOT:-$(git rev-parse --show-toplevel)}"
|
||||
TIMESTAMP="${TIMESTAMP:-$(date -u +%Y%m%dT%H%M%SZ)}"
|
||||
ARTIFACT_ROOT="${ARTIFACT_ROOT:-"$ROOT/ops/devops/artifacts/ci-110/$TIMESTAMP"}"
|
||||
LOG_DIR="$ARTIFACT_ROOT/logs"
|
||||
TRX_DIR="$ARTIFACT_ROOT/trx"
|
||||
|
||||
NUGET_SOURCES_DEFAULT="$ROOT/.nuget/packages;https://api.nuget.org/v3/index.json"
|
||||
NUGET_SOURCES="${NUGET_SOURCES:-$NUGET_SOURCES_DEFAULT}"
|
||||
|
||||
export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1
|
||||
export DOTNET_CLI_TELEMETRY_OPTOUT=1
|
||||
export DOTNET_RESTORE_DISABLE_PARALLEL="${DOTNET_RESTORE_DISABLE_PARALLEL:-1}"
|
||||
|
||||
mkdir -p "$LOG_DIR" "$TRX_DIR"
|
||||
|
||||
log() {
|
||||
printf '[%s] %s\n' "$(date -u +%H:%M:%S)" "$*"
|
||||
}
|
||||
|
||||
ensure_openssl11() {
|
||||
if openssl version 2>/dev/null | grep -q "1\\.1."; then
|
||||
log "OpenSSL 1.1 detected: $(openssl version)"
|
||||
return
|
||||
fi
|
||||
|
||||
if command -v apt-get >/dev/null 2>&1; then
|
||||
log "OpenSSL 1.1 not found; attempting install via apt-get (libssl1.1)"
|
||||
sudo DEBIAN_FRONTEND=noninteractive apt-get update -y >/dev/null || true
|
||||
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y libssl1.1 || true
|
||||
if openssl version 2>/dev/null | grep -q "1\\.1."; then
|
||||
log "OpenSSL 1.1 available after install: $(openssl version)"
|
||||
return
|
||||
fi
|
||||
fi
|
||||
|
||||
log "OpenSSL 1.1 still unavailable. Provide it via LD_LIBRARY_PATH if required."
|
||||
}
|
||||
|
||||
restore_solution() {
|
||||
local sln="$1"
|
||||
log "Restore $sln"
|
||||
dotnet restore "$sln" --source "$NUGET_SOURCES" --verbosity minimal | tee "$LOG_DIR/restore-$(basename "$sln").log"
|
||||
}
|
||||
|
||||
run_test_slice() {
|
||||
local proj="$1"
|
||||
local filter="$2"
|
||||
local name="$3"
|
||||
log "Test $name ($proj, filter='$filter')"
|
||||
dotnet test "$proj" \
|
||||
-c Debug \
|
||||
--no-build \
|
||||
${filter:+--filter "$filter"} \
|
||||
--logger "trx;LogFileName=${name}.trx" \
|
||||
--results-directory "$TRX_DIR" \
|
||||
--blame-hang \
|
||||
--blame-hang-timeout 8m \
|
||||
--blame-hang-dump-type none \
|
||||
| tee "$LOG_DIR/test-${name}.log"
|
||||
}
|
||||
|
||||
main() {
|
||||
log "Starting CI-110 runner; artefacts -> $ARTIFACT_ROOT"
|
||||
ensure_openssl11
|
||||
|
||||
restore_solution "$ROOT/concelier-webservice.slnf"
|
||||
restore_solution "$ROOT/src/Excititor/StellaOps.Excititor.sln"
|
||||
|
||||
# Concelier: lightweight health slice to validate runner + Mongo wiring
|
||||
run_test_slice "$ROOT/src/Concelier/__Tests/StellaOps.Concelier.WebService.Tests/StellaOps.Concelier.WebService.Tests.csproj" \
|
||||
"HealthAndReadyEndpointsRespond" \
|
||||
"concelier-health"
|
||||
|
||||
# Excititor: airgap import surface (chunk-path) smoke
|
||||
run_test_slice "$ROOT/src/Excititor/__Tests/StellaOps.Excititor.WebService.Tests/StellaOps.Excititor.WebService.Tests.csproj" \
|
||||
"FullyQualifiedName~AirgapImportEndpointTests" \
|
||||
"excititor-airgapimport"
|
||||
|
||||
log "Done. TRX files in $TRX_DIR"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -1,11 +0,0 @@
|
||||
# CI-110 runner filters (Concelier/Excititor smoke)
|
||||
|
||||
## Concelier
|
||||
- WebService health: `HealthAndReadyEndpointsRespond`
|
||||
- Storage.Mongo job store: `FullyQualifiedName~MongoJobStore`
|
||||
- WebService orchestrator endpoints: TODO (tests not yet present; add to WebService.Tests then filter with `FullyQualifiedName~Orchestrator`)
|
||||
|
||||
## Excititor
|
||||
- WebService airgap import: `FullyQualifiedName~AirgapImportEndpointTests`
|
||||
|
||||
Artifacts are written under `ops/devops/artifacts/ci-110/<timestamp>/` by `run-ci-110.sh`.
|
||||
@@ -1,25 +0,0 @@
|
||||
# Concelier CI Runner Harness (DEVOPS-CONCELIER-CI-24-101)
|
||||
|
||||
Purpose: provide a deterministic, offline-friendly harness that restores, builds, and runs Concelier WebService + Storage Mongo tests with warmed NuGet cache and TRX/binlog artefacts for downstream sprints (Concelier II/III).
|
||||
|
||||
Usage
|
||||
- From repo root run: `ops/devops/concelier-ci-runner/run-concelier-ci.sh`
|
||||
- Outputs land in `ops/devops/artifacts/concelier-ci/<UTC timestamp>/`:
|
||||
- `build.binlog` (solution build)
|
||||
- `tests/webservice.trx`, `tests/storage.trx` (VSTest results)
|
||||
- per-project `.dmp`/logs if failures occur
|
||||
- `summary.json` (paths + hashes)
|
||||
|
||||
Environment
|
||||
- Defaults: `DOTNET_CLI_TELEMETRY_OPTOUT=1`, `DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1`, `NUGET_PACKAGES=$REPO/.nuget/packages`.
|
||||
- Uses `.nuget/packages` cache (can be overridden via `NUGET_SOURCES`).
|
||||
- No external services required; Mongo2Go provides ephemeral Mongo for tests.
|
||||
|
||||
What it does
|
||||
1) `dotnet restore` + `dotnet build` on `concelier-webservice.slnf` with `/bl`.
|
||||
3) Run WebService and Storage.Mongo test projects with TRX output and without rebuild (`--no-build`).
|
||||
4) Emit a concise `summary.json` listing artefacts and SHA256s for reproducibility.
|
||||
|
||||
Notes
|
||||
- Keep test filters narrow if you need faster runs; edit `TEST_FILTER` env var (default empty = run all tests).
|
||||
- Artefacts are timestamped UTC to keep ordering deterministic in pipelines; consumers should sort by path.
|
||||
@@ -1,73 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Concelier CI runner harness (DEVOPS-CONCELIER-CI-24-101)
|
||||
# Produces warmed-cache restore, build binlog, and TRX outputs for WebService + Storage Mongo tests.
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/../../.." && pwd)"
|
||||
ts="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
out_dir="$repo_root/ops/devops/artifacts/concelier-ci/$ts"
|
||||
logs_dir="$out_dir/tests"
|
||||
mkdir -p "$logs_dir"
|
||||
|
||||
# Deterministic env
|
||||
export DOTNET_CLI_TELEMETRY_OPTOUT=${DOTNET_CLI_TELEMETRY_OPTOUT:-1}
|
||||
export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=${DOTNET_SKIP_FIRST_TIME_EXPERIENCE:-1}
|
||||
export NUGET_PACKAGES=${NUGET_PACKAGES:-$repo_root/.nuget/packages}
|
||||
export NUGET_SOURCES=${NUGET_SOURCES:-"$repo_root/.nuget/packages"}
|
||||
export TEST_FILTER=${TEST_FILTER:-""}
|
||||
export DOTNET_RESTORE_DISABLE_PARALLEL=${DOTNET_RESTORE_DISABLE_PARALLEL:-1}
|
||||
|
||||
mkdir -p "$NUGET_PACKAGES"
|
||||
|
||||
# Restore with deterministic sources
|
||||
restore_sources=()
|
||||
IFS=';' read -ra SRC_ARR <<< "$NUGET_SOURCES"
|
||||
for s in "${SRC_ARR[@]}"; do
|
||||
[[ -n "$s" ]] && restore_sources+=(--source "$s")
|
||||
done
|
||||
|
||||
dotnet restore "$repo_root/concelier-webservice.slnf" --ignore-failed-sources "${restore_sources[@]}"
|
||||
|
||||
# Build with binlog
|
||||
build_binlog="$out_dir/build.binlog"
|
||||
dotnet build "$repo_root/concelier-webservice.slnf" -c Debug /p:ContinuousIntegrationBuild=true /bl:"$build_binlog"
|
||||
|
||||
common_test_args=( -c Debug --no-build --results-directory "$logs_dir" )
|
||||
if [[ -n "$TEST_FILTER" ]]; then
|
||||
common_test_args+=( --filter "$TEST_FILTER" )
|
||||
fi
|
||||
|
||||
# WebService tests
|
||||
web_trx="webservice.trx"
|
||||
dotnet test "$repo_root/src/Concelier/__Tests/StellaOps.Concelier.WebService.Tests/StellaOps.Concelier.WebService.Tests.csproj" \
|
||||
"${common_test_args[@]}" \
|
||||
--logger "trx;LogFileName=$web_trx"
|
||||
|
||||
# Storage Mongo tests
|
||||
storage_trx="storage.trx"
|
||||
dotnet test "$repo_root/src/Concelier/__Tests/StellaOps.Concelier.Storage.Mongo.Tests/StellaOps.Concelier.Storage.Mongo.Tests.csproj" \
|
||||
"${common_test_args[@]}" \
|
||||
--logger "trx;LogFileName=$storage_trx"
|
||||
|
||||
# Summarize artefacts (relative paths to repo root)
|
||||
summary="$out_dir/summary.json"
|
||||
{
|
||||
printf '{\n'
|
||||
printf ' "timestamp_utc": "%s",\n' "$ts"
|
||||
printf ' "build_binlog": "%s",\n' "${build_binlog#${repo_root}/}"
|
||||
printf ' "tests": [\n'
|
||||
printf ' {"project": "WebService", "trx": "%s"},\n' "${logs_dir#${repo_root}/}/$web_trx"
|
||||
printf ' {"project": "Storage.Mongo", "trx": "%s"}\n' "${logs_dir#${repo_root}/}/$storage_trx"
|
||||
printf ' ],\n'
|
||||
printf ' "nuget_packages": "%s",\n' "${NUGET_PACKAGES#${repo_root}/}"
|
||||
printf ' "sources": [\n'
|
||||
for i in "${!SRC_ARR[@]}"; do
|
||||
sep=","; [[ $i -eq $((${#SRC_ARR[@]}-1)) ]] && sep=""
|
||||
printf ' "%s"%s\n' "${SRC_ARR[$i]}" "$sep"
|
||||
done
|
||||
printf ' ]\n'
|
||||
printf '}\n'
|
||||
} > "$summary"
|
||||
|
||||
echo "Artifacts written to ${out_dir#${repo_root}/}"
|
||||
@@ -1,53 +0,0 @@
|
||||
# Concelier LNM Release Plan (DEVOPS-LNM-21-101-REL / 102-REL / 103-REL)
|
||||
|
||||
Scope: package and publish Link-Not-Merge migrations/backfill/object-store seeds for release and offline kits.
|
||||
|
||||
## Artefacts
|
||||
- Migration bundles:
|
||||
- 21-101 shard/index migrations (`EnsureLinkNotMergeShardingAndTtlMigration`)
|
||||
- 21-102 backfill/tombstone/rollback scripts
|
||||
- 21-103 object-store seed bundle (once contract final)
|
||||
- Checksums (`SHA256SUMS`, signed)
|
||||
- SBOMs (spdx.json) for migration runner image/tooling
|
||||
- Cosign attestations for images/bundles
|
||||
- Offline kit slice tarball with all above + DSSE manifest
|
||||
|
||||
## Pipeline outline
|
||||
1) Build migration runner image (dotnet) with migrations baked; generate SBOM; pin digest.
|
||||
2) Export migration scripts/bundles to `artifacts/lnm/`.
|
||||
3) Create offline bundle:
|
||||
- `migrations/21-101/` (DLLs, scripts, README)
|
||||
- `migrations/21-102/` (backfill, rollback, README)
|
||||
- `seeds/object-store/` (placeholder until 21-103 dev output)
|
||||
- `SHA256SUMS` + `.sig`
|
||||
- SBOMs + cosign attestations
|
||||
4) Verification stage:
|
||||
- `dotnet test` on migration runner
|
||||
- `cosign verify-blob` for bundles
|
||||
- `sha256sum --check`
|
||||
5) Publish:
|
||||
- Upload to release bucket + offline kit
|
||||
- Record manifest (hashes, versions, digests)
|
||||
|
||||
## Runbook (apply in staging → prod)
|
||||
1) Take Mongo backup; freeze Concelier ingest.
|
||||
2) Apply 21-101 migrations (shards/TTL) — idempotent; record duration.
|
||||
3) Run 21-102 backfill with `--batch-size=500 --stop-on-error`; capture report hash.
|
||||
4) Validate counts (observations/linksets/events) and shard balance.
|
||||
5) Enable outbox publishers; monitor lag and errors.
|
||||
6) (When ready) apply 21-103 object-store migration: move raw payloads to object store; verify CAS URIs; keep GridFS read-only during move.
|
||||
|
||||
## Rollback
|
||||
- 21-101: restore from backup if shard layout breaks; migrations are idempotent.
|
||||
- 21-102: run rollback script (`ops/devops/scripts/rollback-lnm-backfill.js`); if inconsistent, restore backup.
|
||||
- 21-103: switch back to GridFS URI map; restore seeds.
|
||||
|
||||
## Monitoring/alerts
|
||||
- Migration error count > 0
|
||||
- Mongo oplog lag > 60s during backfill
|
||||
- Outbox backlog growth post-unfreeze
|
||||
|
||||
## Owners
|
||||
- DevOps Guild (pipeline + rollout)
|
||||
- Concelier Storage Guild (migration content)
|
||||
- Platform Security (signing policy)
|
||||
@@ -1,38 +0,0 @@
|
||||
# syntax=docker/dockerfile:1.7
|
||||
# Offline-friendly console CI runner image with pre-baked npm and Playwright caches (DEVOPS-CONSOLE-23-001)
|
||||
ARG BASE_IMAGE=node:20-bookworm-slim
|
||||
ARG APP_DIR=src/Web/StellaOps.Web
|
||||
ARG SOURCE_DATE_EPOCH=1704067200
|
||||
|
||||
FROM ${BASE_IMAGE}
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive \
|
||||
NPM_CONFIG_FUND=false \
|
||||
NPM_CONFIG_AUDIT=false \
|
||||
NPM_CONFIG_PROGRESS=false \
|
||||
SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH} \
|
||||
PLAYWRIGHT_BROWSERS_PATH=/home/node/.cache/ms-playwright \
|
||||
NPM_CONFIG_CACHE=/home/node/.npm \
|
||||
CI=true
|
||||
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends git ca-certificates dumb-init wget curl && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /tmp/console-seed
|
||||
COPY ${APP_DIR}/package.json ${APP_DIR}/package-lock.json ./
|
||||
|
||||
ENV npm_config_cache=/tmp/npm-cache
|
||||
RUN npm ci --cache ${npm_config_cache} --prefer-offline --no-audit --progress=false --ignore-scripts && \
|
||||
PLAYWRIGHT_BROWSERS_PATH=/tmp/ms-playwright npx playwright install chromium --with-deps && \
|
||||
rm -rf node_modules
|
||||
|
||||
RUN install -d -o node -g node /home/node/.npm /home/node/.cache && \
|
||||
mv /tmp/npm-cache /home/node/.npm && \
|
||||
mv /tmp/ms-playwright /home/node/.cache/ms-playwright && \
|
||||
chown -R node:node /home/node/.npm /home/node/.cache
|
||||
|
||||
WORKDIR /workspace
|
||||
USER node
|
||||
ENTRYPOINT ["/usr/bin/dumb-init","--"]
|
||||
CMD ["/bin/bash"]
|
||||
@@ -1,44 +0,0 @@
|
||||
# Console CI runner (offline-friendly)
|
||||
|
||||
Status: runner spec + CI now wired to PRs; runner image scaffold + CI build workflow now available with baked npm + Playwright cache.
|
||||
|
||||
## Runner profile
|
||||
- OS: Ubuntu 22.04 LTS (x86_64) with Docker available for Playwright deps if needed.
|
||||
- Node: 20.x (LTS). Enable corepack; prefer npm (default) to avoid extra downloads.
|
||||
- Caches:
|
||||
- npm: `~/.npm` keyed by `src/Web/package-lock.json` hash.
|
||||
- Playwright: `~/.cache/ms-playwright` pre-seeded with Chromium so `npm test -- --browsers=ChromeHeadless` can run offline. Seed once using `npx playwright install chromium` on a connected runner, then snapshot the directory into the runner image.
|
||||
- Angular build cache: optional `~/.cache/angular` if using angular.json cache; safe to keep.
|
||||
- Artifacts retention: keep lint/test/build outputs 14 days; limit to 500 MB per run (coverage + dist + test reports). Artifacts path: `artifacts/` (dist, coverage, junit/trx if produced).
|
||||
|
||||
## Pipeline steps (expected)
|
||||
1) Checkout
|
||||
2) Node 20 setup with npm cache restore (package-lock at `src/Web/package-lock.json`).
|
||||
3) Install: `npm ci --prefer-offline --no-audit --progress=false` in `src/Web`.
|
||||
4) Lint: `npm run lint -- --no-progress`.
|
||||
5) Unit: `npm test -- --watch=false --browsers=ChromeHeadless --no-progress` (headless Chromium from pre-seeded cache).
|
||||
6) Build: `npm run build -- --configuration=production --progress=false`.
|
||||
7) Artifact collect: `dist/`, `coverage/`, any `test-results/**`.
|
||||
|
||||
## Offline/airgap notes
|
||||
- Do not hit external registries during CI; rely on pre-seeded npm mirror or cached tarballs. Runner image should contain npm cache prime. If mirror is used, set `NPM_CONFIG_REGISTRY=https://registry.npmjs.org` equivalent mirror URL inside the runner; default pipeline does not hard-code it.
|
||||
- Playwright browsers must be pre-baked; the workflow will not download them.
|
||||
|
||||
### Runner image (with baked caches)
|
||||
- Dockerfile: `ops/devops/console/Dockerfile.runner` (Node 20, npm cache, Playwright Chromium cache). Builds with `npm ci` + `playwright install chromium --with-deps` during the image build.
|
||||
- Build locally: `IMAGE_TAG=stellaops/console-runner:offline OUTPUT_TAR=ops/devops/artifacts/console-runner/console-runner.tar ops/devops/console/build-runner-image.sh`
|
||||
- `OUTPUT_TAR` optional; when set, the script saves the image for airgap transport.
|
||||
- Runner expectations: `NPM_CONFIG_CACHE=~/.npm`, `PLAYWRIGHT_BROWSERS_PATH=~/.cache/ms-playwright` (paths already baked). Register the runner with a label (e.g., `console-ci`) and point `.gitea/workflows/console-ci.yml` at that runner pool.
|
||||
- CI build helper: `ops/devops/console/build-runner-image-ci.sh` wraps the build, sets a run-scoped tag, emits metadata JSON, and saves a tarball under `ops/devops/artifacts/console-runner/`.
|
||||
- CI workflow: `.gitea/workflows/console-runner-image.yml` (manual + path-trigger) builds the runner image and uploads the tarball + metadata as an artifact named `console-runner-image-<run_id>`.
|
||||
|
||||
### Seeding Playwright cache (one-time per runner image, host-based option)
|
||||
```bash
|
||||
ops/devops/console/seed_playwright.sh
|
||||
# then bake ~/.cache/ms-playwright into the runner image or mount it on the agent
|
||||
```
|
||||
|
||||
## How to run
|
||||
- PR-triggered via `.gitea/workflows/console-ci.yml`; restrict runners to images with baked Playwright cache.
|
||||
- Manual `workflow_dispatch` remains available for dry runs or cache updates.
|
||||
- To refresh the runner image, run the `console-runner-image` workflow or execute `ops/devops/console/build-runner-image-ci.sh` locally to generate a tarball and metadata for distribution.
|
||||
@@ -1,86 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build console container image with SBOM and optional attestations
|
||||
# Usage: ./build-console-image.sh [tag] [registry]
|
||||
# Example: ./build-console-image.sh 2025.10.0-edge ghcr.io/stellaops
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
|
||||
|
||||
TAG="${1:-$(date +%Y%m%dT%H%M%S)}"
|
||||
REGISTRY="${2:-registry.stella-ops.org/stellaops}"
|
||||
IMAGE_NAME="console"
|
||||
FULL_IMAGE="${REGISTRY}/${IMAGE_NAME}:${TAG}"
|
||||
|
||||
# Freeze timestamps for reproducibility
|
||||
export SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH:-1704067200}
|
||||
|
||||
echo "==> Building console image: ${FULL_IMAGE}"
|
||||
|
||||
# Build using the existing Dockerfile.console
|
||||
docker build \
|
||||
--file "${REPO_ROOT}/ops/devops/docker/Dockerfile.console" \
|
||||
--build-arg APP_DIR=src/Web/StellaOps.Web \
|
||||
--build-arg APP_PORT=8080 \
|
||||
--tag "${FULL_IMAGE}" \
|
||||
--label "org.opencontainers.image.created=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
|
||||
--label "org.opencontainers.image.revision=$(git -C "${REPO_ROOT}" rev-parse HEAD 2>/dev/null || echo 'unknown')" \
|
||||
--label "org.opencontainers.image.source=https://github.com/stellaops/stellaops" \
|
||||
--label "org.opencontainers.image.title=StellaOps Console" \
|
||||
--label "org.opencontainers.image.description=StellaOps Angular Console (non-root nginx)" \
|
||||
"${REPO_ROOT}"
|
||||
|
||||
# Get digest
|
||||
DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' "${FULL_IMAGE}" 2>/dev/null || echo "${FULL_IMAGE}")
|
||||
|
||||
echo "==> Image built: ${FULL_IMAGE}"
|
||||
echo "==> Digest: ${DIGEST}"
|
||||
|
||||
# Output metadata for CI
|
||||
mkdir -p "${SCRIPT_DIR}/../artifacts/console"
|
||||
cat > "${SCRIPT_DIR}/../artifacts/console/build-metadata.json" <<EOF
|
||||
{
|
||||
"image": "${FULL_IMAGE}",
|
||||
"digest": "${DIGEST}",
|
||||
"tag": "${TAG}",
|
||||
"registry": "${REGISTRY}",
|
||||
"buildTime": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
|
||||
"gitCommit": "$(git -C "${REPO_ROOT}" rev-parse HEAD 2>/dev/null || echo 'unknown')",
|
||||
"sourceDateEpoch": "${SOURCE_DATE_EPOCH}"
|
||||
}
|
||||
EOF
|
||||
|
||||
echo "==> Build metadata written to ops/devops/artifacts/console/build-metadata.json"
|
||||
|
||||
# Generate SBOM if syft is available
|
||||
if command -v syft &>/dev/null; then
|
||||
echo "==> Generating SBOM..."
|
||||
syft "${FULL_IMAGE}" -o spdx-json > "${SCRIPT_DIR}/../artifacts/console/console.spdx.json"
|
||||
syft "${FULL_IMAGE}" -o cyclonedx-json > "${SCRIPT_DIR}/../artifacts/console/console.cdx.json"
|
||||
echo "==> SBOMs written to ops/devops/artifacts/console/"
|
||||
else
|
||||
echo "==> Skipping SBOM generation (syft not found)"
|
||||
fi
|
||||
|
||||
# Sign and attest if cosign is available and key is set
|
||||
if command -v cosign &>/dev/null; then
|
||||
if [[ -n "${COSIGN_KEY:-}" ]]; then
|
||||
echo "==> Signing image with cosign..."
|
||||
cosign sign --key "${COSIGN_KEY}" "${FULL_IMAGE}"
|
||||
|
||||
if [[ -f "${SCRIPT_DIR}/../artifacts/console/console.spdx.json" ]]; then
|
||||
echo "==> Attesting SBOM..."
|
||||
cosign attest --predicate "${SCRIPT_DIR}/../artifacts/console/console.spdx.json" \
|
||||
--type spdx --key "${COSIGN_KEY}" "${FULL_IMAGE}"
|
||||
fi
|
||||
echo "==> Image signed and attested"
|
||||
else
|
||||
echo "==> Skipping signing (COSIGN_KEY not set)"
|
||||
fi
|
||||
else
|
||||
echo "==> Skipping signing (cosign not found)"
|
||||
fi
|
||||
|
||||
echo "==> Console image build complete"
|
||||
echo " Image: ${FULL_IMAGE}"
|
||||
@@ -1,44 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# CI-friendly wrapper to build the console runner image with baked npm/Playwright caches
|
||||
# and emit a tarball + metadata for offline distribution.
|
||||
#
|
||||
# Inputs (env):
|
||||
# RUN_ID : unique run identifier (default: $GITHUB_RUN_ID or UTC timestamp)
|
||||
# IMAGE_TAG : optional override of image tag (default: stellaops/console-runner:offline-$RUN_ID)
|
||||
# OUTPUT_TAR : optional override of tarball path (default: ops/devops/artifacts/console-runner/console-runner-$RUN_ID.tar)
|
||||
# APP_DIR : optional override of app directory (default: src/Web/StellaOps.Web)
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
|
||||
RUN_ID="${RUN_ID:-${GITHUB_RUN_ID:-$(date -u +%Y%m%dT%H%M%SZ)}}"
|
||||
APP_DIR="${APP_DIR:-src/Web/StellaOps.Web}"
|
||||
IMAGE_TAG="${IMAGE_TAG:-stellaops/console-runner:offline-$RUN_ID}"
|
||||
OUTPUT_TAR="${OUTPUT_TAR:-$ROOT/ops/devops/artifacts/console-runner/console-runner-$RUN_ID.tar}"
|
||||
META_DIR="$(dirname "$OUTPUT_TAR")"
|
||||
META_JSON="$META_DIR/console-runner-$RUN_ID.json"
|
||||
|
||||
mkdir -p "$META_DIR"
|
||||
|
||||
IMAGE_TAG="$IMAGE_TAG" OUTPUT_TAR="$OUTPUT_TAR" APP_DIR="$APP_DIR" "$ROOT/ops/devops/console/build-runner-image.sh"
|
||||
|
||||
digest="$(docker image inspect --format='{{index .RepoDigests 0}}' "$IMAGE_TAG" || true)"
|
||||
id="$(docker image inspect --format='{{.Id}}' "$IMAGE_TAG" || true)"
|
||||
|
||||
cat > "$META_JSON" <<EOF
|
||||
{
|
||||
"run_id": "$RUN_ID",
|
||||
"image_tag": "$IMAGE_TAG",
|
||||
"image_id": "$id",
|
||||
"repo_digest": "$digest",
|
||||
"output_tar": "$(python - <<PY
|
||||
import os, sys
|
||||
print(os.path.relpath("$OUTPUT_TAR","$ROOT"))
|
||||
PY
|
||||
)"
|
||||
}
|
||||
EOF
|
||||
|
||||
echo "Built $IMAGE_TAG"
|
||||
echo "Saved tarball: $OUTPUT_TAR"
|
||||
echo "Metadata: $META_JSON"
|
||||
@@ -1,29 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Builds the offline console CI runner image with baked npm/Playwright caches.
|
||||
# IMAGE_TAG: docker tag to produce (default: stellaops/console-runner:offline)
|
||||
# OUTPUT_TAR: optional path to save the image tarball for airgap use.
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
|
||||
IMAGE_TAG=${IMAGE_TAG:-stellaops/console-runner:offline}
|
||||
DOCKERFILE=${DOCKERFILE:-ops/devops/console/Dockerfile.runner}
|
||||
APP_DIR=${APP_DIR:-src/Web/StellaOps.Web}
|
||||
OUTPUT_TAR=${OUTPUT_TAR:-}
|
||||
|
||||
if ! command -v docker >/dev/null 2>&1; then
|
||||
echo "docker not found; install Docker/Podman before building the runner image." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
docker build -f "$ROOT/$DOCKERFILE" --build-arg APP_DIR="$APP_DIR" -t "$IMAGE_TAG" "$ROOT"
|
||||
|
||||
if [[ -n "$OUTPUT_TAR" ]]; then
|
||||
mkdir -p "$(dirname "$OUTPUT_TAR")"
|
||||
docker save "$IMAGE_TAG" -o "$OUTPUT_TAR"
|
||||
fi
|
||||
|
||||
echo "Runner image built: $IMAGE_TAG"
|
||||
if [[ -n "$OUTPUT_TAR" ]]; then
|
||||
echo "Saved tarball: $OUTPUT_TAR"
|
||||
fi
|
||||
@@ -1,131 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Package console for offline/airgap deployment
|
||||
# Usage: ./package-offline-bundle.sh [image] [output-dir]
|
||||
# Example: ./package-offline-bundle.sh registry.stella-ops.org/stellaops/console:2025.10.0 ./offline-bundle
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
|
||||
|
||||
IMAGE="${1:-registry.stella-ops.org/stellaops/console:latest}"
|
||||
OUTPUT_DIR="${2:-${SCRIPT_DIR}/../artifacts/console/offline-bundle}"
|
||||
BUNDLE_NAME="console-offline-$(date +%Y%m%dT%H%M%S)"
|
||||
|
||||
# Freeze timestamps
|
||||
export SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH:-1704067200}
|
||||
|
||||
echo "==> Creating offline bundle for: ${IMAGE}"
|
||||
mkdir -p "${OUTPUT_DIR}"
|
||||
|
||||
# Save image as tarball
|
||||
IMAGE_TAR="${OUTPUT_DIR}/${BUNDLE_NAME}.tar"
|
||||
echo "==> Saving image to ${IMAGE_TAR}..."
|
||||
docker save "${IMAGE}" -o "${IMAGE_TAR}"
|
||||
|
||||
# Calculate checksums
|
||||
echo "==> Generating checksums..."
|
||||
cd "${OUTPUT_DIR}"
|
||||
sha256sum "${BUNDLE_NAME}.tar" > "${BUNDLE_NAME}.tar.sha256"
|
||||
|
||||
# Copy Helm values
|
||||
echo "==> Including Helm values overlay..."
|
||||
cp "${REPO_ROOT}/deploy/helm/stellaops/values-console.yaml" "${OUTPUT_DIR}/"
|
||||
|
||||
# Copy Dockerfile for reference
|
||||
cp "${REPO_ROOT}/ops/devops/docker/Dockerfile.console" "${OUTPUT_DIR}/"
|
||||
|
||||
# Generate SBOMs if syft available
|
||||
if command -v syft &>/dev/null; then
|
||||
echo "==> Generating SBOMs..."
|
||||
syft "${IMAGE}" -o spdx-json > "${OUTPUT_DIR}/${BUNDLE_NAME}.spdx.json"
|
||||
syft "${IMAGE}" -o cyclonedx-json > "${OUTPUT_DIR}/${BUNDLE_NAME}.cdx.json"
|
||||
fi
|
||||
|
||||
# Create manifest
|
||||
cat > "${OUTPUT_DIR}/manifest.json" <<EOF
|
||||
{
|
||||
"bundle": "${BUNDLE_NAME}",
|
||||
"image": "${IMAGE}",
|
||||
"imageTarball": "${BUNDLE_NAME}.tar",
|
||||
"checksumFile": "${BUNDLE_NAME}.tar.sha256",
|
||||
"helmValues": "values-console.yaml",
|
||||
"dockerfile": "Dockerfile.console",
|
||||
"sbom": {
|
||||
"spdx": "${BUNDLE_NAME}.spdx.json",
|
||||
"cyclonedx": "${BUNDLE_NAME}.cdx.json"
|
||||
},
|
||||
"createdAt": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
|
||||
"sourceDateEpoch": "${SOURCE_DATE_EPOCH}"
|
||||
}
|
||||
EOF
|
||||
|
||||
# Create load script
|
||||
cat > "${OUTPUT_DIR}/load.sh" <<'LOAD'
|
||||
#!/usr/bin/env bash
|
||||
# Load console image into local Docker daemon
|
||||
set -euo pipefail
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MANIFEST="${SCRIPT_DIR}/manifest.json"
|
||||
|
||||
if [[ ! -f "${MANIFEST}" ]]; then
|
||||
echo "ERROR: manifest.json not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
TARBALL=$(jq -r '.imageTarball' "${MANIFEST}")
|
||||
CHECKSUM_FILE=$(jq -r '.checksumFile' "${MANIFEST}")
|
||||
|
||||
echo "==> Verifying checksum..."
|
||||
cd "${SCRIPT_DIR}"
|
||||
sha256sum -c "${CHECKSUM_FILE}"
|
||||
|
||||
echo "==> Loading image..."
|
||||
docker load -i "${TARBALL}"
|
||||
|
||||
IMAGE=$(jq -r '.image' "${MANIFEST}")
|
||||
echo "==> Image loaded: ${IMAGE}"
|
||||
LOAD
|
||||
chmod +x "${OUTPUT_DIR}/load.sh"
|
||||
|
||||
# Create README
|
||||
cat > "${OUTPUT_DIR}/README.md" <<EOF
|
||||
# Console Offline Bundle
|
||||
|
||||
This bundle contains the StellaOps Console container image and deployment assets
|
||||
for air-gapped environments.
|
||||
|
||||
## Contents
|
||||
|
||||
- \`${BUNDLE_NAME}.tar\` - Docker image tarball
|
||||
- \`${BUNDLE_NAME}.tar.sha256\` - SHA-256 checksum
|
||||
- \`values-console.yaml\` - Helm values overlay
|
||||
- \`Dockerfile.console\` - Reference Dockerfile
|
||||
- \`${BUNDLE_NAME}.spdx.json\` - SPDX SBOM (if generated)
|
||||
- \`${BUNDLE_NAME}.cdx.json\` - CycloneDX SBOM (if generated)
|
||||
- \`manifest.json\` - Bundle manifest
|
||||
- \`load.sh\` - Image load helper script
|
||||
|
||||
## Usage
|
||||
|
||||
1. Transfer this bundle to the air-gapped environment
|
||||
2. Verify checksums: \`sha256sum -c ${BUNDLE_NAME}.tar.sha256\`
|
||||
3. Load image: \`./load.sh\` or \`docker load -i ${BUNDLE_NAME}.tar\`
|
||||
4. Deploy with Helm: \`helm install stellaops ../stellaops -f values-console.yaml\`
|
||||
|
||||
## Image Details
|
||||
|
||||
- Image: \`${IMAGE}\`
|
||||
- Created: $(date -u +%Y-%m-%dT%H:%M:%SZ)
|
||||
- Non-root user: UID 101 (nginx-unprivileged)
|
||||
- Port: 8080
|
||||
|
||||
## Verification
|
||||
|
||||
The image runs as non-root and supports read-only root filesystem.
|
||||
Enable \`readOnlyRootFilesystem: true\` in your security context.
|
||||
EOF
|
||||
|
||||
echo "==> Offline bundle created at: ${OUTPUT_DIR}"
|
||||
echo "==> Contents:"
|
||||
ls -la "${OUTPUT_DIR}"
|
||||
@@ -1,22 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Seeds the Playwright browser cache for offline console CI runs.
|
||||
# Run on a connected runner once, then bake ~/.cache/ms-playwright into the runner image.
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
pushd "$ROOT/src/Web" >/dev/null
|
||||
|
||||
if ! command -v npx >/dev/null; then
|
||||
echo "npx not found; install Node.js 20+ first" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Installing Playwright Chromium to ~/.cache/ms-playwright ..."
|
||||
PLAYWRIGHT_BROWSERS_PATH=${PLAYWRIGHT_BROWSERS_PATH:-~/.cache/ms-playwright}
|
||||
export PLAYWRIGHT_BROWSERS_PATH
|
||||
|
||||
npx playwright install chromium --with-deps
|
||||
|
||||
echo "Done. Cache directory: $PLAYWRIGHT_BROWSERS_PATH"
|
||||
popd >/dev/null
|
||||
@@ -1,25 +0,0 @@
|
||||
### Identity
|
||||
You are an autonomous software engineering agent for StellaOps working in the DevOps crypto services area.
|
||||
|
||||
### Roles
|
||||
- Document author
|
||||
- Backend developer (.NET 10)
|
||||
- Tester/QA automation engineer
|
||||
|
||||
### Required reading
|
||||
- docs/README.md
|
||||
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
|
||||
- docs/modules/devops/architecture.md
|
||||
|
||||
### Working agreements
|
||||
- Scope is limited to `devops/services/crypto/**` unless a sprint explicitly allows cross-module edits.
|
||||
- Keep outputs deterministic; inject time/ID providers and use invariant culture parsing.
|
||||
- Use ASCII-only strings in logs and comments unless explicitly required.
|
||||
- Respect offline-first posture; avoid hard-coded external dependencies.
|
||||
|
||||
### Testing
|
||||
- Add or update tests for any behavior change.
|
||||
- Tag tests with `[Trait("Category", "Unit")]` or `[Trait("Category", "Integration")]` as appropriate.
|
||||
|
||||
### Notes
|
||||
- These services are DevOps utilities; keep configuration explicit and validate options at startup.
|
||||
@@ -1,13 +0,0 @@
|
||||
FROM mcr.microsoft.com/dotnet/sdk:8.0-alpine AS build
|
||||
WORKDIR /src
|
||||
COPY SimCryptoService.csproj .
|
||||
RUN dotnet restore
|
||||
COPY . .
|
||||
RUN dotnet publish -c Release -o /app/publish
|
||||
|
||||
FROM mcr.microsoft.com/dotnet/aspnet:8.0-alpine
|
||||
WORKDIR /app
|
||||
COPY --from=build /app/publish .
|
||||
EXPOSE 8080
|
||||
ENV ASPNETCORE_URLS=http://0.0.0.0:8080
|
||||
ENTRYPOINT ["dotnet", "SimCryptoService.dll"]
|
||||
@@ -1,130 +0,0 @@
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using System.Text.Json.Serialization;
|
||||
|
||||
var builder = WebApplication.CreateBuilder(args);
|
||||
var app = builder.Build();
|
||||
|
||||
// Static key material for simulations (not for production use).
|
||||
using var ecdsa = ECDsa.Create(ECCurve.NamedCurves.nistP256);
|
||||
var ecdsaPublic = ecdsa.ExportSubjectPublicKeyInfo();
|
||||
|
||||
byte[] Sign(string message, string algorithm)
|
||||
{
|
||||
var data = Encoding.UTF8.GetBytes(message);
|
||||
var lower = algorithm.Trim().ToLowerInvariant();
|
||||
var upper = algorithm.Trim().ToUpperInvariant();
|
||||
|
||||
if (lower is "pq.dilithium3" or "pq.falcon512" or "pq.sim" || upper is "DILITHIUM3" or "FALCON512")
|
||||
{
|
||||
return HMACSHA256.HashData(Encoding.UTF8.GetBytes("pq-sim-key"), data);
|
||||
}
|
||||
|
||||
if (lower is "ru.magma.sim" or "ru.kuznyechik.sim" || upper is "GOST12-256" or "GOST12-512")
|
||||
{
|
||||
return HMACSHA256.HashData(Encoding.UTF8.GetBytes("gost-sim-key"), data);
|
||||
}
|
||||
|
||||
if (lower is "sm.sim" or "sm2.sim" || upper is "SM2")
|
||||
{
|
||||
return HMACSHA256.HashData(Encoding.UTF8.GetBytes("sm-sim-key"), data);
|
||||
}
|
||||
|
||||
return ecdsa.SignData(data, HashAlgorithmName.SHA256);
|
||||
}
|
||||
|
||||
bool Verify(string message, string algorithm, byte[] signature)
|
||||
{
|
||||
var data = Encoding.UTF8.GetBytes(message);
|
||||
var lower = algorithm.Trim().ToLowerInvariant();
|
||||
var upper = algorithm.Trim().ToUpperInvariant();
|
||||
|
||||
if (lower is "pq.dilithium3" or "pq.falcon512" or "pq.sim" || upper is "DILITHIUM3" or "FALCON512")
|
||||
{
|
||||
return CryptographicOperations.FixedTimeEquals(HMACSHA256.HashData(Encoding.UTF8.GetBytes("pq-sim-key"), data), signature);
|
||||
}
|
||||
|
||||
if (lower is "ru.magma.sim" or "ru.kuznyechik.sim" || upper is "GOST12-256" or "GOST12-512")
|
||||
{
|
||||
return CryptographicOperations.FixedTimeEquals(HMACSHA256.HashData(Encoding.UTF8.GetBytes("gost-sim-key"), data), signature);
|
||||
}
|
||||
|
||||
if (lower is "sm.sim" or "sm2.sim" || upper is "SM2")
|
||||
{
|
||||
return CryptographicOperations.FixedTimeEquals(HMACSHA256.HashData(Encoding.UTF8.GetBytes("sm-sim-key"), data), signature);
|
||||
}
|
||||
|
||||
return ecdsa.VerifyData(data, signature, HashAlgorithmName.SHA256);
|
||||
}
|
||||
|
||||
app.MapPost("/sign", (SignRequest request) =>
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(request.Algorithm) || string.IsNullOrWhiteSpace(request.Message))
|
||||
{
|
||||
return Results.BadRequest("Algorithm and message are required.");
|
||||
}
|
||||
|
||||
var sig = Sign(request.Message, request.Algorithm);
|
||||
return Results.Json(new SignResponse(Convert.ToBase64String(sig), request.Algorithm));
|
||||
});
|
||||
|
||||
app.MapPost("/verify", (VerifyRequest request) =>
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(request.Algorithm) || string.IsNullOrWhiteSpace(request.Message) || string.IsNullOrWhiteSpace(request.SignatureBase64))
|
||||
{
|
||||
return Results.BadRequest("Algorithm, message, and signature are required.");
|
||||
}
|
||||
|
||||
var sig = Convert.FromBase64String(request.SignatureBase64);
|
||||
var ok = Verify(request.Message, request.Algorithm, sig);
|
||||
return Results.Json(new VerifyResponse(ok, request.Algorithm));
|
||||
});
|
||||
|
||||
app.MapGet("/keys", () =>
|
||||
{
|
||||
return Results.Json(new KeysResponse(
|
||||
Convert.ToBase64String(ecdsaPublic),
|
||||
"nistp256",
|
||||
new[]
|
||||
{
|
||||
"pq.sim",
|
||||
"DILITHIUM3",
|
||||
"FALCON512",
|
||||
"ru.magma.sim",
|
||||
"ru.kuznyechik.sim",
|
||||
"GOST12-256",
|
||||
"GOST12-512",
|
||||
"sm.sim",
|
||||
"SM2",
|
||||
"fips.sim",
|
||||
"eidas.sim",
|
||||
"kcmvp.sim",
|
||||
"world.sim"
|
||||
}));
|
||||
});
|
||||
|
||||
app.Run();
|
||||
|
||||
public record SignRequest(
|
||||
[property: JsonPropertyName("message")] string Message,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
public record SignResponse(
|
||||
[property: JsonPropertyName("signature_b64")] string SignatureBase64,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
public record VerifyRequest(
|
||||
[property: JsonPropertyName("message")] string Message,
|
||||
[property: JsonPropertyName("signature_b64")] string SignatureBase64,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
public record VerifyResponse(
|
||||
[property: JsonPropertyName("ok")] bool Ok,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
public record KeysResponse(
|
||||
[property: JsonPropertyName("public_key_b64")] string PublicKeyBase64,
|
||||
[property: JsonPropertyName("curve")] string Curve,
|
||||
[property: JsonPropertyName("simulated_providers")] IEnumerable<string> Providers);
|
||||
|
||||
public partial class Program { }
|
||||
@@ -1,32 +0,0 @@
|
||||
# Sim Crypto Service · 2025-12-11
|
||||
|
||||
Minimal HTTP service to simulate sovereign crypto providers when licensed hardware or certified modules are unavailable.
|
||||
|
||||
## Endpoints
|
||||
- `POST /sign` — body: `{"message":"<string>","algorithm":"<id>"}`; returns `{"signature_b64":"...","algorithm":"<id>"}`.
|
||||
- `POST /verify` — body: `{"message":"<string>","algorithm":"<id>","signature_b64":"..."}`; returns `{"ok":true/false,"algorithm":"<id>"}`.
|
||||
- `GET /keys` — returns public key info for simulated providers.
|
||||
|
||||
## Supported simulated provider IDs
|
||||
- GOST: `GOST12-256`, `GOST12-512`, `ru.magma.sim`, `ru.kuznyechik.sim` — deterministic HMAC-SHA256.
|
||||
- SM: `SM2`, `sm.sim`, `sm2.sim` — deterministic HMAC-SHA256.
|
||||
- PQ: `DILITHIUM3`, `FALCON512`, `pq.sim` — deterministic HMAC-SHA256.
|
||||
- FIPS/eIDAS/KCMVP/world: `ES256`, `ES384`, `ES512`, `fips.sim`, `eidas.sim`, `kcmvp.sim`, `world.sim` — ECDSA P-256 with a static key.
|
||||
|
||||
## Build & run
|
||||
```bash
|
||||
dotnet run -c Release --project ops/crypto/sim-crypto-service/SimCryptoService.csproj
|
||||
# or
|
||||
docker build -t sim-crypto -f ops/crypto/sim-crypto-service/Dockerfile ops/crypto/sim-crypto-service
|
||||
docker run --rm -p 8080:8080 sim-crypto
|
||||
```
|
||||
|
||||
## Wiring
|
||||
- Set `STELLAOPS_CRYPTO_ENABLE_SIM=1` to append `sim.crypto.remote` to the registry preference order.
|
||||
- Point the provider at the service: `STELLAOPS_CRYPTO_SIM_URL=http://localhost:8080` (or bind `StellaOps:Crypto:Sim:BaseAddress` in config).
|
||||
- `SimRemoteProviderOptions.Algorithms` already includes the IDs above; extend if you need extra aliases.
|
||||
|
||||
## Notes
|
||||
- Replaces the legacy SM-only simulator; use this unified service for SM, PQ, GOST, and FIPS/eIDAS/KCMVP placeholders.
|
||||
- Deterministic HMAC for SM/PQ/GOST; static ECDSA key for the rest. Not for production use.
|
||||
- No licensed binaries are shipped; everything is BCL-only.
|
||||
@@ -1,13 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk.Web">
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<LangVersion>preview</LangVersion>
|
||||
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
|
||||
<AssetTargetFallback></AssetTargetFallback>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<Compile Remove="__Tests/**" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
@@ -1 +0,0 @@
|
||||
global using Xunit;
|
||||
@@ -1,20 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<IsTestProject>true</IsTestProject>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<LangVersion>preview</LangVersion>
|
||||
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<PackageReference Include="FluentAssertions" />
|
||||
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" />
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" />
|
||||
<PackageReference Include="xunit" />
|
||||
<PackageReference Include="xunit.runner.visualstudio" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\SimCryptoService.csproj" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
@@ -1,68 +0,0 @@
|
||||
using System.Net.Http.Json;
|
||||
using System.Text.Json.Serialization;
|
||||
using FluentAssertions;
|
||||
using Microsoft.AspNetCore.Mvc.Testing;
|
||||
|
||||
namespace SimCryptoService.Tests;
|
||||
|
||||
public sealed class SimCryptoServiceTests : IClassFixture<WebApplicationFactory<Program>>
|
||||
{
|
||||
private readonly WebApplicationFactory<Program> _factory;
|
||||
|
||||
public SimCryptoServiceTests(WebApplicationFactory<Program> factory)
|
||||
{
|
||||
_factory = factory;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task SignThenVerify_ReturnsOk()
|
||||
{
|
||||
using var client = _factory.CreateClient();
|
||||
var signResponse = await client.PostAsJsonAsync("/sign", new SignRequest("hello", "SM2"));
|
||||
signResponse.IsSuccessStatusCode.Should().BeTrue();
|
||||
|
||||
var signPayload = await signResponse.Content.ReadFromJsonAsync<SignResponse>();
|
||||
signPayload.Should().NotBeNull();
|
||||
signPayload!.SignatureBase64.Should().NotBeNullOrWhiteSpace();
|
||||
|
||||
var verifyResponse = await client.PostAsJsonAsync("/verify", new VerifyRequest("hello", signPayload.SignatureBase64, "SM2"));
|
||||
verifyResponse.IsSuccessStatusCode.Should().BeTrue();
|
||||
|
||||
var verifyPayload = await verifyResponse.Content.ReadFromJsonAsync<VerifyResponse>();
|
||||
verifyPayload.Should().NotBeNull();
|
||||
verifyPayload!.Ok.Should().BeTrue();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Keys_ReturnsAlgorithmsAndKey()
|
||||
{
|
||||
using var client = _factory.CreateClient();
|
||||
var response = await client.GetFromJsonAsync<KeysResponse>("/keys");
|
||||
response.Should().NotBeNull();
|
||||
response!.PublicKeyBase64.Should().NotBeNullOrWhiteSpace();
|
||||
response.SimulatedProviders.Should().Contain("SM2");
|
||||
response.SimulatedProviders.Should().Contain("GOST12-256");
|
||||
}
|
||||
|
||||
private sealed record SignRequest(
|
||||
[property: JsonPropertyName("message")] string Message,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
private sealed record SignResponse(
|
||||
[property: JsonPropertyName("signature_b64")] string SignatureBase64,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
private sealed record VerifyRequest(
|
||||
[property: JsonPropertyName("message")] string Message,
|
||||
[property: JsonPropertyName("signature_b64")] string SignatureBase64,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
private sealed record VerifyResponse(
|
||||
[property: JsonPropertyName("ok")] bool Ok,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
private sealed record KeysResponse(
|
||||
[property: JsonPropertyName("public_key_b64")] string PublicKeyBase64,
|
||||
[property: JsonPropertyName("curve")] string Curve,
|
||||
[property: JsonPropertyName("simulated_providers")] string[] SimulatedProviders);
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
var baseUrl = Environment.GetEnvironmentVariable("STELLAOPS_CRYPTO_SIM_URL") ?? "http://localhost:8080";
|
||||
var profile = (Environment.GetEnvironmentVariable("SIM_PROFILE") ?? "sm").ToLowerInvariant();
|
||||
var algList = SmokeLogic.ResolveAlgorithms(profile, Environment.GetEnvironmentVariable("SIM_ALGORITHMS"));
|
||||
var message = Environment.GetEnvironmentVariable("SIM_MESSAGE") ?? "stellaops-sim-smoke";
|
||||
|
||||
using var client = new HttpClient { BaseAddress = new Uri(baseUrl) };
|
||||
|
||||
var cts = new CancellationTokenSource(TimeSpan.FromSeconds(20));
|
||||
var failures = new List<string>();
|
||||
|
||||
foreach (var alg in algList)
|
||||
{
|
||||
var (ok, error) = await SmokeLogic.SignAndVerifyAsync(client, alg, message, cts.Token);
|
||||
if (!ok)
|
||||
{
|
||||
failures.Add($"{alg}: {error}");
|
||||
continue;
|
||||
}
|
||||
|
||||
Console.WriteLine($"[ok] {alg} via {baseUrl}");
|
||||
}
|
||||
|
||||
if (failures.Count > 0)
|
||||
{
|
||||
Console.Error.WriteLine("Simulation smoke failed:");
|
||||
foreach (var f in failures)
|
||||
{
|
||||
Console.Error.WriteLine($" - {f}");
|
||||
}
|
||||
|
||||
Environment.Exit(1);
|
||||
}
|
||||
|
||||
Console.WriteLine("Simulation smoke passed.");
|
||||
@@ -1,14 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<LangVersion>preview</LangVersion>
|
||||
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
|
||||
<AssetTargetFallback></AssetTargetFallback>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<Compile Remove="__Tests/**" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
@@ -1,72 +0,0 @@
|
||||
using System.Net.Http.Json;
|
||||
using System.Text.Json.Serialization;
|
||||
|
||||
public static class SmokeLogic
|
||||
{
|
||||
public static IReadOnlyList<string> ResolveAlgorithms(string profile, string? overrideList)
|
||||
{
|
||||
if (!string.IsNullOrWhiteSpace(overrideList))
|
||||
{
|
||||
return overrideList.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
|
||||
}
|
||||
|
||||
return profile switch
|
||||
{
|
||||
"ru-free" or "ru-paid" or "gost" or "ru" => new[] { "GOST12-256", "ru.magma.sim", "ru.kuznyechik.sim" },
|
||||
"sm" or "cn" => new[] { "SM2" },
|
||||
"eidas" => new[] { "ES256" },
|
||||
"fips" => new[] { "ES256" },
|
||||
"kcmvp" => new[] { "ES256" },
|
||||
"pq" => new[] { "pq.sim", "DILITHIUM3", "FALCON512" },
|
||||
_ => new[] { "ES256", "SM2", "pq.sim" }
|
||||
};
|
||||
}
|
||||
|
||||
public static async Task<(bool Ok, string Error)> SignAndVerifyAsync(HttpClient client, string algorithm, string message, CancellationToken ct)
|
||||
{
|
||||
var signPayload = new SignRequest(message, algorithm);
|
||||
var signResponse = await client.PostAsJsonAsync("/sign", signPayload, ct).ConfigureAwait(false);
|
||||
if (!signResponse.IsSuccessStatusCode)
|
||||
{
|
||||
return (false, $"sign failed: {(int)signResponse.StatusCode} {signResponse.ReasonPhrase}");
|
||||
}
|
||||
|
||||
var signResult = await signResponse.Content.ReadFromJsonAsync<SignResponse>(cancellationToken: ct).ConfigureAwait(false);
|
||||
if (signResult is null || string.IsNullOrWhiteSpace(signResult.SignatureBase64))
|
||||
{
|
||||
return (false, "sign returned empty payload");
|
||||
}
|
||||
|
||||
var verifyPayload = new VerifyRequest(message, signResult.SignatureBase64, algorithm);
|
||||
var verifyResponse = await client.PostAsJsonAsync("/verify", verifyPayload, ct).ConfigureAwait(false);
|
||||
if (!verifyResponse.IsSuccessStatusCode)
|
||||
{
|
||||
return (false, $"verify failed: {(int)verifyResponse.StatusCode} {verifyResponse.ReasonPhrase}");
|
||||
}
|
||||
|
||||
var verifyResult = await verifyResponse.Content.ReadFromJsonAsync<VerifyResponse>(cancellationToken: ct).ConfigureAwait(false);
|
||||
if (verifyResult?.Ok is not true)
|
||||
{
|
||||
return (false, "verify returned false");
|
||||
}
|
||||
|
||||
return (true, "");
|
||||
}
|
||||
|
||||
private sealed record SignRequest(
|
||||
[property: JsonPropertyName("message")] string Message,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
private sealed record SignResponse(
|
||||
[property: JsonPropertyName("signature_b64")] string SignatureBase64,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
private sealed record VerifyRequest(
|
||||
[property: JsonPropertyName("message")] string Message,
|
||||
[property: JsonPropertyName("signature_b64")] string SignatureBase64,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
|
||||
private sealed record VerifyResponse(
|
||||
[property: JsonPropertyName("ok")] bool Ok,
|
||||
[property: JsonPropertyName("algorithm")] string Algorithm);
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
global using Xunit;
|
||||
@@ -1,19 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<IsTestProject>true</IsTestProject>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<LangVersion>preview</LangVersion>
|
||||
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<PackageReference Include="FluentAssertions" />
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" />
|
||||
<PackageReference Include="xunit" />
|
||||
<PackageReference Include="xunit.runner.visualstudio" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\SimCryptoSmoke.csproj" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
@@ -1,65 +0,0 @@
|
||||
using System.Net;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using FluentAssertions;
|
||||
|
||||
namespace SimCryptoSmoke.Tests;
|
||||
|
||||
public sealed class SimCryptoSmokeTests
|
||||
{
|
||||
[Fact]
|
||||
public void ResolveAlgorithms_UsesProfileDefaults()
|
||||
{
|
||||
var algs = SmokeLogic.ResolveAlgorithms("gost", null);
|
||||
algs.Should().Contain("GOST12-256");
|
||||
algs.Should().Contain("ru.magma.sim");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ResolveAlgorithms_UsesOverrideList()
|
||||
{
|
||||
var algs = SmokeLogic.ResolveAlgorithms("sm", "ES256,SM2");
|
||||
algs.Should().ContainInOrder(new[] { "ES256", "SM2" });
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task SignAndVerifyAsync_ReturnsOk()
|
||||
{
|
||||
using var client = new HttpClient(new StubHandler())
|
||||
{
|
||||
BaseAddress = new Uri("http://localhost")
|
||||
};
|
||||
|
||||
var result = await SmokeLogic.SignAndVerifyAsync(client, "SM2", "hello", CancellationToken.None);
|
||||
result.Ok.Should().BeTrue();
|
||||
result.Error.Should().BeEmpty();
|
||||
}
|
||||
|
||||
private sealed class StubHandler : HttpMessageHandler
|
||||
{
|
||||
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
|
||||
{
|
||||
var path = request.RequestUri?.AbsolutePath ?? string.Empty;
|
||||
if (path.Equals("/sign", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
return Task.FromResult(BuildJsonResponse(new { signature_b64 = "c2ln", algorithm = "SM2" }));
|
||||
}
|
||||
|
||||
if (path.Equals("/verify", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
return Task.FromResult(BuildJsonResponse(new { ok = true, algorithm = "SM2" }));
|
||||
}
|
||||
|
||||
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.NotFound));
|
||||
}
|
||||
|
||||
private static HttpResponseMessage BuildJsonResponse(object payload)
|
||||
{
|
||||
var json = JsonSerializer.Serialize(payload, new JsonSerializerOptions(JsonSerializerDefaults.Web));
|
||||
return new HttpResponseMessage(HttpStatusCode.OK)
|
||||
{
|
||||
Content = new StringContent(json, Encoding.UTF8, "application/json")
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,25 +0,0 @@
|
||||
### Identity
|
||||
You are an autonomous software engineering agent for StellaOps working in the DevOps CryptoPro service area.
|
||||
|
||||
### Roles
|
||||
- Document author
|
||||
- Backend developer (.NET 10)
|
||||
- Tester/QA automation engineer
|
||||
|
||||
### Required reading
|
||||
- docs/README.md
|
||||
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
|
||||
- docs/modules/devops/architecture.md
|
||||
|
||||
### Working agreements
|
||||
- Scope is limited to `devops/services/cryptopro/**` unless a sprint explicitly allows cross-module edits.
|
||||
- Keep outputs deterministic; inject time/ID providers and use invariant culture parsing.
|
||||
- Use ASCII-only strings in logs and comments unless explicitly required.
|
||||
- Respect offline-first posture; avoid hard-coded external dependencies.
|
||||
|
||||
### Testing
|
||||
- Add or update tests for any behavior change.
|
||||
- Tag tests with `[Trait("Category", "Unit")]` or `[Trait("Category", "Integration")]` as appropriate.
|
||||
|
||||
### Notes
|
||||
- This service targets licensed CryptoPro tooling; keep configuration explicit and validate options at startup.
|
||||
@@ -1,185 +0,0 @@
|
||||
#!/bin/bash
|
||||
# CryptoPro CSP 5.0 R3 Linux installer (deb packages)
|
||||
# Uses locally provided .deb packages under /opt/cryptopro/downloads (host volume).
|
||||
# No Wine dependency. Runs offline against the supplied packages only.
|
||||
#
|
||||
# Env:
|
||||
# CRYPTOPRO_INSTALL_FROM Path to folder with .deb packages (default /opt/cryptopro/downloads)
|
||||
# CRYPTOPRO_ACCEPT_EULA Must be 1 to proceed (default 0 -> hard stop with warning)
|
||||
# CRYPTOPRO_SKIP_APT_FIX Set to 1 to skip `apt-get -f install` (offline strict)
|
||||
# CRYPTOPRO_PACKAGE_FILTER Optional glob (e.g., "cprocsp*amd64.deb") to narrow selection
|
||||
#
|
||||
# Exit codes:
|
||||
# 0 success; 1 missing dir/files; 2 incompatible arch; 3 EULA not accepted.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
INSTALL_FROM="${CRYPTOPRO_INSTALL_FROM:-/opt/cryptopro/downloads}"
|
||||
PACKAGE_FILTER="${CRYPTOPRO_PACKAGE_FILTER:-*.deb}"
|
||||
SKIP_APT_FIX="${CRYPTOPRO_SKIP_APT_FIX:-0}"
|
||||
STAGING_DIR="/tmp/cryptopro-debs"
|
||||
MINIMAL="${CRYPTOPRO_MINIMAL:-1}"
|
||||
INCLUDE_PLUGIN="${CRYPTOPRO_INCLUDE_PLUGIN:-0}"
|
||||
|
||||
arch_from_uname() {
|
||||
local raw
|
||||
raw="$(uname -m)"
|
||||
case "${raw}" in
|
||||
x86_64) echo "amd64" ;;
|
||||
aarch64) echo "arm64" ;;
|
||||
arm64) echo "arm64" ;;
|
||||
i386|i686) echo "i386" ;;
|
||||
*) echo "${raw}" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
HOST_ARCH="$(dpkg --print-architecture 2>/dev/null || arch_from_uname)"
|
||||
|
||||
log() {
|
||||
echo "[$(date -u '+%Y-%m-%dT%H:%M:%SZ')] [cryptopro-install] $*"
|
||||
}
|
||||
|
||||
log_err() {
|
||||
echo "[$(date -u '+%Y-%m-%dT%H:%M:%SZ')] [cryptopro-install] [ERROR] $*" >&2
|
||||
}
|
||||
|
||||
require_eula() {
|
||||
if [[ "${CRYPTOPRO_ACCEPT_EULA:-0}" != "1" ]]; then
|
||||
log_err "License not accepted. Set CRYPTOPRO_ACCEPT_EULA=1 only if you hold a valid CryptoPro license for these binaries and agree to the vendor EULA."
|
||||
exit 3
|
||||
fi
|
||||
}
|
||||
|
||||
maybe_extract_bundle() {
|
||||
# Prefer a bundle that matches host arch in filename, otherwise first *.tgz
|
||||
mapfile -t TGZ < <(find "${INSTALL_FROM}" -maxdepth 1 -type f -name "*.tgz" -print 2>/dev/null | sort)
|
||||
if [[ ${#TGZ[@]} -eq 0 ]]; then
|
||||
return
|
||||
fi
|
||||
local chosen=""
|
||||
for candidate in "${TGZ[@]}"; do
|
||||
if [[ "${candidate}" == *"${HOST_ARCH}"* ]]; then
|
||||
chosen="${candidate}"
|
||||
break
|
||||
fi
|
||||
done
|
||||
if [[ -z "${chosen}" ]]; then
|
||||
chosen="${TGZ[0]}"
|
||||
fi
|
||||
log "Extracting bundle ${chosen} into ${STAGING_DIR}"
|
||||
rm -rf "${STAGING_DIR}"
|
||||
mkdir -p "${STAGING_DIR}"
|
||||
tar -xf "${chosen}" -C "${STAGING_DIR}"
|
||||
# If bundle contains a single subfolder, use it as install root
|
||||
local subdir
|
||||
subdir="$(find "${STAGING_DIR}" -maxdepth 1 -type d ! -path "${STAGING_DIR}" | head -n1)"
|
||||
if [[ -n "${subdir}" ]]; then
|
||||
INSTALL_FROM="${subdir}"
|
||||
else
|
||||
INSTALL_FROM="${STAGING_DIR}"
|
||||
fi
|
||||
}
|
||||
|
||||
gather_packages() {
|
||||
if [[ ! -d "${INSTALL_FROM}" ]]; then
|
||||
log_err "Package directory not found: ${INSTALL_FROM}"
|
||||
exit 1
|
||||
fi
|
||||
maybe_extract_bundle
|
||||
mapfile -t PKGS < <(find "${INSTALL_FROM}" -maxdepth 2 -type f -name "${PACKAGE_FILTER}" -print 2>/dev/null | sort)
|
||||
if [[ ${#PKGS[@]} -eq 0 ]]; then
|
||||
log_err "No .deb packages found in ${INSTALL_FROM} (filter=${PACKAGE_FILTER})"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
apply_minimal_filter() {
|
||||
if [[ "${MINIMAL}" != "1" ]]; then
|
||||
return
|
||||
fi
|
||||
local -a keep_exact=(
|
||||
"lsb-cprocsp-base"
|
||||
"lsb-cprocsp-ca-certs"
|
||||
"lsb-cprocsp-capilite-64"
|
||||
"lsb-cprocsp-kc1-64"
|
||||
"lsb-cprocsp-pkcs11-64"
|
||||
"lsb-cprocsp-rdr-64"
|
||||
"cprocsp-curl-64"
|
||||
"cprocsp-pki-cades-64"
|
||||
"cprocsp-compat-debian"
|
||||
)
|
||||
if [[ "${INCLUDE_PLUGIN}" == "1" ]]; then
|
||||
keep_exact+=("cprocsp-pki-plugin-64" "cprocsp-rdr-gui-gtk-64")
|
||||
fi
|
||||
local -a filtered=()
|
||||
for pkg in "${PKGS[@]}"; do
|
||||
local name
|
||||
name="$(dpkg-deb -f "${pkg}" Package 2>/dev/null || basename "${pkg}")"
|
||||
for wanted in "${keep_exact[@]}"; do
|
||||
if [[ "${name}" == "${wanted}" ]]; then
|
||||
filtered+=("${pkg}")
|
||||
break
|
||||
fi
|
||||
done
|
||||
done
|
||||
if [[ ${#filtered[@]} -gt 0 ]]; then
|
||||
log "Applying minimal package set (CRYPTOPRO_MINIMAL=1); kept ${#filtered[@]} of ${#PKGS[@]}"
|
||||
PKGS=("${filtered[@]}")
|
||||
else
|
||||
log "Minimal filter yielded no matches; using full package set"
|
||||
fi
|
||||
}
|
||||
|
||||
filter_by_arch() {
|
||||
FILTERED=()
|
||||
for pkg in "${PKGS[@]}"; do
|
||||
local pkg_arch
|
||||
pkg_arch="$(dpkg-deb -f "${pkg}" Architecture 2>/dev/null || echo "unknown")"
|
||||
if [[ "${pkg_arch}" == "all" || "${pkg_arch}" == "${HOST_ARCH}" ]]; then
|
||||
FILTERED+=("${pkg}")
|
||||
else
|
||||
log "Skipping ${pkg} (arch=${pkg_arch}, host=${HOST_ARCH})"
|
||||
fi
|
||||
done
|
||||
if [[ ${#FILTERED[@]} -eq 0 ]]; then
|
||||
log_err "No packages match host architecture ${HOST_ARCH}"
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
print_matrix() {
|
||||
log "Discovered packages (arch filter: host=${HOST_ARCH}):"
|
||||
for pkg in "${FILTERED[@]}"; do
|
||||
local name ver arch
|
||||
name="$(dpkg-deb -f "${pkg}" Package 2>/dev/null || basename "${pkg}")"
|
||||
ver="$(dpkg-deb -f "${pkg}" Version 2>/dev/null || echo "unknown")"
|
||||
arch="$(dpkg-deb -f "${pkg}" Architecture 2>/dev/null || echo "unknown")"
|
||||
echo " - ${name} ${ver} (${arch}) <- ${pkg}"
|
||||
done
|
||||
}
|
||||
|
||||
install_packages() {
|
||||
log "Installing ${#FILTERED[@]} package(s) from ${INSTALL_FROM}"
|
||||
if ! dpkg -i "${FILTERED[@]}"; then
|
||||
if [[ "${SKIP_APT_FIX}" == "1" ]]; then
|
||||
log_err "dpkg reported errors and CRYPTOPRO_SKIP_APT_FIX=1; aborting."
|
||||
exit 1
|
||||
fi
|
||||
log "Resolving dependencies with apt-get -f install (may require network if deps missing locally)"
|
||||
apt-get update >/dev/null
|
||||
DEBIAN_FRONTEND=noninteractive apt-get -y -f install
|
||||
fi
|
||||
log "CryptoPro packages installed. Verify with: dpkg -l | grep cprocsp"
|
||||
}
|
||||
|
||||
main() {
|
||||
require_eula
|
||||
gather_packages
|
||||
apply_minimal_filter
|
||||
filter_by_arch
|
||||
print_matrix
|
||||
install_packages
|
||||
log "Installation finished. For headless/server use on Ubuntu 22.04 (amd64), the 'linux-amd64_deb.tgz' bundle is preferred and auto-selected."
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -1,16 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk.Web">
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<PublishSingleFile>true</PublishSingleFile>
|
||||
<SelfContained>true</SelfContained>
|
||||
<RuntimeIdentifier>linux-x64</RuntimeIdentifier>
|
||||
<InvariantGlobalization>true</InvariantGlobalization>
|
||||
<EnableTrimAnalyzer>false</EnableTrimAnalyzer>
|
||||
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<Compile Remove="__Tests/**" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
@@ -1,36 +0,0 @@
|
||||
# syntax=docker/dockerfile:1.7
|
||||
FROM mcr.microsoft.com/dotnet/nightly/sdk:10.0 AS build
|
||||
WORKDIR /src
|
||||
COPY ops/cryptopro/linux-csp-service/CryptoProLinuxApi.csproj .
|
||||
RUN dotnet restore CryptoProLinuxApi.csproj
|
||||
COPY ops/cryptopro/linux-csp-service/ .
|
||||
RUN dotnet publish CryptoProLinuxApi.csproj -c Release -r linux-x64 --self-contained true \
|
||||
/p:PublishSingleFile=true /p:DebugType=none /p:DebugSymbols=false -o /app/publish
|
||||
|
||||
FROM ubuntu:22.04
|
||||
|
||||
ARG CRYPTOPRO_ACCEPT_EULA=0
|
||||
ENV DEBIAN_FRONTEND=noninteractive \
|
||||
CRYPTOPRO_ACCEPT_EULA=${CRYPTOPRO_ACCEPT_EULA} \
|
||||
CRYPTOPRO_MINIMAL=1
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# System deps for CryptoPro installer
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends tar xz-utils ca-certificates && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# CryptoPro packages (provided in repo) and installer
|
||||
COPY opt/cryptopro/downloads/*.tgz /opt/cryptopro/downloads/
|
||||
COPY ops/cryptopro/install-linux-csp.sh /usr/local/bin/install-linux-csp.sh
|
||||
RUN chmod +x /usr/local/bin/install-linux-csp.sh
|
||||
|
||||
# Install CryptoPro CSP (requires CRYPTOPRO_ACCEPT_EULA=1 at build/runtime)
|
||||
RUN CRYPTOPRO_ACCEPT_EULA=${CRYPTOPRO_ACCEPT_EULA} /usr/local/bin/install-linux-csp.sh
|
||||
|
||||
# Copy published .NET app
|
||||
COPY --from=build /app/publish/ /app/
|
||||
|
||||
EXPOSE 8080
|
||||
ENTRYPOINT ["/app/CryptoProLinuxApi"]
|
||||
@@ -1,120 +0,0 @@
|
||||
using System.Diagnostics;
|
||||
using System.Text.Json.Serialization;
|
||||
|
||||
var builder = WebApplication.CreateSlimBuilder(args);
|
||||
builder.Services.ConfigureHttpJsonOptions(opts =>
|
||||
{
|
||||
opts.SerializerOptions.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull;
|
||||
});
|
||||
|
||||
var app = builder.Build();
|
||||
|
||||
const string CsptestPath = "/opt/cprocsp/bin/amd64/csptest";
|
||||
|
||||
app.MapGet("/health", () =>
|
||||
{
|
||||
if (!File.Exists(CsptestPath))
|
||||
{
|
||||
return Results.Problem(statusCode: 500, detail: "csptest not found; ensure CryptoPro CSP is installed");
|
||||
}
|
||||
|
||||
return Results.Ok(new { status = "ok", csptest = CsptestPath });
|
||||
});
|
||||
|
||||
app.MapGet("/license", () =>
|
||||
{
|
||||
var result = RunProcess([CsptestPath, "-keyset", "-info"], allowFailure: true);
|
||||
return Results.Json(result);
|
||||
});
|
||||
|
||||
app.MapPost("/hash", async (HashRequest request) =>
|
||||
{
|
||||
byte[] data;
|
||||
try
|
||||
{
|
||||
data = Convert.FromBase64String(request.DataBase64);
|
||||
}
|
||||
catch (FormatException)
|
||||
{
|
||||
return Results.BadRequest(new { error = "Invalid base64" });
|
||||
}
|
||||
|
||||
var inputPath = Path.GetTempFileName();
|
||||
var outputPath = Path.GetTempFileName();
|
||||
await File.WriteAllBytesAsync(inputPath, data);
|
||||
|
||||
var result = RunProcess([CsptestPath, "-hash", "-alg", "GOST12_256", "-in", inputPath, "-out", outputPath], allowFailure: true);
|
||||
string? digestBase64 = null;
|
||||
if (File.Exists(outputPath))
|
||||
{
|
||||
var digestBytes = await File.ReadAllBytesAsync(outputPath);
|
||||
digestBase64 = Convert.ToBase64String(digestBytes);
|
||||
}
|
||||
|
||||
TryDelete(inputPath);
|
||||
TryDelete(outputPath);
|
||||
|
||||
return Results.Json(new
|
||||
{
|
||||
result.ExitCode,
|
||||
result.Output,
|
||||
digest_b64 = digestBase64
|
||||
});
|
||||
});
|
||||
|
||||
app.MapPost("/keyset/init", (KeysetRequest request) =>
|
||||
{
|
||||
var name = string.IsNullOrWhiteSpace(request.Name) ? "default" : request.Name!;
|
||||
var result = RunProcess([CsptestPath, "-keyset", "-newkeyset", "-container", name, "-keytype", "none"], allowFailure: true);
|
||||
return Results.Json(result);
|
||||
});
|
||||
|
||||
app.Run("http://0.0.0.0:8080");
|
||||
|
||||
static void TryDelete(string path)
|
||||
{
|
||||
try { File.Delete(path); } catch { /* ignore */ }
|
||||
}
|
||||
|
||||
static ProcessResult RunProcess(string[] args, bool allowFailure = false)
|
||||
{
|
||||
try
|
||||
{
|
||||
var psi = new ProcessStartInfo
|
||||
{
|
||||
FileName = args[0],
|
||||
RedirectStandardOutput = true,
|
||||
RedirectStandardError = true,
|
||||
UseShellExecute = false,
|
||||
ArgumentList = { }
|
||||
};
|
||||
for (var i = 1; i < args.Length; i++)
|
||||
{
|
||||
psi.ArgumentList.Add(args[i]);
|
||||
}
|
||||
|
||||
using var proc = Process.Start(psi)!;
|
||||
var output = proc.StandardOutput.ReadToEnd();
|
||||
output += proc.StandardError.ReadToEnd();
|
||||
proc.WaitForExit();
|
||||
if (proc.ExitCode != 0 && !allowFailure)
|
||||
{
|
||||
throw new InvalidOperationException($"Command failed with exit {proc.ExitCode}: {output}");
|
||||
}
|
||||
return new ProcessResult(proc.ExitCode, output);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
if (!allowFailure)
|
||||
{
|
||||
throw;
|
||||
}
|
||||
return new ProcessResult(-1, ex.ToString());
|
||||
}
|
||||
}
|
||||
|
||||
sealed record HashRequest([property: JsonPropertyName("data_b64")] string DataBase64);
|
||||
sealed record KeysetRequest([property: JsonPropertyName("name")] string? Name);
|
||||
sealed record ProcessResult(int ExitCode, string Output);
|
||||
|
||||
public partial class Program { }
|
||||
@@ -1,33 +0,0 @@
|
||||
# CryptoPro Linux CSP Service (.NET minimal API)
|
||||
|
||||
Minimal HTTP wrapper around the Linux CryptoPro CSP binaries to prove installation and hash operations.
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
docker build -t cryptopro-linux-csp -f ops/cryptopro/linux-csp-service/Dockerfile .
|
||||
```
|
||||
|
||||
`CRYPTOPRO_ACCEPT_EULA` defaults to `0` (build will fail); set to `1` only if you hold a valid CryptoPro license and accept the vendor EULA:
|
||||
|
||||
```bash
|
||||
docker build -t cryptopro-linux-csp \
|
||||
--build-arg CRYPTOPRO_ACCEPT_EULA=1 \
|
||||
-f ops/cryptopro/linux-csp-service/Dockerfile .
|
||||
```
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
docker run --rm -p 18080:8080 --name cryptopro-linux-csp-test cryptopro-linux-csp
|
||||
```
|
||||
|
||||
Endpoints:
|
||||
- `GET /health` — checks `csptest` presence.
|
||||
- `GET /license` — runs `csptest -keyset -info` (reports errors if no keyset/token present).
|
||||
- `POST /hash` with `{"data_b64":"<base64>"}` — hashes using `csptest -hash -alg GOST12_256`.
|
||||
- `POST /keyset/init` with optional `{"name":"<container>"}` — creates an empty keyset (`-keytype none`) to silence missing-container warnings.
|
||||
|
||||
Notes:
|
||||
- Uses the provided CryptoPro `.tgz` bundles under `opt/cryptopro/downloads`. Do not set `CRYPTOPRO_ACCEPT_EULA=1` unless you are licensed to use these binaries.
|
||||
- Minimal, headless install; browser/plugin packages are not included.
|
||||
@@ -1,20 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<IsTestProject>true</IsTestProject>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<LangVersion>preview</LangVersion>
|
||||
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<PackageReference Include="FluentAssertions" />
|
||||
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" />
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" />
|
||||
<PackageReference Include="xunit" />
|
||||
<PackageReference Include="xunit.runner.visualstudio" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\CryptoProLinuxApi.csproj" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
@@ -1,77 +0,0 @@
|
||||
using System.Net;
|
||||
using System.Net.Http.Json;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using FluentAssertions;
|
||||
using Microsoft.AspNetCore.Mvc.Testing;
|
||||
|
||||
namespace CryptoProLinuxApi.Tests;
|
||||
|
||||
public sealed class CryptoProLinuxApiTests : IClassFixture<WebApplicationFactory<Program>>
|
||||
{
|
||||
private readonly HttpClient _client;
|
||||
|
||||
public CryptoProLinuxApiTests(WebApplicationFactory<Program> factory)
|
||||
{
|
||||
_client = factory.CreateClient();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Health_ReportsStatus()
|
||||
{
|
||||
var response = await _client.GetAsync("/health");
|
||||
if (response.StatusCode == HttpStatusCode.OK)
|
||||
{
|
||||
using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync());
|
||||
doc.RootElement.GetProperty("status").GetString().Should().Be("ok");
|
||||
doc.RootElement.GetProperty("csptest").GetString().Should().NotBeNullOrWhiteSpace();
|
||||
return;
|
||||
}
|
||||
|
||||
response.StatusCode.Should().Be(HttpStatusCode.InternalServerError);
|
||||
var body = await response.Content.ReadAsStringAsync();
|
||||
body.Contains("csptest", StringComparison.OrdinalIgnoreCase).Should().BeTrue();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task License_ReturnsResultShape()
|
||||
{
|
||||
var response = await _client.GetAsync("/license");
|
||||
response.IsSuccessStatusCode.Should().BeTrue();
|
||||
|
||||
using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync());
|
||||
doc.RootElement.GetProperty("exitCode").ValueKind.Should().Be(JsonValueKind.Number);
|
||||
doc.RootElement.GetProperty("output").ValueKind.Should().Be(JsonValueKind.String);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Hash_InvalidBase64_ReturnsBadRequest()
|
||||
{
|
||||
var response = await _client.PostAsJsonAsync("/hash", new { data_b64 = "not-base64" });
|
||||
response.StatusCode.Should().Be(HttpStatusCode.BadRequest);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Hash_ValidBase64_ReturnsResultShape()
|
||||
{
|
||||
var payload = Convert.ToBase64String(Encoding.UTF8.GetBytes("test"));
|
||||
var response = await _client.PostAsJsonAsync("/hash", new { data_b64 = payload });
|
||||
response.IsSuccessStatusCode.Should().BeTrue();
|
||||
|
||||
using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync());
|
||||
doc.RootElement.GetProperty("exitCode").ValueKind.Should().Be(JsonValueKind.Number);
|
||||
doc.RootElement.GetProperty("output").ValueKind.Should().Be(JsonValueKind.String);
|
||||
doc.RootElement.GetProperty("digest_b64").ValueKind.Should().BeOneOf(JsonValueKind.Null, JsonValueKind.String);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task KeysetInit_ReturnsResultShape()
|
||||
{
|
||||
var response = await _client.PostAsJsonAsync("/keyset/init", new { name = "test" });
|
||||
response.IsSuccessStatusCode.Should().BeTrue();
|
||||
|
||||
using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync());
|
||||
doc.RootElement.GetProperty("exitCode").ValueKind.Should().Be(JsonValueKind.Number);
|
||||
doc.RootElement.GetProperty("output").ValueKind.Should().Be(JsonValueKind.String);
|
||||
}
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
global using Xunit;
|
||||
@@ -1,21 +0,0 @@
|
||||
# DevPortal Build & Offline — Agent Charter
|
||||
|
||||
## Mission
|
||||
Automate deterministic developer portal builds (online/offline), enforce accessibility/performance budgets, and publish nightly offline bundles with checksums and provenance.
|
||||
|
||||
## Scope
|
||||
- CI pipeline for `devportal` (pnpm install, lint, type-check, unit, a11y, Lighthouse perf, caching).
|
||||
- Offline/nightly build (`devportal --offline`) with artifact retention and checksum manifest.
|
||||
- Accessibility checks (axe/pa11y) and link checking for docs/content.
|
||||
- Performance budgets via Lighthouse (P95) recorded per commit.
|
||||
|
||||
## Working Agreements
|
||||
- Use pnpm with a locked store; no network during build steps beyond configured registries/mirrors.
|
||||
- Keep outputs deterministic: pinned deps, `NODE_OPTIONS=--enable-source-maps`, UTC timestamps.
|
||||
- Artifacts stored under `out/devportal/<run-id>` with `SHA256SUMS` manifest.
|
||||
- Update sprint entries when task states change; record evidence bundle paths in Execution Log.
|
||||
|
||||
## Required Reading
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
- `docs/modules/devops/architecture.md`
|
||||
- `docs/modules/ui/architecture.md`
|
||||
@@ -1,32 +0,0 @@
|
||||
groups:
|
||||
- name: evidence-locker
|
||||
rules:
|
||||
- alert: EvidenceLockerRetentionDrift
|
||||
expr: evidence_retention_days != 180
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
team: devops
|
||||
annotations:
|
||||
summary: "Evidence locker retention drift"
|
||||
description: "Configured retention {{ $value }}d differs from target 180d."
|
||||
|
||||
- alert: EvidenceLockerWormDisabled
|
||||
expr: evidence_worm_enabled == 0
|
||||
for: 5m
|
||||
labels:
|
||||
severity: critical
|
||||
team: devops
|
||||
annotations:
|
||||
summary: "WORM/immutability disabled"
|
||||
description: "Evidence locker WORM not enabled."
|
||||
|
||||
- alert: EvidenceLockerBackupLag
|
||||
expr: (time() - evidence_last_backup_seconds) > 3600
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
team: devops
|
||||
annotations:
|
||||
summary: "Evidence locker backup lag > 1h"
|
||||
description: "Last backup older than 1 hour."
|
||||
@@ -1,23 +0,0 @@
|
||||
{
|
||||
"title": "Evidence Locker",
|
||||
"time": { "from": "now-24h", "to": "now" },
|
||||
"panels": [
|
||||
{
|
||||
"type": "stat",
|
||||
"title": "WORM enabled",
|
||||
"targets": [{ "expr": "evidence_worm_enabled" }]
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"title": "Retention days",
|
||||
"targets": [{ "expr": "evidence_retention_days" }]
|
||||
},
|
||||
{
|
||||
"type": "stat",
|
||||
"title": "Backup lag (seconds)",
|
||||
"targets": [{ "expr": "time() - evidence_last_backup_seconds" }]
|
||||
}
|
||||
],
|
||||
"schemaVersion": 39,
|
||||
"version": 1
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
version: '3.8'
|
||||
services:
|
||||
minio:
|
||||
image: minio/minio:RELEASE.2024-10-08T09-56-18Z
|
||||
command: server /data --console-address ":9001"
|
||||
environment:
|
||||
MINIO_ROOT_USER: exportci
|
||||
MINIO_ROOT_PASSWORD: exportci123
|
||||
ports:
|
||||
- "9000:9000"
|
||||
- "9001:9001"
|
||||
volumes:
|
||||
- minio-data:/data
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
volumes:
|
||||
minio-data:
|
||||
driver: local
|
||||
@@ -1,23 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
MINIO_ENDPOINT=${MINIO_ENDPOINT:-http://localhost:9000}
|
||||
MINIO_ACCESS_KEY=${MINIO_ACCESS_KEY:-exportci}
|
||||
MINIO_SECRET_KEY=${MINIO_SECRET_KEY:-exportci123}
|
||||
BUCKET=${BUCKET:-export-ci}
|
||||
TMP=$(mktemp)
|
||||
cleanup(){ rm -f "$TMP"; }
|
||||
trap cleanup EXIT
|
||||
|
||||
cat > "$TMP" <<'DATA'
|
||||
{"id":"exp-001","object":"s3://export-ci/sample-export.ndjson","status":"ready"}
|
||||
DATA
|
||||
|
||||
export AWS_ACCESS_KEY_ID="$MINIO_ACCESS_KEY"
|
||||
export AWS_SECRET_ACCESS_KEY="$MINIO_SECRET_KEY"
|
||||
export AWS_EC2_METADATA_DISABLED=true
|
||||
|
||||
if ! aws --endpoint-url "$MINIO_ENDPOINT" s3 ls "s3://$BUCKET" >/dev/null 2>&1; then
|
||||
aws --endpoint-url "$MINIO_ENDPOINT" s3 mb "s3://$BUCKET"
|
||||
fi
|
||||
aws --endpoint-url "$MINIO_ENDPOINT" s3 cp "$TMP" "s3://$BUCKET/sample-export.ndjson"
|
||||
echo "Seeded $BUCKET/sample-export.ndjson"
|
||||
@@ -1,51 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
# Smoke tests for Trivy compatibility and OCI distribution for Export Center.
|
||||
ROOT=${ROOT:-$(cd "$(dirname "$0")/../.." && pwd)}
|
||||
ARTifacts=${ARTifacts:-$ROOT/out/export-smoke}
|
||||
mkdir -p "$ARTifacts"
|
||||
|
||||
# 1) Trivy DB import compatibility
|
||||
TRIVY_VERSION="0.52.2"
|
||||
TRIVY_BIN="$ARTifacts/trivy"
|
||||
if [[ ! -x "$TRIVY_BIN" ]]; then
|
||||
curl -fsSL "https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz" -o "$ARTifacts/trivy.tgz"
|
||||
tar -xzf "$ARTifacts/trivy.tgz" -C "$ARTifacts" trivy
|
||||
fi
|
||||
"$TRIVY_BIN" module db import --help > "$ARTifacts/trivy-import-help.txt"
|
||||
|
||||
# 2) OCI distribution check (local registry)
|
||||
REGISTRY_PORT=${REGISTRY_PORT:-5005}
|
||||
REGISTRY_DIR="$ARTifacts/registry"
|
||||
mkdir -p "$REGISTRY_DIR"
|
||||
podman run --rm -d -p "${REGISTRY_PORT}:5000" --name export-registry -v "$REGISTRY_DIR":/var/lib/registry registry:2
|
||||
trap 'podman rm -f export-registry >/dev/null 2>&1 || true' EXIT
|
||||
sleep 2
|
||||
|
||||
echo '{"schemaVersion":2,"manifests":[]}' > "$ARTifacts/empty-index.json"
|
||||
DIGEST=$(sha256sum "$ARTifacts/empty-index.json" | awk '{print $1}')
|
||||
mkdir -p "$ARTifacts/blobs/sha256"
|
||||
cp "$ARTifacts/empty-index.json" "$ARTifacts/blobs/sha256/$DIGEST"
|
||||
|
||||
# Push blob and manifest via curl
|
||||
cat > "$ARTifacts/manifest.json" <<JSON
|
||||
{
|
||||
"schemaVersion": 2,
|
||||
"mediaType": "application/vnd.oci.image.manifest.v1+json",
|
||||
"config": {
|
||||
"mediaType": "application/vnd.oci.image.config.v1+json",
|
||||
"size": 2,
|
||||
"digest": "sha256:d4735e3a265e16eee03f59718b9b5d03d68c8ffa19c2f8f71b66e08d6c6f2c1a"
|
||||
},
|
||||
"layers": []
|
||||
}
|
||||
JSON
|
||||
MAN_DIGEST=$(sha256sum "$ARTifacts/manifest.json" | awk '{print $1}')
|
||||
|
||||
curl -sSf -X PUT "http://localhost:${REGISTRY_PORT}/v2/export-smoke/blobs/uploads/" -H 'Content-Length: 0' -o "$ARTifacts/upload-location.txt"
|
||||
UPLOAD_URL=$(cat "$ARTifacts/upload-location.txt" | tr -d '\r\n')
|
||||
|
||||
curl -sSf -X PUT "${UPLOAD_URL}?digest=sha256:${MAN_DIGEST}" --data-binary "@$ARTifacts/manifest.json"
|
||||
|
||||
curl -sSf "http://localhost:${REGISTRY_PORT}/v2/export-smoke/manifests/sha256:${MAN_DIGEST}" -o "$ARTifacts/manifest.pull.json"
|
||||
echo "trivy smoke + oci registry ok" > "$ARTifacts/result.txt"
|
||||
@@ -1,42 +0,0 @@
|
||||
groups:
|
||||
- name: exporter
|
||||
rules:
|
||||
- alert: ExporterThroughputLow
|
||||
expr: rate(exporter_jobs_processed_total[5m]) < 1
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
team: devops
|
||||
annotations:
|
||||
summary: "Exporter throughput low"
|
||||
description: "Processed <1 job/s over last 5m (current {{ $value }})."
|
||||
|
||||
- alert: ExporterFailuresHigh
|
||||
expr: rate(exporter_jobs_failed_total[5m]) / rate(exporter_jobs_processed_total[5m]) > 0.02
|
||||
for: 5m
|
||||
labels:
|
||||
severity: critical
|
||||
team: devops
|
||||
annotations:
|
||||
summary: "Exporter failure rate >2%"
|
||||
description: "Failure rate {{ $value | humanizePercentage }} over last 5m."
|
||||
|
||||
- alert: ExporterLatencyP95High
|
||||
expr: histogram_quantile(0.95, sum(rate(exporter_job_duration_seconds_bucket[5m])) by (le)) > 3
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
team: devops
|
||||
annotations:
|
||||
summary: "Exporter job p95 latency high"
|
||||
description: "Job p95 latency {{ $value }}s over last 5m (threshold 3s)."
|
||||
|
||||
- alert: ExporterQueueDepthHigh
|
||||
expr: exporter_queue_depth > 500
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
team: devops
|
||||
annotations:
|
||||
summary: "Exporter queue depth high"
|
||||
description: "Queue depth {{ $value }} exceeds 500 for >10m."
|
||||
@@ -1,29 +0,0 @@
|
||||
{
|
||||
"title": "Exporter Overview",
|
||||
"time": { "from": "now-24h", "to": "now" },
|
||||
"panels": [
|
||||
{
|
||||
"type": "stat",
|
||||
"title": "Queue depth",
|
||||
"targets": [{ "expr": "exporter_queue_depth" }]
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"title": "Jobs processed / failed",
|
||||
"targets": [
|
||||
{ "expr": "rate(exporter_jobs_processed_total[5m])", "legendFormat": "processed" },
|
||||
{ "expr": "rate(exporter_jobs_failed_total[5m])", "legendFormat": "failed" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"title": "Job duration p50/p95",
|
||||
"targets": [
|
||||
{ "expr": "histogram_quantile(0.5, sum(rate(exporter_job_duration_seconds_bucket[5m])) by (le))", "legendFormat": "p50" },
|
||||
{ "expr": "histogram_quantile(0.95, sum(rate(exporter_job_duration_seconds_bucket[5m])) by (le))", "legendFormat": "p95" }
|
||||
]
|
||||
}
|
||||
],
|
||||
"schemaVersion": 39,
|
||||
"version": 1
|
||||
}
|
||||
@@ -1,64 +0,0 @@
|
||||
# Findings Ledger Docker Compose overlay
|
||||
# Append to or reference from your main compose file
|
||||
#
|
||||
# Usage:
|
||||
# docker compose -f docker-compose.yaml -f ops/devops/findings-ledger/compose/docker-compose.ledger.yaml up -d
|
||||
|
||||
services:
|
||||
findings-ledger:
|
||||
image: stellaops/findings-ledger:${STELLA_VERSION:-2025.11.0}
|
||||
restart: unless-stopped
|
||||
env_file:
|
||||
- ./env/ledger.${STELLAOPS_ENV:-dev}.env
|
||||
environment:
|
||||
ASPNETCORE_URLS: http://0.0.0.0:8080
|
||||
ASPNETCORE_ENVIRONMENT: ${ASPNETCORE_ENVIRONMENT:-Production}
|
||||
# Database connection (override via env file or secrets)
|
||||
# LEDGER__DB__CONNECTIONSTRING: see secrets
|
||||
# Observability
|
||||
LEDGER__OBSERVABILITY__ENABLED: "true"
|
||||
LEDGER__OBSERVABILITY__OTLPENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4317}
|
||||
# Merkle anchoring
|
||||
LEDGER__MERKLE__ANCHORINTERVAL: "00:05:00"
|
||||
LEDGER__MERKLE__EXTERNALIZE: ${LEDGER_MERKLE_EXTERNALIZE:-false}
|
||||
# Attachments
|
||||
LEDGER__ATTACHMENTS__MAXSIZEBYTES: "104857600" # 100MB
|
||||
LEDGER__ATTACHMENTS__ALLOWEGRESS: ${LEDGER_ATTACHMENTS_ALLOWEGRESS:-true}
|
||||
ports:
|
||||
- "${LEDGER_PORT:-8188}:8080"
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health/ready"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 15s
|
||||
volumes:
|
||||
- ledger-data:/app/data
|
||||
- ./etc/ledger/appsettings.json:/app/appsettings.json:ro
|
||||
networks:
|
||||
- stellaops
|
||||
|
||||
# Migration job (run before starting ledger)
|
||||
findings-ledger-migrations:
|
||||
image: stellaops/findings-ledger-migrations:${STELLA_VERSION:-2025.11.0}
|
||||
command: ["--connection", "${LEDGER__DB__CONNECTIONSTRING}"]
|
||||
env_file:
|
||||
- ./env/ledger.${STELLAOPS_ENV:-dev}.env
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- stellaops
|
||||
profiles:
|
||||
- migrations
|
||||
|
||||
volumes:
|
||||
ledger-data:
|
||||
driver: local
|
||||
|
||||
networks:
|
||||
stellaops:
|
||||
external: true
|
||||
@@ -1,24 +0,0 @@
|
||||
# Findings Ledger - Development Environment
|
||||
# Copy to ledger.local.env and customize for local dev
|
||||
|
||||
# Database connection
|
||||
LEDGER__DB__CONNECTIONSTRING=Host=postgres;Port=5432;Database=findings_ledger_dev;Username=ledger;Password=change_me_dev;
|
||||
|
||||
# Attachment encryption key (AES-256, base64 encoded)
|
||||
# Generate with: openssl rand -base64 32
|
||||
LEDGER__ATTACHMENTS__ENCRYPTIONKEY=
|
||||
|
||||
# Merkle anchor signing (optional in dev)
|
||||
LEDGER__MERKLE__SIGNINGKEY=
|
||||
|
||||
# Authority service endpoint (for JWT validation)
|
||||
LEDGER__AUTHORITY__BASEURL=http://authority:8080
|
||||
|
||||
# Logging level
|
||||
Logging__LogLevel__Default=Debug
|
||||
Logging__LogLevel__Microsoft=Information
|
||||
Logging__LogLevel__StellaOps=Debug
|
||||
|
||||
# Feature flags
|
||||
LEDGER__FEATURES__ENABLEATTACHMENTS=true
|
||||
LEDGER__FEATURES__ENABLEAUDITLOG=true
|
||||
@@ -1,40 +0,0 @@
|
||||
# Findings Ledger - Production Environment
|
||||
# Secrets should be injected from secrets manager, not committed
|
||||
|
||||
# Database connection (inject from secrets manager)
|
||||
# LEDGER__DB__CONNECTIONSTRING=
|
||||
|
||||
# Attachment encryption key (inject from secrets manager)
|
||||
# LEDGER__ATTACHMENTS__ENCRYPTIONKEY=
|
||||
|
||||
# Merkle anchor signing (inject from secrets manager)
|
||||
# LEDGER__MERKLE__SIGNINGKEY=
|
||||
|
||||
# Authority service endpoint
|
||||
LEDGER__AUTHORITY__BASEURL=http://authority:8080
|
||||
|
||||
# Logging level
|
||||
Logging__LogLevel__Default=Warning
|
||||
Logging__LogLevel__Microsoft=Warning
|
||||
Logging__LogLevel__StellaOps=Information
|
||||
|
||||
# Feature flags
|
||||
LEDGER__FEATURES__ENABLEATTACHMENTS=true
|
||||
LEDGER__FEATURES__ENABLEAUDITLOG=true
|
||||
|
||||
# Observability
|
||||
LEDGER__OBSERVABILITY__ENABLED=true
|
||||
LEDGER__OBSERVABILITY__METRICSPORT=9090
|
||||
|
||||
# Merkle anchoring
|
||||
LEDGER__MERKLE__ANCHORINTERVAL=00:05:00
|
||||
LEDGER__MERKLE__EXTERNALIZE=false
|
||||
|
||||
# Attachments
|
||||
LEDGER__ATTACHMENTS__MAXSIZEBYTES=104857600
|
||||
LEDGER__ATTACHMENTS__ALLOWEGRESS=false
|
||||
|
||||
# Air-gap staleness thresholds (seconds)
|
||||
LEDGER__AIRGAP__ADVISORYSTALETHRESHOLD=604800
|
||||
LEDGER__AIRGAP__VEXSTALETHRESHOLD=604800
|
||||
LEDGER__AIRGAP__POLICYSTALETHRESHOLD=86400
|
||||
@@ -1,20 +0,0 @@
|
||||
apiVersion: v2
|
||||
name: stellaops-findings-ledger
|
||||
version: 0.1.0
|
||||
appVersion: "2025.11.0"
|
||||
description: Findings Ledger service for StellaOps platform - event-sourced findings storage with Merkle anchoring.
|
||||
type: application
|
||||
keywords:
|
||||
- findings
|
||||
- ledger
|
||||
- event-sourcing
|
||||
- merkle
|
||||
- attestation
|
||||
maintainers:
|
||||
- name: StellaOps Team
|
||||
email: platform@stellaops.io
|
||||
dependencies:
|
||||
- name: postgresql
|
||||
version: "14.x"
|
||||
repository: https://charts.bitnami.com/bitnami
|
||||
condition: postgresql.enabled
|
||||
@@ -1,80 +0,0 @@
|
||||
{{/*
|
||||
Expand the name of the chart.
|
||||
*/}}
|
||||
{{- define "findings-ledger.name" -}}
|
||||
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Create a default fully qualified app name.
|
||||
*/}}
|
||||
{{- define "findings-ledger.fullname" -}}
|
||||
{{- if .Values.fullnameOverride }}
|
||||
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
|
||||
{{- else }}
|
||||
{{- $name := default .Chart.Name .Values.nameOverride }}
|
||||
{{- if contains $name .Release.Name }}
|
||||
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
|
||||
{{- else }}
|
||||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Create chart name and version as used by the chart label.
|
||||
*/}}
|
||||
{{- define "findings-ledger.chart" -}}
|
||||
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Common labels
|
||||
*/}}
|
||||
{{- define "findings-ledger.labels" -}}
|
||||
helm.sh/chart: {{ include "findings-ledger.chart" . }}
|
||||
{{ include "findings-ledger.selectorLabels" . }}
|
||||
{{- if .Chart.AppVersion }}
|
||||
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
|
||||
{{- end }}
|
||||
app.kubernetes.io/managed-by: {{ .Release.Service }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Selector labels
|
||||
*/}}
|
||||
{{- define "findings-ledger.selectorLabels" -}}
|
||||
app.kubernetes.io/name: {{ include "findings-ledger.name" . }}
|
||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
app.kubernetes.io/component: ledger
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Create the name of the service account to use
|
||||
*/}}
|
||||
{{- define "findings-ledger.serviceAccountName" -}}
|
||||
{{- if .Values.serviceAccount.create }}
|
||||
{{- default (include "findings-ledger.fullname" .) .Values.serviceAccount.name }}
|
||||
{{- else }}
|
||||
{{- default "default" .Values.serviceAccount.name }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Database connection string - from secret or constructed
|
||||
*/}}
|
||||
{{- define "findings-ledger.databaseConnectionString" -}}
|
||||
{{- if .Values.database.connectionStringSecret }}
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ .Values.database.connectionStringSecret }}
|
||||
key: {{ .Values.database.connectionStringKey }}
|
||||
{{- else if .Values.postgresql.enabled }}
|
||||
value: "Host={{ .Release.Name }}-postgresql;Port=5432;Database={{ .Values.postgresql.auth.database }};Username={{ .Values.postgresql.auth.username }};Password=$(POSTGRES_PASSWORD);"
|
||||
{{- else }}
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ .Values.secrets.name }}
|
||||
key: LEDGER__DB__CONNECTIONSTRING
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -1,19 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: {{ include "findings-ledger.fullname" . }}-config
|
||||
labels:
|
||||
{{- include "findings-ledger.labels" . | nindent 4 }}
|
||||
data:
|
||||
appsettings.json: |
|
||||
{
|
||||
"Logging": {
|
||||
"LogLevel": {
|
||||
"Default": "Information",
|
||||
"Microsoft": "Warning",
|
||||
"Microsoft.Hosting.Lifetime": "Information",
|
||||
"StellaOps": "Information"
|
||||
}
|
||||
},
|
||||
"AllowedHosts": "*"
|
||||
}
|
||||
@@ -1,122 +0,0 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {{ include "findings-ledger.fullname" . }}
|
||||
labels:
|
||||
{{- include "findings-ledger.labels" . | nindent 4 }}
|
||||
spec:
|
||||
replicas: {{ .Values.replicaCount }}
|
||||
selector:
|
||||
matchLabels:
|
||||
{{- include "findings-ledger.selectorLabels" . | nindent 6 }}
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
|
||||
labels:
|
||||
{{- include "findings-ledger.selectorLabels" . | nindent 8 }}
|
||||
spec:
|
||||
serviceAccountName: {{ include "findings-ledger.serviceAccountName" . }}
|
||||
securityContext:
|
||||
{{- toYaml .Values.podSecurityContext | nindent 8 }}
|
||||
containers:
|
||||
- name: ledger
|
||||
securityContext:
|
||||
{{- toYaml .Values.securityContext | nindent 12 }}
|
||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: {{ .Values.service.port }}
|
||||
protocol: TCP
|
||||
{{- if .Values.observability.metricsEnabled }}
|
||||
- name: metrics
|
||||
containerPort: {{ .Values.service.metricsPort }}
|
||||
protocol: TCP
|
||||
{{- end }}
|
||||
env:
|
||||
- name: ASPNETCORE_URLS
|
||||
value: "http://0.0.0.0:{{ .Values.service.port }}"
|
||||
- name: ASPNETCORE_ENVIRONMENT
|
||||
value: "Production"
|
||||
# Database
|
||||
- name: LEDGER__DB__CONNECTIONSTRING
|
||||
{{- include "findings-ledger.databaseConnectionString" . | nindent 14 }}
|
||||
# Observability
|
||||
- name: LEDGER__OBSERVABILITY__ENABLED
|
||||
value: {{ .Values.observability.enabled | quote }}
|
||||
- name: LEDGER__OBSERVABILITY__OTLPENDPOINT
|
||||
value: {{ .Values.observability.otlpEndpoint | quote }}
|
||||
# Merkle anchoring
|
||||
- name: LEDGER__MERKLE__ANCHORINTERVAL
|
||||
value: {{ .Values.merkle.anchorInterval | quote }}
|
||||
- name: LEDGER__MERKLE__EXTERNALIZE
|
||||
value: {{ .Values.merkle.externalize | quote }}
|
||||
# Attachments
|
||||
- name: LEDGER__ATTACHMENTS__MAXSIZEBYTES
|
||||
value: {{ .Values.attachments.maxSizeBytes | quote }}
|
||||
- name: LEDGER__ATTACHMENTS__ALLOWEGRESS
|
||||
value: {{ .Values.attachments.allowEgress | quote }}
|
||||
- name: LEDGER__ATTACHMENTS__ENCRYPTIONKEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: {{ .Values.secrets.name }}
|
||||
key: LEDGER__ATTACHMENTS__ENCRYPTIONKEY
|
||||
# Authority
|
||||
- name: LEDGER__AUTHORITY__BASEURL
|
||||
value: {{ .Values.authority.baseUrl | quote }}
|
||||
# Air-gap thresholds
|
||||
- name: LEDGER__AIRGAP__ADVISORYSTALETHRESHOLD
|
||||
value: {{ .Values.airgap.advisoryStaleThreshold | quote }}
|
||||
- name: LEDGER__AIRGAP__VEXSTALETHRESHOLD
|
||||
value: {{ .Values.airgap.vexStaleThreshold | quote }}
|
||||
- name: LEDGER__AIRGAP__POLICYSTALETHRESHOLD
|
||||
value: {{ .Values.airgap.policyStaleThreshold | quote }}
|
||||
# Features
|
||||
- name: LEDGER__FEATURES__ENABLEATTACHMENTS
|
||||
value: {{ .Values.features.enableAttachments | quote }}
|
||||
- name: LEDGER__FEATURES__ENABLEAUDITLOG
|
||||
value: {{ .Values.features.enableAuditLog | quote }}
|
||||
{{- with .Values.extraEnv }}
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
{{- with .Values.extraEnvFrom }}
|
||||
envFrom:
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: {{ .Values.probes.readiness.path }}
|
||||
port: http
|
||||
initialDelaySeconds: {{ .Values.probes.readiness.initialDelaySeconds }}
|
||||
periodSeconds: {{ .Values.probes.readiness.periodSeconds }}
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: {{ .Values.probes.liveness.path }}
|
||||
port: http
|
||||
initialDelaySeconds: {{ .Values.probes.liveness.initialDelaySeconds }}
|
||||
periodSeconds: {{ .Values.probes.liveness.periodSeconds }}
|
||||
resources:
|
||||
{{- toYaml .Values.resources | nindent 12 }}
|
||||
volumeMounts:
|
||||
- name: tmp
|
||||
mountPath: /tmp
|
||||
- name: data
|
||||
mountPath: /app/data
|
||||
volumes:
|
||||
- name: tmp
|
||||
emptyDir: {}
|
||||
- name: data
|
||||
emptyDir: {}
|
||||
{{- with .Values.nodeSelector }}
|
||||
nodeSelector:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.affinity }}
|
||||
affinity:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- with .Values.tolerations }}
|
||||
tolerations:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
@@ -1,43 +0,0 @@
|
||||
{{- if .Values.migrations.enabled }}
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: {{ include "findings-ledger.fullname" . }}-migrations
|
||||
labels:
|
||||
{{- include "findings-ledger.labels" . | nindent 4 }}
|
||||
app.kubernetes.io/component: migrations
|
||||
annotations:
|
||||
"helm.sh/hook": pre-install,pre-upgrade
|
||||
"helm.sh/hook-weight": "-5"
|
||||
"helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
|
||||
spec:
|
||||
backoffLimit: 3
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
{{- include "findings-ledger.selectorLabels" . | nindent 8 }}
|
||||
app.kubernetes.io/component: migrations
|
||||
spec:
|
||||
serviceAccountName: {{ include "findings-ledger.serviceAccountName" . }}
|
||||
securityContext:
|
||||
{{- toYaml .Values.podSecurityContext | nindent 8 }}
|
||||
restartPolicy: Never
|
||||
containers:
|
||||
- name: migrations
|
||||
securityContext:
|
||||
{{- toYaml .Values.securityContext | nindent 12 }}
|
||||
image: "{{ .Values.migrations.image.repository }}:{{ .Values.migrations.image.tag }}"
|
||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||
args:
|
||||
- "--connection"
|
||||
- "$(LEDGER__DB__CONNECTIONSTRING)"
|
||||
env:
|
||||
- name: LEDGER__DB__CONNECTIONSTRING
|
||||
{{- include "findings-ledger.databaseConnectionString" . | nindent 14 }}
|
||||
resources:
|
||||
{{- toYaml .Values.migrations.resources | nindent 12 }}
|
||||
{{- with .Values.nodeSelector }}
|
||||
nodeSelector:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -1,21 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{ include "findings-ledger.fullname" . }}
|
||||
labels:
|
||||
{{- include "findings-ledger.labels" . | nindent 4 }}
|
||||
spec:
|
||||
type: {{ .Values.service.type }}
|
||||
ports:
|
||||
- port: {{ .Values.service.port }}
|
||||
targetPort: http
|
||||
protocol: TCP
|
||||
name: http
|
||||
{{- if .Values.observability.metricsEnabled }}
|
||||
- port: {{ .Values.service.metricsPort }}
|
||||
targetPort: metrics
|
||||
protocol: TCP
|
||||
name: metrics
|
||||
{{- end }}
|
||||
selector:
|
||||
{{- include "findings-ledger.selectorLabels" . | nindent 4 }}
|
||||
@@ -1,12 +0,0 @@
|
||||
{{- if .Values.serviceAccount.create -}}
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: {{ include "findings-ledger.serviceAccountName" . }}
|
||||
labels:
|
||||
{{- include "findings-ledger.labels" . | nindent 4 }}
|
||||
{{- with .Values.serviceAccount.annotations }}
|
||||
annotations:
|
||||
{{- toYaml . | nindent 4 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -1,151 +0,0 @@
|
||||
# Default values for stellaops-findings-ledger
|
||||
|
||||
image:
|
||||
repository: stellaops/findings-ledger
|
||||
tag: "2025.11.0"
|
||||
pullPolicy: IfNotPresent
|
||||
|
||||
replicaCount: 1
|
||||
|
||||
service:
|
||||
type: ClusterIP
|
||||
port: 8080
|
||||
metricsPort: 9090
|
||||
|
||||
# Database configuration
|
||||
database:
|
||||
# External PostgreSQL connection (preferred for production)
|
||||
# Set connectionStringSecret to use existing secret
|
||||
connectionStringSecret: ""
|
||||
connectionStringKey: "LEDGER__DB__CONNECTIONSTRING"
|
||||
# Or provide connection details directly (not recommended for prod)
|
||||
host: "postgres"
|
||||
port: 5432
|
||||
database: "findings_ledger"
|
||||
username: "ledger"
|
||||
# password via secret only
|
||||
|
||||
# Built-in PostgreSQL (dev/testing only)
|
||||
postgresql:
|
||||
enabled: false
|
||||
auth:
|
||||
username: ledger
|
||||
database: findings_ledger
|
||||
|
||||
# Secrets configuration
|
||||
secrets:
|
||||
# Name of secret containing sensitive values
|
||||
name: "findings-ledger-secrets"
|
||||
# Expected keys in secret:
|
||||
# LEDGER__DB__CONNECTIONSTRING
|
||||
# LEDGER__ATTACHMENTS__ENCRYPTIONKEY
|
||||
# LEDGER__MERKLE__SIGNINGKEY (optional)
|
||||
|
||||
# Observability
|
||||
observability:
|
||||
enabled: true
|
||||
otlpEndpoint: "http://otel-collector:4317"
|
||||
metricsEnabled: true
|
||||
|
||||
# Merkle anchoring
|
||||
merkle:
|
||||
anchorInterval: "00:05:00"
|
||||
externalize: false
|
||||
# externalAnchorEndpoint: ""
|
||||
|
||||
# Attachments
|
||||
attachments:
|
||||
maxSizeBytes: 104857600 # 100MB
|
||||
allowEgress: true
|
||||
# encryptionKey via secret
|
||||
|
||||
# Air-gap configuration
|
||||
airgap:
|
||||
advisoryStaleThreshold: 604800 # 7 days
|
||||
vexStaleThreshold: 604800 # 7 days
|
||||
policyStaleThreshold: 86400 # 1 day
|
||||
|
||||
# Authority integration
|
||||
authority:
|
||||
baseUrl: "http://authority:8080"
|
||||
|
||||
# Feature flags
|
||||
features:
|
||||
enableAttachments: true
|
||||
enableAuditLog: true
|
||||
|
||||
# Resource limits
|
||||
resources:
|
||||
requests:
|
||||
cpu: "500m"
|
||||
memory: "1Gi"
|
||||
limits:
|
||||
cpu: "2"
|
||||
memory: "4Gi"
|
||||
|
||||
# Probes
|
||||
probes:
|
||||
readiness:
|
||||
path: /health/ready
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 10
|
||||
liveness:
|
||||
path: /health/live
|
||||
initialDelaySeconds: 15
|
||||
periodSeconds: 20
|
||||
|
||||
# Pod configuration
|
||||
nodeSelector: {}
|
||||
tolerations: []
|
||||
affinity: {}
|
||||
|
||||
# Extra environment variables
|
||||
extraEnv: []
|
||||
# - name: CUSTOM_VAR
|
||||
# value: "value"
|
||||
|
||||
extraEnvFrom: []
|
||||
# - secretRef:
|
||||
# name: additional-secrets
|
||||
|
||||
# Migration job
|
||||
migrations:
|
||||
enabled: true
|
||||
image:
|
||||
repository: stellaops/findings-ledger-migrations
|
||||
tag: "2025.11.0"
|
||||
resources:
|
||||
requests:
|
||||
cpu: "100m"
|
||||
memory: "256Mi"
|
||||
limits:
|
||||
cpu: "500m"
|
||||
memory: "512Mi"
|
||||
|
||||
# Service account
|
||||
serviceAccount:
|
||||
create: true
|
||||
name: ""
|
||||
annotations: {}
|
||||
|
||||
# Pod security context
|
||||
podSecurityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
fsGroup: 1000
|
||||
|
||||
# Container security context
|
||||
securityContext:
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
|
||||
# Ingress (optional)
|
||||
ingress:
|
||||
enabled: false
|
||||
className: ""
|
||||
annotations: {}
|
||||
hosts: []
|
||||
tls: []
|
||||
@@ -1,158 +0,0 @@
|
||||
# Findings Ledger Offline Kit
|
||||
|
||||
This directory contains manifests and scripts for deploying Findings Ledger in air-gapped/offline environments.
|
||||
|
||||
## Contents
|
||||
|
||||
```
|
||||
offline-kit/
|
||||
├── README.md # This file
|
||||
├── manifest.yaml # Offline bundle manifest
|
||||
├── images/ # Container image tarballs (populated at build)
|
||||
│ └── .gitkeep
|
||||
├── migrations/ # Database migration scripts
|
||||
│ └── .gitkeep
|
||||
├── dashboards/ # Grafana dashboard JSON exports
|
||||
│ └── findings-ledger.json
|
||||
├── alerts/ # Prometheus alert rules
|
||||
│ └── findings-ledger-alerts.yaml
|
||||
└── scripts/
|
||||
├── import-images.sh # Load container images
|
||||
├── run-migrations.sh # Apply database migrations
|
||||
└── verify-install.sh # Post-install verification
|
||||
```
|
||||
|
||||
## Building the Offline Kit
|
||||
|
||||
Use the platform offline kit builder:
|
||||
|
||||
```bash
|
||||
# From repository root
|
||||
python ops/offline-kit/build_offline_kit.py \
|
||||
--include ledger \
|
||||
--version 2025.11.0 \
|
||||
--output dist/offline-kit-ledger-2025.11.0.tar.gz
|
||||
```
|
||||
|
||||
## Installation Steps
|
||||
|
||||
### 1. Transfer and Extract
|
||||
|
||||
```bash
|
||||
# On air-gapped host
|
||||
tar xzf offline-kit-ledger-*.tar.gz
|
||||
cd offline-kit-ledger-*
|
||||
```
|
||||
|
||||
### 2. Load Container Images
|
||||
|
||||
```bash
|
||||
./scripts/import-images.sh
|
||||
# Loads: stellaops/findings-ledger, stellaops/findings-ledger-migrations
|
||||
```
|
||||
|
||||
### 3. Run Database Migrations
|
||||
|
||||
```bash
|
||||
export LEDGER__DB__CONNECTIONSTRING="Host=...;Database=...;..."
|
||||
./scripts/run-migrations.sh
|
||||
```
|
||||
|
||||
### 4. Deploy Service
|
||||
|
||||
Choose deployment method:
|
||||
|
||||
**Docker Compose:**
|
||||
```bash
|
||||
cp ../compose/env/ledger.prod.env ./ledger.env
|
||||
# Edit ledger.env with local values
|
||||
docker compose -f ../compose/docker-compose.ledger.yaml up -d
|
||||
```
|
||||
|
||||
**Helm:**
|
||||
```bash
|
||||
helm upgrade --install findings-ledger ../helm \
|
||||
-f values-offline.yaml \
|
||||
--set image.pullPolicy=Never
|
||||
```
|
||||
|
||||
### 5. Verify Installation
|
||||
|
||||
```bash
|
||||
./scripts/verify-install.sh
|
||||
```
|
||||
|
||||
## Configuration Notes
|
||||
|
||||
### Sealed Mode
|
||||
|
||||
In air-gapped environments, configure:
|
||||
|
||||
```yaml
|
||||
# Disable outbound attachment egress
|
||||
LEDGER__ATTACHMENTS__ALLOWEGRESS: "false"
|
||||
|
||||
# Set appropriate staleness thresholds
|
||||
LEDGER__AIRGAP__ADVISORYSTALETHRESHOLD: "604800" # 7 days
|
||||
LEDGER__AIRGAP__VEXSTALETHRESHOLD: "604800"
|
||||
LEDGER__AIRGAP__POLICYSTALETHRESHOLD: "86400" # 1 day
|
||||
```
|
||||
|
||||
### Merkle Anchoring
|
||||
|
||||
For offline environments without external anchoring:
|
||||
|
||||
```yaml
|
||||
LEDGER__MERKLE__EXTERNALIZE: "false"
|
||||
```
|
||||
|
||||
Keep local Merkle roots and export periodically for audit.
|
||||
|
||||
## Backup & Restore
|
||||
|
||||
See `docs/modules/findings-ledger/deployment.md` for full backup/restore procedures.
|
||||
|
||||
Quick reference:
|
||||
```bash
|
||||
# Backup
|
||||
pg_dump -Fc --dbname="$LEDGER_DB" --file ledger-$(date -u +%Y%m%d).dump
|
||||
|
||||
# Restore
|
||||
pg_restore -C -d postgres ledger-YYYYMMDD.dump
|
||||
|
||||
# Replay projections
|
||||
dotnet run --project tools/LedgerReplayHarness -- \
|
||||
--connection "$LEDGER_DB" --tenant all
|
||||
```
|
||||
|
||||
## Observability
|
||||
|
||||
Import the provided dashboards into your local Grafana instance:
|
||||
|
||||
```bash
|
||||
# Import via Grafana API or UI
|
||||
curl -X POST http://grafana:3000/api/dashboards/db \
|
||||
-H "Content-Type: application/json" \
|
||||
-d @dashboards/findings-ledger.json
|
||||
```
|
||||
|
||||
Apply alert rules to Prometheus:
|
||||
```bash
|
||||
cp alerts/findings-ledger-alerts.yaml /etc/prometheus/rules.d/
|
||||
# Reload Prometheus
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Issue | Resolution |
|
||||
| --- | --- |
|
||||
| Migration fails | Check DB connectivity; verify user has CREATE/ALTER privileges |
|
||||
| Health check fails | Check logs: `docker logs findings-ledger` or `kubectl logs -l app.kubernetes.io/name=findings-ledger` |
|
||||
| Metrics not visible | Verify OTLP endpoint is reachable or use Prometheus scrape |
|
||||
| Staleness warnings | Import fresh advisory/VEX bundles via Mirror |
|
||||
|
||||
## Support
|
||||
|
||||
- Platform docs: `docs/modules/findings-ledger/`
|
||||
- Offline operation: `docs/24_OFFLINE_KIT.md`
|
||||
- Air-gap mode: `docs/airgap/`
|
||||
@@ -1,122 +0,0 @@
|
||||
# Findings Ledger Prometheus Alert Rules
|
||||
# Apply to Prometheus: cp findings-ledger-alerts.yaml /etc/prometheus/rules.d/
|
||||
|
||||
groups:
|
||||
- name: findings-ledger
|
||||
rules:
|
||||
# Service availability
|
||||
- alert: FindingsLedgerDown
|
||||
expr: up{job="findings-ledger"} == 0
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "Findings Ledger service is down"
|
||||
description: "Findings Ledger service has been unreachable for more than 2 minutes."
|
||||
|
||||
# Write latency
|
||||
- alert: FindingsLedgerHighWriteLatency
|
||||
expr: histogram_quantile(0.95, sum(rate(ledger_write_latency_seconds_bucket{job="findings-ledger"}[5m])) by (le)) > 1
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "Findings Ledger write latency is high"
|
||||
description: "95th percentile write latency exceeds 1 second for 5 minutes. Current: {{ $value | humanizeDuration }}"
|
||||
|
||||
- alert: FindingsLedgerCriticalWriteLatency
|
||||
expr: histogram_quantile(0.95, sum(rate(ledger_write_latency_seconds_bucket{job="findings-ledger"}[5m])) by (le)) > 5
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "Findings Ledger write latency is critically high"
|
||||
description: "95th percentile write latency exceeds 5 seconds. Current: {{ $value | humanizeDuration }}"
|
||||
|
||||
# Projection lag
|
||||
- alert: FindingsLedgerProjectionLag
|
||||
expr: ledger_projection_lag_seconds{job="findings-ledger"} > 30
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "Findings Ledger projection lag is high"
|
||||
description: "Projection lag exceeds 30 seconds for 5 minutes. Current: {{ $value | humanizeDuration }}"
|
||||
|
||||
- alert: FindingsLedgerCriticalProjectionLag
|
||||
expr: ledger_projection_lag_seconds{job="findings-ledger"} > 300
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "Findings Ledger projection lag is critically high"
|
||||
description: "Projection lag exceeds 5 minutes. Current: {{ $value | humanizeDuration }}"
|
||||
|
||||
# Merkle anchoring
|
||||
- alert: FindingsLedgerMerkleAnchorStale
|
||||
expr: time() - ledger_merkle_last_anchor_timestamp_seconds{job="findings-ledger"} > 600
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "Findings Ledger Merkle anchor is stale"
|
||||
description: "No Merkle anchor created in the last 10 minutes. Last anchor: {{ $value | humanizeTimestamp }}"
|
||||
|
||||
- alert: FindingsLedgerMerkleAnchorFailed
|
||||
expr: increase(ledger_merkle_anchor_failures_total{job="findings-ledger"}[15m]) > 0
|
||||
for: 0m
|
||||
labels:
|
||||
severity: warning
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "Findings Ledger Merkle anchoring failed"
|
||||
description: "Merkle anchor operation failed. Check logs for details."
|
||||
|
||||
# Database connectivity
|
||||
- alert: FindingsLedgerDatabaseErrors
|
||||
expr: increase(ledger_database_errors_total{job="findings-ledger"}[5m]) > 5
|
||||
for: 2m
|
||||
labels:
|
||||
severity: warning
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "Findings Ledger database errors detected"
|
||||
description: "More than 5 database errors in the last 5 minutes."
|
||||
|
||||
# Attachment storage
|
||||
- alert: FindingsLedgerAttachmentStorageErrors
|
||||
expr: increase(ledger_attachment_storage_errors_total{job="findings-ledger"}[15m]) > 0
|
||||
for: 0m
|
||||
labels:
|
||||
severity: warning
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "Findings Ledger attachment storage errors"
|
||||
description: "Attachment storage operation failed. Check encryption keys and storage connectivity."
|
||||
|
||||
# Air-gap staleness (for offline environments)
|
||||
- alert: FindingsLedgerAdvisoryStaleness
|
||||
expr: ledger_airgap_advisory_staleness_seconds{job="findings-ledger"} > 604800
|
||||
for: 1h
|
||||
labels:
|
||||
severity: warning
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "Advisory data is stale in air-gapped environment"
|
||||
description: "Advisory data is older than 7 days. Import fresh data from Mirror."
|
||||
|
||||
- alert: FindingsLedgerVexStaleness
|
||||
expr: ledger_airgap_vex_staleness_seconds{job="findings-ledger"} > 604800
|
||||
for: 1h
|
||||
labels:
|
||||
severity: warning
|
||||
service: findings-ledger
|
||||
annotations:
|
||||
summary: "VEX data is stale in air-gapped environment"
|
||||
description: "VEX data is older than 7 days. Import fresh data from Mirror."
|
||||
@@ -1,185 +0,0 @@
|
||||
{
|
||||
"__inputs": [
|
||||
{
|
||||
"name": "DS_PROMETHEUS",
|
||||
"label": "Prometheus",
|
||||
"description": "",
|
||||
"type": "datasource",
|
||||
"pluginId": "prometheus",
|
||||
"pluginName": "Prometheus"
|
||||
}
|
||||
],
|
||||
"__requires": [
|
||||
{
|
||||
"type": "grafana",
|
||||
"id": "grafana",
|
||||
"name": "Grafana",
|
||||
"version": "9.0.0"
|
||||
},
|
||||
{
|
||||
"type": "datasource",
|
||||
"id": "prometheus",
|
||||
"name": "Prometheus",
|
||||
"version": "1.0.0"
|
||||
}
|
||||
],
|
||||
"annotations": {
|
||||
"list": []
|
||||
},
|
||||
"description": "Findings Ledger service metrics and health",
|
||||
"editable": true,
|
||||
"fiscalYearStartMonth": 0,
|
||||
"graphTooltip": 0,
|
||||
"id": null,
|
||||
"links": [],
|
||||
"liveNow": false,
|
||||
"panels": [
|
||||
{
|
||||
"collapsed": false,
|
||||
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 0 },
|
||||
"id": 1,
|
||||
"panels": [],
|
||||
"title": "Health Overview",
|
||||
"type": "row"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"mappings": [
|
||||
{ "options": { "0": { "color": "red", "index": 1, "text": "DOWN" }, "1": { "color": "green", "index": 0, "text": "UP" } }, "type": "value" }
|
||||
],
|
||||
"thresholds": { "mode": "absolute", "steps": [{ "color": "red", "value": null }, { "color": "green", "value": 1 }] }
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": { "h": 4, "w": 4, "x": 0, "y": 1 },
|
||||
"id": 2,
|
||||
"options": { "colorMode": "value", "graphMode": "none", "justifyMode": "auto", "orientation": "auto", "reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }, "textMode": "auto" },
|
||||
"pluginVersion": "9.0.0",
|
||||
"targets": [{ "expr": "up{job=\"findings-ledger\"}", "refId": "A" }],
|
||||
"title": "Service Status",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
|
||||
"fieldConfig": {
|
||||
"defaults": { "color": { "mode": "palette-classic" }, "unit": "short" },
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": { "h": 4, "w": 4, "x": 4, "y": 1 },
|
||||
"id": 3,
|
||||
"options": { "colorMode": "value", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }, "textMode": "auto" },
|
||||
"pluginVersion": "9.0.0",
|
||||
"targets": [{ "expr": "ledger_events_total{job=\"findings-ledger\"}", "refId": "A" }],
|
||||
"title": "Total Events",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
|
||||
"fieldConfig": {
|
||||
"defaults": { "color": { "mode": "thresholds" }, "unit": "s", "thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }, { "color": "yellow", "value": 1 }, { "color": "red", "value": 5 }] } },
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": { "h": 4, "w": 4, "x": 8, "y": 1 },
|
||||
"id": 4,
|
||||
"options": { "colorMode": "value", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }, "textMode": "auto" },
|
||||
"pluginVersion": "9.0.0",
|
||||
"targets": [{ "expr": "ledger_projection_lag_seconds{job=\"findings-ledger\"}", "refId": "A" }],
|
||||
"title": "Projection Lag",
|
||||
"type": "stat"
|
||||
},
|
||||
{
|
||||
"collapsed": false,
|
||||
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 5 },
|
||||
"id": 10,
|
||||
"panels": [],
|
||||
"title": "Write Performance",
|
||||
"type": "row"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
|
||||
"fieldConfig": {
|
||||
"defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "drawStyle": "line", "fillOpacity": 10, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "never", "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "unit": "s" },
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 6 },
|
||||
"id": 11,
|
||||
"options": { "legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom", "showLegend": true }, "tooltip": { "mode": "multi", "sort": "none" } },
|
||||
"pluginVersion": "9.0.0",
|
||||
"targets": [
|
||||
{ "expr": "histogram_quantile(0.50, sum(rate(ledger_write_latency_seconds_bucket{job=\"findings-ledger\"}[5m])) by (le))", "legendFormat": "p50", "refId": "A" },
|
||||
{ "expr": "histogram_quantile(0.95, sum(rate(ledger_write_latency_seconds_bucket{job=\"findings-ledger\"}[5m])) by (le))", "legendFormat": "p95", "refId": "B" },
|
||||
{ "expr": "histogram_quantile(0.99, sum(rate(ledger_write_latency_seconds_bucket{job=\"findings-ledger\"}[5m])) by (le))", "legendFormat": "p99", "refId": "C" }
|
||||
],
|
||||
"title": "Write Latency",
|
||||
"type": "timeseries"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
|
||||
"fieldConfig": {
|
||||
"defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "drawStyle": "line", "fillOpacity": 10, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "never", "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "unit": "ops" },
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 6 },
|
||||
"id": 12,
|
||||
"options": { "legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom", "showLegend": true }, "tooltip": { "mode": "multi", "sort": "none" } },
|
||||
"pluginVersion": "9.0.0",
|
||||
"targets": [{ "expr": "rate(ledger_events_total{job=\"findings-ledger\"}[5m])", "legendFormat": "events/s", "refId": "A" }],
|
||||
"title": "Event Write Rate",
|
||||
"type": "timeseries"
|
||||
},
|
||||
{
|
||||
"collapsed": false,
|
||||
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 14 },
|
||||
"id": 20,
|
||||
"panels": [],
|
||||
"title": "Merkle Anchoring",
|
||||
"type": "row"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
|
||||
"fieldConfig": {
|
||||
"defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "drawStyle": "line", "fillOpacity": 10, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "never", "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "unit": "s" },
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 15 },
|
||||
"id": 21,
|
||||
"options": { "legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom", "showLegend": true }, "tooltip": { "mode": "multi", "sort": "none" } },
|
||||
"pluginVersion": "9.0.0",
|
||||
"targets": [
|
||||
{ "expr": "histogram_quantile(0.50, sum(rate(ledger_merkle_anchor_duration_seconds_bucket{job=\"findings-ledger\"}[5m])) by (le))", "legendFormat": "p50", "refId": "A" },
|
||||
{ "expr": "histogram_quantile(0.95, sum(rate(ledger_merkle_anchor_duration_seconds_bucket{job=\"findings-ledger\"}[5m])) by (le))", "legendFormat": "p95", "refId": "B" }
|
||||
],
|
||||
"title": "Anchor Duration",
|
||||
"type": "timeseries"
|
||||
},
|
||||
{
|
||||
"datasource": { "type": "prometheus", "uid": "${DS_PROMETHEUS}" },
|
||||
"fieldConfig": {
|
||||
"defaults": { "color": { "mode": "thresholds" }, "unit": "short", "thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }] } },
|
||||
"overrides": []
|
||||
},
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 15 },
|
||||
"id": 22,
|
||||
"options": { "colorMode": "value", "graphMode": "area", "justifyMode": "auto", "orientation": "auto", "reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false }, "textMode": "auto" },
|
||||
"pluginVersion": "9.0.0",
|
||||
"targets": [{ "expr": "ledger_merkle_anchors_total{job=\"findings-ledger\"}", "refId": "A" }],
|
||||
"title": "Total Anchors",
|
||||
"type": "stat"
|
||||
}
|
||||
],
|
||||
"refresh": "30s",
|
||||
"schemaVersion": 37,
|
||||
"style": "dark",
|
||||
"tags": ["stellaops", "findings-ledger"],
|
||||
"templating": { "list": [] },
|
||||
"time": { "from": "now-1h", "to": "now" },
|
||||
"timepicker": {},
|
||||
"timezone": "utc",
|
||||
"title": "Findings Ledger",
|
||||
"uid": "findings-ledger",
|
||||
"version": 1,
|
||||
"weekStart": ""
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
# Container image tarballs populated at build time by offline-kit builder
|
||||
@@ -1,106 +0,0 @@
|
||||
# Findings Ledger Offline Kit Manifest
|
||||
# Version: 2025.11.0
|
||||
# Generated: 2025-12-07
|
||||
|
||||
apiVersion: stellaops.io/v1
|
||||
kind: OfflineKitManifest
|
||||
metadata:
|
||||
name: findings-ledger
|
||||
version: "2025.11.0"
|
||||
description: Findings Ledger service for event-sourced findings storage with Merkle anchoring
|
||||
|
||||
spec:
|
||||
components:
|
||||
- name: findings-ledger
|
||||
type: service
|
||||
image: stellaops/findings-ledger:2025.11.0
|
||||
digest: "" # Populated at build time
|
||||
|
||||
- name: findings-ledger-migrations
|
||||
type: job
|
||||
image: stellaops/findings-ledger-migrations:2025.11.0
|
||||
digest: "" # Populated at build time
|
||||
|
||||
dependencies:
|
||||
- name: postgresql
|
||||
version: ">=14.0"
|
||||
type: database
|
||||
required: true
|
||||
|
||||
- name: otel-collector
|
||||
version: ">=0.80.0"
|
||||
type: service
|
||||
required: false
|
||||
description: Optional for telemetry export
|
||||
|
||||
migrations:
|
||||
- version: "001"
|
||||
file: migrations/001_initial_schema.sql
|
||||
checksum: "" # Populated at build time
|
||||
- version: "002"
|
||||
file: migrations/002_merkle_tables.sql
|
||||
checksum: ""
|
||||
- version: "003"
|
||||
file: migrations/003_attachments.sql
|
||||
checksum: ""
|
||||
- version: "004"
|
||||
file: migrations/004_projections.sql
|
||||
checksum: ""
|
||||
- version: "005"
|
||||
file: migrations/005_airgap_imports.sql
|
||||
checksum: ""
|
||||
- version: "006"
|
||||
file: migrations/006_evidence_snapshots.sql
|
||||
checksum: ""
|
||||
- version: "007"
|
||||
file: migrations/007_timeline_events.sql
|
||||
checksum: ""
|
||||
- version: "008"
|
||||
file: migrations/008_attestation_pointers.sql
|
||||
checksum: ""
|
||||
|
||||
dashboards:
|
||||
- name: findings-ledger
|
||||
file: dashboards/findings-ledger.json
|
||||
checksum: ""
|
||||
|
||||
alerts:
|
||||
- name: findings-ledger-alerts
|
||||
file: alerts/findings-ledger-alerts.yaml
|
||||
checksum: ""
|
||||
|
||||
configuration:
|
||||
required:
|
||||
- key: LEDGER__DB__CONNECTIONSTRING
|
||||
description: PostgreSQL connection string
|
||||
secret: true
|
||||
- key: LEDGER__ATTACHMENTS__ENCRYPTIONKEY
|
||||
description: AES-256 encryption key for attachments (base64)
|
||||
secret: true
|
||||
|
||||
optional:
|
||||
- key: LEDGER__MERKLE__SIGNINGKEY
|
||||
description: Signing key for Merkle root attestations
|
||||
secret: true
|
||||
- key: LEDGER__OBSERVABILITY__OTLPENDPOINT
|
||||
description: OpenTelemetry collector endpoint
|
||||
default: http://otel-collector:4317
|
||||
- key: LEDGER__MERKLE__ANCHORINTERVAL
|
||||
description: Merkle anchor interval (TimeSpan)
|
||||
default: "00:05:00"
|
||||
- key: LEDGER__AIRGAP__ADVISORYSTALETHRESHOLD
|
||||
description: Advisory staleness threshold in seconds
|
||||
default: "604800"
|
||||
|
||||
verification:
|
||||
healthEndpoint: /health/ready
|
||||
metricsEndpoint: /metrics
|
||||
expectedMetrics:
|
||||
- ledger_write_latency_seconds
|
||||
- ledger_projection_lag_seconds
|
||||
- ledger_merkle_anchor_duration_seconds
|
||||
- ledger_events_total
|
||||
|
||||
checksums:
|
||||
algorithm: sha256
|
||||
manifest: "" # Populated at build time
|
||||
@@ -1 +0,0 @@
|
||||
# Database migration SQL scripts copied from StellaOps.FindingsLedger.Migrations
|
||||
@@ -1,131 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Import Findings Ledger container images into local Docker/containerd
|
||||
# Usage: ./import-images.sh [registry-prefix]
|
||||
#
|
||||
# Example:
|
||||
# ./import-images.sh # Loads as stellaops/*
|
||||
# ./import-images.sh myregistry.local/ # Loads and tags as myregistry.local/stellaops/*
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
IMAGES_DIR="${SCRIPT_DIR}/../images"
|
||||
REGISTRY_PREFIX="${1:-}"
|
||||
|
||||
# Color output helpers
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
log_info() { echo -e "${GREEN}[INFO]${NC} $*"; }
|
||||
log_warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
|
||||
log_error() { echo -e "${RED}[ERROR]${NC} $*"; }
|
||||
|
||||
# Detect container runtime
|
||||
detect_runtime() {
|
||||
if command -v docker &>/dev/null; then
|
||||
echo "docker"
|
||||
elif command -v nerdctl &>/dev/null; then
|
||||
echo "nerdctl"
|
||||
elif command -v podman &>/dev/null; then
|
||||
echo "podman"
|
||||
else
|
||||
log_error "No container runtime found (docker, nerdctl, podman)"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
RUNTIME=$(detect_runtime)
|
||||
log_info "Using container runtime: $RUNTIME"
|
||||
|
||||
# Load images from tarballs
|
||||
load_images() {
|
||||
local count=0
|
||||
|
||||
for tarball in "${IMAGES_DIR}"/*.tar; do
|
||||
if [[ -f "$tarball" ]]; then
|
||||
log_info "Loading image from: $(basename "$tarball")"
|
||||
|
||||
if $RUNTIME load -i "$tarball"; then
|
||||
((count++))
|
||||
else
|
||||
log_error "Failed to load: $tarball"
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ $count -eq 0 ]]; then
|
||||
log_warn "No image tarballs found in $IMAGES_DIR"
|
||||
log_warn "Run the offline kit builder first to populate images"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "Loaded $count image(s)"
|
||||
}
|
||||
|
||||
# Re-tag images with custom registry prefix
|
||||
retag_images() {
|
||||
if [[ -z "$REGISTRY_PREFIX" ]]; then
|
||||
log_info "No registry prefix specified, skipping re-tag"
|
||||
return 0
|
||||
fi
|
||||
|
||||
local images=(
|
||||
"stellaops/findings-ledger"
|
||||
"stellaops/findings-ledger-migrations"
|
||||
)
|
||||
|
||||
for image in "${images[@]}"; do
|
||||
# Get the loaded tag
|
||||
local loaded_tag
|
||||
loaded_tag=$($RUNTIME images --format '{{.Repository}}:{{.Tag}}' | grep "^${image}:" | head -1)
|
||||
|
||||
if [[ -n "$loaded_tag" ]]; then
|
||||
local new_tag="${REGISTRY_PREFIX}${loaded_tag}"
|
||||
log_info "Re-tagging: $loaded_tag -> $new_tag"
|
||||
$RUNTIME tag "$loaded_tag" "$new_tag"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# Verify loaded images
|
||||
verify_images() {
|
||||
log_info "Verifying loaded images..."
|
||||
|
||||
local images=(
|
||||
"stellaops/findings-ledger"
|
||||
"stellaops/findings-ledger-migrations"
|
||||
)
|
||||
|
||||
local missing=0
|
||||
for image in "${images[@]}"; do
|
||||
if $RUNTIME images --format '{{.Repository}}' | grep -q "^${REGISTRY_PREFIX}${image}$"; then
|
||||
log_info " ✓ ${REGISTRY_PREFIX}${image}"
|
||||
else
|
||||
log_error " ✗ ${REGISTRY_PREFIX}${image} not found"
|
||||
((missing++))
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ $missing -gt 0 ]]; then
|
||||
log_error "$missing image(s) missing"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "All images verified"
|
||||
}
|
||||
|
||||
main() {
|
||||
log_info "Findings Ledger - Image Import"
|
||||
log_info "=============================="
|
||||
|
||||
load_images
|
||||
retag_images
|
||||
verify_images
|
||||
|
||||
log_info "Image import complete"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -1,125 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run Findings Ledger database migrations
|
||||
# Usage: ./run-migrations.sh [connection-string]
|
||||
#
|
||||
# Environment variables:
|
||||
# LEDGER__DB__CONNECTIONSTRING - PostgreSQL connection string (if not provided as arg)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MIGRATIONS_DIR="${SCRIPT_DIR}/../migrations"
|
||||
|
||||
# Color output helpers
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
log_info() { echo -e "${GREEN}[INFO]${NC} $*"; }
|
||||
log_warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
|
||||
log_error() { echo -e "${RED}[ERROR]${NC} $*"; }
|
||||
|
||||
# Get connection string
|
||||
CONNECTION_STRING="${1:-${LEDGER__DB__CONNECTIONSTRING:-}}"
|
||||
|
||||
if [[ -z "$CONNECTION_STRING" ]]; then
|
||||
log_error "Connection string required"
|
||||
echo "Usage: $0 <connection-string>"
|
||||
echo " or set LEDGER__DB__CONNECTIONSTRING environment variable"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Detect container runtime
|
||||
detect_runtime() {
|
||||
if command -v docker &>/dev/null; then
|
||||
echo "docker"
|
||||
elif command -v nerdctl &>/dev/null; then
|
||||
echo "nerdctl"
|
||||
elif command -v podman &>/dev/null; then
|
||||
echo "podman"
|
||||
else
|
||||
log_error "No container runtime found"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
RUNTIME=$(detect_runtime)
|
||||
|
||||
# Run migrations via container
|
||||
run_migrations_container() {
|
||||
log_info "Running migrations via container..."
|
||||
|
||||
$RUNTIME run --rm \
|
||||
-e "LEDGER__DB__CONNECTIONSTRING=${CONNECTION_STRING}" \
|
||||
--network host \
|
||||
stellaops/findings-ledger-migrations:2025.11.0 \
|
||||
--connection "$CONNECTION_STRING"
|
||||
}
|
||||
|
||||
# Alternative: Run migrations via psql (if dotnet not available)
|
||||
run_migrations_psql() {
|
||||
log_info "Running migrations via psql..."
|
||||
|
||||
if ! command -v psql &>/dev/null; then
|
||||
log_error "psql not found and container runtime unavailable"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Parse connection string for psql
|
||||
# Expected format: Host=...;Port=...;Database=...;Username=...;Password=...
|
||||
local host port database username password
|
||||
host=$(echo "$CONNECTION_STRING" | grep -oP 'Host=\K[^;]+')
|
||||
port=$(echo "$CONNECTION_STRING" | grep -oP 'Port=\K[^;]+' || echo "5432")
|
||||
database=$(echo "$CONNECTION_STRING" | grep -oP 'Database=\K[^;]+')
|
||||
username=$(echo "$CONNECTION_STRING" | grep -oP 'Username=\K[^;]+')
|
||||
password=$(echo "$CONNECTION_STRING" | grep -oP 'Password=\K[^;]+')
|
||||
|
||||
export PGPASSWORD="$password"
|
||||
|
||||
for migration in "${MIGRATIONS_DIR}"/*.sql; do
|
||||
if [[ -f "$migration" ]]; then
|
||||
log_info "Applying: $(basename "$migration")"
|
||||
psql -h "$host" -p "$port" -U "$username" -d "$database" -f "$migration"
|
||||
fi
|
||||
done
|
||||
|
||||
unset PGPASSWORD
|
||||
}
|
||||
|
||||
verify_connection() {
|
||||
log_info "Verifying database connection..."
|
||||
|
||||
# Try container-based verification
|
||||
if $RUNTIME run --rm \
|
||||
--network host \
|
||||
postgres:14-alpine \
|
||||
pg_isready -h "$(echo "$CONNECTION_STRING" | grep -oP 'Host=\K[^;]+')" \
|
||||
-p "$(echo "$CONNECTION_STRING" | grep -oP 'Port=\K[^;]+' || echo 5432)" \
|
||||
&>/dev/null; then
|
||||
log_info "Database connection verified"
|
||||
return 0
|
||||
fi
|
||||
|
||||
log_warn "Could not verify database connection (may still work)"
|
||||
return 0
|
||||
}
|
||||
|
||||
main() {
|
||||
log_info "Findings Ledger - Database Migrations"
|
||||
log_info "======================================"
|
||||
|
||||
verify_connection
|
||||
|
||||
# Prefer container-based migrations
|
||||
if $RUNTIME image inspect stellaops/findings-ledger-migrations:2025.11.0 &>/dev/null; then
|
||||
run_migrations_container
|
||||
else
|
||||
log_warn "Migration image not found, falling back to psql"
|
||||
run_migrations_psql
|
||||
fi
|
||||
|
||||
log_info "Migrations complete"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -1,70 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Verify Findings Ledger installation
|
||||
# Usage: ./verify-install.sh [service-url]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SERVICE_URL="${1:-http://localhost:8188}"
|
||||
|
||||
# Color output helpers
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m'
|
||||
|
||||
log_info() { echo -e "${GREEN}[INFO]${NC} $*"; }
|
||||
log_warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
|
||||
log_error() { echo -e "${RED}[ERROR]${NC} $*"; }
|
||||
log_pass() { echo -e "${GREEN} ✓${NC} $*"; }
|
||||
log_fail() { echo -e "${RED} ✗${NC} $*"; }
|
||||
|
||||
CHECKS_PASSED=0
|
||||
CHECKS_FAILED=0
|
||||
|
||||
run_check() {
|
||||
local name="$1"
|
||||
local cmd="$2"
|
||||
|
||||
if eval "$cmd" &>/dev/null; then
|
||||
log_pass "$name"
|
||||
((CHECKS_PASSED++))
|
||||
else
|
||||
log_fail "$name"
|
||||
((CHECKS_FAILED++))
|
||||
fi
|
||||
}
|
||||
|
||||
main() {
|
||||
log_info "Findings Ledger - Installation Verification"
|
||||
log_info "==========================================="
|
||||
log_info "Service URL: $SERVICE_URL"
|
||||
echo ""
|
||||
|
||||
log_info "Health Checks:"
|
||||
run_check "Readiness endpoint" "curl -sf ${SERVICE_URL}/health/ready"
|
||||
run_check "Liveness endpoint" "curl -sf ${SERVICE_URL}/health/live"
|
||||
|
||||
echo ""
|
||||
log_info "Metrics Checks:"
|
||||
run_check "Metrics endpoint available" "curl -sf ${SERVICE_URL}/metrics | head -1"
|
||||
run_check "ledger_write_latency_seconds present" "curl -sf ${SERVICE_URL}/metrics | grep -q ledger_write_latency_seconds"
|
||||
run_check "ledger_projection_lag_seconds present" "curl -sf ${SERVICE_URL}/metrics | grep -q ledger_projection_lag_seconds"
|
||||
run_check "ledger_merkle_anchor_duration_seconds present" "curl -sf ${SERVICE_URL}/metrics | grep -q ledger_merkle_anchor_duration_seconds"
|
||||
|
||||
echo ""
|
||||
log_info "API Checks:"
|
||||
run_check "OpenAPI spec available" "curl -sf ${SERVICE_URL}/swagger/v1/swagger.json | head -1"
|
||||
|
||||
echo ""
|
||||
log_info "========================================"
|
||||
log_info "Results: ${CHECKS_PASSED} passed, ${CHECKS_FAILED} failed"
|
||||
|
||||
if [[ $CHECKS_FAILED -gt 0 ]]; then
|
||||
log_error "Some checks failed. Review service logs for details."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
log_info "All checks passed. Installation verified."
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -1,42 +0,0 @@
|
||||
# Graph Indexer Release/Offline Bundle Plan (DEVOPS-GRAPH-INDEX-28-010-REL)
|
||||
|
||||
## Goals
|
||||
- Publish signed Helm/Compose bundles for Graph Indexer with offline parity.
|
||||
- Provide SBOM + attestations for images/charts and reproducible artefacts for air-gap kits.
|
||||
|
||||
## Artefacts
|
||||
- Helm chart + values overrides (offline/airgap).
|
||||
- Docker/OCI images (indexer, api) pinned by digest.
|
||||
- SBOMs (SPDX JSON) for images and chart.
|
||||
- Cosign attestations for images and chart tarball.
|
||||
- Offline bundle: tarball containing images (oras layout), charts, values, SBOMs, attestations, and `SHA256SUMS`.
|
||||
|
||||
## Pipeline outline
|
||||
1) **Build** images (indexer + api) with SBOM generation (`syft`), tag and record digests.
|
||||
2) **Sign** images with cosign key (KMS for online; file key for offline bundle) and produce attestations.
|
||||
3) **Chart package**: render chart, package to `.tgz`, generate SBOM for chart, sign with cosign.
|
||||
4) **Compose export**: render Compose file with pinned digests and non-root users.
|
||||
5) **Bundle**: assemble offline tarball:
|
||||
- `images/` oras layout with signed images
|
||||
- `charts/graph-indexer.tgz` + signature
|
||||
- `compose/graph-indexer.yml` (pinned digests)
|
||||
- `sboms/` for images + chart
|
||||
- `attestations/` (cosign bundles)
|
||||
- `SHA256SUMS` and `SHA256SUMS.sig`
|
||||
6) **Verify step**: pipeline stage runs `cosign verify`, `sha256sum --check`, and `helm template` smoke render with airgap values.
|
||||
7) **Publish**: upload to artefact store + offline kit; write manifest with hashes/versions.
|
||||
|
||||
## Security/hardening
|
||||
- Non-root images, read-only rootfs, drop NET_RAW, seccomp default.
|
||||
- Telemetry disabled; no registry pulls at runtime.
|
||||
- mTLS between indexer and dependencies (documented values).
|
||||
|
||||
## Evidence to capture
|
||||
- Image digests, SBOM hashes, cosign verification logs.
|
||||
- Bundle `SHA256SUMS` and signed manifest.
|
||||
- Helm/Compose render outputs (short).
|
||||
|
||||
## Owners
|
||||
- DevOps Guild (build/pipeline)
|
||||
- Graph Indexer Guild (chart/values)
|
||||
- Platform Security (signing policy)
|
||||
@@ -1,128 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build Findings Ledger export pack
|
||||
# Usage: ./build-pack.sh [--snapshot-id <id>] [--sign] [--output <dir>]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
ROOT=$(cd "$(dirname "$0")/../../.." && pwd)
|
||||
OUT_DIR="${OUT_DIR:-$ROOT/out/ledger/packs}"
|
||||
SNAPSHOT_ID="${SNAPSHOT_ID:-$(date +%Y%m%d%H%M%S)}"
|
||||
CREATED="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
SIGN=0
|
||||
|
||||
# Parse args
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
--snapshot-id) SNAPSHOT_ID="$2"; shift 2 ;;
|
||||
--output) OUT_DIR="$2"; shift 2 ;;
|
||||
--sign) SIGN=1; shift ;;
|
||||
*) shift ;;
|
||||
esac
|
||||
done
|
||||
|
||||
mkdir -p "$OUT_DIR/staging"
|
||||
|
||||
echo "==> Building Ledger Pack"
|
||||
echo " Snapshot ID: $SNAPSHOT_ID"
|
||||
echo " Output: $OUT_DIR"
|
||||
|
||||
# Key resolution for signing
|
||||
resolve_key() {
|
||||
if [[ -n "${COSIGN_PRIVATE_KEY_B64:-}" ]]; then
|
||||
local tmp_key="$OUT_DIR/.cosign.key"
|
||||
echo "$COSIGN_PRIVATE_KEY_B64" | base64 -d > "$tmp_key"
|
||||
chmod 600 "$tmp_key"
|
||||
echo "$tmp_key"
|
||||
elif [[ -f "$ROOT/tools/cosign/cosign.key" ]]; then
|
||||
echo "$ROOT/tools/cosign/cosign.key"
|
||||
elif [[ "${COSIGN_ALLOW_DEV_KEY:-0}" == "1" && -f "$ROOT/tools/cosign/cosign.dev.key" ]]; then
|
||||
echo "[info] Using development key" >&2
|
||||
echo "$ROOT/tools/cosign/cosign.dev.key"
|
||||
else
|
||||
echo ""
|
||||
fi
|
||||
}
|
||||
|
||||
# Create pack structure
|
||||
STAGE="$OUT_DIR/staging/$SNAPSHOT_ID"
|
||||
mkdir -p "$STAGE/findings" "$STAGE/metadata" "$STAGE/signatures"
|
||||
|
||||
# Create placeholder data (replace with actual Ledger export)
|
||||
cat > "$STAGE/findings/findings.ndjson" <<EOF
|
||||
{"id": "placeholder-1", "type": "infrastructure-ready", "created": "$CREATED"}
|
||||
EOF
|
||||
|
||||
cat > "$STAGE/metadata/snapshot.json" <<EOF
|
||||
{
|
||||
"snapshotId": "$SNAPSHOT_ID",
|
||||
"created": "$CREATED",
|
||||
"format": "ledger-pack-v1",
|
||||
"status": "infrastructure-ready",
|
||||
"note": "Replace with actual Ledger snapshot export"
|
||||
}
|
||||
EOF
|
||||
|
||||
# Generate manifest
|
||||
sha256() { sha256sum "$1" | awk '{print $1}'; }
|
||||
|
||||
cat > "$STAGE/manifest.json" <<EOF
|
||||
{
|
||||
"schemaVersion": "1.0.0",
|
||||
"packId": "$SNAPSHOT_ID",
|
||||
"created": "$CREATED",
|
||||
"format": "ledger-pack-v1",
|
||||
"contents": {
|
||||
"findings": {"path": "findings/findings.ndjson", "format": "ndjson"},
|
||||
"metadata": {"path": "metadata/snapshot.json", "format": "json"}
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# Generate provenance
|
||||
cat > "$STAGE/provenance.json" <<EOF
|
||||
{
|
||||
"_type": "https://in-toto.io/Statement/v1",
|
||||
"subject": [{"name": "snapshot-$SNAPSHOT_ID.pack.tar.gz", "digest": {"sha256": "pending"}}],
|
||||
"predicateType": "https://slsa.dev/provenance/v1",
|
||||
"predicate": {
|
||||
"buildDefinition": {
|
||||
"buildType": "https://stella-ops.org/ledger-pack/v1",
|
||||
"internalParameters": {"snapshotId": "$SNAPSHOT_ID", "created": "$CREATED"}
|
||||
},
|
||||
"runDetails": {"builder": {"id": "https://stella-ops.org/ledger-pack-builder"}}
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# Create pack tarball
|
||||
PACK_TAR="$OUT_DIR/snapshot-$SNAPSHOT_ID.pack.tar.gz"
|
||||
tar -czf "$PACK_TAR" -C "$STAGE" .
|
||||
|
||||
# Update provenance with actual hash
|
||||
PACK_HASH=$(sha256 "$PACK_TAR")
|
||||
sed -i "s/\"sha256\": \"pending\"/\"sha256\": \"$PACK_HASH\"/" "$STAGE/provenance.json" 2>/dev/null || \
|
||||
sed -i '' "s/\"sha256\": \"pending\"/\"sha256\": \"$PACK_HASH\"/" "$STAGE/provenance.json"
|
||||
|
||||
# Generate checksums
|
||||
cd "$OUT_DIR"
|
||||
sha256sum "snapshot-$SNAPSHOT_ID.pack.tar.gz" > "snapshot-$SNAPSHOT_ID.SHA256SUMS"
|
||||
|
||||
# Sign if requested
|
||||
if [[ $SIGN -eq 1 ]]; then
|
||||
KEY_FILE=$(resolve_key)
|
||||
if [[ -n "$KEY_FILE" ]] && command -v cosign &>/dev/null; then
|
||||
echo "==> Signing pack..."
|
||||
COSIGN_PASSWORD="${COSIGN_PASSWORD:-}" cosign sign-blob \
|
||||
--key "$KEY_FILE" \
|
||||
--bundle "$OUT_DIR/snapshot-$SNAPSHOT_ID.dsse.json" \
|
||||
--tlog-upload=false --yes "$PACK_TAR" 2>/dev/null || echo "[info] Signing skipped"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Cleanup
|
||||
rm -rf "$OUT_DIR/staging"
|
||||
[[ -f "$OUT_DIR/.cosign.key" ]] && rm -f "$OUT_DIR/.cosign.key"
|
||||
|
||||
echo "==> Pack build complete"
|
||||
echo " Pack: $PACK_TAR"
|
||||
echo " Checksums: $OUT_DIR/snapshot-$SNAPSHOT_ID.SHA256SUMS"
|
||||
@@ -1,61 +0,0 @@
|
||||
# Findings Ledger API Deprecation Policy
|
||||
# DEVOPS-LEDGER-OAS-63-001-REL
|
||||
|
||||
version: "1.0.0"
|
||||
created: "2025-12-14"
|
||||
|
||||
policy:
|
||||
# Minimum deprecation notice period
|
||||
notice_period_days: 90
|
||||
|
||||
# Supported API versions
|
||||
supported_versions:
|
||||
- version: "v1"
|
||||
status: "current"
|
||||
sunset_date: null
|
||||
# Future versions will be added here
|
||||
|
||||
# Deprecation workflow
|
||||
workflow:
|
||||
- stage: "announce"
|
||||
description: "Add deprecation notice to API responses and docs"
|
||||
actions:
|
||||
- "Add Sunset header to deprecated endpoints"
|
||||
- "Update OpenAPI spec with deprecation annotations"
|
||||
- "Notify consumers via changelog"
|
||||
|
||||
- stage: "warn"
|
||||
description: "Emit warnings in logs and metrics"
|
||||
duration_days: 30
|
||||
actions:
|
||||
- "Log deprecation warnings"
|
||||
- "Increment deprecation_usage_total metric"
|
||||
- "Send email to registered consumers"
|
||||
|
||||
- stage: "sunset"
|
||||
description: "Remove deprecated endpoints"
|
||||
actions:
|
||||
- "Return 410 Gone for removed endpoints"
|
||||
- "Update SDK to remove deprecated methods"
|
||||
- "Archive endpoint documentation"
|
||||
|
||||
# HTTP headers for deprecation
|
||||
headers:
|
||||
sunset: "Sunset"
|
||||
deprecation: "Deprecation"
|
||||
link: "Link"
|
||||
|
||||
# Metrics to track
|
||||
metrics:
|
||||
- name: "ledger_api_deprecation_usage_total"
|
||||
type: "counter"
|
||||
labels: ["endpoint", "version", "consumer"]
|
||||
description: "Usage count of deprecated endpoints"
|
||||
|
||||
- name: "ledger_api_version_requests_total"
|
||||
type: "counter"
|
||||
labels: ["version"]
|
||||
description: "Requests per API version"
|
||||
|
||||
# Current deprecations (none yet)
|
||||
deprecations: []
|
||||
@@ -1,56 +0,0 @@
|
||||
# Findings Ledger OpenAPI Infrastructure
|
||||
|
||||
## Scope
|
||||
Infrastructure for Ledger OAS lint, publish, SDK generation, and deprecation governance.
|
||||
|
||||
## Tasks Covered
|
||||
- DEVOPS-LEDGER-OAS-61-001-REL: Lint/diff/publish gates
|
||||
- DEVOPS-LEDGER-OAS-61-002-REL: `.well-known/openapi` validation
|
||||
- DEVOPS-LEDGER-OAS-62-001-REL: SDK generation/signing
|
||||
- DEVOPS-LEDGER-OAS-63-001-REL: Deprecation governance
|
||||
|
||||
## File Structure
|
||||
```
|
||||
ops/devops/ledger/
|
||||
├── oas-infrastructure.md (this file)
|
||||
├── validate-oas.sh # Lint + validate OAS spec
|
||||
├── generate-sdk.sh # Generate and sign SDK
|
||||
├── publish-oas.sh # Publish to .well-known
|
||||
└── deprecation-policy.yaml # Deprecation rules
|
||||
|
||||
.gitea/workflows/
|
||||
├── ledger-oas-ci.yml # OAS lint/validate/diff
|
||||
├── ledger-sdk-release.yml # SDK generation
|
||||
└── ledger-oas-publish.yml # Publish spec
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
- Findings Ledger OpenAPI spec at `api/ledger/openapi.yaml`
|
||||
- Version info in spec metadata
|
||||
- Examples for each endpoint
|
||||
|
||||
## Usage
|
||||
|
||||
### Validate OAS
|
||||
```bash
|
||||
./ops/devops/ledger/validate-oas.sh api/ledger/openapi.yaml
|
||||
```
|
||||
|
||||
### Generate SDK
|
||||
```bash
|
||||
# Dev mode
|
||||
COSIGN_ALLOW_DEV_KEY=1 ./ops/devops/ledger/generate-sdk.sh
|
||||
|
||||
# Production
|
||||
./ops/devops/ledger/generate-sdk.sh
|
||||
```
|
||||
|
||||
### Publish to .well-known
|
||||
```bash
|
||||
./ops/devops/ledger/publish-oas.sh --environment staging
|
||||
```
|
||||
|
||||
## Outputs
|
||||
- `out/ledger/sdk/` - Generated SDK packages
|
||||
- `out/ledger/oas/` - Validated spec + diff reports
|
||||
- `out/ledger/deprecation/` - Deprecation reports
|
||||
@@ -1,58 +0,0 @@
|
||||
# Findings Ledger Packs Infrastructure
|
||||
|
||||
## Scope
|
||||
Infrastructure for snapshot/time-travel export packaging and signing.
|
||||
|
||||
## Tasks Covered
|
||||
- DEVOPS-LEDGER-PACKS-42-001-REL: Snapshot/time-travel export packaging
|
||||
- DEVOPS-LEDGER-PACKS-42-002-REL: Pack signing + integrity verification
|
||||
|
||||
## Components
|
||||
|
||||
### 1. Pack Builder
|
||||
Creates deterministic export packs from Ledger snapshots.
|
||||
|
||||
```bash
|
||||
# Build pack from snapshot
|
||||
./ops/devops/ledger/build-pack.sh --snapshot-id <id> --output out/ledger/packs/
|
||||
|
||||
# Dev mode with signing
|
||||
COSIGN_ALLOW_DEV_KEY=1 ./ops/devops/ledger/build-pack.sh --sign
|
||||
```
|
||||
|
||||
### 2. Pack Verifier
|
||||
Verifies pack integrity and signatures.
|
||||
|
||||
```bash
|
||||
# Verify pack
|
||||
./ops/devops/ledger/verify-pack.sh out/ledger/packs/snapshot-*.pack.tar.gz
|
||||
```
|
||||
|
||||
### 3. Time-Travel Export
|
||||
Creates point-in-time exports for compliance/audit.
|
||||
|
||||
```bash
|
||||
# Export at specific timestamp
|
||||
./ops/devops/ledger/time-travel-export.sh --timestamp 2025-12-01T00:00:00Z
|
||||
```
|
||||
|
||||
## Pack Format
|
||||
```
|
||||
snapshot-<id>.pack.tar.gz
|
||||
├── manifest.json # Pack metadata + checksums
|
||||
├── findings/ # Finding records (NDJSON)
|
||||
├── metadata/ # Scan metadata
|
||||
├── provenance.json # SLSA provenance
|
||||
└── signatures/
|
||||
├── manifest.dsse.json # DSSE signature
|
||||
└── SHA256SUMS # Checksums
|
||||
```
|
||||
|
||||
## CI Workflows
|
||||
- `ledger-packs-ci.yml` - Build and verify packs
|
||||
- `ledger-packs-release.yml` - Sign and publish packs
|
||||
|
||||
## Prerequisites
|
||||
- Ledger snapshot schema finalized
|
||||
- Storage contract defined
|
||||
- Pack format specification
|
||||
@@ -1,80 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Validate Findings Ledger OpenAPI spec
|
||||
# Usage: ./validate-oas.sh [spec-path]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
ROOT=$(cd "$(dirname "$0")/../../.." && pwd)
|
||||
SPEC_PATH="${1:-$ROOT/api/ledger/openapi.yaml}"
|
||||
OUT_DIR="${OUT_DIR:-$ROOT/out/ledger/oas}"
|
||||
|
||||
mkdir -p "$OUT_DIR"
|
||||
|
||||
echo "==> Validating Ledger OpenAPI Spec"
|
||||
echo " Spec: $SPEC_PATH"
|
||||
|
||||
# Check if spec exists
|
||||
if [[ ! -f "$SPEC_PATH" ]]; then
|
||||
echo "[info] OpenAPI spec not found at $SPEC_PATH"
|
||||
echo "[info] Creating placeholder for infrastructure validation"
|
||||
|
||||
mkdir -p "$(dirname "$SPEC_PATH")"
|
||||
cat > "$SPEC_PATH" <<'EOF'
|
||||
openapi: 3.1.0
|
||||
info:
|
||||
title: Findings Ledger API
|
||||
version: 0.0.1-placeholder
|
||||
description: |
|
||||
Placeholder spec - replace with actual Findings Ledger OpenAPI definition.
|
||||
Infrastructure is ready for validation once spec is provided.
|
||||
paths:
|
||||
/health:
|
||||
get:
|
||||
summary: Health check
|
||||
responses:
|
||||
'200':
|
||||
description: OK
|
||||
EOF
|
||||
echo "[info] Placeholder spec created"
|
||||
fi
|
||||
|
||||
# Lint with spectral if available
|
||||
if command -v spectral &>/dev/null; then
|
||||
echo "==> Running Spectral lint..."
|
||||
spectral lint "$SPEC_PATH" --output "$OUT_DIR/lint-report.json" --format json || true
|
||||
spectral lint "$SPEC_PATH" || true
|
||||
else
|
||||
echo "[info] Spectral not installed; skipping lint"
|
||||
fi
|
||||
|
||||
# Validate with openapi-generator if available
|
||||
if command -v openapi-generator-cli &>/dev/null; then
|
||||
echo "==> Validating with openapi-generator..."
|
||||
openapi-generator-cli validate -i "$SPEC_PATH" > "$OUT_DIR/validation-report.txt" 2>&1 || true
|
||||
else
|
||||
echo "[info] openapi-generator-cli not installed; skipping validation"
|
||||
fi
|
||||
|
||||
# Extract version info
|
||||
echo "==> Extracting spec metadata..."
|
||||
if command -v yq &>/dev/null; then
|
||||
VERSION=$(yq '.info.version' "$SPEC_PATH")
|
||||
TITLE=$(yq '.info.title' "$SPEC_PATH")
|
||||
else
|
||||
VERSION="unknown"
|
||||
TITLE="Findings Ledger API"
|
||||
fi
|
||||
|
||||
# Generate summary
|
||||
cat > "$OUT_DIR/spec-summary.json" <<EOF
|
||||
{
|
||||
"specPath": "$SPEC_PATH",
|
||||
"title": "$TITLE",
|
||||
"version": "$VERSION",
|
||||
"validatedAt": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
|
||||
"status": "validated"
|
||||
}
|
||||
EOF
|
||||
|
||||
echo "==> Validation complete"
|
||||
echo " Summary: $OUT_DIR/spec-summary.json"
|
||||
@@ -1,46 +0,0 @@
|
||||
# Orchestrator Infra Bootstrap (DEVOPS-ORCH-32-001)
|
||||
|
||||
## Components
|
||||
- Postgres 16 (state/config)
|
||||
- Mongo 7 (job ledger history)
|
||||
- NATS 2.10 JetStream (queue/bus)
|
||||
|
||||
Compose file: `ops/devops/orchestrator/docker-compose.orchestrator.yml`
|
||||
|
||||
## Quick start (offline-friendly)
|
||||
```bash
|
||||
# bring up infra
|
||||
COMPOSE_FILE=ops/devops/orchestrator/docker-compose.orchestrator.yml docker compose up -d
|
||||
|
||||
# smoke check and emit connection strings
|
||||
scripts/orchestrator/smoke.sh
|
||||
cat out/orchestrator-smoke/readiness.txt
|
||||
|
||||
# synthetic probe (postgres/mongo/nats health)
|
||||
scripts/orchestrator/probe.sh
|
||||
cat out/orchestrator-probe/status.txt
|
||||
|
||||
# replay readiness (restart then smoke)
|
||||
scripts/orchestrator/replay-smoke.sh
|
||||
```
|
||||
|
||||
Connection strings
|
||||
- Postgres: `postgres://orch:orchpass@localhost:55432/orchestrator`
|
||||
- Mongo: `mongodb://localhost:57017`
|
||||
- NATS: `nats://localhost:4222`
|
||||
|
||||
## Observability
|
||||
- Alerts: `ops/devops/orchestrator/alerts.yaml`
|
||||
- Grafana dashboard: `ops/devops/orchestrator/grafana/orchestrator-overview.json`
|
||||
- Metrics expected: `job_queue_depth`, `job_failures_total`, `lease_extensions_total`, `job_latency_seconds_bucket`.
|
||||
- Runbook: `ops/devops/orchestrator/incident-response.md`
|
||||
- Synthetic probes: `scripts/orchestrator/probe.sh` (writes `out/orchestrator-probe/status.txt`).
|
||||
- Replay smoke: `scripts/orchestrator/replay-smoke.sh` (idempotent restart + smoke).
|
||||
|
||||
## CI hook (suggested)
|
||||
Add a workflow step (or local cron) to run `scripts/orchestrator/smoke.sh` with `SKIP_UP=1` against existing infra and publish the `readiness.txt` artifact for traceability.
|
||||
|
||||
## Notes
|
||||
- Uses fixed ports for determinism; adjust via COMPOSE overrides if needed.
|
||||
- Data volumes: `orch_pg_data`, `orch_mongo_data` (docker volumes).
|
||||
- No external downloads beyond base images; pin images to specific tags above.
|
||||
@@ -1,69 +0,0 @@
|
||||
groups:
|
||||
- name: orchestrator-core
|
||||
rules:
|
||||
- alert: OrchestratorQueueDepthHigh
|
||||
expr: job_queue_depth > 500
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
service: orchestrator
|
||||
annotations:
|
||||
summary: "Queue depth high"
|
||||
description: "job_queue_depth exceeded 500 for 10m"
|
||||
- alert: OrchestratorFailuresHigh
|
||||
expr: rate(job_failures_total[5m]) > 5
|
||||
for: 5m
|
||||
labels:
|
||||
severity: critical
|
||||
service: orchestrator
|
||||
annotations:
|
||||
summary: "Job failures elevated"
|
||||
description: "Failure rate above 5/min in last 5m"
|
||||
- alert: OrchestratorLeaseStall
|
||||
expr: rate(lease_extensions_total[5m]) == 0 and job_queue_depth > 0
|
||||
for: 5m
|
||||
labels:
|
||||
severity: critical
|
||||
service: orchestrator
|
||||
annotations:
|
||||
summary: "Leases stalled"
|
||||
description: "No lease renewals while queue has items"
|
||||
- alert: OrchestratorDLQDepthHigh
|
||||
expr: job_dlq_depth > 10
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
service: orchestrator
|
||||
annotations:
|
||||
summary: "DLQ depth high"
|
||||
description: "Dead-letter queue depth above 10 for 10m"
|
||||
- alert: OrchestratorBackpressure
|
||||
expr: avg_over_time(rate_limiter_backpressure_ratio[5m]) > 0.5
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
service: orchestrator
|
||||
annotations:
|
||||
summary: "Backpressure elevated"
|
||||
description: "Rate limiter backpressure >50% over 5m"
|
||||
- alert: OrchestratorErrorCluster
|
||||
expr: sum by(jobType) (rate(job_failures_total[5m])) > 3
|
||||
for: 5m
|
||||
labels:
|
||||
severity: critical
|
||||
service: orchestrator
|
||||
annotations:
|
||||
summary: "Error cluster detected"
|
||||
description: "Failure rate >3/min for a job type"
|
||||
- alert: OrchestratorFailureBurnRateHigh
|
||||
expr: |
|
||||
(rate(job_failures_total[5m]) / clamp_min(rate(job_processed_total[5m]), 1)) > 0.02
|
||||
and
|
||||
(rate(job_failures_total[30m]) / clamp_min(rate(job_processed_total[30m]), 1)) > 0.01
|
||||
for: 10m
|
||||
labels:
|
||||
severity: critical
|
||||
service: orchestrator
|
||||
annotations:
|
||||
summary: "Failure burn rate breaching SLO"
|
||||
description: "5m/30m failure burn rate above 2%/1% SLO; investigate upstream jobs and dependencies."
|
||||
@@ -1,50 +0,0 @@
|
||||
version: "3.9"
|
||||
services:
|
||||
orchestrator-postgres:
|
||||
image: postgres:18.1-alpine
|
||||
environment:
|
||||
POSTGRES_USER: orch
|
||||
POSTGRES_PASSWORD: orchpass
|
||||
POSTGRES_DB: orchestrator
|
||||
volumes:
|
||||
- orch_pg_data:/var/lib/postgresql/data
|
||||
ports:
|
||||
- "55432:5432"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U orch"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
restart: unless-stopped
|
||||
|
||||
orchestrator-mongo:
|
||||
image: mongo:7
|
||||
command: ["mongod", "--quiet", "--storageEngine=wiredTiger"]
|
||||
ports:
|
||||
- "57017:27017"
|
||||
volumes:
|
||||
- orch_mongo_data:/data/db
|
||||
healthcheck:
|
||||
test: ["CMD", "mongosh", "--quiet", "--eval", "db.adminCommand('ping')"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
restart: unless-stopped
|
||||
|
||||
orchestrator-nats:
|
||||
image: nats:2.10-alpine
|
||||
ports:
|
||||
- "5422:4222"
|
||||
- "5822:8222"
|
||||
command: ["-js", "-m", "8222"]
|
||||
healthcheck:
|
||||
test: ["CMD", "nats", "--server", "localhost:4222", "ping"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
orch_pg_data:
|
||||
orch_mongo_data:
|
||||
|
||||
@@ -1,63 +0,0 @@
|
||||
{
|
||||
"schemaVersion": 39,
|
||||
"title": "Orchestrator Overview",
|
||||
"panels": [
|
||||
{
|
||||
"type": "stat",
|
||||
"title": "Queue Depth",
|
||||
"datasource": "Prometheus",
|
||||
"fieldConfig": {"defaults": {"unit": "none"}},
|
||||
"targets": [{"expr": "sum(job_queue_depth)"}]
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"title": "Queue Depth by Job Type",
|
||||
"datasource": "Prometheus",
|
||||
"targets": [{"expr": "job_queue_depth"}],
|
||||
"fieldConfig": {"defaults": {"unit": "none"}}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"title": "Failures per minute",
|
||||
"datasource": "Prometheus",
|
||||
"targets": [{"expr": "rate(job_failures_total[5m])"}],
|
||||
"fieldConfig": {"defaults": {"unit": "short"}}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"title": "Leases per second",
|
||||
"datasource": "Prometheus",
|
||||
"targets": [{"expr": "rate(lease_extensions_total[5m])"}],
|
||||
"fieldConfig": {"defaults": {"unit": "ops"}}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"title": "Job latency p95",
|
||||
"datasource": "Prometheus",
|
||||
"targets": [{"expr": "histogram_quantile(0.95, sum(rate(job_latency_seconds_bucket[5m])) by (le))"}],
|
||||
"fieldConfig": {"defaults": {"unit": "s"}}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"title": "DLQ depth",
|
||||
"datasource": "Prometheus",
|
||||
"targets": [{"expr": "job_dlq_depth"}],
|
||||
"fieldConfig": {"defaults": {"unit": "none"}}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"title": "Backpressure ratio",
|
||||
"datasource": "Prometheus",
|
||||
"targets": [{"expr": "rate_limiter_backpressure_ratio"}],
|
||||
"fieldConfig": {"defaults": {"unit": "percentunit"}}
|
||||
},
|
||||
{
|
||||
"type": "timeseries",
|
||||
"title": "Failures by job type",
|
||||
"datasource": "Prometheus",
|
||||
"targets": [{"expr": "rate(job_failures_total[5m])"}],
|
||||
"fieldConfig": {"defaults": {"unit": "short"}}
|
||||
}
|
||||
],
|
||||
"time": {"from": "now-6h", "to": "now"}
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
# Orchestrator Incident Response & GA Readiness
|
||||
|
||||
## Alert links
|
||||
- Prometheus rules: `ops/devops/orchestrator/alerts.yaml` (includes burn-rate).
|
||||
- Dashboard: `ops/devops/orchestrator/grafana/orchestrator-overview.json`.
|
||||
|
||||
## Runbook (by alert)
|
||||
- **QueueDepthHigh / DLQDepthHigh**
|
||||
- Check backlog cause: slow workers vs. downstream dependency.
|
||||
- Scale workers + clear DLQ after snapshot; if DLQ cause is transient, replay via `replay-smoke.sh` after fixes.
|
||||
- **FailuresHigh / ErrorCluster / FailureBurnRateHigh**
|
||||
- Inspect failing job type from alert labels.
|
||||
- Pause new dispatch for the job type; ship hotfix or rollback offending worker image.
|
||||
- Validate with `scripts/orchestrator/probe.sh` then `smoke.sh` to ensure infra is healthy.
|
||||
- **LeaseStall**
|
||||
- Look for stuck locks in Postgres `locks` view; force release or restart the worker set.
|
||||
- Confirm NATS health (probe) and worker heartbeats.
|
||||
- **Backpressure**
|
||||
- Increase rate-limit budgets temporarily; ensure backlog drains; restore defaults after stability.
|
||||
|
||||
## Synthetic checks
|
||||
- `scripts/orchestrator/probe.sh` — psql ping, mongo ping, NATS pub/ping; writes `out/orchestrator-probe/status.txt`.
|
||||
- `scripts/orchestrator/smoke.sh` — end-to-end infra smoke, emits readiness.
|
||||
- `scripts/orchestrator/replay-smoke.sh` — restart stack then run smoke to prove restart/replay works.
|
||||
|
||||
## GA readiness checklist
|
||||
- [ ] Burn-rate alerting enabled in Prometheus/Alertmanager (see `alerts.yaml` rule `OrchestratorFailureBurnRateHigh`).
|
||||
- [ ] Dashboard imported and linked in on-call rotation.
|
||||
- [ ] Synthetic probe cron in CI/ops runner publishing `status.txt` artifact daily.
|
||||
- [ ] Replay smoke scheduled post-deploy to validate persistence/volumes.
|
||||
- [ ] Backup/restore for Postgres & Mongo verified weekly (not automated here).
|
||||
- [ ] NATS JetStream retention + DLQ policy reviewed and documented.
|
||||
|
||||
## Escalation
|
||||
- Primary: Orchestrator on-call.
|
||||
- Secondary: DevOps Guild (release).
|
||||
- Page when any critical alert persists >15m or dual criticals fire simultaneously.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user