CD/CD consolidation

devops/attestation/ALERTS.md

@@ -0,0 +1,24 @@
+# Attestation Alerts & Dashboards (DEVOPS-ATTEST-75-001)
+
+## Prometheus alert rules
+File: `ops/devops/attestation/attestation-alerts.yaml`
+- `AttestorSignLatencyP95High`: p95 signing latency > 2s for 5m.
+- `AttestorVerifyLatencyP95High`: p95 verification latency > 2s for 5m.
+- `AttestorVerifyFailureRate`: verification failures / requests > 2% over 5m.
+- `AttestorKeyRotationStale`: key not rotated in 30d.
+
+Metrics expected:
+- `attestor_sign_duration_seconds_bucket`
+- `attestor_verify_duration_seconds_bucket`
+- `attestor_verify_failures_total`
+- `attestor_verify_requests_total`
+- `attestor_key_last_rotated_seconds` (gauge of Unix epoch seconds of last rotation)
+
+## Grafana
+File: `ops/devops/attestation/grafana/attestation-latency.json`
+- Panels: signing p50/p95, verification p50/p95, failure rate, key-age gauge, last 24h error counts.
+
+## Runbook
+- Verify exporters scrape `attestor-*` metrics from Attestor service.
+- Ensure alertmanager routes `team=devops` to on-call.
+- Key rotation alert: rotate via standard KMS workflow; acknowledge alert after new metric value observed.

devops/attestation/README.md

@@ -0,0 +1,10 @@
+# Attestor CI/Secrets (DEVOPS-ATTEST-73-001/002)
+
+Artifacts added for the DevOps attestation track:
+
+- `ci.yml` — GitHub Actions workflow (parity stub) that restores/builds/tests Attestor solution and uploads test artefacts. Offline/airgap friendly when mirrored into local runner; set DOTNET_* envs for determinism.
+- Secrets storage plan:
+  - Use KMS-backed cosign key refs (e.g., `azurekms://...` or `awskms://...`).
+  - Store ref in CI secret `ATTESTOR_COSIGN_KEY`; pipeline passes via env and never writes key material to disk.
+  - Audit logs: enable KMS audit + CI job logs; avoid plaintext key dumps.
+- Next steps: wire `.gitea/workflows/attestor-ci.yml` to mirror this job, add `cosign sign-blob` stage for DSSE envelopes, and publish artefacts to `ops/devops/artifacts/attestor/<ts>/` with checksums.

devops/attestation/attestation-alerts.yaml

@@ -0,0 +1,43 @@
+groups:
+  - name: attestor-latency
+    rules:
+      - alert: AttestorSignLatencyP95High
+        expr: histogram_quantile(0.95, sum(rate(attestor_sign_duration_seconds_bucket[5m])) by (le)) > 2
+        for: 5m
+        labels:
+          severity: warning
+          team: devops
+        annotations:
+          summary: "Attestor signing latency p95 high"
+          description: "Signing p95 is {{ $value }}s over the last 5m (threshold 2s)."
+      - alert: AttestorVerifyLatencyP95High
+        expr: histogram_quantile(0.95, sum(rate(attestor_verify_duration_seconds_bucket[5m])) by (le)) > 2
+        for: 5m
+        labels:
+          severity: warning
+          team: devops
+        annotations:
+          summary: "Attestor verification latency p95 high"
+          description: "Verification p95 is {{ $value }}s over the last 5m (threshold 2s)."
+  - name: attestor-errors
+    rules:
+      - alert: AttestorVerifyFailureRate
+        expr: rate(attestor_verify_failures_total[5m]) / rate(attestor_verify_requests_total[5m]) > 0.02
+        for: 5m
+        labels:
+          severity: critical
+          team: devops
+        annotations:
+          summary: "Attestor verification failure rate above 2%"
+          description: "Verification failure rate is {{ $value | humanizePercentage }} over last 5m."
+  - name: attestor-keys
+    rules:
+      - alert: AttestorKeyRotationStale
+        expr: (time() - attestor_key_last_rotated_seconds) > 60*60*24*30
+        for: 10m
+        labels:
+          severity: warning
+          team: devops
+        annotations:
+          summary: "Attestor signing key rotation overdue"
+          description: "Signing key has not rotated in >30d ({{ $value }} seconds)."

devops/attestation/ci.yml

@@ -0,0 +1,38 @@
+name: Attestor CI
+
+on:
+  workflow_dispatch:
+  push:
+    paths:
+      - 'src/Attestor/**'
+      - '.gitea/workflows/attestor-ci.yml'
+      - 'ops/devops/attestation/**'
+
+jobs:
+  build-test:
+    runs-on: ubuntu-latest
+    env:
+      DOTNET_NOLOGO: 1
+      DOTNET_CLI_TELEMETRY_OPTOUT: 1
+    steps:
+      - uses: actions/checkout@v4
+      - name: Setup .NET 10
+        uses: actions/setup-dotnet@v4
+        with:
+          dotnet-version: '10.0.x'
+      - name: Restore
+        run: dotnet restore src/Attestor/StellaOps.Attestor.sln
+      - name: Build
+        run: dotnet build --no-restore -c Release src/Attestor/StellaOps.Attestor.sln
+      - name: Test
+        run: dotnet test --no-build -c Release src/Attestor/StellaOps.Attestor.sln
+      - name: Publish artefacts
+        if: always()
+        run: |
+          mkdir -p out/ci/attestor
+          find src/Attestor -name '*.trx' -o -name '*.xml' | tar -czf out/ci/attestor/test-artifacts.tgz -T-
+      - name: Upload artefacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: attestor-ci-artifacts
+          path: out/ci/attestor/test-artifacts.tgz

devops/attestation/grafana/attestation-latency.json

@@ -0,0 +1,38 @@
+{
+  "title": "Attestor Latency & Errors",
+  "time": { "from": "now-24h", "to": "now" },
+  "panels": [
+    {
+      "type": "timeseries",
+      "title": "Signing latency p50/p95",
+      "targets": [
+        { "expr": "histogram_quantile(0.5, sum(rate(attestor_sign_duration_seconds_bucket[5m])) by (le))", "legendFormat": "p50" },
+        { "expr": "histogram_quantile(0.95, sum(rate(attestor_sign_duration_seconds_bucket[5m])) by (le))", "legendFormat": "p95" }
+      ]
+    },
+    {
+      "type": "timeseries",
+      "title": "Verification latency p50/p95",
+      "targets": [
+        { "expr": "histogram_quantile(0.5, sum(rate(attestor_verify_duration_seconds_bucket[5m])) by (le))", "legendFormat": "p50" },
+        { "expr": "histogram_quantile(0.95, sum(rate(attestor_verify_duration_seconds_bucket[5m])) by (le))", "legendFormat": "p95" }
+      ]
+    },
+    {
+      "type": "timeseries",
+      "title": "Verification failure rate",
+      "targets": [
+        { "expr": "rate(attestor_verify_failures_total[5m]) / rate(attestor_verify_requests_total[5m])", "legendFormat": "failure rate" }
+      ]
+    },
+    {
+      "type": "stat",
+      "title": "Key age (days)",
+      "targets": [
+        { "expr": "(time() - attestor_key_last_rotated_seconds) / 86400" }
+      ]
+    }
+  ],
+  "schemaVersion": 39,
+  "version": 1
+}

devops/attestation/witness-plan.md

@@ -0,0 +1,57 @@
+# Transparency Log Witness Deployment Plan (DEVOPS-ATTEST-74-001)
+
+## Goals
+- Deploy and monitor a Sigstore-compatible witness for Rekor v1/v2 logs (and air-gap mirrors).
+- Provide offline-ready configs and evidence (hashes, DSSE attestations) for bootstrap packs.
+
+## Scope
+- Environments: staging → prod (online), sealed/offline mirror (optional, read-only).
+- Witness duties: verify inclusion proofs, publish checkpoints/signed STHs, expose metrics and health.
+
+## Architecture
+- Witness binary (sigstore/witness or equivalent) in a hardened container:
+  - Non-root user, read-only rootfs, seccomp/AppArmor defaults.
+  - TLS with mTLS between witness and collector; optional OIDC for admin endpoints.
+- Inputs:
+  - Rekor base URL(s) + public keys.
+  - Mirror CAR path + signature (for air-gap).
+- Outputs:
+  - Signed checkpoints (STH) rotated hourly; stored in object storage + DSSE manifest.
+  - Metrics: Prometheus `/metrics` endpoint (request latency, verify failures, checkpoint age).
+  - Logs: JSON, structured, no PII.
+
+## Deployment steps
+1) Build/pull witness image (pin digest); generate SBOM + cosign attestations.
+2) Create config:
+   - `rekor_urls`: prod/staging
+   - `rekor_keys`: PEMs
+   - `checkpoint_interval`: 1h
+   - `mirror_path` (optional): `/data/rekor-mirror.car`
+   - `signer`: KMS ref or file key (sealed-mode uses file key from bootstrap pack)
+3) Helm/Compose template:
+   - read-only rootfs, drop NET_RAW, memory/cpu limits
+   - PVC for checkpoints (`/var/lib/witness/checkpoints`)
+   - Service exposing HTTPS + `/metrics`
+4) CI:
+   - Lint chart
+   - Run e2e: start Rekor test instance, run witness, verify checkpoint written, verify metrics non-zero.
+   - Publish image SBOM/attestations and chart checksums.
+5) Monitoring/alerts:
+   - `witness_verify_failures_total` > 0 over 5m
+   - `witness_checkpoint_age_seconds` > 5400
+   - `witness_backfill_queue_depth` (if supported) above threshold
+
+## Offline/air-gap mode
+- Consume signed Rekor mirror (CAR + manifest) from bootstrap pack.
+- Run witness in verify-only mode against mirror; disable outbound network.
+- Emit checkpoints signed with offline key; store in mirror bundle for audit.
+
+## Evidence to capture
+- Image digest, SBOM hash, chart checksum.
+- Signed checkpoint sample and DSSE manifest.
+- CI e2e logs and metrics sample (scrape output).
+
+## Owners
+- Build/deploy: DevOps Guild
+- Keys/config: Platform Security
+- Observability: Observability Guild