feat: Add UI benchmark driver and scenarios for graph interactions
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
- Introduced `ui_bench_driver.mjs` to read scenarios and fixture manifest, generating a deterministic run plan. - Created `ui_bench_plan.md` outlining the purpose, scope, and next steps for the benchmark. - Added `ui_bench_scenarios.json` containing various scenarios for graph UI interactions. - Implemented tests for CLI commands, ensuring bundle verification and telemetry defaults. - Developed schemas for orchestrator components, including replay manifests and event envelopes. - Added mock API for risk management, including listing and statistics functionalities. - Implemented models for risk profiles and query options to support the new API.
This commit is contained in:
50
ops/devops/aoc/supersedes-rollout.md
Normal file
50
ops/devops/aoc/supersedes-rollout.md
Normal file
@@ -0,0 +1,50 @@
|
||||
# Supersedes backfill rollout plan (DEVOPS-AOC-19-101)
|
||||
|
||||
Scope: Concelier Link-Not-Merge backfill and supersedes processing once advisory_raw idempotency index is in staging.
|
||||
|
||||
## Preconditions
|
||||
- Idempotency index verified in staging (`advisory_raw` duplicate inserts rejected; log hash recorded).
|
||||
- LNM migrations 21-101/102 applied (shards, TTL, tombstones).
|
||||
- Event transport to NATS/Redis disabled during backfill to avoid noisy downstream replays.
|
||||
- Offline kit mirror includes current hashes for `advisory_raw` and backfill bundle.
|
||||
|
||||
## Rollout steps (staging → prod)
|
||||
1) **Freeze window** (announce 24h prior)
|
||||
- Pause Concelier ingest workers (`CONCELIER_INGEST_ENABLED=false`).
|
||||
- Stop outbox publisher or point to blackhole NATS subject.
|
||||
2) **Dry-run (staging)**
|
||||
- Run backfill job with `--dry-run` to emit counts only.
|
||||
- Verify: new supersedes records count == expected; no write errors; idempotency violations = 0.
|
||||
- Capture logs + SHA256 of generated report.
|
||||
3) **Prod execution**
|
||||
- Run backfill job with `--batch-size=500` and `--stop-on-error`.
|
||||
- Monitor: insert rate, error rate, Mongo oplog lag; target <5% CPU on primary.
|
||||
4) **Validation**
|
||||
- Run consistency check:
|
||||
- `advisory_observations` count stable (no drop).
|
||||
- Supersedes edges present for all prior conflicts.
|
||||
- Idempotency index hit rate <0.1%.
|
||||
- Run API spot check: `/advisories/summary` returns supersedes metadata; `advisory.linkset.updated` events absent during freeze.
|
||||
5) **Unfreeze**
|
||||
- Re-enable ingest + outbox publisher.
|
||||
- Trigger single `advisory.observation.updated@1` replay to confirm event path is healthy.
|
||||
|
||||
## Rollback
|
||||
- If errors >0 or idempotency violations observed:
|
||||
- Stop job, keep ingest paused.
|
||||
- Run rollback script `ops/devops/scripts/rollback-lnm-backfill.js` to remove supersedes/tombstones inserted in current window.
|
||||
- Restore Mongo from last checkpointed snapshot if rollback script fails.
|
||||
|
||||
## Evidence to capture
|
||||
- Job command + arguments.
|
||||
- SHA256 of backfill bundle and report.
|
||||
- Idempotency violation count.
|
||||
- Post-run consistency report (JSON) stored under `ops/devops/artifacts/aoc-supersedes/<timestamp>/`.
|
||||
|
||||
## Monitoring/Alerts
|
||||
- Add temporary Grafana panel for idempotency violations and Mongo ops/sec during job.
|
||||
- Alert if job runtime exceeds 2h or if oplog lag > 60s.
|
||||
|
||||
## Owners
|
||||
- Run: DevOps Guild
|
||||
- Approvals: Concelier Storage Guild + Platform Security
|
||||
57
ops/devops/attestation/witness-plan.md
Normal file
57
ops/devops/attestation/witness-plan.md
Normal file
@@ -0,0 +1,57 @@
|
||||
# Transparency Log Witness Deployment Plan (DEVOPS-ATTEST-74-001)
|
||||
|
||||
## Goals
|
||||
- Deploy and monitor a Sigstore-compatible witness for Rekor v1/v2 logs (and air-gap mirrors).
|
||||
- Provide offline-ready configs and evidence (hashes, DSSE attestations) for bootstrap packs.
|
||||
|
||||
## Scope
|
||||
- Environments: staging → prod (online), sealed/offline mirror (optional, read-only).
|
||||
- Witness duties: verify inclusion proofs, publish checkpoints/signed STHs, expose metrics and health.
|
||||
|
||||
## Architecture
|
||||
- Witness binary (sigstore/witness or equivalent) in a hardened container:
|
||||
- Non-root user, read-only rootfs, seccomp/AppArmor defaults.
|
||||
- TLS with mTLS between witness and collector; optional OIDC for admin endpoints.
|
||||
- Inputs:
|
||||
- Rekor base URL(s) + public keys.
|
||||
- Mirror CAR path + signature (for air-gap).
|
||||
- Outputs:
|
||||
- Signed checkpoints (STH) rotated hourly; stored in object storage + DSSE manifest.
|
||||
- Metrics: Prometheus `/metrics` endpoint (request latency, verify failures, checkpoint age).
|
||||
- Logs: JSON, structured, no PII.
|
||||
|
||||
## Deployment steps
|
||||
1) Build/pull witness image (pin digest); generate SBOM + cosign attestations.
|
||||
2) Create config:
|
||||
- `rekor_urls`: prod/staging
|
||||
- `rekor_keys`: PEMs
|
||||
- `checkpoint_interval`: 1h
|
||||
- `mirror_path` (optional): `/data/rekor-mirror.car`
|
||||
- `signer`: KMS ref or file key (sealed-mode uses file key from bootstrap pack)
|
||||
3) Helm/Compose template:
|
||||
- read-only rootfs, drop NET_RAW, memory/cpu limits
|
||||
- PVC for checkpoints (`/var/lib/witness/checkpoints`)
|
||||
- Service exposing HTTPS + `/metrics`
|
||||
4) CI:
|
||||
- Lint chart
|
||||
- Run e2e: start Rekor test instance, run witness, verify checkpoint written, verify metrics non-zero.
|
||||
- Publish image SBOM/attestations and chart checksums.
|
||||
5) Monitoring/alerts:
|
||||
- `witness_verify_failures_total` > 0 over 5m
|
||||
- `witness_checkpoint_age_seconds` > 5400
|
||||
- `witness_backfill_queue_depth` (if supported) above threshold
|
||||
|
||||
## Offline/air-gap mode
|
||||
- Consume signed Rekor mirror (CAR + manifest) from bootstrap pack.
|
||||
- Run witness in verify-only mode against mirror; disable outbound network.
|
||||
- Emit checkpoints signed with offline key; store in mirror bundle for audit.
|
||||
|
||||
## Evidence to capture
|
||||
- Image digest, SBOM hash, chart checksum.
|
||||
- Signed checkpoint sample and DSSE manifest.
|
||||
- CI e2e logs and metrics sample (scrape output).
|
||||
|
||||
## Owners
|
||||
- Build/deploy: DevOps Guild
|
||||
- Keys/config: Platform Security
|
||||
- Observability: Observability Guild
|
||||
53
ops/devops/concelier/lnm-release-plan.md
Normal file
53
ops/devops/concelier/lnm-release-plan.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# Concelier LNM Release Plan (DEVOPS-LNM-21-101-REL / 102-REL / 103-REL)
|
||||
|
||||
Scope: package and publish Link-Not-Merge migrations/backfill/object-store seeds for release and offline kits.
|
||||
|
||||
## Artefacts
|
||||
- Migration bundles:
|
||||
- 21-101 shard/index migrations (`EnsureLinkNotMergeShardingAndTtlMigration`)
|
||||
- 21-102 backfill/tombstone/rollback scripts
|
||||
- 21-103 object-store seed bundle (once contract final)
|
||||
- Checksums (`SHA256SUMS`, signed)
|
||||
- SBOMs (spdx.json) for migration runner image/tooling
|
||||
- Cosign attestations for images/bundles
|
||||
- Offline kit slice tarball with all above + DSSE manifest
|
||||
|
||||
## Pipeline outline
|
||||
1) Build migration runner image (dotnet) with migrations baked; generate SBOM; pin digest.
|
||||
2) Export migration scripts/bundles to `artifacts/lnm/`.
|
||||
3) Create offline bundle:
|
||||
- `migrations/21-101/` (DLLs, scripts, README)
|
||||
- `migrations/21-102/` (backfill, rollback, README)
|
||||
- `seeds/object-store/` (placeholder until 21-103 dev output)
|
||||
- `SHA256SUMS` + `.sig`
|
||||
- SBOMs + cosign attestations
|
||||
4) Verification stage:
|
||||
- `dotnet test` on migration runner
|
||||
- `cosign verify-blob` for bundles
|
||||
- `sha256sum --check`
|
||||
5) Publish:
|
||||
- Upload to release bucket + offline kit
|
||||
- Record manifest (hashes, versions, digests)
|
||||
|
||||
## Runbook (apply in staging → prod)
|
||||
1) Take Mongo backup; freeze Concelier ingest.
|
||||
2) Apply 21-101 migrations (shards/TTL) — idempotent; record duration.
|
||||
3) Run 21-102 backfill with `--batch-size=500 --stop-on-error`; capture report hash.
|
||||
4) Validate counts (observations/linksets/events) and shard balance.
|
||||
5) Enable outbox publishers; monitor lag and errors.
|
||||
6) (When ready) apply 21-103 object-store migration: move raw payloads to object store; verify CAS URIs; keep GridFS read-only during move.
|
||||
|
||||
## Rollback
|
||||
- 21-101: restore from backup if shard layout breaks; migrations are idempotent.
|
||||
- 21-102: run rollback script (`ops/devops/scripts/rollback-lnm-backfill.js`); if inconsistent, restore backup.
|
||||
- 21-103: switch back to GridFS URI map; restore seeds.
|
||||
|
||||
## Monitoring/alerts
|
||||
- Migration error count > 0
|
||||
- Mongo oplog lag > 60s during backfill
|
||||
- Outbox backlog growth post-unfreeze
|
||||
|
||||
## Owners
|
||||
- DevOps Guild (pipeline + rollout)
|
||||
- Concelier Storage Guild (migration content)
|
||||
- Platform Security (signing policy)
|
||||
42
ops/devops/graph-indexer/release-plan.md
Normal file
42
ops/devops/graph-indexer/release-plan.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Graph Indexer Release/Offline Bundle Plan (DEVOPS-GRAPH-INDEX-28-010-REL)
|
||||
|
||||
## Goals
|
||||
- Publish signed Helm/Compose bundles for Graph Indexer with offline parity.
|
||||
- Provide SBOM + attestations for images/charts and reproducible artefacts for air-gap kits.
|
||||
|
||||
## Artefacts
|
||||
- Helm chart + values overrides (offline/airgap).
|
||||
- Docker/OCI images (indexer, api) pinned by digest.
|
||||
- SBOMs (SPDX JSON) for images and chart.
|
||||
- Cosign attestations for images and chart tarball.
|
||||
- Offline bundle: tarball containing images (oras layout), charts, values, SBOMs, attestations, and `SHA256SUMS`.
|
||||
|
||||
## Pipeline outline
|
||||
1) **Build** images (indexer + api) with SBOM generation (`syft`), tag and record digests.
|
||||
2) **Sign** images with cosign key (KMS for online; file key for offline bundle) and produce attestations.
|
||||
3) **Chart package**: render chart, package to `.tgz`, generate SBOM for chart, sign with cosign.
|
||||
4) **Compose export**: render Compose file with pinned digests and non-root users.
|
||||
5) **Bundle**: assemble offline tarball:
|
||||
- `images/` oras layout with signed images
|
||||
- `charts/graph-indexer.tgz` + signature
|
||||
- `compose/graph-indexer.yml` (pinned digests)
|
||||
- `sboms/` for images + chart
|
||||
- `attestations/` (cosign bundles)
|
||||
- `SHA256SUMS` and `SHA256SUMS.sig`
|
||||
6) **Verify step**: pipeline stage runs `cosign verify`, `sha256sum --check`, and `helm template` smoke render with airgap values.
|
||||
7) **Publish**: upload to artefact store + offline kit; write manifest with hashes/versions.
|
||||
|
||||
## Security/hardening
|
||||
- Non-root images, read-only rootfs, drop NET_RAW, seccomp default.
|
||||
- Telemetry disabled; no registry pulls at runtime.
|
||||
- mTLS between indexer and dependencies (documented values).
|
||||
|
||||
## Evidence to capture
|
||||
- Image digests, SBOM hashes, cosign verification logs.
|
||||
- Bundle `SHA256SUMS` and signed manifest.
|
||||
- Helm/Compose render outputs (short).
|
||||
|
||||
## Owners
|
||||
- DevOps Guild (build/pipeline)
|
||||
- Graph Indexer Guild (chart/values)
|
||||
- Platform Security (signing policy)
|
||||
48
ops/devops/scanner-java/release-plan.md
Normal file
48
ops/devops/scanner-java/release-plan.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Java Analyzer Release Plan (DEVOPS-SCANNER-JAVA-21-011-REL)
|
||||
|
||||
## Goal
|
||||
Publish the Java analyzer plug-in with signed artifacts and offline-ready bundles for CLI/Offline Kit.
|
||||
|
||||
## Inputs
|
||||
- Analyzer JAR(s) + native helpers from dev task 21-011.
|
||||
- SBOM (SPDX JSON) for plugin + native components.
|
||||
- Test suite outputs (unit + integration).
|
||||
|
||||
## Artifacts
|
||||
- OCI image (optional) or zip bundle containing:
|
||||
- `analyzer.jar`
|
||||
- `lib/` natives (if any)
|
||||
- `LICENSE`, `NOTICE`
|
||||
- `SBOM` (spdx.json)
|
||||
- `SIGNATURES` (cosign/PGP)
|
||||
- Cosign attestations for OCI/zip (provenance + SBOM).
|
||||
- Checksums: `SHA256SUMS`, `SHA256SUMS.sig`.
|
||||
- Offline kit slice: tarball with bundle + attestations + SBOM.
|
||||
|
||||
## Pipeline steps
|
||||
1) **Build**: run gradle/mvn with `--offline` using vendored deps; produce JAR + natives.
|
||||
2) **SBOM**: `syft packages -o spdx-json` over build output.
|
||||
3) **Package**: zip bundle with fixed ordering (`zip -X`) and normalized timestamps (`SOURCE_DATE_EPOCH`).
|
||||
4) **Sign**:
|
||||
- cosign sign blob (zip) and/or image.
|
||||
- generate in-toto provenance (SLSA level 1) referencing git commit + toolchain hashes.
|
||||
5) **Checksums**: create `SHA256SUMS` and sign with cosign/PGP.
|
||||
6) **Verify stage**: pipeline step runs `cosign verify-blob`, `sha256sum --check`, and `syft validate spdx`.
|
||||
7) **Publish**:
|
||||
- Upload to artifact store (release bucket) with metadata (version, commit, digest).
|
||||
- Produce offline kit slice tarball (`scanner-java-<ver>-offline.tgz`) containing bundle, SBOM, attestations, checksums.
|
||||
|
||||
## Security/hardening
|
||||
- Non-root build container; disable gradle/mvn network (`--offline`).
|
||||
- Strip debug info unless required; ensure reproducible JAR (sorted entries, normalized timestamps).
|
||||
- Telemetry disabled.
|
||||
|
||||
## Evidence to capture
|
||||
- Bundle SHA256, cosign signatures, provenance statement.
|
||||
- SBOM hash.
|
||||
- Verification logs from pipeline.
|
||||
|
||||
## Owners
|
||||
- Build/pipeline: DevOps Guild
|
||||
- Signing policy: Platform Security
|
||||
- Consumer integration: CLI Guild / Offline Kit Guild
|
||||
26
ops/devops/telemetry/tests/manifest-valid.json
Normal file
26
ops/devops/telemetry/tests/manifest-valid.json
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"schemaVersion": "1.0.0",
|
||||
"bundleId": "00000000-0000-0000-0000-000000000001",
|
||||
"createdAt": "2025-12-01T00:00:00Z",
|
||||
"profileHash": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"collectorVersion": "otelcol/1.0.0",
|
||||
"sealedMode": true,
|
||||
"redactionManifest": "redaction-manifest.json",
|
||||
"manifestHashAlgorithm": "sha256",
|
||||
"timeAnchor": {
|
||||
"type": "rfc3161",
|
||||
"value": "dummy-token"
|
||||
},
|
||||
"artifacts": [
|
||||
{
|
||||
"path": "logs.ndjson",
|
||||
"sha256": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
|
||||
"mediaType": "application/x-ndjson",
|
||||
"size": 123
|
||||
}
|
||||
],
|
||||
"dsseEnvelope": {
|
||||
"hash": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc",
|
||||
"location": "bundle.dsse.json"
|
||||
}
|
||||
}
|
||||
1
ops/devops/telemetry/tests/telemetry-bundle.sha256
Normal file
1
ops/devops/telemetry/tests/telemetry-bundle.sha256
Normal file
@@ -0,0 +1 @@
|
||||
6e3fedbf183aece5dfa14a90ebce955e2887d36747c424e628dc2cc03bcb0ed3 ops/devops/telemetry/tests/manifest-valid.json
|
||||
69
ops/devops/telemetry/verify-telemetry-bundle.sh
Normal file
69
ops/devops/telemetry/verify-telemetry-bundle.sh
Normal file
@@ -0,0 +1,69 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Minimal offline verifier for telemetry bundles (v1)
|
||||
# Exits:
|
||||
# 0 success
|
||||
# 21 checksum/manifest missing
|
||||
# 22 checksum mismatch
|
||||
# 23 schema validation failed
|
||||
|
||||
BUNDLE=${1:-}
|
||||
if [[ -z "$BUNDLE" ]]; then
|
||||
echo "Usage: $0 path/to/telemetry-bundle.tar" >&2
|
||||
exit 64
|
||||
fi
|
||||
|
||||
WORKDIR=$(mktemp -d)
|
||||
cleanup() { rm -rf "$WORKDIR"; }
|
||||
trap cleanup EXIT
|
||||
|
||||
tar --extract --file "$BUNDLE" --directory "$WORKDIR"
|
||||
|
||||
MANIFEST="$WORKDIR/telemetry-bundle.json"
|
||||
HASHES="$WORKDIR/telemetry-bundle.sha256"
|
||||
|
||||
if [[ ! -f "$MANIFEST" || ! -f "$HASHES" ]]; then
|
||||
echo "Missing manifest or checksum file." >&2
|
||||
exit 21
|
||||
fi
|
||||
|
||||
# Verify checksums
|
||||
pushd "$WORKDIR" >/dev/null
|
||||
if ! sha256sum --quiet --check telemetry-bundle.sha256; then
|
||||
echo "Checksum mismatch." >&2
|
||||
exit 22
|
||||
fi
|
||||
popd >/dev/null
|
||||
|
||||
# JSON schema validation (optional if jsonschema not present).
|
||||
if command -v python >/dev/null 2>&1; then
|
||||
SCHEMA_DIR="$(cd "$(dirname "$0")/../../docs/modules/telemetry/schemas" && pwd)"
|
||||
SCHEMA_FILE="$SCHEMA_DIR/telemetry-bundle.schema.json"
|
||||
if [[ -f "$SCHEMA_FILE" ]]; then
|
||||
python - "$MANIFEST" "$SCHEMA_FILE" <<'PY'
|
||||
import json, sys
|
||||
from jsonschema import validate, Draft202012Validator
|
||||
|
||||
manifest_path = sys.argv[1]
|
||||
schema_path = sys.argv[2]
|
||||
with open(manifest_path, 'r', encoding='utf-8') as f:
|
||||
manifest = json.load(f)
|
||||
with open(schema_path, 'r', encoding='utf-8') as f:
|
||||
schema = json.load(f)
|
||||
Draft202012Validator.check_schema(schema)
|
||||
validate(manifest, schema)
|
||||
PY
|
||||
if [[ $? -ne 0 ]]; then
|
||||
echo "Schema validation failed." >&2
|
||||
exit 23
|
||||
fi
|
||||
else
|
||||
echo "Schema file not found ($SCHEMA_FILE); skipping validation." >&2
|
||||
fi
|
||||
else
|
||||
echo "jsonschema validation skipped (requires python + jsonschema)." >&2
|
||||
fi
|
||||
|
||||
echo "Telemetry bundle verified." >&2
|
||||
exit 0
|
||||
Reference in New Issue
Block a user