stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 1 Releases Wiki Activity

feat(docs): Add comprehensive documentation for Vexer, Vulnerability Explorer, and Zastava modules

Browse Source 
- Introduced AGENTS.md, README.md, TASKS.md, and implementation_plan.md for Vexer, detailing mission, responsibilities, key components, and operational notes.
- Established similar documentation structure for Vulnerability Explorer and Zastava modules, including their respective workflows, integrations, and observability notes.
- Created risk scoring profiles documentation outlining the core workflow, factor model, governance, and deliverables.
- Ensured all modules adhere to the Aggregation-Only Contract and maintain determinism and provenance in outputs.
This commit is contained in:
master
2025-10-30 00:09:39 +02:00
parent 3154c67978
commit 7b5bdcf4d3
503 changed files with 16136 additions and 54638 deletions
Download Patch File Download Diff File Expand all files Collapse all files

320
docs/deploy/containers.md
View File

@@ -1,160 +1,160 @@
# Container Deployment Guide — AOC Update
> **Audience:** DevOps Guild, platform operators deploying StellaOps services.
> **Scope:** Deployment configuration changes required by the Aggregation-Only Contract (AOC), including schema validators, guard environment flags, and verifier identities.
This guide supplements existing deployment manuals with AOC-specific configuration. It assumes familiarity with the base Compose/Helm manifests described in `ops/deployment/` and `docs/ARCHITECTURE_DEVOPS.md`.
---
## 1 · Schema validator enablement
### 1.1 MongoDB validators
- Apply JSON schema validators to `advisory_raw` and `vex_raw` collections before enabling AOC guards.
- Before enabling validators or the idempotency index, run the duplicate audit helper to confirm no conflicting raw advisories remain:
```bash
mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'
```
Resolve any reported rows prior to rollout.
- Use the migration script provided in `ops/devops/scripts/apply-aoc-validators.js`:
```bash
kubectl exec -n concelier deploy/concelier-mongo -- \
mongo concelier ops/devops/scripts/apply-aoc-validators.js
kubectl exec -n excititor deploy/excititor-mongo -- \
mongo excititor ops/devops/scripts/apply-aoc-validators.js
```
- Validators enforce required fields (`tenant`, `source`, `upstream`, `linkset`) and reject forbidden keys at DB level.
- Rollback plan: validators are applied with `validationLevel: moderate`—downgrade via the same script with `--remove` if required.
### 1.2 Migration order
1. Deploy validators in maintenance window.
2. Roll out Concelier/Excititor images with guard middleware enabled (`AOC_GUARD_ENABLED=true`).
3. Run smoke tests (`stella sources ingest --dry-run` fixtures) before resuming production ingestion.
### 1.3 Supersedes backfill verification
1. **Duplicate audit:** Confirm `mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'` reports no conflicts before restarting Concelier with the new migrations.
2. **Post-migration check:** After the service restarts, validate that `db.advisory` is a view pointing to `advisory_backup_20251028`:
```bash
mongo concelier --quiet --eval 'db.getCollectionInfos({ name: "advisory" })[0]'
```
The `type` should be `"view"` and `options.viewOn` should equal `"advisory_backup_20251028"`.
3. **Supersedes chain spot-check:** Inspect a sample set to ensure deterministic chaining:
```bash
mongo concelier --quiet --eval '
db.advisory_raw.aggregate([
{ $match: { "upstream.upstream_id": { $exists: true } } },
{ $sort: { "tenant": 1, "source.vendor": 1, "upstream.upstream_id": 1, "upstream.retrieved_at": 1 } },
{ $limit: 5 },
{ $project: { _id: 1, supersedes: 1 } }
]).forEach(printjson)'
```
Each revision should reference the previous `_id` (or `null` for the first revision). Record findings in the change ticket before proceeding to production.
---
## 2 · Container environment flags
Add the following environment variables to Concelier/Excititor deployments:
| Variable | Default | Description |
|----------|---------|-------------|
| `AOC_GUARD_ENABLED` | `true` | Enables `AOCWriteGuard` interception. Set `false` only for controlled rollback. |
| `AOC_ALLOW_SUPERSEDES_RETROFIT` | `false` | Allows temporary supersedes backfill during migration. Remove after cutover. |
| `AOC_METRICS_ENABLED` | `true` | Emits `ingestion_write_total`, `aoc_violation_total`, etc. |
| `AOC_TENANT_HEADER` | `X-Stella-Tenant` | Header name expected from Gateway. |
| `AOC_VERIFIER_USER` | `stella-aoc-verify` | Read-only service user used by UI/CLI verification. |
Compose snippet:
```yaml
environment:
- AOC_GUARD_ENABLED=true
- AOC_ALLOW_SUPERSEDES_RETROFIT=false
- AOC_METRICS_ENABLED=true
- AOC_TENANT_HEADER=X-Stella-Tenant
- AOC_VERIFIER_USER=stella-aoc-verify
```
Ensure `AOC_VERIFIER_USER` exists in Authority with `aoc:verify` scope and no write permissions.
---
## 3 · Verifier identity
- Create a dedicated client (`stella-aoc-verify`) via Authority bootstrap:
```yaml
clients:
- clientId: stella-aoc-verify
grantTypes: [client_credentials]
scopes: [aoc:verify, advisory:read, vex:read]
tenants: [default]
```
- Store credentials in secret store (`Kubernetes Secret`, `Docker swarm secret`).
- Bind credentials to `stella aoc verify` CI jobs and Console verification service.
- Rotate quarterly; document in `ops/authority-key-rotation.md`.
---
## 4 · Deployment steps
1. **Pre-checks:** Confirm database backups, alerting in maintenance mode, and staging environment validated.
2. **Apply validators:** Run scripts per § 1.1.
3. **Update manifests:** Inject environment variables (§ 2) and mount guard configuration configmaps.
4. **Redeploy services:** Rolling restart Concelier/Excititor pods. Monitor `ingestion_write_total` for steady throughput.
5. **Seed verifier:** Deploy read-only verifier user and store credentials.
6. **Run verification:** Execute `stella aoc verify --since 24h` and ensure exit code `0`.
7. **Update dashboards:** Point Grafana panels to new metrics (`aoc_violation_total`).
8. **Record handoff:** Capture console screenshots and verification logs for release notes.
---
## 5 · Offline Kit updates
- Ship validator scripts with Offline Kit (`offline-kit/scripts/apply-aoc-validators.js`).
- Include pre-generated verification reports for air-gapped deployments.
- Document offline CLI workflow in bundle README referencing `docs/cli/cli-reference.md`.
- Ensure `stella-aoc-verify` credentials are scoped to offline tenant and rotated during bundle refresh.
---
## 6 · Rollback plan
1. Disable guard via `AOC_GUARD_ENABLED=false` on Concelier/Excititor and rollout.
2. Remove validators with the migration script (`--remove`).
3. Pause verification jobs to prevent noise.
4. Investigate and remediate upstream issues before re-enabling guards.
---
## 7 · References
- [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md)
- [Authority scopes & tenancy](../security/authority-scopes.md)
- [Observability guide](../observability/observability.md)
- [CLI AOC commands](../cli/cli-reference.md)
- [Concelier architecture](../ARCHITECTURE_CONCELIER.md)
- [Excititor architecture](../ARCHITECTURE_EXCITITOR.md)
---
## 8 · Compliance checklist
- [ ] Validators documented and scripts referenced for online/offline deployments.
- [ ] Environment variables cover guard enablement, metrics, and tenant header.
- [ ] Read-only verifier user installation steps included.
- [ ] Offline kit instructions align with validator/verification workflow.
- [ ] Rollback procedure captured.
- [ ] Cross-links to AOC docs, Authority scopes, and observability guides present.
- [ ] DevOps Guild sign-off tracked (owner: @devops-guild, due 2025-10-29).
---
*Last updated: 2025-10-26 (Sprint 19).*
# Container Deployment Guide — AOC Update
> **Audience:** DevOps Guild, platform operators deploying StellaOps services.
> **Scope:** Deployment configuration changes required by the Aggregation-Only Contract (AOC), including schema validators, guard environment flags, and verifier identities.
This guide supplements existing deployment manuals with AOC-specific configuration. It assumes familiarity with the base Compose/Helm manifests described in `ops/deployment/` and `docs/modules/devops/architecture.md`.
---
## 1 · Schema validator enablement
### 1.1 MongoDB validators
- Apply JSON schema validators to `advisory_raw` and `vex_raw` collections before enabling AOC guards.
- Before enabling validators or the idempotency index, run the duplicate audit helper to confirm no conflicting raw advisories remain:
```bash
mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'
```
Resolve any reported rows prior to rollout.
- Use the migration script provided in `ops/devops/scripts/apply-aoc-validators.js`:
```bash
kubectl exec -n concelier deploy/concelier-mongo -- \
mongo concelier ops/devops/scripts/apply-aoc-validators.js
kubectl exec -n excititor deploy/excititor-mongo -- \
mongo excititor ops/devops/scripts/apply-aoc-validators.js
```
- Validators enforce required fields (`tenant`, `source`, `upstream`, `linkset`) and reject forbidden keys at DB level.
- Rollback plan: validators are applied with `validationLevel: moderate`—downgrade via the same script with `--remove` if required.
### 1.2 Migration order
1. Deploy validators in maintenance window.
2. Roll out Concelier/Excititor images with guard middleware enabled (`AOC_GUARD_ENABLED=true`).
3. Run smoke tests (`stella sources ingest --dry-run` fixtures) before resuming production ingestion.
### 1.3 Supersedes backfill verification
1. **Duplicate audit:** Confirm `mongo concelier ops/devops/scripts/check-advisory-raw-duplicates.js --eval 'var LIMIT=200;'` reports no conflicts before restarting Concelier with the new migrations.
2. **Post-migration check:** After the service restarts, validate that `db.advisory` is a view pointing to `advisory_backup_20251028`:
```bash
mongo concelier --quiet --eval 'db.getCollectionInfos({ name: "advisory" })[0]'
```
The `type` should be `"view"` and `options.viewOn` should equal `"advisory_backup_20251028"`.
3. **Supersedes chain spot-check:** Inspect a sample set to ensure deterministic chaining:
```bash
mongo concelier --quiet --eval '
db.advisory_raw.aggregate([
{ $match: { "upstream.upstream_id": { $exists: true } } },
{ $sort: { "tenant": 1, "source.vendor": 1, "upstream.upstream_id": 1, "upstream.retrieved_at": 1 } },
{ $limit: 5 },
{ $project: { _id: 1, supersedes: 1 } }
]).forEach(printjson)'
```
Each revision should reference the previous `_id` (or `null` for the first revision). Record findings in the change ticket before proceeding to production.
---
## 2 · Container environment flags
Add the following environment variables to Concelier/Excititor deployments:
| Variable | Default | Description |
|----------|---------|-------------|
| `AOC_GUARD_ENABLED` | `true` | Enables `AOCWriteGuard` interception. Set `false` only for controlled rollback. |
| `AOC_ALLOW_SUPERSEDES_RETROFIT` | `false` | Allows temporary supersedes backfill during migration. Remove after cutover. |
| `AOC_METRICS_ENABLED` | `true` | Emits `ingestion_write_total`, `aoc_violation_total`, etc. |
| `AOC_TENANT_HEADER` | `X-Stella-Tenant` | Header name expected from Gateway. |
| `AOC_VERIFIER_USER` | `stella-aoc-verify` | Read-only service user used by UI/CLI verification. |
Compose snippet:
```yaml
environment:
- AOC_GUARD_ENABLED=true
- AOC_ALLOW_SUPERSEDES_RETROFIT=false
- AOC_METRICS_ENABLED=true
- AOC_TENANT_HEADER=X-Stella-Tenant
- AOC_VERIFIER_USER=stella-aoc-verify
```
Ensure `AOC_VERIFIER_USER` exists in Authority with `aoc:verify` scope and no write permissions.
---
## 3 · Verifier identity
- Create a dedicated client (`stella-aoc-verify`) via Authority bootstrap:
```yaml
clients:
- clientId: stella-aoc-verify
grantTypes: [client_credentials]
scopes: [aoc:verify, advisory:read, vex:read]
tenants: [default]
```
- Store credentials in secret store (`Kubernetes Secret`, `Docker swarm secret`).
- Bind credentials to `stella aoc verify` CI jobs and Console verification service.
- Rotate quarterly; document in `ops/authority-key-rotation.md`.
---
## 4 · Deployment steps
1. **Pre-checks:** Confirm database backups, alerting in maintenance mode, and staging environment validated.
2. **Apply validators:** Run scripts per § 1.1.
3. **Update manifests:** Inject environment variables (§ 2) and mount guard configuration configmaps.
4. **Redeploy services:** Rolling restart Concelier/Excititor pods. Monitor `ingestion_write_total` for steady throughput.
5. **Seed verifier:** Deploy read-only verifier user and store credentials.
6. **Run verification:** Execute `stella aoc verify --since 24h` and ensure exit code `0`.
7. **Update dashboards:** Point Grafana panels to new metrics (`aoc_violation_total`).
8. **Record handoff:** Capture console screenshots and verification logs for release notes.
---
## 5 · Offline Kit updates
- Ship validator scripts with Offline Kit (`offline-kit/scripts/apply-aoc-validators.js`).
- Include pre-generated verification reports for air-gapped deployments.
- Document offline CLI workflow in bundle README referencing `docs/modules/cli/guides/cli-reference.md`.
- Ensure `stella-aoc-verify` credentials are scoped to offline tenant and rotated during bundle refresh.
---
## 6 · Rollback plan
1. Disable guard via `AOC_GUARD_ENABLED=false` on Concelier/Excititor and rollout.
2. Remove validators with the migration script (`--remove`).
3. Pause verification jobs to prevent noise.
4. Investigate and remediate upstream issues before re-enabling guards.
---
## 7 · References
- [Aggregation-Only Contract reference](../ingestion/aggregation-only-contract.md)
- [Authority scopes & tenancy](../security/authority-scopes.md)
- [Observability guide](../observability/observability.md)
- [CLI AOC commands](../modules/cli/guides/cli-reference.md)
- [Concelier architecture](../modules/concelier/architecture.md)
- [Excititor architecture](../modules/excititor/architecture.md)
---
## 8 · Compliance checklist
- [ ] Validators documented and scripts referenced for online/offline deployments.
- [ ] Environment variables cover guard enablement, metrics, and tenant header.
- [ ] Read-only verifier user installation steps included.
- [ ] Offline kit instructions align with validator/verification workflow.
- [ ] Rollback procedure captured.
- [ ] Cross-links to AOC docs, Authority scopes, and observability guides present.
- [ ] DevOps Guild sign-off tracked (owner: @devops-guild, due 2025-10-29).
---
*Last updated: 2025-10-26 (Sprint 19).*