stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 1 Releases Wiki Activity

feat: Add new projects to solution and implement contract testing documentation

Browse Source 
- Added "StellaOps.Policy.Engine", "StellaOps.Cartographer", and "StellaOps.SbomService" projects to the StellaOps solution.
- Created AGENTS.md to outline the Contract Testing Guild Charter, detailing mission, scope, and definition of done.
- Established TASKS.md for the Contract Testing Task Board, outlining tasks for Sprint 62 and Sprint 63 related to mock servers and replay testing.
This commit is contained in:
master
2025-10-27 07:57:55 +02:00
parent 1e41ba7ffa
commit 651b8e0fa3
355 changed files with 17276 additions and 1160 deletions
Download Patch File Download Diff File Expand all files Collapse all files

474
.gitea/workflows/build-test-deploy.yml
View File

@@ -1,40 +1,40 @@
# .gitea/workflows/build-test-deploy.yml
# Unified CI/CD workflow for git.stella-ops.org (Feedser monorepo)
name: Build Test Deploy
on:
push:
branches: [ main ]
paths:
- 'src/**'
- 'docs/**'
- 'scripts/**'
- 'Directory.Build.props'
- 'Directory.Build.targets'
- 'global.json'
- '.gitea/workflows/**'
pull_request:
branches: [ main, develop ]
paths:
- 'src/**'
- 'docs/**'
- 'scripts/**'
- '.gitea/workflows/**'
workflow_dispatch:
inputs:
force_deploy:
description: 'Ignore branch checks and run the deploy stage'
required: false
default: 'false'
type: boolean
env:
DOTNET_VERSION: '10.0.100-rc.1.25451.107'
BUILD_CONFIGURATION: Release
CI_CACHE_ROOT: /data/.cache/stella-ops/feedser
RUNNER_TOOL_CACHE: /toolcache
# .gitea/workflows/build-test-deploy.yml
# Unified CI/CD workflow for git.stella-ops.org (Feedser monorepo)
name: Build Test Deploy
on:
push:
branches: [ main ]
paths:
- 'src/**'
- 'docs/**'
- 'scripts/**'
- 'Directory.Build.props'
- 'Directory.Build.targets'
- 'global.json'
- '.gitea/workflows/**'
pull_request:
branches: [ main, develop ]
paths:
- 'src/**'
- 'docs/**'
- 'scripts/**'
- '.gitea/workflows/**'
workflow_dispatch:
inputs:
force_deploy:
description: 'Ignore branch checks and run the deploy stage'
required: false
default: 'false'
type: boolean
env:
DOTNET_VERSION: '10.0.100-rc.1.25451.107'
BUILD_CONFIGURATION: Release
CI_CACHE_ROOT: /data/.cache/stella-ops/feedser
RUNNER_TOOL_CACHE: /toolcache
jobs:
profile-validation:
runs-on: ubuntu-22.04
@@ -58,24 +58,24 @@ jobs:
PUBLISH_DIR: ${{ github.workspace }}/artifacts/publish/webservice
AUTHORITY_PUBLISH_DIR: ${{ github.workspace }}/artifacts/publish/authority
TEST_RESULTS_DIR: ${{ github.workspace }}/artifacts/test-results
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup .NET ${{ env.DOTNET_VERSION }}
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ env.DOTNET_VERSION }}
include-prerelease: true
- name: Restore dependencies
run: dotnet restore src/StellaOps.Feedser.sln
- name: Build solution (warnings as errors)
run: dotnet build src/StellaOps.Feedser.sln --configuration $BUILD_CONFIGURATION --no-restore -warnaserror
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup .NET ${{ env.DOTNET_VERSION }}
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ env.DOTNET_VERSION }}
include-prerelease: true
- name: Restore dependencies
run: dotnet restore src/StellaOps.Feedser.sln
- name: Build solution (warnings as errors)
run: dotnet build src/StellaOps.Feedser.sln --configuration $BUILD_CONFIGURATION --no-restore -warnaserror
- name: Run unit and integration tests
run: |
mkdir -p "$TEST_RESULTS_DIR"
@@ -118,11 +118,11 @@ jobs:
CAPTURED_AT="$(date -u +"%Y-%m-%dT%H:%M:%SZ")"
dotnet run \
--project bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj \
--project src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj \
--configuration $BUILD_CONFIGURATION \
-- \
--repo-root . \
--baseline bench/Scanner.Analyzers/baseline.csv \
--baseline src/StellaOps.Bench/Scanner.Analyzers/baseline.csv \
--out "$PERF_OUTPUT_DIR/latest.csv" \
--json "$PERF_OUTPUT_DIR/report.json" \
--prom "$PERF_OUTPUT_DIR/metrics.prom" \
@@ -269,7 +269,7 @@ PY
path: ${{ env.AUTHORITY_PUBLISH_DIR }}
if-no-files-found: error
retention-days: 7
- name: Upload test results
if: always()
uses: actions/upload-artifact@v4
@@ -297,27 +297,27 @@ PY
env:
DOCS_OUTPUT_DIR: ${{ github.workspace }}/artifacts/docs-site
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install documentation dependencies
run: |
python -m pip install --upgrade pip
python -m pip install markdown pygments
- name: Render documentation bundle
run: |
python scripts/render_docs.py --source docs --output "$DOCS_OUTPUT_DIR" --clean
- name: Upload documentation artifact
uses: actions/upload-artifact@v4
with:
name: feedser-docs-site
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install documentation dependencies
run: |
python -m pip install --upgrade pip
python -m pip install markdown pygments
- name: Render documentation bundle
run: |
python scripts/render_docs.py --source docs --output "$DOCS_OUTPUT_DIR" --clean
- name: Upload documentation artifact
uses: actions/upload-artifact@v4
with:
name: feedser-docs-site
path: ${{ env.DOCS_OUTPUT_DIR }}
if-no-files-found: error
retention-days: 7
@@ -326,7 +326,7 @@ PY
runs-on: ubuntu-22.04
needs: build-test
env:
BENCH_DIR: bench/Scanner.Analyzers
BENCH_DIR: src/StellaOps.Bench/Scanner.Analyzers
steps:
- name: Checkout repository
uses: actions/checkout@v4
@@ -412,163 +412,163 @@ PY
needs.scanner-perf.result == 'success' &&
(
(github.event_name == 'push' && github.ref == 'refs/heads/main') ||
github.event_name == 'workflow_dispatch'
)
environment: staging
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
sparse-checkout: |
scripts
.gitea/workflows
sparse-checkout-cone-mode: true
- name: Check if deployment should proceed
id: check-deploy
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
if [ "${{ github.event.inputs.force_deploy }}" = "true" ]; then
echo "should-deploy=true" >> $GITHUB_OUTPUT
echo "✅ Manual deployment requested"
else
echo "should-deploy=false" >> $GITHUB_OUTPUT
echo "ℹ️ Manual dispatch without force_deploy=true — skipping"
fi
elif [ "${{ github.ref }}" = "refs/heads/main" ]; then
echo "should-deploy=true" >> $GITHUB_OUTPUT
echo "✅ Deploying latest main branch build"
else
echo "should-deploy=false" >> $GITHUB_OUTPUT
echo "ℹ️ Deployment restricted to main branch"
fi
- name: Resolve deployment credentials
id: params
if: steps.check-deploy.outputs.should-deploy == 'true'
run: |
missing=()
host="${{ secrets.STAGING_DEPLOYMENT_HOST }}"
if [ -z "$host" ]; then host="${{ vars.STAGING_DEPLOYMENT_HOST }}"; fi
if [ -z "$host" ]; then host="${{ secrets.DEPLOYMENT_HOST }}"; fi
if [ -z "$host" ]; then host="${{ vars.DEPLOYMENT_HOST }}"; fi
if [ -z "$host" ]; then missing+=("STAGING_DEPLOYMENT_HOST"); fi
user="${{ secrets.STAGING_DEPLOYMENT_USERNAME }}"
if [ -z "$user" ]; then user="${{ vars.STAGING_DEPLOYMENT_USERNAME }}"; fi
if [ -z "$user" ]; then user="${{ secrets.DEPLOYMENT_USERNAME }}"; fi
if [ -z "$user" ]; then user="${{ vars.DEPLOYMENT_USERNAME }}"; fi
if [ -z "$user" ]; then missing+=("STAGING_DEPLOYMENT_USERNAME"); fi
path="${{ secrets.STAGING_DEPLOYMENT_PATH }}"
if [ -z "$path" ]; then path="${{ vars.STAGING_DEPLOYMENT_PATH }}"; fi
docs_path="${{ secrets.STAGING_DOCS_PATH }}"
if [ -z "$docs_path" ]; then docs_path="${{ vars.STAGING_DOCS_PATH }}"; fi
key="${{ secrets.STAGING_DEPLOYMENT_KEY }}"
if [ -z "$key" ]; then key="${{ secrets.DEPLOYMENT_KEY }}"; fi
if [ -z "$key" ]; then key="${{ vars.STAGING_DEPLOYMENT_KEY }}"; fi
if [ -z "$key" ]; then key="${{ vars.DEPLOYMENT_KEY }}"; fi
if [ -z "$key" ]; then missing+=("STAGING_DEPLOYMENT_KEY"); fi
if [ ${#missing[@]} -gt 0 ]; then
echo "❌ Missing deployment configuration: ${missing[*]}"
exit 1
fi
key_file="$RUNNER_TEMP/staging_deploy_key"
printf '%s\n' "$key" > "$key_file"
chmod 600 "$key_file"
echo "host=$host" >> $GITHUB_OUTPUT
echo "user=$user" >> $GITHUB_OUTPUT
echo "path=$path" >> $GITHUB_OUTPUT
echo "docs-path=$docs_path" >> $GITHUB_OUTPUT
echo "key-file=$key_file" >> $GITHUB_OUTPUT
- name: Download service artifact
if: steps.check-deploy.outputs.should-deploy == 'true' && steps.params.outputs.path != ''
uses: actions/download-artifact@v4
with:
name: feedser-publish
path: artifacts/service
- name: Download documentation artifact
if: steps.check-deploy.outputs.should-deploy == 'true' && steps.params.outputs['docs-path'] != ''
uses: actions/download-artifact@v4
with:
name: feedser-docs-site
path: artifacts/docs
- name: Install rsync
if: steps.check-deploy.outputs.should-deploy == 'true'
run: |
if command -v rsync >/dev/null 2>&1; then
exit 0
fi
CACHE_DIR="${CI_CACHE_ROOT:-/tmp}/apt"
mkdir -p "$CACHE_DIR"
KEY="rsync-$(lsb_release -rs 2>/dev/null || echo unknown)"
DEB_DIR="$CACHE_DIR/$KEY"
mkdir -p "$DEB_DIR"
if ls "$DEB_DIR"/rsync*.deb >/dev/null 2>&1; then
apt-get update
apt-get install -y --no-install-recommends "$DEB_DIR"/libpopt0*.deb "$DEB_DIR"/rsync*.deb
else
apt-get update
apt-get download rsync libpopt0
mv rsync*.deb libpopt0*.deb "$DEB_DIR"/
dpkg -i "$DEB_DIR"/libpopt0*.deb "$DEB_DIR"/rsync*.deb || apt-get install -f -y
fi
- name: Deploy service bundle
if: steps.check-deploy.outputs.should-deploy == 'true' && steps.params.outputs.path != ''
env:
HOST: ${{ steps.params.outputs.host }}
USER: ${{ steps.params.outputs.user }}
TARGET: ${{ steps.params.outputs.path }}
KEY_FILE: ${{ steps.params.outputs['key-file'] }}
run: |
SERVICE_DIR="artifacts/service/feedser-publish"
if [ ! -d "$SERVICE_DIR" ]; then
echo "❌ Service artifact directory missing ($SERVICE_DIR)"
exit 1
fi
echo "🚀 Deploying Feedser web service to $HOST:$TARGET"
rsync -az --delete \
-e "ssh -i $KEY_FILE -o StrictHostKeyChecking=no" \
"$SERVICE_DIR"/ \
"$USER@$HOST:$TARGET/"
- name: Deploy documentation bundle
if: steps.check-deploy.outputs.should-deploy == 'true' && steps.params.outputs['docs-path'] != ''
env:
HOST: ${{ steps.params.outputs.host }}
USER: ${{ steps.params.outputs.user }}
DOCS_TARGET: ${{ steps.params.outputs['docs-path'] }}
KEY_FILE: ${{ steps.params.outputs['key-file'] }}
run: |
DOCS_DIR="artifacts/docs/feedser-docs-site"
if [ ! -d "$DOCS_DIR" ]; then
echo "❌ Documentation artifact directory missing ($DOCS_DIR)"
exit 1
fi
echo "📚 Deploying documentation bundle to $HOST:$DOCS_TARGET"
rsync -az --delete \
-e "ssh -i $KEY_FILE -o StrictHostKeyChecking=no" \
"$DOCS_DIR"/ \
"$USER@$HOST:$DOCS_TARGET/"
- name: Deployment summary
if: steps.check-deploy.outputs.should-deploy == 'true'
run: |
echo "✅ Deployment completed"
echo " Host: ${{ steps.params.outputs.host }}"
echo " Service path: ${{ steps.params.outputs.path || '(skipped)' }}"
echo " Docs path: ${{ steps.params.outputs['docs-path'] || '(skipped)' }}"
github.event_name == 'workflow_dispatch'
)
environment: staging
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
sparse-checkout: |
scripts
.gitea/workflows
sparse-checkout-cone-mode: true
- name: Check if deployment should proceed
id: check-deploy
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
if [ "${{ github.event.inputs.force_deploy }}" = "true" ]; then
echo "should-deploy=true" >> $GITHUB_OUTPUT
echo "✅ Manual deployment requested"
else
echo "should-deploy=false" >> $GITHUB_OUTPUT
echo "ℹ️ Manual dispatch without force_deploy=true — skipping"
fi
elif [ "${{ github.ref }}" = "refs/heads/main" ]; then
echo "should-deploy=true" >> $GITHUB_OUTPUT
echo "✅ Deploying latest main branch build"
else
echo "should-deploy=false" >> $GITHUB_OUTPUT
echo "ℹ️ Deployment restricted to main branch"
fi
- name: Resolve deployment credentials
id: params
if: steps.check-deploy.outputs.should-deploy == 'true'
run: |
missing=()
host="${{ secrets.STAGING_DEPLOYMENT_HOST }}"
if [ -z "$host" ]; then host="${{ vars.STAGING_DEPLOYMENT_HOST }}"; fi
if [ -z "$host" ]; then host="${{ secrets.DEPLOYMENT_HOST }}"; fi
if [ -z "$host" ]; then host="${{ vars.DEPLOYMENT_HOST }}"; fi
if [ -z "$host" ]; then missing+=("STAGING_DEPLOYMENT_HOST"); fi
user="${{ secrets.STAGING_DEPLOYMENT_USERNAME }}"
if [ -z "$user" ]; then user="${{ vars.STAGING_DEPLOYMENT_USERNAME }}"; fi
if [ -z "$user" ]; then user="${{ secrets.DEPLOYMENT_USERNAME }}"; fi
if [ -z "$user" ]; then user="${{ vars.DEPLOYMENT_USERNAME }}"; fi
if [ -z "$user" ]; then missing+=("STAGING_DEPLOYMENT_USERNAME"); fi
path="${{ secrets.STAGING_DEPLOYMENT_PATH }}"
if [ -z "$path" ]; then path="${{ vars.STAGING_DEPLOYMENT_PATH }}"; fi
docs_path="${{ secrets.STAGING_DOCS_PATH }}"
if [ -z "$docs_path" ]; then docs_path="${{ vars.STAGING_DOCS_PATH }}"; fi
key="${{ secrets.STAGING_DEPLOYMENT_KEY }}"
if [ -z "$key" ]; then key="${{ secrets.DEPLOYMENT_KEY }}"; fi
if [ -z "$key" ]; then key="${{ vars.STAGING_DEPLOYMENT_KEY }}"; fi
if [ -z "$key" ]; then key="${{ vars.DEPLOYMENT_KEY }}"; fi
if [ -z "$key" ]; then missing+=("STAGING_DEPLOYMENT_KEY"); fi
if [ ${#missing[@]} -gt 0 ]; then
echo "❌ Missing deployment configuration: ${missing[*]}"
exit 1
fi
key_file="$RUNNER_TEMP/staging_deploy_key"
printf '%s\n' "$key" > "$key_file"
chmod 600 "$key_file"
echo "host=$host" >> $GITHUB_OUTPUT
echo "user=$user" >> $GITHUB_OUTPUT
echo "path=$path" >> $GITHUB_OUTPUT
echo "docs-path=$docs_path" >> $GITHUB_OUTPUT
echo "key-file=$key_file" >> $GITHUB_OUTPUT
- name: Download service artifact
if: steps.check-deploy.outputs.should-deploy == 'true' && steps.params.outputs.path != ''
uses: actions/download-artifact@v4
with:
name: feedser-publish
path: artifacts/service
- name: Download documentation artifact
if: steps.check-deploy.outputs.should-deploy == 'true' && steps.params.outputs['docs-path'] != ''
uses: actions/download-artifact@v4
with:
name: feedser-docs-site
path: artifacts/docs
- name: Install rsync
if: steps.check-deploy.outputs.should-deploy == 'true'
run: |
if command -v rsync >/dev/null 2>&1; then
exit 0
fi
CACHE_DIR="${CI_CACHE_ROOT:-/tmp}/apt"
mkdir -p "$CACHE_DIR"
KEY="rsync-$(lsb_release -rs 2>/dev/null || echo unknown)"
DEB_DIR="$CACHE_DIR/$KEY"
mkdir -p "$DEB_DIR"
if ls "$DEB_DIR"/rsync*.deb >/dev/null 2>&1; then
apt-get update
apt-get install -y --no-install-recommends "$DEB_DIR"/libpopt0*.deb "$DEB_DIR"/rsync*.deb
else
apt-get update
apt-get download rsync libpopt0
mv rsync*.deb libpopt0*.deb "$DEB_DIR"/
dpkg -i "$DEB_DIR"/libpopt0*.deb "$DEB_DIR"/rsync*.deb || apt-get install -f -y
fi
- name: Deploy service bundle
if: steps.check-deploy.outputs.should-deploy == 'true' && steps.params.outputs.path != ''
env:
HOST: ${{ steps.params.outputs.host }}
USER: ${{ steps.params.outputs.user }}
TARGET: ${{ steps.params.outputs.path }}
KEY_FILE: ${{ steps.params.outputs['key-file'] }}
run: |
SERVICE_DIR="artifacts/service/feedser-publish"
if [ ! -d "$SERVICE_DIR" ]; then
echo "❌ Service artifact directory missing ($SERVICE_DIR)"
exit 1
fi
echo "🚀 Deploying Feedser web service to $HOST:$TARGET"
rsync -az --delete \
-e "ssh -i $KEY_FILE -o StrictHostKeyChecking=no" \
"$SERVICE_DIR"/ \
"$USER@$HOST:$TARGET/"
- name: Deploy documentation bundle
if: steps.check-deploy.outputs.should-deploy == 'true' && steps.params.outputs['docs-path'] != ''
env:
HOST: ${{ steps.params.outputs.host }}
USER: ${{ steps.params.outputs.user }}
DOCS_TARGET: ${{ steps.params.outputs['docs-path'] }}
KEY_FILE: ${{ steps.params.outputs['key-file'] }}
run: |
DOCS_DIR="artifacts/docs/feedser-docs-site"
if [ ! -d "$DOCS_DIR" ]; then
echo "❌ Documentation artifact directory missing ($DOCS_DIR)"
exit 1
fi
echo "📚 Deploying documentation bundle to $HOST:$DOCS_TARGET"
rsync -az --delete \
-e "ssh -i $KEY_FILE -o StrictHostKeyChecking=no" \
"$DOCS_DIR"/ \
"$USER@$HOST:$DOCS_TARGET/"
- name: Deployment summary
if: steps.check-deploy.outputs.should-deploy == 'true'
run: |
echo "✅ Deployment completed"
echo " Host: ${{ steps.params.outputs.host }}"
echo " Service path: ${{ steps.params.outputs.path || '(skipped)' }}"
echo " Docs path: ${{ steps.params.outputs['docs-path'] || '(skipped)' }}"
- name: Deployment skipped summary
if: steps.check-deploy.outputs.should-deploy != 'true'
run: |

7
AGENTS.md
View File

@@ -96,16 +96,17 @@ You main characteristics:
Maintains this agent framework, templates, and per‑directory guides; assists parallelization and reviews.
## 5.2) Work-in-parallel rules (important)
## 5.2) Work rules (important)
- **Directory ownership**: Each agent works **only inside its module directory**. Cross‑module edits require a brief handshake in issues/PR description.
- **Scoping**: Use each module’s `AGENTS.md` and `TASKS.md` to plan; autonomous agents must read `src/AGENTS.md` and the module docs before acting.
- **Determinism**: Sort keys, normalize timestamps to UTC ISO‑8601, avoid non‑deterministic data in exports and tests.
- **Status tracking**: Update your module’s `TASKS.md` as you progress (TODO → DOING → DONE/BLOCKED). Before starting of actual work - ensure you have set the task to DOING. When complete or stop update the status in corresponding TASKS.md and in ./SPRINTS.md and ./EXECPLAN.md file.
- **Status tracking**: Update your module’s `TASKS.md` as you progress (TODO → DOING → DONE/BLOCKED). Before starting of actual work - ensure you have set the task to DOING. When complete or stop update the status in corresponding TASKS.md and in ./SPRINTS.md file.
- **Coordination**: In case task is discovered as blocked on other team or task, according TASKS.md files that dependency is on needs to be changed by adding new tasks describing the requirement. the current task must be updated as completed. In case task changes, scope or requirements or rules - other documentations needs be updated accordingly.
- **Sprint synchronization**: When given task seek for relevant directory to work on from SPRINTS.md. Confirm its state on both SPRINTS.md and EXECPLAN.md and the relevant TASKS.md file. Always check the AGENTS.md in the relevant TASKS.md directory.
- **Sprint synchronization**: When given task seek for relevant directory to work on from SPRINTS.md. Confirm its state on both SPRINTS.md and the relevant TASKS.md file. Always check the AGENTS.md in the relevant TASKS.md directory.
- **Tests**: Add/extend fixtures and unit tests per change; never regress determinism or precedence.
- **Test layout**: Use module-specific projects in `StellaOps.Concelier.<Component>.Tests`; shared fixtures/harnesses live in `StellaOps.Concelier.Testing`.
- **Execution autonomous**: In case you need to continue with more than one options just continue sequentially, unless the continue requires design decision.
- **EPIC reference**: In case task (on relevant TASKS.md file) references in direct or indirect way an EPIC - then seek for epic document like ./EPIC_*.md. There will be more information how to implement the task.
---

524
EPIC_1.md Normal file
View File

@@ -0,0 +1,524 @@
Here’s the full write‑up you can drop into your repo as the canonical reference for Epic 1. It’s written in clean product‑doc style so it’s safe to check in as Markdown. No fluff, just everything you need to build, test, and police it.
---
# Epic 1: Aggregation‑Only Contract (AOC) Enforcement
> Short name: **AOC enforcement**
> Services touched: **Conseiller (advisory ingestion), Excitator (VEX ingestion), Web API, Workers, Policy Engine, CLI, Console, Authority**
> Data stores: **MongoDB primary, optional Redis/NATS for jobs**
---
## 1) What it is
**Aggregation‑Only Contract (AOC)** is the ingestion covenant for StellaOps. It defines a hard boundary between **collection** and **interpretation**:
* **Ingestion (Conseiller/Excitator)** only **collects** data and preserves it as immutable raw facts with provenance. It does not decide, merge, normalize, prioritize, or assign severity. It may compute **links** that help future joins (aliases, PURLs, CPEs), but never derived judgments.
* **Policy evaluation** is the only place where merges, deduplication, consensus, severity computation, and status folding are allowed. It’s reproducible and traceable.
The AOC establishes:
* **Immutable raw stores**: `advisory_raw` and `vex_raw` documents with full provenance, signatures, checksums, and upstream identifiers.
* **Linksets**: machine‑generated join hints (aliases, PURLs, CPEs, CVE/GHSA IDs) that never change the underlying source content.
* **Invariants**: a strict set of “never do this in ingestion” rules enforced by schema validation, runtime guards, and CI checks.
* **AOC Verifier**: a build‑time and runtime watchdog that blocks non‑compliant code and data writes.
This epic delivers: schemas, guards, error codes, APIs, tests, migration, docs, and ops dashboards to make AOC non‑negotiable across the platform.
---
## 2) Why
AOC makes results **auditable, deterministic, and organization‑specific**. Source vendors disagree; your policies decide. By removing hidden heuristics from ingestion, we avoid unexplainable risk changes, race conditions between collectors, and vendor bias. Policy‑time evaluation yields reproducible deltas with complete “why” traces.
---
## 3) How it should work (deep details)
### 3.1 Core invariants
The following must be true for every write to `advisory_raw` and `vex_raw` and for every ingestion pipeline:
1. **No severity in ingestion**
* Forbidden fields: `severity`, `cvss`, `cvss_vector`, `effective_status`, `effective_range`, `merged_from`, `consensus_provider`, `reachability`, `asset_criticality`, `risk_score`.
2. **No merges or de‑dups in ingestion**
* No combining two upstream advisories into one. No picking a single truth when multiple VEX statements exist.
3. **Provenance is mandatory**
* Every raw doc includes `provenance` and `signature/checksum`.
4. **Idempotent upserts**
* Same upstream document (by `upstream_id` + `source` + `content_hash`) must not create duplicates.
5. **Append‑only versioning**
* Revisions from the source create new immutable documents with `supersedes` pointers; no in‑place edits.
6. **Linkset only**
* Ingestion can compute and store a `linkset` for join performance. Linkset does not alter or infer severity/status.
7. **Policy‑time only for effective findings**
* Only the Policy Engine can write `effective_finding_*` materializations.
8. **Schema safety**
* Strict JSON schema validation at DB level; unknown fields reject writes.
9. **Clock discipline**
* Timestamps are UTC, monotonic within a batch; collectors record `fetched_at` and `received_at`.
### 3.2 Data model
#### 3.2.1 `advisory_raw` (Mongo collection)
```json
{
"_id": "advisory_raw:osv:GHSA-xxxx-....:v3",
"source": {
"vendor": "OSV",
"stream": "github",
"api": "https://api.osv.dev/v1/.../GHSA-...",
"collector_version": "conseiller/1.7.3"
},
"upstream": {
"upstream_id": "GHSA-xxxx-....",
"document_version": "2024-09-01T12:13:14Z",
"fetched_at": "2025-01-02T03:04:05Z",
"received_at": "2025-01-02T03:04:06Z",
"content_hash": "sha256:...",
"signature": {
"present": true,
"format": "dsse",
"key_id": "rekor:.../key/abc",
"sig": "base64..."
}
},
"content": {
"format": "OSV",
"spec_version": "1.6",
"raw": { /* full upstream JSON, unmodified */ }
},
"identifiers": {
"cve": ["CVE-2023-1234"],
"ghsa": ["GHSA-xxxx-...."],
"aliases": ["CVE-2023-1234", "GHSA-xxxx-...."]
},
"linkset": {
"purls": ["pkg:npm/lodash@4.17.21", "pkg:maven/..."],
"cpes": ["cpe:2.3:a:..."],
"references": [
{"type":"advisory","url":"https://..."},
{"type":"fix","url":"https://..."}
],
"reconciled_from": ["content.raw.affected.ranges", "content.raw.pkg"]
},
"supersedes": "advisory_raw:osv:GHSA-xxxx-....:v2",
"tenant": "default"
}
```
> Note: No `severity`, no `cvss`, no `effective_*`. If the upstream payload includes CVSS, it stays inside `content.raw` and is not promoted or normalized at ingestion.
#### 3.2.2 `vex_raw` (Mongo collection)
```json
{
"_id": "vex_raw:vendorX:doc-123:v4",
"source": {
"vendor": "VendorX",
"stream": "vex",
"api": "https://.../vex/doc-123",
"collector_version": "excitator/0.9.2"
},
"upstream": {
"upstream_id": "doc-123",
"document_version": "2025-01-15T08:09:10Z",
"fetched_at": "2025-01-16T01:02:03Z",
"received_at": "2025-01-16T01:02:03Z",
"content_hash": "sha256:...",
"signature": { "present": true, "format": "cms", "key_id": "kid:...", "sig": "..." }
},
"content": {
"format": "CycloneDX-VEX", // or "CSAF-VEX"
"spec_version": "1.5",
"raw": { /* full upstream VEX */ }
},
"identifiers": {
"statements": [
{
"advisory_ids": ["CVE-2023-1234","GHSA-..."],
"component_purls": ["pkg:deb/openssl@1.1.1"],
"status": "not_affected",
"justification": "component_not_present"
}
]
},
"linkset": {
"purls": ["pkg:deb/openssl@1.1.1"],
"cves": ["CVE-2023-1234"],
"ghsas": ["GHSA-..."]
},
"supersedes": "vex_raw:vendorX:doc-123:v3",
"tenant": "default"
}
```
> VEX statuses remain as raw facts. No cross‑provider consensus is computed here.
### 3.3 Database validation
* MongoDB JSON Schema validators on both collections:
* Reject forbidden fields at the top level.
* Enforce presence of `source`, `upstream`, `content`, `linkset`, `tenant`.
* Enforce string formats for timestamps and hashes.
### 3.4 Write paths
1. **Collector fetches upstream**
* Normalize transport (gzip/json), compute `content_hash`, verify signature if available.
2. **Build raw doc**
* Populate `source`, `upstream`, `content.raw`, `identifiers`, `linkset`.
3. **Idempotent upsert**
* Lookup by `(source.vendor, upstream.upstream_id, upstream.content_hash)`. If exists, skip; if new content hash, insert new revision with `supersedes`.
4. **AOC guard**
* Runtime interceptor inspects write payload; if any forbidden field detected, reject with `ERR_AOC_001`.
5. **Metrics**
* Emit `ingestion_write_ok` or `ingestion_write_reject` with reason code.
### 3.5 Read paths (ingestion scope)
* Allow only listing, getting raw docs, and searching by linkset. No endpoints return “effective findings” from ingestion services.
### 3.6 Error codes
| Code | Meaning | HTTP |
| ------------- | ------------------------------------------------------------ | ---- |
| `ERR_AOC_001` | Forbidden field present (severity/consensus/normalized data) | 400 |
| `ERR_AOC_002` | Merge attempt detected (multiple upstreams fused) | 400 |
| `ERR_AOC_003` | Idempotency violation (duplicate without supersedes) | 409 |
| `ERR_AOC_004` | Missing provenance fields | 422 |
| `ERR_AOC_005` | Signature/checksum mismatch | 422 |
| `ERR_AOC_006` | Attempt to write effective findings from ingestion context | 403 |
| `ERR_AOC_007` | Unknown top‑level fields (schema violation) | 400 |
### 3.7 AOC Verifier
A build‑time and runtime safeguard:
* **Static checks (CI)**
* Block imports of `*.Policy*` or `*.Merge*` from ingestion modules.
* AST lint rule: any write to `advisory_raw` or `vex_raw` setting a forbidden key fails the build.
* **Runtime checks**
* Repository layer interceptor inspects documents before insert/update; rejects forbidden fields and multi‑source merges.
* **Drift detection job**
* Nightly job scans newest N docs; if violation found, pages ops and blocks new pipeline runs.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 3.8 Indexing strategy
* `advisory_raw`:
* `{ "identifiers.cve": 1 }`, `{ "identifiers.ghsa": 1 }`, `{ "linkset.purls": 1 }`, `{ "source.vendor": 1, "upstream.upstream_id": 1, "upstream.content_hash": 1 }` (unique), `{ "tenant": 1 }`.
* `vex_raw`:
* `{ "identifiers.statements.advisory_ids": 1 }`, `{ "linkset.purls": 1 }`, `{ "source.vendor": 1, "upstream.upstream_id": 1, "upstream.content_hash": 1 }` (unique), `{ "tenant": 1 }`.
### 3.9 Interaction with Policy Engine
* Policy Engine pulls raw docs by identifiers/linksets and computes:
* De‑dup/merge per policy
* Consensus for VEX statements
* Severity normalization and risk scoring
* Writes **only** to `effective_finding_{policyId}` collections.
A dedicated write guard refuses `effective_finding_*` writes from any caller that isn’t the Policy Engine service identity.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 3.10 Migration plan
1. **Freeze ingestion writes** except raw pass‑through.
2. **Backfill**: copy existing ingestion collections to `_backup_*`.
3. **Strip forbidden fields** from raw copies, move them into a temporary `advisory_view_legacy` used only by Policy Engine for parity.
4. **Enable DB schema validators**.
5. **Run collectors** in dry‑run; ensure only allowed keys land.
6. **Switch Policy Engine** to pull exclusively from `*_raw` and to compute everything else.
7. **Delete legacy normalized fields** in ingestion codepaths.
8. **Enable runtime guards** and CI lint.
### 3.11 Observability
* Metrics:
* `aoc_violation_total{code=...}`, `ingestion_write_total{result=ok|reject}`, `ingestion_signature_verified_total{result=ok|fail}`, `ingestion_latency_seconds`, `advisory_revision_count`.
* Tracing: span `ingest.fetch`, `ingest.transform`, `ingest.write`, `aoc.guard`.
* Logs: include `tenant`, `source.vendor`, `upstream.upstream_id`, `content_hash`, `correlation_id`.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 3.12 Security and tenancy
* Every raw doc carries a `tenant` field.
* Authority enforces `advisory:write` and `vex:write` scopes for ingestion endpoints.
* Cross‑tenant reads/writes are blocked by default.
* Secrets never logged; signatures verified with pinned trust stores.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 3.13 CLI and Console behavior
* **CLI**
* `stella sources ingest --dry-run` prints would‑write payload and explicitly shows that no severity/status fields are present.
* `stella aoc verify` scans last K documents and reports violations with exit codes.
* **Console**
* Sources dashboard shows AOC pass/fail per job, most recent violation codes, and a drill‑down to the offending document.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 4) API surface (ingestion scope)
### 4.1 Conseiller (Advisories)
* `POST /ingest/advisory`
* Body: raw upstream advisory with metadata; server constructs document, not the client.
* Rejections: `ERR_AOC_00x` per table above.
* `GET /advisories/raw/{id}`
* `GET /advisories/raw?cve=CVE-...&purl=pkg:...&tenant=...`
* `GET /advisories/raw/{id}/provenance`
* `POST /aoc/verify?since=ISO8601` returns summary stats and first N violations.
### 4.2 Excitator (VEX)
* `POST /ingest/vex`
* `GET /vex/raw/{id}`
* `GET /vex/raw?advisory_id=CVE-...&purl=pkg:...`
* `POST /aoc/verify?since=ISO8601`
All endpoints require `tenant` scope and appropriate `:write` or `:read`.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 5) Example: end‑to‑end flow
1. Collector fetches `GHSA-1234` from OSV.
2. Build `advisory_raw` with linkset PURLs.
3. Insert; AOC guard approves.
4. Policy Engine later evaluates SBOM `S-42` under `policy P-7`, reads raw advisory and any VEX raw docs, computes effective findings, and writes to `effective_finding_P-7`.
5. CLI `stella aoc verify --since 24h` returns `0` violations.
---
## 6) Implementation tasks
Breakdown by component with exact work items. Each section ends with the imposed sentence you requested.
### 6.1 Conseiller (advisory ingestion, WS + Worker)
* [ ] Add Mongo JSON schema validation for `advisory_raw`.
* [ ] Implement repository layer with **write interceptors** that reject forbidden fields.
* [ ] Compute `linkset` from upstream using deterministic mappers.
* [ ] Enforce idempotency by unique index on `(source.vendor, upstream.upstream_id, upstream.content_hash, tenant)`.
* [ ] Remove any normalization pipelines; relocate to Policy Engine.
* [ ] Add `POST /ingest/advisory` and `GET /advisories/raw*` endpoints with Authority scope checks.
* [ ] Emit observability metrics and traces.
* [ ] Unit tests: schema violations, idempotency, supersedes chain, forbidden fields.
* [ ] Integration tests: large batch ingest, linkset correctness against golden fixtures.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.2 Excitator (VEX ingestion, WS + Worker)
* [ ] Add Mongo JSON schema validation for `vex_raw`.
* [ ] Implement repository layer guard identical to Conseiller.
* [ ] Deterministic `linkset` extraction for advisory IDs and PURLs.
* [ ] Endpoints `POST /ingest/vex`, `GET /vex/raw*` with scopes.
* [ ] Remove any consensus or folding logic; leave VEX statements as raw.
* [ ] Tests as per Conseiller, with rich fixtures for CycloneDX‑VEX and CSAF.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.3 Web API shared library
* [ ] Define `AOCForbiddenKeys` and export for both services.
* [ ] Provide `AOCWriteGuard` middleware and `AOCError` types.
* [ ] Provide `ProvenanceBuilder` utility.
* [ ] Provide `SignatureVerifier` and `Checksum` helpers.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.4 Policy Engine
* [ ] Block any import/use from ingestion modules by lint rule.
* [ ] Add hard gate on `effective_finding_*` writes that verifies caller identity is Policy Engine.
* [ ] Update readers to pull fields only from `content.raw`, `identifiers`, `linkset`, not any legacy normalized fields.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.5 Authority
* [ ] Introduce scopes: `advisory:write`, `advisory:read`, `vex:write`, `vex:read`, `aoc:verify`.
* [ ] Add `tenant` claim propagation to ingestion services.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.6 CLI
* [ ] `stella sources ingest --dry-run` and `stella aoc verify` commands.
* [ ] Exit codes mapping to `ERR_AOC_00x`.
* [ ] JSON output schema including violation list.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.7 Console
* [ ] Sources dashboard tiles: last run, AOC violations, top error codes.
* [ ] Drill‑down page rendering offending doc with highlight on forbidden keys.
* [ ] “Verify last 24h” action calling the AOC Verifier endpoint.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.8 CI/CD
* [ ] AST linter to forbid writes of banned keys in ingestion modules.
* [ ] Unit test coverage gates for AOC guard code.
* [ ] Pipeline stage that runs `stella aoc verify` against seeded DB snapshots.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Documentation changes (create/update these files)
1. **`/docs/ingestion/aggregation-only-contract.md`**
* Add: philosophy, invariants, schemas for `advisory_raw`/`vex_raw`, error codes, linkset definition, examples, idempotency rules, supersedes, API references, migration steps, observability, security.
2. **`/docs/architecture/overview.md`**
* Update system diagram to show AOC boundary and raw stores; add sequence diagram: fetch → guard → raw insert → policy evaluation.
3. **`/docs/architecture/policy-engine.md`**
* Clarify ingestion boundary; list inputs consumed from raw; note that any severity/consensus is policy‑time only.
4. **`/docs/ui/console.md`**
* Add Sources dashboard section: AOC tiles and violation drill‑down.
5. **`/docs/cli/cli-reference.md`**
* Add `stella aoc verify` and `stella sources ingest --dry-run` usage and exit codes.
6. **`/docs/observability/observability.md`**
* Document new metrics, traces, logs keys for AOC.
7. **`/docs/security/authority-scopes.md`**
* Add new scopes and tenancy enforcement for ingestion endpoints.
8. **`/docs/deploy/containers.md`**
* Note DB validators must be enabled; environment flags for AOC guards; read‑only user for verify endpoint.
Each file should include a “Compliance checklist” subsection for AOC.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 8) Acceptance criteria
* DB validators are active and reject writes with forbidden fields.
* AOC runtime guards log and reject violations with correct error codes.
* CI linter prevents shipping code that writes forbidden keys to raw stores.
* Ingestion of known fixture sets results in zero normalized fields outside `content.raw`.
* Policy Engine is the only writer of `effective_finding_*` materializations.
* CLI `stella aoc verify` returns success on clean datasets and non‑zero on seeded violations.
* Console shows AOC status and violation drill‑downs.
---
## 9) Risks and mitigations
* **Collector drift**: new upstream fields tempt developers to normalize.
* Mitigation: CI linter + guard + schema validators; require RFC to extend linkset.
* **Performance impact**: extra validation on write.
* Mitigation: guard is O(number of keys) and schema check is bounded; indexes sized appropriately.
* **Migration complexity**: moving legacy normalized fields out.
* Mitigation: temporary `advisory_view_legacy` for parity; stepwise cutover.
* **Tenant leakage**: missing tenant on write.
* Mitigation: schema requires `tenant`; middleware injects and asserts.
---
## 10) Test plan
* **Unit tests**
* Guard rejects forbidden keys; idempotency; supersedes chain; provenance required.
* Signature verification paths: good, bad, absent.
* **Property tests**
* Randomized upstream docs never produce forbidden keys at top level.
* **Integration tests**
* Batch ingest of 50k advisories: throughput, zero violations.
* Mixed VEX sources with contradictory statements remain separate in raw.
* **Contract tests**
* Policy Engine refuses to run without raw inputs; writes only to `effective_finding_*`.
* **End‑to‑end**
* Seed SBOM + advisories + VEX; ensure findings are identical pre/post migration.
---
## 11) Developer checklists
**Definition of Ready**
* Upstream spec reference attached.
* Linkset mappers defined.
* Example fixtures added.
**Definition of Done**
* DB validators deployed and tested.
* Runtime guards enabled.
* CI linter merged and enforced.
* Docs updated (files in section 7).
* Metrics visible on dashboard.
* CLI verify passes.
---
## 12) Glossary
* **Raw document**: exact upstream content plus provenance, with join hints.
* **Linkset**: PURLs/CPEs/IDs extracted to accelerate joins later.
* **Supersedes**: pointer from a newer raw doc to the previous revision of the same upstream doc.
* **Policy‑time**: evaluation phase where merges, consensus, and severity are computed.
* **AOC**: Aggregation‑Only Contract.
---
### Final imposed reminder
**Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**

566
EPIC_10.md Normal file
View File

@@ -0,0 +1,566 @@
Fine. Here’s your next brick of “maximum documentation.” Try not to drop it on your foot.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
# Epic 10: Export Center (JSON, Trivy DB, Mirror bundles)
**Short name:** `Export Center`
**Primary service:** `exporter`
**Surfaces:** Console (Web UI), CLI, Web API
**Touches:** Conseiller (Feedser), Excitator (Vexer), SBOM Service, Policy Engine, VEX Consensus Lens, Findings Ledger, Authority (authN/Z), Object Storage, Orchestrator, Signing/Attestation, Telemetry
**AOC ground rule:** Conseiller and Excitator aggregate but never merge. The Export Center serializes evidence and policy results; it does not rewrite or “improve” your data in-flight.
---
## 1) What it is
The Export Center is the unified system for packaging StellaOps data into portable, verifiable bundles:
* **Canonical JSON exports** for advisories, VEX, SBOMs, findings, and policy-evaluation snapshots.
* **Trivy DB compatible bundles** so downstream scanners can use Stella’s curated vulnerability knowledge without custom integrations.
* **Mirror bundles** for air‑gapped or disconnected environments containing raw evidence, normalized records, indexes, and policy snapshots, with provenance, signatures, and optional encryption.
It centralizes format adapters, compliance and provenance, scheduling, versioning, and distribution (download, OCI push, file share). Every export is reproducible by `run_id`, cryptographically signed, and audit‑traceable back to source artifacts.
---
## 2) Why (brief)
Teams need to move results into other scanners, CI systems, and isolated networks without babysitting ten different scripts. The Export Center gives a single, policy‑aware, verifiable exit point that doesn’t surprise compliance or set your ops team on fire.
---
## 3) How it should work (maximum detail)
### 3.1 Capabilities
* **Profiles**
* `json:raw` canonical JSON lines for advisories, VEX, SBOMs.
* `json:policy` adds policy evaluation results (allow/deny/risk, rationales).
* `trivy:db` Trivy DB‑compatible export (core vulnerability DB).
* `trivy:java-db` optional Java ecosystem DB export if enabled.
* `mirror:full` air‑gap bundle with raw + normalized + indexes + policy + VEX consensus + SBOMs.
* `mirror:delta` incremental bundle based on a previous export manifest.
* **Scope selectors**
* By time window, product, ecosystem, package, image digest, repository, tenant, tag.
* Include/exclude SBOMs, advisories, VEX, findings, policy snapshots.
* **Policy awareness**
* Optionally bake a **policy snapshot** into the bundle, including the policy version, inputs and decision traces.
* Can export **raw** evidence only (AOC) or **raw + evaluated**.
* **Provenance & signing**
* Generate attestation metadata with source URIs, artifact hashes, schema versions, export profile and filters.
* Sign manifests and bundle using configured KMS; support detached and in‑bundle signatures.
* **Distribution**
* Download via Console or API (streaming).
* Push to OCI registry as an artifact/image with annotations.
* Write to object storage prefix for batch pickup.
* **Scheduling & automation**
* One‑off, cron, and event‑triggered exports (e.g., after a “VEX consensus recompute” run).
* Retention policies and automatic expiry for old bundles.
* **Observability**
* Export metrics, throughput, size, downstream pull counts (when pushed to registry with report backs).
### 3.2 Architecture
* **exporter (service)**
* Orchestrates export runs, gathers records from the ledger and indexes, calls format adapters, writes bundles, signs, and publishes distribution tasks.
* Stateless workers pull “export.jobs” from the orchestrator, stream data, and write manifests into object storage.
* **format adapters**
* Pluggable adapters:
* `adapter-json`: canonicalized JSONL writers per record type.
* `adapter-trivy`: translates Stella’s normalized advisory model into Trivy DB format (and Java DB if enabled).
* `adapter-mirror`: constructs a portable filesystem/OCI layout with manifests, indexes, and data subtrees.
* **manifesting & provenance**
* `export.json` (export manifest): profile, filters, counts, schema versions, content checksums, start/finish times, inputs list (artifact ids + hashes).
* `provenance.json`: full chain back to source runs and artifacts; linked signatures.
* **distribution engines**
* `dist-http` streaming for downloads.
* `dist-oci` layer writer with descriptors and annotations.
* `dist-objstore` for staging to buckets.
* **security**
* Tenant scoping, RBAC on export creation and retrieval.
* Optional in‑bundle encryption (age/AES‑GCM) with key wrapping using KMS.
### 3.3 Data model (selected tables)
* `export_profiles`
* `id`, `name`, `kind` (`json|trivy|mirror`), `variant` (`raw|policy|db|java-db|full|delta`), `config_json`, `created_by`, `created_at`.
* `export_runs`
* `id`, `profile_id`, `trigger` (`manual|schedule|event|api`), `state`, `filters_json`, `policy_version`, `started_at`, `finished_at`, `artifact_uri`, `size_bytes`, `sig_uri`, `provenance_uri`, `tenant_id`, `error_class`, `error_message`.
* `export_inputs`
* Link table between `export_runs` and source artifacts. `export_run_id`, `artifact_id`, `hash`.
* `export_distributions`
* `export_run_id`, `type` (`download|oci|objstore`), `target`, `state`, `meta_json`, `created_at`, `updated_at`.
### 3.4 Canonical file layouts
**JSON profile output**
Directory layout under export root:
```
/export/
export.json # export manifest
provenance.json # provenance and source artifact chain
signatures/
export.sig # detached signature for export.json
advisories/
normalized.jsonl # normalized advisory records
vex/
normalized.jsonl # normalized VEX records
sboms/
<subject>/sbom.spdx.json # one per subject; SPDX JSON or CycloneDX JSON
findings/
policy_evaluated.jsonl # if profile=json:policy
```
**Trivy DB profile output**
Produced as a compressed artifact:
```
/export/
export.json
provenance.json
trivy/
db.bundle # Trivy DB compatible archive
java-db.bundle # optional Java DB bundle (if enabled)
signatures/
trivy-db.sig
```
Notes:
* The adapter keeps an internal mapping of Stella normalized fields to Trivy’s expected fields and namespaces. The mapping is versioned to track upstream schema evolution.
**Mirror bundle (filesystem layout)**
```
/mirror/
manifest.yaml # high-level bundle manifest (profile, filters, counts)
export.json # same as JSON profile
provenance.json
indexes/
advisories.index.json # quick lookups (pkg -> advisory ids)
vex.index.json
sbom.index.json
advisories/raw/...
advisories/normalized/...
vex/raw/...
vex/normalized/...
sboms/raw/...
sboms/graph/...
policy/
snapshot.yaml # full policy set used for evaluation
evaluations.jsonl # decision outputs if requested
consensus/
vex_consensus.jsonl
signatures/
manifest.sig
export.sig
README.md
```
**Mirror bundle (OCI layout)**
Following standard OCI image artifact layout with annotations (`org.opencontainers.artifact.description`, `com.stella.export.profile`, `com.stella.export.filters`), and manifest lists for large bundles.
### 3.5 Export workflow
1. **Plan**
Exporter computes candidates based on filters. For `mirror:delta`, compares with previous manifest to compute changes.
2. **Stream & write**
Records are streamed from the Findings Ledger and stores. Writers are forward‑only, emitting JSONL or adapter‑specific structures, chunked for memory safety.
3. **Sign & attest**
Once all content hashes are stable, Export Center writes `export.json`, `provenance.json`, and signs using KMS. Optional encryption wraps data layers.
4. **Distribute**
Depending on profile settings, it exposes a download URL, pushes an OCI artifact, or writes to object storage. Distribution metadata is recorded.
5. **Audit & retention**
Run, manifest, and signatures are immutable. Retention policy prunes large data folders after N days with manifests retained longer.
### 3.6 APIs
```
POST /export/profiles
GET /export/profiles?kind=&variant=
GET /export/profiles/{id}
PATCH /export/profiles/{id}
DELETE /export/profiles/{id}
POST /export/runs
GET /export/runs?state=&profile_id=&from=&to=&tenant_id=
GET /export/runs/{run_id}
POST /export/runs/{run_id}/cancel
GET /export/runs/{run_id}/download # presigned URL or streaming
POST /export/runs/{run_id}/distribute # { "type":"oci|objstore", "target":"..." }
GET /export/runs/{run_id}/manifest # export.json
GET /export/runs/{run_id}/provenance
GET /export/runs/{run_id}/signatures
GET /export/metrics/overview
WS /export/streams/updates
```
**Request example (create run):**
```json
{
"profile_id": "prof_json_policy",
"filters": {
"time_from": "2025-01-01T00:00:00Z",
"time_to": "2025-01-31T23:59:59Z",
"ecosystems": ["pypi", "npm"],
"include": ["advisories", "vex", "sboms", "findings"]
},
"distribution": { "type": "download" },
"policy_version": "pol-v1.8.2"
}
```
### 3.7 CLI
```
stella export profiles list --kind mirror
stella export profiles create --file profile.yaml
stella export run create --profile prof_json_policy --from 2025-01-01 --to 2025-01-31 --include advisories,vex,sboms --download
stella export run status <run-id>
stella export run cancel <run-id>
stella export run get <run-id> --manifest
stella export run download <run-id> --out export-jan.tar.zst
stella export distribute <run-id> --oci ghcr.io/org/stella-export:jan2025
stella export verify <bundle> --manifest export.json --sig signatures/export.sig
```
Exit codes: `0` ok, `2` bad args, `4` not found, `5` denied, `6` integrity failed, `8` export error.
### 3.8 RBAC & security
* Roles:
* `Export.Viewer`: list runs, download completed bundles.
* `Export.Operator`: create runs, cancel, schedule, distribute.
* `Export.Admin`: manage profiles, set retention, manage signing keys.
* Tenancy:
* Every run and artifact scoped by tenant; cross‑tenant export is disallowed.
* Secrets:
* KMS references for signing and encryption; never store private keys.
* PII & redaction:
* Exporters must not include secrets or credentials. Redaction rules enforced at writer level with schema‑based allowlists.
### 3.9 Observability
* Metrics:
* `export_bytes_total{profile,tenant}`
* `export_records_total{type}`
* `export_duration_ms{profile}`
* `export_failures_total{error_class}`
* `export_distributions_total{type}`
* `export_verify_fail_total`
* Traces:
* Spans per export phase: plan, stream, write, sign, distribute; baggage includes `export_run_id`.
* Logs:
* Structured JSON with counts, sizes, hashes, and redaction hints.
### 3.10 Performance targets
* Stream throughput ≥ 25k records/sec per worker for JSONL writing with compression.
* Trivy bundle generation for 1M advisories ≤ 8 minutes on a standard worker.
* Mirror delta export for 5% change set ≤ 2 minutes.
### 3.11 Edge cases & behavior
* **Schema drift**: adapter refuses to emit unknown fields without explicit mapping; run fails with `error_class=schema_mismatch`.
* **Oversized bundles**: automatic sharding by time or content type; mirror OCI uses multi‑manifest indices.
* **Missing policy snapshot**: profile `json:policy` will auto‑pull latest version unless pinned; pinning is recommended for reproducibility.
* **Duplicate evidence**: writers dedupe by artifact hash and advisory id; AOC forbids merging.
* **Air‑gap encryption**: if `encrypt=true`, mirror bundles require recipient public key material; decryption tooling documented.
---
## 4) Implementation plan
### 4.1 Modules
* **New service:** `src/StellaOps.ExportCenter`
* `api/` REST + WS
* `planner/` scope planning, delta computation, sampling
* `adapters/`
* `json/` canonical writers
* `trivy/` db builders and schema mapping
* `mirror/` fs/OCI builders, sharding, delta logic
* `signing/` KMS clients, attestations
* `dist/` download streaming, OCI push, object storage writer
* `state/` repositories, migrations
* `metrics/`, `audit/`, `security/`
* **SDK/CLI**
* `src/StellaOps.Cli` subcommands with streaming download and verification.
* **Console**
* `console/apps/export-center/` pages:
* Overview, Profiles, Runs, Run Detail, Distributions, Settings.
* Components: `ExportPlanPreview`, `ProfileEditor`, `RunDiff`, `VerifyPanel`.
* **Existing services updates**
* Findings Ledger: new paginated streaming endpoints for advisories/VEX/SBOM/findings by filters and snapshots.
* Policy Engine: “policy snapshot” exportable endpoint.
* VEX Lens: consensus snapshot endpoint.
### 4.2 Packaging & deployment
* Containers:
* `stella/exporter:<ver>`
* `stella/exporter-worker:<ver>` (optional separated worker pool)
* Helm:
* WS replicas, concurrent export limits, default compression (`zstd`), default retention, KMS settings, OCI creds secrets.
* DB migrations:
* Create `export_*` tables with proper indices (tenant, time, state).
### 4.3 Rollout
* Phase 1: JSON (raw, policy) and Mirror (full) as download only.
* Phase 2: Trivy DB adapters, OCI distribution.
* Phase 3: Mirror deltas, encryption, verification tooling, scheduling.
---
## 5) Documentation changes
Create/update the following docs; each page must end with the imposed rule statement.
1. `/docs/export-center/overview.md`
Purpose, profiles, supported targets, AOC alignment, security model.
2. `/docs/export-center/architecture.md`
Service components, adapters, manifests, signing, distribution flows.
3. `/docs/export-center/profiles.md`
Profile schemas, examples, versioning, compatibility notes.
4. `/docs/export-center/api.md`
All endpoints with request/response examples and error codes.
5. `/docs/export-center/cli.md`
Commands with examples, scripts for CI/CD, verification.
6. `/docs/export-center/mirror-bundles.md`
Filesystem and OCI layouts, delta exports, encryption, air‑gap import guide.
7. `/docs/export-center/trivy-adapter.md`
Field mapping, supported ecosystems, compatibility and test matrix.
8. `/docs/export-center/provenance-and-signing.md`
Manifest format, attestation details, verification process.
9. `/docs/operations/export-runbook.md`
Common failures, recovery, tuning, capacity planning.
10. `/docs/security/export-hardening.md`
RBAC, tenant isolation, secret redaction, encryption keys.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 6) Engineering tasks
### Backend: exporter
* [ ] Migrations for `export_profiles`, `export_runs`, `export_inputs`, `export_distributions`.
* [ ] Planner to resolve filters to iterators over advisory/VEX/SBOM/findings datasets with pagination.
* [ ] JSON adapter: canonical JSONL writers with schema normalization and redaction enforcement.
* [ ] Policy snapshot embedder: pull policy version and evaluation outputs when requested.
* [ ] Trivy adapter: implement schema mapping, writer, integrity validation, and compatibility version flag.
* [ ] Mirror adapter: filesystem and OCI writer, sharding, manifest creation, delta computation.
* [ ] Signing/attestation using KMS; detached and embedded options.
* [ ] Distribution engines: download streaming, OCI push, object storage staging.
* [ ] API layer with async export run handling and WebSocket updates.
* [ ] Rate limit and concurrency controls per tenant/profile.
* [ ] Audit logging for all create/cancel/distribute/verify actions.
### Integrations
* [ ] Findings Ledger streaming APIs for each content type.
* [ ] Policy Engine endpoint to return deterministic policy snapshot and decision set by run.
* [ ] VEX Lens endpoint to expose consensus snapshot.
### Console
* [ ] Profiles CRUD with validation and test preview.
* [ ] Create Run wizard with live count estimates and storage footprint prediction.
* [ ] Runs list + detail page with manifest, provenance, and quick verify.
* [ ] Download and distribution actions with progress and logs.
* [ ] Verification panel to check signatures and hashes client‑side.
### CLI
* [ ] `stella export` commands as defined; include `verify` that checks signatures and hashes.
* [ ] Auto‑resume of interrupted downloads with range requests.
* [ ] Friendly error messages for schema mismatch and verification failure.
### Observability
* [ ] Metrics and traces per §3.9; dashboards for throughput, durations, failures, sizes.
* [ ] Alerts for export failure rate and verify failures.
### Security & RBAC
* [ ] Enforce tenant scoping at query level; fuzz tests for leakage.
* [ ] Role matrix checks on each API; Console hides forbidden actions.
* [ ] Encryption test vectors and key rotation procedure.
### Docs
* [ ] Author all files in §5 with concrete examples and diagrams.
* [ ] Cross‑link from Orchestrator, Policy Studio, VEX Lens, and SBOM docs to the Export Center pages.
* [ ] Append imposed rule line at the end of each page.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Implementation notes per profile
### 7.1 JSON: raw
* **Content:** advisories.normalized, vex.normalized, sboms (SPDX/CycloneDX), optional findings.raw.
* **Normalization:** enforce Stella field casing, timestamps in RFC3339, unicode NFC.
* **Compression:** `.jsonl.zst` per file to allow split/merge.
### 7.2 JSON: policy
* **Adds:** `policy_snapshot` and `findings.policy_evaluated.jsonl` with decision, rule_id, rationale, inputs fingerprint.
* **Determinism:** include `policy_version` and `inputs_hash`; replays should match exactly.
### 7.3 Trivy DB
* **Mapping:**
* Package name, ecosystem, version ranges, CVE/CWE/aliases, severity mapping, vendor statements, fixed versions.
* Ensure namespace mapping avoids collisions (e.g., distro vs ecosystem).
* **Compatibility:** version flag in manifest; adapter throws if upstream schema version not supported.
* **Validation:** run post‑build sanity checks (counts, required indexes).
### 7.4 Mirror: full/delta
* **Full:** everything needed to spin up an isolated read‑only Stella mirror with local search.
* **Delta:** compute changed/added/removed advisory ids, VEX statements, SBOM subjects; update indexes and manifest with `base_export_id`.
* **Encryption:** if enabled, encrypt data subtrees; leave `manifest.yaml` unencrypted for discoverability unless `strict=true`.
---
## 8) Acceptance criteria
* Operators can create an export with filters, download it, verify signatures, and trace back to source artifacts via provenance.
* Trivy adapter produces a bundle consumable by Trivy without custom flags (basic validation in CI).
* Mirror bundle imports successfully in an air‑gapped “mirror‑reader” sample app and serves queries from indexes.
* Policy‑aware exports include deterministic decisions matching the specified `policy_version`.
* RBAC prevents a Viewer from creating or canceling exports; tenancy prevents cross‑tenant leakage.
* Metrics and dashboards show per‑profile throughput and error classes; alerts trigger on failure spikes.
* Export retries are idempotent and do not duplicate content; hashes stable across re‑runs with identical inputs.
---
## 9) Risks & mitigations
* **Upstream schema changes break Trivy export.**
Mitigation: versioned adapter with compatibility gate; integration tests against known fixtures; fail early with clear remediation.
* **Bundle size explosion.**
Mitigation: zstd compression, sharding, delta exports, content‑addressed storage reuse for mirror OCI.
* **Data leakage via exports.**
Mitigation: strict allowlist schemas, redaction filters, RBAC, tenant scoping tests, encryption for mirror.
* **Non‑deterministic policy outputs.**
Mitigation: pin policy version and inputs hash; snapshot embedded rules; deterministic evaluation mode only.
* **Slow downloads/timeouts.**
Mitigation: streaming with range support, resumable downloads, CDN integration if needed.
---
## 10) Test plan
* **Unit**
Schema normalization; Trivy mapping; mirror delta computation; manifest hashing; signing.
* **Integration**
End‑to‑end export with each profile; verify bundles; replay determinism of policy exports; OCI push/pull.
* **Compatibility**
Validate Trivy bundle against a matrix of versions; import mirror bundle into a reference reader and run queries.
* **Security**
Tenant isolation fuzzing; redaction checks; encryption round‑trip; signature verification with rotated keys.
* **Performance**
Large dataset generation; parallel writer stress; OCI multi‑manifest publishing; download resume under packet loss.
* **Chaos**
Kill exporter mid‑write; ensure resume or clean failure without partial corrupt bundles.
---
## 11) Philosophy
* **Ports, not prisons.** Exports should free your data to move with integrity and context, not trap it in a proprietary maze.
* **Reproducible or it didn’t happen.** Every bit derived from known inputs, signed, traceable.
* **Air‑gap is a first‑class citizen.** Mirror bundles are not an afterthought; they’re how serious orgs actually run.
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**

BIN
EPIC_11.md Normal file
View File

Binary file not shown.

BIN
EPIC_12.md Normal file
View File

Binary file not shown.

BIN
EPIC_13.md Normal file
View File

Binary file not shown.

BIN
EPIC_14.md Normal file
View File

Binary file not shown.

BIN
EPIC_15.md Normal file
View File

Binary file not shown.

1
EPIC_16.md Normal file
View File

@@ -0,0 +1 @@
See `docs/airgap/EPIC_16_AIRGAP_MODE.md` for the full Epic 16 specification.

1
EPIC_17.md Normal file
View File

@@ -0,0 +1 @@
See `docs/api/EPIC_17_SDKS_OPENAPI.md` for the complete Epic 17 specification.

1
EPIC_18.md Normal file
View File

@@ -0,0 +1 @@
See `docs/risk/EPIC_18_RISK_PROFILES.md` for the complete Epic 18 specification.

1
EPIC_19.md Normal file
View File

@@ -0,0 +1 @@
See `docs/attestor/EPIC_19_ATTESTOR_CONSOLE.md` for the complete Epic 19 specification.

567
EPIC_2.md Normal file
View File

@@ -0,0 +1,567 @@
Fine. Here’s the next epic, written so you can paste it straight into the repo without having to babysit me. Same structure as before, maximum detail, zero hand‑waving.
---
# Epic 2: Policy Engine & Policy Editor (VEX + Advisory Application Rules)
> Short name: **Policy Engine v2**
> Services touched: **Policy Engine, Web API, Console (Policy Editor), CLI, Conseiller, Excitator, SBOM Service, Authority, Workers/Scheduler**
> Data stores: **MongoDB (policies, runs, effective findings), optional Redis/NATS for jobs**
---
## 1) What it is
This epic delivers the **organization‑specific decision layer** for Stella. Ingestion is now AOC‑compliant (Epic 1). That means advisories and VEX arrive as immutable raw facts. This epic builds the place where those facts become **effective findings** under policies you control.
Core deliverables:
* **Policy Engine**: deterministic evaluator that applies rule sets to inputs:
* Inputs: `advisory_raw`, `vex_raw`, SBOMs, optional telemetry hooks (reachability stubs), org metadata.
* Outputs: `effective_finding_{policyId}` materializations, with full explanation traces.
* **Policy Editor (Console + CLI)**: versioned policy authoring, simulation, review/approval workflow, and change diffs.
* **Rules DSL v1**: safe, declarative language for VEX application, advisory normalization, and risk scoring. No arbitrary code execution, no network calls.
* **Run Orchestrator**: incremental re‑evaluation when new raw facts or SBOM changes arrive; efficient partial updates.
The philosophy is boring on purpose: policy is a **pure function of inputs**. Same inputs and same policy yield the same outputs, every time, on every machine. If you want drama, watch reality TV, not your risk pipeline.
---
## 2) Why
* Vendors disagree, contexts differ, and your tolerance for risk is not universal.
* VEX means nothing until you decide **how** to apply it to **your** assets.
* Auditors care about the “why.” You’ll need consistent, replayable answers, with traces.
* Security teams need **simulation** before rollouts, and **diffs** after.
---
## 3) How it should work (deep details)
### 3.1 Data model
#### 3.1.1 Policy documents (Mongo: `policies`)
```json
{
"_id": "policy:P-7:v3",
"policy_id": "P-7",
"version": 3,
"name": "Default Org Policy",
"status": "approved", // draft | submitted | approved | archived
"owned_by": "team:sec-plat",
"valid_from": "2025-01-15T00:00:00Z",
"valid_to": null,
"dsl": {
"syntax": "stella-dsl@1",
"source": "rule-set text or compiled IR ref"
},
"metadata": {
"description": "Baseline scoring + VEX precedence",
"tags": ["baseline","vex","cvss"]
},
"provenance": {
"created_by": "user:ali",
"created_at": "2025-01-15T08:00:00Z",
"submitted_by": "user:kay",
"approved_by": "user:root",
"approval_at": "2025-01-16T10:00:00Z",
"checksum": "sha256:..."
},
"tenant": "default"
}
```
Constraints:
* `status=approved` is required to run in production.
* Version increments are append‑only. Old versions remain runnable for replay.
#### 3.1.2 Policy runs (Mongo: `policy_runs`)
```json
{
"_id": "run:P-7:2025-02-20T12:34:56Z:abcd",
"policy_id": "P-7",
"policy_version": 3,
"inputs": {
"sbom_set": ["sbom:S-42"],
"advisory_cursor": "2025-02-20T00:00:00Z",
"vex_cursor": "2025-02-20T00:00:00Z"
},
"mode": "incremental", // full | incremental | simulate
"stats": {
"components": 1742,
"advisories_considered": 9210,
"vex_considered": 1187,
"rules_fired": 68023,
"findings_out": 4321
},
"trace": {
"location": "blob://traces/run-.../index.json",
"sampling": "smart-10pct"
},
"status": "succeeded", // queued | running | failed | succeeded | canceled
"started_at": "2025-02-20T12:34:56Z",
"finished_at": "2025-02-20T12:35:41Z",
"tenant": "default"
}
```
#### 3.1.3 Effective findings (Mongo: `effective_finding_P-7`)
```json
{
"_id": "P-7:S-42:pkg:npm/lodash@4.17.21:CVE-2021-23337",
"policy_id": "P-7",
"policy_version": 3,
"sbom_id": "S-42",
"component_purl": "pkg:npm/lodash@4.17.21",
"advisory_ids": ["CVE-2021-23337", "GHSA-..."],
"status": "affected", // affected | not_affected | fixed | under_investigation | suppressed
"severity": {
"normalized": "High",
"score": 7.5,
"vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N",
"rationale": "cvss_base(OSV) + vendor_weighting + env_modifiers"
},
"rationale": [
{"rule":"vex.precedence","detail":"VendorX not_affected justified=component_not_present wins"},
{"rule":"advisory.cvss.normalization","detail":"mapped GHSA severity to CVSS 3.1 = 7.5"}
],
"references": {
"advisory_raw_ids": ["advisory_raw:osv:GHSA-...:v3"],
"vex_raw_ids": ["vex_raw:VendorX:doc-123:v4"]
},
"run_id": "run:P-7:2025-02-20T12:34:56Z:abcd",
"tenant": "default"
}
```
Write protection: only the **Policy Engine** service identity may write any `effective_finding_*` collection.
---
### 3.2 Rules DSL v1 (stella‑dsl@1)
**Design goals**
* Declarative, composable, deterministic.
* No loops, no network IO, no non‑deterministic time.
* Policy authors see readable text; the engine compiles to a safe IR.
**Concepts**
* **WHEN** condition matches a tuple `(sbom_component, advisory, optional vex_statements)`
* **THEN** actions set `status`, compute `severity`, attach `rationale`, or `suppress` with reason.
* **Profiles** for severity and scoring; **Maps** for vendor weighting; **Guards** for VEX justification.
**Mini‑grammar (subset)**
```
policy "Default Org Policy" syntax "stella-dsl@1" {
profile severity {
map vendor_weight {
source "GHSA" => +0.5
source "OSV" => +0.0
source "VendorX" => -0.2
}
env base_cvss {
if env.runtime == "serverless" then -0.5
if env.exposure == "internal-only" then -1.0
}
}
rule vex_precedence {
when vex.any(status in ["not_affected","fixed"])
and vex.justification in ["component_not_present","vulnerable_code_not_present"]
then status := vex.status
because "VEX strong justification prevails";
}
rule advisory_to_cvss {
when advisory.source in ["GHSA","OSV"]
then severity := normalize_cvss(advisory)
because "Map vendor severity or CVSS vector";
}
rule reachability_soft_suppress {
when severity.normalized <= "Medium"
and telemetry.reachability == "none"
then status := "suppressed"
because "not reachable and low severity";
}
}
```
**Built‑ins** (non‑exhaustive)
* `normalize_cvss(advisory)` maps GHSA/OSV/CSAF severity fields to CVSS v3.1 numbers when possible; otherwise vendor‑to‑numeric mapping table in policy.
* `vex.any(...)` tests across matching VEX statements for the same `(component, advisory)`.
* `telemetry.*` is an optional input namespace reserved for future reachability data; if absent, expressions evaluate to `unknown` (no effect).
**Determinism**
* Rules are evaluated in **stable order**: explicit `priority` attribute or lexical order.
* **First‑match** semantics for conflicting status unless `combine` is used.
* Severity computations are pure; numeric maps are part of policy document.
---
### 3.3 Evaluation model
1. **Selection**
* For each SBOM component PURL, find candidate advisories from `advisory_raw` via linkset PURLs or identifiers.
* For each pair `(component, advisory)`, load all matching VEX facts from `vex_raw`.
2. **Context assembly**
* Build an evaluation context from:
* `sbom_component`: PURL, licenses, relationships.
* `advisory`: source, identifiers, references, embedded vendor severity (kept in `content.raw`).
* `vex`: list of statements with status and justification.
* `env`: org‑specific env vars configured per policy run (e.g., exposure).
* Optional `telemetry` if available.
3. **Rule execution**
* Compile DSL to IR once per policy version; cache.
* Execute rules per tuple; record which rules fired and the order.
* If no rule sets status, default is `affected`.
* If no rule sets severity, default severity uses `normalize_cvss(advisory)` with vendor defaults.
4. **Materialization**
* Write to `effective_finding_{policyId}` with `rationale` chain and references to raw docs.
* Emit per‑tuple trace events; sample and store full traces per run.
5. **Incremental updates**
* A watch job observes new `advisory_raw` and `vex_raw` inserts and SBOM deltas.
* The orchestrator computes the affected tuples and re‑evaluates only those.
6. **Replay**
* Any `policy_run` is fully reproducible by `(policy_id, version, input set, cursors)`.
---
### 3.4 VEX application semantics
* **Precedence**: a `not_affected` with strong justification (`component_not_present`, `vulnerable_code_not_present`, `fix_not_required`) wins unless another rule explicitly overrides by environment context.
* **Scoping**: VEX statements often specify product/component scope. Matching uses PURL equivalence and version ranges extracted during ingestion linkset generation.
* **Conflicts**: If multiple VEX statements conflict, the default is **most‑specific scope wins** (component > product > vendor), then newest `document_version`. Policies can override with explicit rules.
* **Explainability**: Every VEX‑driven decision records which statement IDs were considered and which one won.
---
### 3.5 Advisory normalization rules
* **Vendor severity mapping**: Map GHSA levels or CSAF product‑tree severities to CVSS‑like numeric bands via policy maps.
* **CVSS vector use**: If a valid vector exists in `content.raw`, parse and compute; apply policy modifiers from `profile severity`.
* **Temporal/environment modifiers**: Optional reductions for network exposure, isolation, or compensating controls, all encoded in policy.
---
### 3.6 Performance and scale
* Partition evaluation by SBOM ID and hash ranges of PURLs.
* Pre‑index `advisory_raw.linkset.purls` and `vex_raw.linkset.purls` (already in Epic 1).
* Use streaming iterators; avoid loading entire SBOM or advisory sets into memory.
* Materialize only changed findings (diff‑aware writes).
* Target: 100k components, 1M advisories considered, 5 minutes incremental SLA on commodity hardware.
---
### 3.7 Error codes
| Code | Meaning | HTTP |
| ------------- | ----------------------------------------------------- | ---- |
| `ERR_POL_001` | Policy syntax error | 400 |
| `ERR_POL_002` | Policy not approved for run | 403 |
| `ERR_POL_003` | Missing inputs (SBOM/advisory/vex fetch failed) | 424 |
| `ERR_POL_004` | Determinism guard triggered (non‑pure function usage) | 500 |
| `ERR_POL_005` | Write denied to effective findings (caller invalid) | 403 |
| `ERR_POL_006` | Run canceled or timed out | 408 |
---
### 3.8 Observability
* Metrics:
* `policy_compile_seconds`, `policy_run_seconds{mode=...}`, `rules_fired_total`, `findings_written_total`, `vex_overrides_total`, `simulate_diff_total{delta=up|down|unchanged}`.
* Tracing:
* Spans: `policy.compile`, `policy.select`, `policy.eval`, `policy.materialize`.
* Logs:
* Include `policy_id`, `version`, `run_id`, `sbom_id`, `component_purl`, `advisory_id`, `vex_count`, `rule_hits`.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
### 3.9 Security and tenancy
* Only users with `policy:write` can create/modify policies.
* `policy:approve` is a separate privileged role.
* Only Policy Engine service identity has `effective:write`.
* Tenancy is explicit on all documents and queries.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 4) API surface
### 4.1 Policy CRUD and lifecycle
* `POST /policies` create draft
* `GET /policies?status=...` list
* `GET /policies/{policyId}/versions/{v}` fetch
* `POST /policies/{policyId}/submit` move draft to submitted
* `POST /policies/{policyId}/approve` approve version
* `POST /policies/{policyId}/archive` archive version
### 4.2 Compilation and validation
* `POST /policies/{policyId}/versions/{v}/compile`
* Returns IR checksum, syntax diagnostics, rule stats.
### 4.3 Runs
* `POST /policies/{policyId}/runs` body: `{mode, sbom_set, advisory_cursor?, vex_cursor?, env?}`
* `GET /policies/{policyId}/runs/{runId}` status + stats
* `POST /policies/{policyId}/simulate` returns **diff** vs current approved version on a sample SBOM set.
### 4.4 Findings and explanations
* `GET /findings/{policyId}?sbom_id=S-42&status=affected&severity=High+Critical`
* `GET /findings/{policyId}/{findingId}/explain` returns ordered rule hits and linked raw IDs.
All endpoints require tenant scoping and appropriate `policy:*` or `findings:*` roles.
---
## 5) Console (Policy Editor) and CLI behavior
**Console**
* Monaco‑style editor with DSL syntax highlighting, lint, quick docs.
* Side‑by‑side **Simulation** panel: show count of affected findings before/after.
* Approval workflow: submit, review comments, approve with rationale.
* Diffs: show rule‑wise changes and estimated impact.
* Read‑only run viewer: heatmap of rules fired, top suppressions, VEX wins.
**CLI**
* `stella policy new --name "Default Org Policy"`
* `stella policy edit P-7` opens local editor -> `submit`
* `stella policy approve P-7 --version 3`
* `stella policy simulate P-7 --sbom S-42 --env exposure=internal-only`
* `stella findings ls --policy P-7 --sbom S-42 --status affected`
Exit codes map to `ERR_POL_*`.
---
## 6) Implementation tasks
### 6.1 Policy Engine service
* [ ] Implement DSL parser and IR compiler (`stella-dsl@1`).
* [ ] Build evaluator with stable ordering and first‑match semantics.
* [ ] Implement selection joiners for SBOM↔advisory↔vex using linksets.
* [ ] Materialization writer with upsert‑only semantics to `effective_finding_{policyId}`.
* [ ] Determinism guard (ban wall‑clock, network, and RNG during eval).
* [ ] Incremental orchestrator listening to advisory/vex/SBOM change streams.
* [ ] Trace emitter with rule‑hit sampling.
* [ ] Unit tests, property tests, golden fixtures; perf tests to target SLA.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.2 Web API
* [ ] Policy CRUD, compile, run, simulate, findings, explain endpoints.
* [ ] Pagination, filters, and tenant enforcement on all list endpoints.
* [ ] Error mapping to `ERR_POL_*`.
* [ ] Rate limits on simulate endpoints.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.3 Console (Policy Editor)
* [ ] Editor with DSL syntax highlighting and inline diagnostics.
* [ ] Simulation UI with pre/post counts and top deltas.
* [ ] Approval workflow UI with audit trail.
* [ ] Run viewer dashboards (rule heatmap, VEX wins, suppressions).
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.4 CLI
* [ ] New commands: `policy new|edit|submit|approve|simulate`, `findings ls|get`.
* [ ] Json/YAML output formats for CI consumption.
* [ ] Non‑zero exits on syntax errors or simulation failures; map to `ERR_POL_*`.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.5 Conseiller & Excitator integration
* [ ] Provide search endpoints optimized for policy selection (batch by PURLs and IDs).
* [ ] Harden linkset extraction to maximize join recall.
* [ ] Add cursors for incremental selection windows per run.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.6 SBOM Service
* [ ] Ensure fast PURL index and component metadata projection for policy queries.
* [ ] Provide relationship graph API for future transitive logic.
* [ ] Emit change events on SBOM updates.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.7 Authority
* [ ] Define scopes: `policy:write`, `policy:approve`, `policy:run`, `findings:read`, `effective:write`.
* [ ] Issue service identity for Policy Engine with `effective:write` only.
* [ ] Enforce tenant claims at gateway.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 6.8 CI/CD
* [ ] Lint policy DSL in PRs; block invalid syntax.
* [ ] Run `simulate` against golden SBOMs to detect explosive deltas.
* [ ] Determinism CI: two runs with identical seeds produce identical outputs.
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Documentation changes (create/update these files)
1. **`/docs/policy/overview.md`**
* What the Policy Engine is, high‑level concepts, inputs, outputs, determinism.
2. **`/docs/policy/dsl.md`**
* Full grammar, built‑ins, examples, best practices, anti‑patterns.
3. **`/docs/policy/lifecycle.md`**
* Draft → submitted → approved → archived, roles, and audit trail.
4. **`/docs/policy/runs.md`**
* Run modes, incremental mechanics, cursors, replay.
5. **`/docs/api/policy.md`**
* Endpoints, request/response schemas, error codes.
6. **`/docs/cli/policy.md`**
* Command usage, examples, exit codes, JSON output contracts.
7. **`/docs/ui/policy-editor.md`**
* Screens, workflows, simulation, diffs, approvals.
8. **`/docs/architecture/policy-engine.md`**
* Detailed sequence diagrams, selection/join strategy, materialization schema.
9. **`/docs/observability/policy.md`**
* Metrics, tracing, logs, sample dashboards.
10. **`/docs/security/policy-governance.md`**
* Scopes, approvals, tenancy, least privilege.
11. **`/docs/examples/policies/`**
* `baseline.pol`, `serverless.pol`, `internal-only.pol`, each with commentary.
12. **`/docs/faq/policy-faq.md`**
* Common pitfalls, VEX conflict handling, determinism gotchas.
Each file includes a **Compliance checklist** for authors and reviewers.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 8) Acceptance criteria
* Policies are versioned, approvable, and compilable; invalid DSL blocks merges.
* Engine produces deterministic outputs with full rationale chains.
* VEX precedence rules work per spec and are overridable by policy.
* Simulation yields accurate pre/post deltas and diffs.
* Only Policy Engine can write to `effective_finding_*`.
* Incremental runs pick up new advisories/VEX/SBOM changes without full re‑runs.
* Console and CLI cover authoring, simulation, approval, and retrieval.
* Observability dashboards show rule hits, VEX wins, and run timings.
---
## 9) Risks and mitigations
* **Policy sprawl**: too many similar policies.
* Mitigation: templates, policy inheritance in v1.1, tagging, ownership metadata.
* **Non‑determinism creep**: someone sneaks wall‑clock or network into evaluation.
* Mitigation: determinism guard, static analyzer, and CI replay check.
* **Join miss‑rate**: weak linksets cause under‑matching.
* Mitigation: linkset strengthening in ingestion, PURL equivalence tables, monitoring for “zero‑hit” rates.
* **Approval bottlenecks**: blocked rollouts.
* Mitigation: RBAC with delegated approvers and time‑boxed SLAs.
---
## 10) Test plan
* **Unit**: parser, compiler, evaluator; conflict resolution; precedence.
* **Property**: random policies over synthetic inputs; ensure no panics and stable outputs.
* **Golden**: fixed SBOM + curated advisories/VEX → expected findings; compare every run.
* **Performance**: large SBOMs with heavy rule sets; assert run times and memory ceilings.
* **Integration**: end‑to‑end simulate → approve → run → diff; verify write protections.
* **Chaos**: inject malformed VEX, missing advisories; ensure graceful degradation and clear errors.
---
## 11) Developer checklists
**Definition of Ready**
* Policy grammar finalized; examples prepared.
* Linkset join queries benchmarked.
* Owner and approvers assigned.
**Definition of Done**
* All APIs live with RBAC.
* CLI and Console features shipped.
* Determinism and golden tests green.
* Observability dashboards deployed.
* Docs in section 7 merged.
* Two real org policies migrated and in production.
---
## 12) Glossary
* **Policy**: versioned rule set controlling status and severity.
* **DSL**: domain‑specific language used to express rules.
* **Run**: a single evaluation execution with defined inputs and outputs.
* **Simulation**: a run that doesn’t write findings; returns diffs.
* **Materialization**: persisted effective findings for fast queries.
* **Determinism**: same inputs + same policy = same outputs. Always.
---
### Final imposed reminder
**Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**

531
EPIC_3.md Normal file
View File

@@ -0,0 +1,531 @@
Here’s Epic 3 in the same “paste‑into‑repo” format: exhaustive, implementation‑ready, and aligned with the AOC model plus the Policy Engine from the previous epics.
---
# Epic 3: StellaOps Console (Web UI over WebServices)
> Short name: **StellaOps Console**
> Services touched: **Web API Gateway, Authority (authN/Z), Policy Engine, SBOM Service, Conseiller (Feedser), Excitator (Vexer), Scheduler/Workers, Telemetry**
> Data stores used via APIs: **MongoDB (reads only from UI), object storage for traces**, optional **Redis/NATS** for live updates
> Deliverable: **TypeScript/React web application** with a component library and feature modules, packaged as container images and static builds
---
## 1) What it is
**StellaOps Console** is the first‑party Web UI for all Stella WebServices. It provides a cohesive, role‑aware surface for:
* Viewing raw AOC facts (advisories, VEX, SBOMs) without mutation.
* Applying and simulating policies (VEX application rules, advisory normalization) then exploring **effective findings**.
* Navigating SBOMs as graphs, zooming into components, and seeing linked advisories/VEX with clear precedence.
* Running and monitoring evaluations, auditing why decisions were made, and exporting evidence.
* Administering tenants, users and roles, API tokens, and integrations.
* Publishing a self‑hostable Console image and a **Download & Install** page covering all product containers.
The Console is a read‑write client for allowable operations (policy authoring, run orchestration, approvals), and strictly read‑only for **raw facts** per the AOC enforcement. It is **not** a new API; it is a UI over the existing ones with strong guardrails and deterministic behavior.
---
## 2) Why
* Teams need a single, consistent interface to explore SBOMs, advisories, VEX, and policy outcomes.
* Audits require visible provenance, replayable evidence, and explanation chains.
* Policy creation and simulation are safer when you can see deltas and traces.
* Many workflows benefit from visual tools: graph explorers, diff views, and step‑wise wizards.
* Not everyone wants to live in the CLI all day. Parity and choice matter.
---
## 3) How it should work (maximum detail)
### 3.1 Information architecture
Top‑level navigation with a tenant context picker:
1. **Dashboard**: high‑level posture and recent changes.
2. **SBOMs**: catalog, search, and **Graph Explorer**.
3. **Advisories & VEX**: raw fact browsers with aggregation‑not‑merge semantics.
4. **Findings**: policy‑materialized findings with filters and explanations.
5. **Policies**: editor, simulation, versioning, approvals.
6. **Runs**: orchestration, live progress, history, diffs.
7. **Reports & Export**: evidence packages, CSV/JSON exports.
8. **Admin**: users/roles, tokens, SSO, tenants, registries, settings.
9. **Downloads**: product containers and installation instructions.
Global elements:
* **Global Filters**: policy version, environment profile, severity band, time window.
* **Search Bar**: PURL, CVE/GHSA IDs, SBOM IDs.
* **Live Status**: background jobs, queue lag, last sync cursors.
* **Help & Docs**: contextual deep links into `/docs/*`.
### 3.2 Navigation & routes
```
/dashboard
/sboms
/sboms/:sbomId
/sboms/:sbomId/graph
/advisories
/advisories/:advisoryId (shows all linked sources; aggregation only)
/vex
/vex/:vexId
/findings?policy=:pid&sbom=:sid&status=:st&severity=:sev
/findings/:findingId/explain
/policies
/policies/:policyId/versions/:v
/policies/:policyId/simulate
/runs
/runs/:runId
/reports
/admin/users
/admin/roles
/admin/tenants
/admin/integrations
/admin/tokens
/downloads
```
### 3.3 Core feature modules
#### 3.3.1 Dashboard
* Cards: “Findings by severity,” “VEX overrides in last 24h,” “New advisories linked,” “Run health,” “Policy changes.”
* Click‑through to filtered views.
* Data sources: aggregated endpoints exposed by Web API (no client‑side aggregation over large sets).
#### 3.3.2 SBOM Explorer (catalog + graph)
* **Catalog**: table with SBOM ID, artifact name/version, source, ingest time, component count, last evaluation per policy.
* **Detail**: components tabular view with paging; filters by package type, license, scope.
* **Graph Explorer**:
* Interactive canvas with pan/zoom, focus on component, dependency paths, reachability placeholders.
* Overlay toggles: highlight components with affected findings; show VEX “not_affected” zones; show licenses risk overlay.
* **Policy overlays**: toggle between policy versions to see in‑place severity/status changes.
* **Actions**: export component list, copy PURL, open related findings.
**AOC alignment**: SBOM content is immutable; any edits are proposed as new SBOM versions upstream. UI displays raw SBOM JSON in a read‑only side panel.
#### 3.3.3 Advisories & VEX browsers
* **Advisories list**:
* Left panel: filters by source (OSV, GHSA, CSAF vendors, NVD), published/modified time, affected ecosystem.
* Middle panel: **aggregation group** keyed by linkset identity (same vulnerability across sources). No merging; show a roll‑up with per‑source chips.
* Right panel: selected advisory source view with raw JSON, references, CVSS vectors, and “linked SBOM components” sample.
* Severity shown three ways: vendor‑reported, normalized (per mapping), and **effective** under the currently selected policy.
* **VEX list**:
* Filters by vendor, product, status, justification, scope.
* Detail panel: all statements applying to the same `(component, advisory)` tuple, with precedence logic visualization and the statement that won under the current policy.
* Raw JSON viewer for each document.
**Strict rule**: Conseiller and Excitator are visualized as **aggregators only**. No UI affordance suggests server‑side merging. All links route to raw documents with reference IDs.
#### 3.3.4 Findings
* Virtualized table supporting millions of rows via server‑side pagination and cursoring.
* Columns: policy, SBOM, component PURL, advisory IDs (chips for each source), status, severity, last updated, rationale count.
* Row click → **Explain** view with rule hits in order, references to advisories/VEX used, and links to raw docs and trace blobs.
* Bulk export with query replay (the export API re‑runs the same filters on the server and streams CSV/JSON).
#### 3.3.5 Policies
* Embedded **Policy Editor** (from Epic 2) with Monaco features, simulation panel, diffs, and approval workflow.
* Pre‑commit lint and compile; cannot submit with syntax errors.
* Simulation results show increase/decrease unchanged counts, top rules impacting results, and sample affected components.
#### 3.3.6 Runs
* Queue view: queued/running/succeeded/failed with timestamps and SLA hints.
* Live progress with **SSE/WebSocket** updates: tuples processed, rules fired, findings materialized.
* Diff view between runs for the same policy and SBOM set.
* Retry and cancel actions as allowed by RBAC.
#### 3.3.7 Reports & Export
* Evidence bundle creation: include policy version, run ID, sample traces, and result slices.
* Export templates (CSV for management, JSON/NDJSON for SIEM ingestion).
* Signed export manifests with checksums.
#### 3.3.8 Admin
* Users & Roles: invite, disable, role mapping.
* Tenants: create, switch, default policy bindings.
* Tokens: create scoped API tokens with expirations.
* Integrations: configure SSO (OIDC), registries, webhooks.
* Settings: environment defaults for policy evaluation (exposure, runtime hints).
#### 3.3.9 Downloads
* List of official Docker images: `stella-console`, `stella-api`, `conseiller`, `excitator`, `sbom-svc`, `policy-engine`, etc.
* Version matrix, pull commands, Helm chart snippet, offline tarballs, and system requirements.
* Link to `/docs/install/docker.md` and `/docs/deploy/console.md`.
### 3.4 UX flows (key tasks)
* **Triage a vulnerability**: search CVE → open roll‑up → view all sources → jump to affected findings → open Explain → see VEX precedence → decide if policy change is needed → simulate policy → if good, submit and request approval → run → verify new findings.
* **Investigate SBOM**: open SBOM → Graph Explorer → highlight affected nodes under policy P‑X vN → click a node → see linked advisories + VEX → open Explain for a specific finding.
* **Audit evidence**: open run → download evidence bundle with policy, run metadata, traces, and effective finding slice.
* **Onboard team**: invite users, set roles, define default tenant policies, give read‑only access to auditors.
### 3.5 CLI vs UI parity
Create `/docs/cli-vs-ui-parity.md` with a matrix. Principle:
* All **read** capabilities must exist in both CLI and UI.
* All **policy lifecycle** actions exist in both.
* Long‑running operations can be initiated in UI and monitored in either surface.
### 3.6 Security & auth
* Auth: OIDC with PKCE; short‑lived ID tokens; silent refresh.
* RBAC enforced by the API; UI only gates affordances and never trusts itself.
* CSRF not applicable for token‑based APIs; still set robust **CSP**, **X‑Frame‑Options**, and **Referrer‑Policy**.
* Tenancy: every API call includes tenant header; UI shows explicit tenant badge.
* Sensitive pages require **fresh auth** (re‑prompt).
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 3.7 Accessibility & i18n
* WCAG 2.1 AA: keyboard nav, focus indicators, ARIA for tables and graphs, color‑contrast tests.
* i18n scaffolding via ICU messages; English shipped first; content keys stored in code, translations as JSON resources.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 3.8 Performance
* Use server‑side pagination and cursoring everywhere; never fetch unbounded lists.
* Virtualized tables and lazy panels.
* Graph Explorer loads neighborhood windows, not whole graphs.
* Cache with TanStack Query; deduplicate requests; stale‑while‑revalidate.
* Performance budgets in CI (Lighthouse): TTI < 3.5s on reference hardware.
### 3.9 Error handling & offline
* Error boundaries per feature; retry buttons; copyable request IDs.
* Network loss → banner + read‑only cached views where safe.
* Clear messages for **AOC** constraints: raw facts cannot be edited.
### 3.10 Telemetry & observability
* UI event telemetry to internal sink (no third‑party beacons by default).
* Metrics: UI API latency percentiles, error rates, SSE subscription health.
* Feature flags to dark‑launch modules.
---
## 4) APIs consumed (representative)
* `GET /sboms`, `GET /sboms/{id}`, `GET /sboms/{id}/components?cursor=...`
* `GET /advisories?source=...`, `GET /advisories/{id}`, `GET /advisories/{id}/linked`
* `GET /vex?status=...`, `GET /vex/{id}`
* `GET /findings/{policyId}` and `GET /findings/{policyId}/{findingId}/explain`
* `POST /policies`, `POST /policies/{id}/compile`, `POST /policies/{id}/simulate`, `POST /policies/{id}/approve`
* `POST /policies/{id}/runs`, `GET /policies/{id}/runs/{runId}` with SSE for progress
* `POST /exports` for evidence bundles
* `GET /auth/user`, `GET /auth/tenants`, `POST /admin/users`, `POST /admin/tokens`
All calls include tenant scope headers and bearer tokens from Authority.
---
## 5) Implementation plan
### 5.1 Frontend architecture
* **Framework**: Next.js 14 (App Router) with TypeScript.
* **State/data**: TanStack Query for server state; Redux only if a global app state proves necessary.
* **UI toolkit**: Internal **Stella UI** component library (headless + primitives) with CSS variables and design tokens.
* **Visualization**: D3 for graph, Monaco for policy editing.
* **Testing**: Playwright (E2E), Vitest/Jest (unit), Storybook (components), Lighthouse (perf).
* **i18n**: `@formatjs/intl` + message catalogs.
* **Packaging**: static build served by Node adapter behind the API gateway; also a `stella-console` Docker image.
**Repo layout**
```
/console
/apps/web
/packages/ui # design system & components
/packages/api # typed API clients (OpenAPI codegen)
/packages/features # feature modules (sboms, advisories, vex, findings, policies, runs, admin)
/packages/utils
/e2e
/storybook
```
### 5.2 Design System (packages/ui)
* Foundation tokens: color, spacing, typography, elevation; dark/light modes.
* Components: AppShell, Nav, DataTable (virtualized), Badge/Chip, Tabs, Drawer, GraphCanvas, CodeViewer (read‑only JSON), Form primitives, Modal, Toast, Pill filters.
* Accessibility baked into components; snapshot and interaction tests in Storybook.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 5.3 Feature modules
Each module has:
* `routes.tsx` pages, `api` data hooks, `components`, `tests`, `docs link`.
* Query keys standardized for caching and invalidation.
**SBOMs**
* Hooks: `useSboms`, `useSbom(id)`, `useComponents(sbomId, query)`.
* GraphCanvas using neighborhood loaders: `/sboms/:id/graph?center=:purl&depth=1..3`.
**Advisories**
* `useAdvisories(filters)` and `useAdvisory(id)` plus `useLinkedAdvisories(id)`.
* UI explicitly shows aggregation groups; never collapses sources into one record.
**VEX**
* `useVex(filters)`, `useVexDoc(id)`, `useVexForTuple(purl, advisoryId)` for precedence views.
**Findings**
* `useFindings(policyId, filters, cursor)` and `useFinding(findingId)`.
* Explain viewer reading `/findings/:policyId/:findingId/explain`.
**Policies**
* Monaco editor wrapper; compile/simulate actions; approval dialog.
* Diff viewer using the compiler’s diagnostics and rule stats.
**Runs**
* `useRuns`, `useRun(runId)` + SSE hook `useRunProgress(runId)`.
**Admin**
* `useUsers`, `useRoles`, `useTenants`, `useTokens`, `useIntegrations`.
**Downloads**
* Static page with dynamic image tags fetched from registry metadata endpoint; copy‑able commands and Helm snippets.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 5.4 Live updates
* SSE/WebSocket client with backoff, heartbeat, and re‑subscribe logic.
* Only Runs and slim “ticker” endpoints use live channels; everything else is HTTP pull with caching.
### 5.5 Security
* OIDC PKCE flow; token storage in memory; refresh via hidden iframe or refresh endpoint.
* CSP locked to same‑origin, with hashes for inline scripts from Next.
* Feature flags control admin features visibility; RBAC double‑checked on server responses.
### 5.6 Packaging & distribution
* `stella-console:<version>` image built in CI; Nginx or Node serve.
* Helm chart values include Authority issuer, API base URL, tenant defaults.
* Offline bundle artifact for air‑gapped deployments.
---
## 6) Documentation changes (create/update)
1. **`/docs/ui/console-overview.md`**
* Purpose, IA, tenant model, role mapping, AOC alignment.
2. **`/docs/ui/navigation.md`**
* Route map, global filters, keyboard shortcuts, deep links.
3. **`/docs/ui/sbom-explorer.md`**
* Catalog, detail, Graph Explorer, overlays, exports.
4. **`/docs/ui/advisories-and-vex.md`**
* Aggregation‑not‑merge, multi‑source roll‑ups, raw viewers.
5. **`/docs/ui/findings.md`**
* Filters, table semantics, explain view, exports.
6. **`/docs/ui/policies.md`**
* Editor, simulation, diffs, approvals, links to DSL docs.
7. **`/docs/ui/runs.md`**
* Queue, live progress, diffs, retries, evidence bundles.
8. **`/docs/ui/admin.md`**
* Users, roles, tenants, tokens, integrations.
9. **`/docs/ui/downloads.md`**
* Containers list, versions, pull/install commands, air‑gapped flow.
10. **`/docs/deploy/console.md`**
* Helm, ingress, TLS, CSP, environment variables, health checks.
11. **`/docs/install/docker.md`**
* All container images, pull commands, compose/Helm examples.
12. **`/docs/security/console-security.md`**
* OIDC, RBAC, CSP, tenancy, evidence of least privilege.
13. **`/docs/observability/ui-telemetry.md`**
* UI metrics, logs, dashboards, feature flags.
14. **`/docs/cli-vs-ui-parity.md`**
* Matrix of operations and surfaces.
15. **`/docs/architecture/console.md`**
* Frontend architecture, packages, data flow diagrams, SSE design.
16. **`/docs/accessibility.md`**
* WCAG checklist, testing tools, color tokens.
17. **`/docs/examples/ui-tours.md`**
* Task‑centric walkthroughs: triage, audit, policy rollout.
> Each document includes a “Compliance checklist” section.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Tasks (tracked per team)
### 7.1 Console scaffold & infra
* [ ] Initialize Next.js 14 TypeScript app with App Router.
* [ ] Set up TanStack Query, Auth context, Error boundaries, Toasts.
* [ ] Integrate OIDC client; implement login/logout, tenant picker.
* [ ] Add design tokens and base components in `packages/ui`.
* [ ] Configure CI: build, test, lint, type‑check, Lighthouse budgets.
* [ ] Build `stella-console` container image and Helm chart.
### 7.2 Typed API client
* [ ] Generate clients from OpenAPI; wrap with hooks in `packages/api`.
* [ ] Centralize retry, error mapping, tenant header injection.
### 7.3 Feature delivery
**SBOMs**
* [ ] Catalog page with filters, server pagination.
* [ ] SBOM detail with components table.
* [ ] Graph Explorer with overlays and neighborhood loaders.
* [ ] Raw JSON viewer drawer.
**Advisories & VEX**
* [ ] Advisory aggregation list; per‑source chips; raw view.
* [ ] VEX list with filters; precedence explainer per tuple.
* [ ] Link outs to Findings and SBOMs.
**Findings**
* [ ] Virtualized table; filters; saved views.
* [ ] Explain view: rules fired, references, trace links.
* [ ] Export actions (CSV/JSON stream).
**Policies**
* [ ] Monaco editor with syntax/diagnostics; compile and simulate.
* [ ] Diff and impact panel; submit and approve workflow.
* [ ] Run from simulation context.
**Runs**
* [ ] Runs list; run detail with SSE progress.
* [ ] Diff between runs; evidence bundle download.
**Admin**
* [ ] Users/roles CRUD; token issuance; tenant management.
* [ ] Integrations: OIDC config form; registry connections.
* [ ] Settings for environment defaults.
**Downloads**
* [ ] Registry tag fetch, pull commands, Helm snippet generator.
* [ ] Air‑gapped instructions and offline bundle download.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 7.4 Quality gates
* [ ] Playwright E2E for core flows: triage, simulate, approve, run, explain.
* [ ] Storybook with a11y addon and interaction tests.
* [ ] Lighthouse CI budgets met; perf regressions block merges.
* [ ] i18n scaffolding ready; all strings externalized.
* [ ] Security checks: CSP effective, OIDC flows tested, RBAC enforced.
### 7.5 Docs tasks
* [ ] Populate all docs listed in section 6 with screenshots and animated GIFs.
* [ ] Add “CLI vs UI” parity matrix and keep it in CI to detect drift.
* [ ] Add “AOC user guide” callouts explaining raw fact immutability across pages.
---
## 8) Feature flags
* `ui.graph-explorer`
* `ui.policy-editor`
* `ui.ai-assist` (off by default; when enabled, renders a right‑rail for human‑in‑the‑loop summaries)
* `ui.downloads`
Flag definitions and defaults live in `/docs/observability/ui-telemetry.md` and config map.
---
## 9) Acceptance criteria
* Console ships as a container image with Helm deployment and a static build option.
* SBOM Explorer visualizes graphs and overlays policy outcomes without page crashes on large SBOMs.
* Advisories/VEX browsers display **aggregation only**, never merge sources; raw document viewers are present.
* Findings view supports server‑side pagination and Explain with rule traces.
* Policy Editor compiles, simulates, diffs, and supports approval workflows.
* Runs page shows live progress and enables evidence exports.
* Admin handles users, roles, tenants, tokens, and OIDC configuration.
* Downloads page lists all images and installation paths.
* All pages meet a11y checks and pass Lighthouse budgets.
* RBAC enforced in UI affordances and validated by API responses.
---
## 10) Risks and mitigations
* **Graph performance** on very large SBOMs → use neighborhood windows and server filters; cap depth.
* **UI/CLI drift** → parity matrix in CI; failing check blocks release.
* **Overfetching** → TanStack caching, cursor‑based endpoints, and strict data‑layer reviews.
* **Scope creep** in Admin → feature‑flag granular sections, ship iteratively.
* **AOC confusion** → constant raw/derived labeling and “view raw” toggles.
---
## 11) Test plan
* **Unit**: hooks and components; data adapters; graph layout utils.
* **E2E**: Playwright flows for triage, simulation→approval→run→explain, admin RBAC.
* **A11y**: axe‑core in CI and manual keyboard checks.
* **Perf**: Lighthouse against seeded data; visual regression on Storybook.
* **Security**: OIDC happy and unhappy paths, CSP violation tests, SSRF resistance for downloads metadata.
* **Resilience**: simulate API timeouts; verify error boundaries and retries.
---
## 12) Non‑goals (this epic)
* No server‑side report authoring engine beyond export templates.
* No proprietary graph database; server remains RESTful with indexed queries.
* No speculative automatic policy changes; all edits remain human‑driven.
---
## 13) Philosophy and guiding principles
* **AOC first**: the UI respects facts vs decisions. Raw content is immutable and visible.
* **Deterministic outcomes**: what you see equals what the Policy Engine produced, with an explanation you can export.
* **Explainability** over cleverness: every badge, color, and status maps to a rule and a source.
* **Parity**: UI is not a second‑class citizen, and the CLI is not an afterthought.
* **Composability**: modules are independent packages with clear contracts and tests.
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**

409
EPIC_4.md Normal file
View File

@@ -0,0 +1,409 @@
Here’s Epic 4 in the same paste‑into‑repo, implementation‑ready style as the prior epics. It’s exhaustive, formal, and slots directly into the existing AOC model, Policy Engine, and Console.
---
# Epic 4: Policy Studio (author, version, simulate)
> Short name: **Policy Studio**
> Services touched: **Policy Engine**, **Policy Registry** (new), **Web API Gateway**, **Authority** (authN/Z), **Scheduler/Workers**, **SBOM Service**, **Conseiller (Feedser)**, **Excitator (Vexer)**, **Telemetry**
> Surfaces: **Console (Web UI)** feature module, **CLI**, **CI hooks**
> Deliverables: Authoring workspace, policy versioning, static checks, simulation at scale, reviews/approvals, signing/publishing, promotion
---
## 1) What it is
**Policy Studio** is the end‑to‑end system for creating, evolving, and safely rolling out the rules that turn AOC facts (SBOM, advisories, VEX) into **effective findings**. It provides:
* A **workspace** where authors write policies in the DSL (Epic 2), with linting, autocompletion, snippets, and templates.
* A **Policy Registry** that stores immutable versions, compiled artifacts, metadata, provenance, and signatures.
* **Simulation** at two levels: quick local samples and large batch simulations across real SBOM inventories with full deltas.
* A **review/approval** workflow with comments, diffs, required approvers, and promotion to environments (dev/test/prod).
* **Publishing** semantics: signed, immutable versions bound to tenants; rollback and deprecation.
* Tight integration with **Explain** traces so any change can show exactly which rules fired and why outcomes shifted.
The Studio respects **AOC enforcement**: policies never edit or merge facts. They only interpret facts and produce determinations consistent with precedence rules defined in the DSL.
---
## 2) Why
* Policy errors are expensive. Authors need safe sandboxes, deterministic builds, and evidence before rollout.
* Auditors require immutability, provenance, and reproducibility from “source policy” to “effective finding.”
* Teams want gradual rollout: simulate, canary, promote, observe, rollback.
* Policy knowledge should be modular, reusable, and testable, not tribal.
---
## 3) How it should work (maximum detail)
### 3.1 Domain model
* **PolicyPackage**: `{name, tenant, description, owners[], tags[], created_at}`
* **PolicyVersion** (immutable): `{package, semver, source_sha, compiled_sha, status: draft|review|approved|published|deprecated|archived, created_by, created_at, signatures[], changelog, metadata{}}`
* **Workspace**: mutable working area for authors; holds unversioned edits until compiled.
* **CompilationArtifact**: `{policy_version, compiler_version, diagnostics[], rule_index[], symbol_table}`
* **SimulationSpec**: `{policy_version|workspace, sbom_selector, time_window?, environment?, sample_size?, severity_floor?, includes{advisories?, vex?}}`
* **SimulationRun**: `{run_id, spec, started_at, finished_at, result{counts_before, counts_after, top_deltas[], by_rule_hit[], sample_explains[]}}`
* **Review**: `{policy_version, required_approvers[], votes[], comments[], files_changed[], diffs[]}`
* **Promotion**: `{policy_version, environment: dev|test|prod, promoted_by, promoted_at, rollout_strategy: All|Percent|TenantSubset}`
* **Attestation**: OIDC‑backed signature metadata binding `source_sha` and `compiled_sha` to an actor and time.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 3.2 Authoring workflow
1. **Create** a workspace from a template (e.g., “Default Risk Model,” “License Tilted,” “Cloud‑Native SBOM”).
2. **Edit** in the Studio: Monaco editor with DSL grammar, intelligent completion for predicates, policies, attributes.
3. **Lint & compile** locally: semantic checks, forbidden rules detection, policy size limits, constant‑folding.
4. **Unit tests**: run policy test cases on bundled fixtures and golden expectations.
5. **Quick simulate** on selected SBOMs (10–50 items) to preview counts, examples, and rule heatmap.
6. **Propose version**: bump semver, enter changelog; create a **PolicyVersion** in `review` with compiled artifacts.
7. **Review & approval**: side‑by‑side diff, comments, required approvers enforced by RBAC.
8. **Batch simulation**: run at scale across tenant inventory; produce deltas, sample explainer evidence.
9. **Publish**: sign and move to `published`; optional **Promotion** to target environment(s).
10. **Run** evaluation with the selected policy version; verify outcomes; optionally promote to default.
11. **Rollback**: select an older version; promotion updates references without mutating older versions.
### 3.3 Editing experience (Console)
* **Three‑pane layout**: file tree, editor, diagnostics/simulation.
* **Features**: autocomplete from symbol table, in‑editor docs on hover, go‑to definition, rule references, rename symbols across files, snippet library, policy templates.
* **Validations**:
* AOC guardrails: no edit/merge actions on source facts, only interpretation.
* Precedence correctness: if rules conflict, studio shows explicit order and effective winner.
* Severity floor and normalization mapping validated against registry configuration.
* **Diagnostics panel**: errors, warnings, performance hints (e.g., “predicate X loads N advisories per component; consider indexing”).
* **Rule heatmap**: during simulation, bar chart of rule firings and the objects they impact.
* **Explain sampler**: click any delta bucket to open a sampled finding with full trace.
### 3.4 Simulation
* **Quick Sim**: synchronous; runs in browser‑orchestrated job against API, constrained by `sample_size`.
* **Batch Sim**: asynchronous run in workers:
* Input selection: all SBOMs, labels, artifact regex, last N ingests, or a curated set.
* Outputs: counts by severity before/after, by status, top deltas by component and advisory, rule heatmap, top K affected artifacts.
* Evidence: NDJSON of sampled findings with traces; CSV summary; signed result manifest.
* Guardrails: cannot publish if batch sim drift > configurable threshold without an override justification.
### 3.5 Versioning & promotion
* Semver enforced: `major` implies compatibility break (e.g., precedence changes), `minor` adds rules, `patch` fixes.
* **Immutable**: after `published`, the version cannot change; deprecate instead.
* **Environment bindings**: dev/test/prod mapping per tenant; default policy per environment.
* **Canary**: promote to a subset of tenants or artifacts; the Runs page displays A/B comparisons.
### 3.6 Review & approval
* Require N approvers by role; self‑approval optionally prohibited.
* Line and file comments; overall decision with justification.
* Review snapshot captures: diffs, diagnostics, simulation summary.
* Webhooks to notify external systems of review events.
### 3.7 RBAC (Authority)
Roles per tenant:
* **Policy Author**: create/edit workspace, quick sim, propose versions.
* **Policy Reviewer**: comment, request changes, approve/reject.
* **Policy Approver**: final approve, publish.
* **Policy Operator**: promote, rollback, schedule runs.
* **Read‑only Auditor**: view everything, download evidence.
All actions server‑checked; UI only hides affordances.
### 3.8 CLI + CI integration
CLI verbs (examples):
```
stella policy init --template default
stella policy lint
stella policy compile
stella policy test --golden ./tests
stella policy simulate --sboms label:prod --sample 1000
stella policy version bump --level minor --changelog "Normalize GHSA CVSS"
stella policy submit --reviewers alice@example.com,bob@example.com
stella policy approve --version 1.3.0
stella policy publish --version 1.3.0 --sign
stella policy promote --version 1.3.0 --env test --percent 20
stella policy rollback --env prod --to 1.2.1
```
CI usage:
* Lint, compile, and run unit tests on PRs that modify `/policies/**`.
* Optionally trigger **Batch Sim** against a staging inventory and post a Markdown report to the PR.
* Block merge if diagnostics include errors or drift exceeds thresholds.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 3.9 APIs (representative)
* `POST /policies/workspaces` create from template
* `PUT /policies/workspaces/{id}/files` edit source files
* `POST /policies/workspaces/{id}/compile` get diagnostics + compiled artifact
* `POST /policies/workspaces/{id}/simulate` quick sim
* `POST /policies/versions` create version from workspace with semver + changelog
* `GET /policies/versions/{id}` fetch version + diagnostics + sim summary
* `POST /policies/versions/{id}/reviews` open review
* `POST /policies/versions/{id}/approve` record approval
* `POST /policies/versions/{id}/publish` sign + publish
* `POST /policies/versions/{id}/promote` bind to env/canary
* `POST /policies/versions/{id}/simulate-batch` start batch sim (async)
* `GET /policies/simulations/{run_id}` get sim results and artifacts
* `GET /policies/registry` list packages/versions, status and bindings
All calls require tenant scoping and RBAC.
### 3.10 Storage & data
* **Policy Registry DB** (MongoDB): packages, versions, workspaces, metadata.
* **Object storage**: source bundles, compiled artifacts, simulation result bundles, evidence.
* **Indexing**: compound indexes by `{tenant, package}`, `{tenant, status}`, `{tenant, environment}`.
* **Retention**: configurable retention for workspaces and simulation artifacts; versions never deleted, only archived.
### 3.11 Evidence & provenance
* Every published version has:
* `source_sha` (content digest of the policy source bundle)
* `compiled_sha` (digest of compiled artifact)
* Attestation: signed envelope binding digests to an identity, time, and tenant.
* Links to the exact compiler version, inputs, and environment.
### 3.12 Observability
* Metrics: compile time, diagnostics rate, simulation queue depth, delta magnitude distribution, approval latencies.
* Logs: structured events for lifecycle transitions.
* Traces: long simulations emit span per shard.
### 3.13 Performance & scale
* Compilation should complete under 3 seconds for typical policies; warn at 10s.
* Batch sim uses workers with partitioning by SBOM id; results reduced by the API.
* Memory guardrails on rule execution; deny policies that exceed configured complexity limits.
### 3.14 Security
* OIDC‑backed signing and attestation.
* Policy sources are scanned on upload for secrets; blocked if found.
* Strict CSP in Studio pages; tokens stored in memory, not localStorage.
* Tenant isolation in buckets and DB collections.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 4) Implementation plan
### 4.1 Services
* **Policy Registry (new microservice)**
* REST API and background workers for batch simulation orchestration.
* Stores workspaces, versions, metadata, bindings, reviews.
* Generates signed attestations at publish time.
* Coordinates with **Policy Engine** for compile/simulate invocations.
* **Policy Engine (existing)**
* Expose compile and simulate endpoints with deterministic outputs.
* Provide rule coverage, symbol table, and explain traces for samples.
* **Web API Gateway**
* Routes requests; injects tenant context; enforces RBAC.
### 4.2 Console (Web UI) feature module
* `packages/features/policies` (shared with Epic 3):
* **Studio** routes: `/policies/studio`, `/policies/:id/versions/:v/edit`, `/simulate`, `/review`.
* Monaco editor wrapper for DSL with hover docs, autocomplete.
* Diff viewer, diagnostics, heatmap, explain sampler, review UI.
### 4.3 CLI
* New commands under `stella policy *`; typed client generated from OpenAPI.
* Outputs machine‑readable JSON and pretty tables.
### 4.4 Workers
* **Simulation workers**: pull shards of SBOMs, run policy, emit partials, reduce into result bundle.
* **Notification worker**: sends webhooks on review, approval, publish, promote.
---
## 5) Documentation changes (create/update)
1. **`/docs/policy/studio-overview.md`**
* Concepts, roles, lifecycle, glossary.
2. **`/docs/policy/authoring.md`**
* Workspace, templates, snippets, lint rules, best practices.
3. **`/docs/policy/versioning-and-publishing.md`**
* Semver, immutability, deprecation, rollback, attestations.
4. **`/docs/policy/simulation.md`**
* Quick vs batch sim, selection strategies, thresholds, evidence artifacts.
5. **`/docs/policy/review-and-approval.md`**
* Required approvers, comments, webhooks, audit trail.
6. **`/docs/policy/promotion.md`**
* Environments, canary, default policy binding, rollback.
7. **`/docs/policy/cli.md`**
* Command reference with examples and JSON outputs.
8. **`/docs/policy/api.md`**
* REST endpoints, request/response schemas, error codes.
9. **`/docs/security/policy-attestations.md`**
* Signatures, digests, verifier steps.
10. **`/docs/architecture/policy-registry.md`**
* Service design, schemas, queues, failure modes.
11. **`/docs/observability/policy-telemetry.md`**
* Metrics, logs, tracing, dashboards.
12. **`/docs/runbooks/policy-incident.md`**
* Rolling back a bad policy, freezing publishes, forensic steps.
13. **`/docs/examples/policy-templates.md`**
* Ready‑made templates and snippet catalog.
14. **`/docs/aoc/aoc-guardrails.md`**
* How Studio enforces AOC in authoring and review.
Each doc ends with a “Compliance checklist.”
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 6) Tasks
### 6.1 Backend: Policy Registry
* [ ] Define OpenAPI spec for Registry (workspaces, versions, reviews, sim).
* [ ] Implement workspace storage and file CRUD.
* [ ] Integrate with Policy Engine compile endpoint; return diagnostics, symbol table.
* [ ] Implement quick simulation with request limits.
* [ ] Implement batch simulation orchestration: enqueue shards, collect results, reduce deltas, store artifacts.
* [ ] Implement review model: comments, required approvers, decisions.
* [ ] Implement publish: sign, persist attestation, set status=published.
* [ ] Implement promotion bindings per tenant/environment; canary subsets.
* [ ] RBAC checks for all endpoints.
* [ ] Unit/integration tests; load tests for batch sim.
### 6.2 Policy Engine enhancements
* [ ] Return rule coverage and firing counts with compile/simulate.
* [ ] Return symbol table and inline docs for editor autocomplete.
* [ ] Expose deterministic Explain traces for sampled findings.
* [ ] Enforce complexity/time limits and report breaches.
### 6.3 Console (Web UI)
* [ ] Build Studio editor wrapper with Monaco + DSL language server hooks.
* [ ] Implement file tree, snippets, templates, hotkeys, search/replace.
* [ ] Diagnostics panel with jump‑to‑line, quick fixes.
* [ ] Simulation panel: quick sim UI, charts, heatmap, sample explains.
* [ ] Review UI: diff, comments, approvals, status badges.
* [ ] Publish & Promote flows with confirmation and post‑actions.
* [ ] Batch sim results pages with export buttons.
* [ ] Accessibility audits and keyboard‑only authoring flow.
### 6.4 CLI
* [ ] Implement commands listed in 3.8 with rich help and examples.
* [ ] Add `--json` flag for machine consumption; emit stable schemas.
* [ ] Exit codes aligned with CI usage (lint errors → non‑zero).
### 6.5 CI/CD & Security
* [ ] Add CI job that runs `stella policy lint/compile/test` on PRs.
* [ ] Optional job that triggers batch sim against staging inventory; post summary to PR.
* [ ] Policy source secret scanning; block on findings.
* [ ] Signing keys configuration; verify pipeline for attestation on publish.
### 6.6 Docs
* [ ] Write all docs in section 5 with screenshots and CLI transcripts.
* [ ] Add cookbook examples and templates in `/docs/examples/policy-templates.md`.
* [ ] Wire contextual Help links from Studio to relevant docs.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Acceptance criteria
* Authors can create, edit, lint, compile policies with inline diagnostics and autocomplete.
* Quick simulation produces counts, rule heatmap, and sample explains within UI.
* Batch simulation scales across large SBOM sets, producing deltas and downloadable evidence.
* Review requires configured approvers; comments and diffs are preserved.
* Publish generates immutable, signed versions with attestations.
* Promotion binds versions to environments and supports canary and rollback.
* CLI supports full lifecycle and is usable in CI.
* All actions are tenant‑scoped, RBAC‑enforced, and logged.
* AOC guardrails prevent any mutation of raw facts.
* Documentation shipped and linked contextually from the Studio.
---
## 8) Risks & mitigations
* **Policy complexity causes timeouts** → compile‑time complexity scoring, execution limits, early diagnostics.
* **Simulation cost at scale** → sharding and streaming reducers; sampling; configurable quotas.
* **RBAC misconfiguration** → server‑enforced checks, defense‑in‑depth tests, deny‑by‑default.
* **Attestation key management** → OIDC‑backed signatures; auditable verifier tool; time‑boxed credentials.
* **Editor usability** → language server with accurate completions; docs on hover; snippet library.
---
## 9) Test plan
* **Unit**: compiler adapters, registry models, reviewers workflow, CLI options.
* **Integration**: compile→simulate→publish→promote on seeded data.
* **E2E**: Playwright flows for author→review→batch sim→publish→promote→rollback.
* **Performance**: load test batch simulation with 100k components spread across SBOMs.
* **Security**: RBAC matrix tests; secret scanning; signing and verification.
* **Determinism**: same inputs produce identical `compiled_sha` and simulation summaries.
---
## 10) Feature flags
* `policy.studio` (enables editor and quick sim)
* `policy.batch-sim`
* `policy.canary-promotion`
* `policy.signature-required` (enforce signing on publish)
Flags documented in `/docs/observability/policy-telemetry.md`.
---
## 11) Non‑goals (this epic)
* Building a general IDE for arbitrary languages; the editor is purpose‑built for the DSL.
* Auto‑generated policies from AI without human approval.
* Cross‑tenant policies; all policies are tenant‑scoped.
---
## 12) Philosophy
* **Safety first**: it’s cheaper to prevent a bad policy than to fix its fallout.
* **Determinism**: same inputs, same outputs, verifiably.
* **Immutability**: versions and evidence are forever; we deprecate, not mutate.
* **Transparency**: every change is explainable with traces and proofs.
* **Reusability**: templates, snippets, and tests turn policy from art into engineering.
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**

431
EPIC_5.md Normal file
View File

@@ -0,0 +1,431 @@
Here’s Epic 5 in the same paste‑into‑repo, implementation‑ready format as the prior epics. It’s exhaustive, formal, and designed to slot into AOC, Policy Engine, Conseiller/Excitator, and the Console.
---
# Epic 5: SBOM Graph Explorer
> Short name: **Graph Explorer**
> Services touched: **SBOM Service**, **Graph Indexer** (new), **Graph API** (new), **Policy Engine**, **Conseiller (Feedser)**, **Excitator (Vexer)**, **Web API Gateway**, **Authority** (authN/Z), **Workers/Scheduler**, **Telemetry**
> Surfaces: **Console (Web UI)** graph module, **CLI**, **Exports**
> Deliverables: Interactive graph UI with semantic zoom, saved queries, policy/VEX/advisory overlays, diff views, impact analysis, exports
---
## 1) What it is
**SBOM Graph Explorer** is the interactive, tenant‑scoped view of all supply‑chain relationships the platform knows about, rendered as a navigable graph. It connects:
* **Artifacts** (applications, images, libs), **Packages/Versions**, **Files/Paths**, **Licenses**, **Advisories** (from Conseiller), **VEX statements** (from Excitator), **Provenance** (builds, sources), and **Policies** (overlays of determinations)
* **Edges** like `depends_on`, `contains`, `built_from`, `declared_in`, `affected_by`, `vex_exempts`, `governs_with`
* **Time/version** dimension: multiple SBOM snapshots with diffs
It’s built for investigation and review: find where a vulnerable package enters; see which apps are impacted; understand why a finding exists; simulate a policy version and see the delta. The explorer observes **AOC enforcement**: it never mutates facts; it aggregates and visualizes them. Only the Policy Engine may classify, and classification is displayed as overlays.
---
## 2) Why
* SBOMs are graphs. Tables flatten what matters and hide transitive risk.
* Engineers, security, and auditors need impact answers quickly: “What pulls in `log4j:2.17` and where is it at runtime?”
* Policy/VEX/advisory interactions are nuanced. A visual overlay makes precedence and outcomes obvious.
* Review is collaborative; you need saved queries, deep links, exports, and consistent evidence.
---
## 3) How it should work (maximum detail)
### 3.1 Domain model
**Nodes** (typed, versioned, tenant‑scoped):
* `Artifact`: application, service, container image, library, module
* `Package`: name + ecosystem (purl), `PackageVersion` with resolved version
* `File`: path within artifact or image layer
* `License`: SPDX id
* `Advisory`: normalized advisory id (GHSA, CVE, vendor), source = Conseiller
* `VEX`: statement with product context, status, justification, source = Excitator
* `SBOM`: ingestion unit; includes metadata (tool, sha, build info)
* `PolicyDetermination`: materialized view of Policy Engine results (read‑only overlay)
* `Build`: provenance, commit, workflow run
* `Source`: repo, tag, commit
**Edges** (directed):
* `declared_in` (PackageVersion → SBOM)
* `contains` (Artifact → PackageVersion | File)
* `depends_on` (PackageVersion → PackageVersion) with scope attr (prod|dev|test|optional)
* `built_from` (Artifact → Build), `provenance_of` (Build → Source)
* `affected_by` (PackageVersion → Advisory) with range semantics
* `vex_exempts` (Advisory ↔ VEX) scoped by product/component
* `licensed_under` (Artifact|PackageVersion → License)
* `governs_with` (Artifact|PackageVersion → PolicyDetermination)
* `derived_from` (SBOM → SBOM) for superseding snapshots
**Identity & versioning**
* Every node has a stable key: `{tenant}:{type}:{natural_id}` (e.g., purl for packages, digest for images).
* SBOM snapshots are immutable; edges carry `valid_from`/`valid_to` for time travel and diffing.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 3.2 User capabilities (end‑to‑end)
* **Search & Navigate**: global search (purls, CVEs, repos, licenses), keyboard nav, breadcrumbs, semantic zoom.
* **Lenses**: toggle views (Security, License, Provenance, Runtime vs Dev, Policy effect).
* **Overlays**:
* **Advisory overlay**: show affected nodes/edges with source, severity, ranges.
* **VEX overlay**: show suppressions/justifications; collapse exempted paths.
* **Policy overlay**: choose a policy version; nodes/edges reflect determinations (severity, status) with explain sampling.
* **Impact analysis**: pick a vulnerable node; highlight upstream/downstream dependents, scope filters, shortest/all paths with constraints.
* **Diff view**: compare SBOM A vs B; show added/removed nodes/edges, changed versions, changed determinations.
* **Saved queries**: visual builder + JSON query; shareable permalinks scoped by tenant and environment.
* **Exports**: GraphML, CSV edge list, NDJSON of findings, PNG/SVG snapshot.
* **Evidence details**: side panel with raw facts, advisory links, VEX statements, policy explain trace, provenance.
* **Accessibility**: tab‑navigable, high‑contrast, screen‑reader labels for nodes and sidebars.
### 3.3 Query model
* **Visual builder** for common queries:
* “Show all paths from Artifact X to Advisory Y up to depth 6.”
* “All runtime dependencies with license = GPL‑3.0.”
* “All artifacts affected by GHSA‑… with no applicable VEX.”
* “Which SBOMs introduced/removed `openssl` between build 120 and 130?”
* **JSON query** (internal, POST body) with:
* `start`: list of node selectors (type + id or attributes)
* `expand`: edge types and depth, direction, scope filters
* `where`: predicates on node/edge attributes
* `overlay`: policy version id, advisory sources, VEX filters
* `limit`: nodes, edges, timebox, cost budget
* **Cost control**: server estimates cost, denies or pages results; UI streams partial graph tiles.
### 3.4 UI architecture (Console)
* **Canvas**: WebGL renderer with level‑of‑detail, edge bundling, and label culling; deterministic layout when possible (seeded).
* **Semantic zoom**:
* Far: clusters by artifact/repo/ecosystem, color by lens
* Mid: package groups, advisory badges, license swatches
* Near: concrete versions, direct edges, inline badges for policy determinations
* **Panels**:
* Left: search, filters, lens selector, saved queries
* Right: details, explain trace, evidence tabs (Advisory/VEX/Policy/Provenance)
* Bottom: query expression, diagnostics, performance/stream status
* **Diff mode**: split or overlay, color legend (add/remove/changed), filter by node type.
* **Deep links**: URL encodes query + viewport; shareable respecting RBAC.
* **Keyboard**: space drag, +/- zoom, F to focus, G to expand neighbors, P to show paths.
### 3.5 Back‑end architecture
**Graph Indexer (new)**
* Consumes SBOM ingests, Conseiller advisories, Excitator VEX statements, Policy Engine determinations (read‑only).
* Projects facts into a **property graph** persisted in:
* Primary: document store + adjacency sets (e.g., Mongo collections + compressed adjacency lists)
* Optional driver for graph DB backends if needed (pluggable)
* Maintains materialized aggregates: degree, critical paths cache, affected artifact counts, license distribution.
* Emits **graph snapshots** per SBOM with lineage to original ingestion.
**Graph API (new)**
* Endpoints for search, neighbor expansion, path queries, diffs, overlays, exports.
* Streaming responses for large graphs (chunked NDJSON tiles).
* Cost accounting + quotas per tenant.
**Workers**
* **Centrality & clustering** precompute on idle: betweenness approximations, connected components, Louvain clusters.
* **Diff compute** on new SBOM ingestion pairs (previous vs current).
* **Overlay materialization** cache for popular policy versions.
**Policy Engine integration**
* Graph API requests can specify a policy version.
* For sampled nodes, the API fetches explain traces; for counts, uses precomputed overlay materializations where available.
**AOC enforcement**
* Graph Indexer never merges or edits advisories/VEX; it links them and exposes overlays that the Policy Engine evaluates.
* Conseiller and Excitator remain authoritative sources; severities come from Policy‑governed normalization.
### 3.6 APIs (representative)
* `GET /graph/search?q=...&type=package|artifact|advisory|license`
* `POST /graph/query` ⇒ stream tiles `{nodes[], edges[], stats, cursor}`
* `POST /graph/paths` body: `{from, to, depth<=6, constraints{scope, runtime_only}}`
* `POST /graph/diff` body: `{sbom_a, sbom_b, filters}`
* `GET /graph/snapshots/{sbom_id}` ⇒ graph metadata, counts, top advisories
* `POST /graph/export` body: `{format: graphml|csv|ndjson|png|svg, query|snapshot}`
* `GET /graph/saved` / `POST /graph/saved` save and list tenant queries
* `GET /graph/overlays/policy/{version_id}` ⇒ summary stats for caching
All endpoints tenant‑scoped, RBAC‑checked. Timeouts and pagination by server. Errors return structured diagnostics.
### 3.7 CLI
```
stella sbom graph search "purl:pkg:maven/org.apache.logging.log4j/log4j-core"
stella sbom graph query --file ./query.json --export graphml > graph.graphml
stella sbom graph impacted --advisory GHSA-xxxx --runtime-only --limit 100
stella sbom graph paths --from artifact:service-a --to advisory:GHSA-xxxx --depth 5 --policy 1.3.0
stella sbom graph diff --sbom-a 2025-03-15T10:00Z --sbom-b 2025-03-22T10:00Z --export csv > diff.csv
stella sbom graph save --name "openssl-runtime" --file ./query.json
```
Exit codes: 0 ok, 2 query validation error, 3 over‑budget, 4 not found, 5 RBAC denied.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
### 3.8 Performance & scale
* **Progressive loading**: server pages tiles by BFS frontier; client renders incrementally.
* **Viewport culling**: only visible nodes/edges in canvas; offscreen demoted to aggregates.
* **Level‑of‑detail**: simplified glyphs and collapsed clusters at distance.
* **Query budgets**: per‑tenant rate + node/edge caps; interactive paths limited to depth ≤ 6.
* **Caching**: hot queries memoized per tenant + overlay version; diffs precomputed for consecutive SBOMs.
### 3.9 Security
* Multi‑tenant isolation at storage and API layers.
* RBAC roles:
* **Viewer**: browse graphs, saved queries
* **Investigator**: run queries, export data
* **Operator**: configure budgets, purge caches
* **Auditor**: download evidence bundles
* Input validation for query JSON; deny disallowed edge traversals; strict CSP in web app.
### 3.10 Observability
* Metrics: tile latency, nodes/edges per tile, cache hit rate, query denials, memory pressure.
* Logs: structured, include query hash, cost, truncation flags.
* Traces: server spans per stage (parse, plan, fetch, overlay, stream).
### 3.11 Accessibility & UX guarantees
* Keyboard complete, ARIA roles for graph and panels, high‑contrast theme.
* Deterministic layout on reload for shareable investigations.
### 3.12 Data retention
* Graph nodes derived from SBOMs share retention with SBOM artifacts; overlays are ephemeral caches.
* Saved queries retained until deleted; references to missing objects show warnings.
---
## 4) Implementation plan
### 4.1 Services
* **Graph Indexer (new microservice)**
* Subscribes to SBOM ingest events, Conseiller advisory updates, Excitator VEX updates, Policy overlay materializations.
* Builds adjacency lists and node documents; computes aggregates and clusters.
* **Graph API (new microservice)**
* Validates and executes queries; streams tiles; composes overlays; serves diffs and exports.
* Integrates with Policy Engine for explain sample retrieval.
* **SBOM Service (existing)**
* Emits ingestion events with SBOM ids and lineage; exposes SBOM metadata to Graph API.
* **Web API Gateway**
* Routes `/graph/*`, injects tenant context, enforces RBAC.
### 4.2 Console (Web UI) feature module
* `packages/features/graph-explorer`
* Canvas renderer (WebGL), panels, query builder, diff mode, overlays, exports.
* Deep‑link router and viewport state serializer.
### 4.3 Workers
* Centrality/clustering worker, diff worker, overlay materialization worker.
* Schedules on low‑traffic windows; backpressure aware.
### 4.4 Data model (storage)
* Collections:
* `graph_nodes`: `{_id, tenant, type, natural_id, attrs, degree, created_at, updated_at}`
* `graph_edges`: `{_id, tenant, from_id, to_id, type, attrs, valid_from, valid_to}`
* `graph_snapshots`: per‑SBOM node/edge references
* `graph_saved_queries`: `{_id, tenant, name, query_json, created_by}`
* `graph_overlays_cache`: keyed by `{tenant, policy_version, hash(query)}`
* Indexes: compound on `{tenant, type, natural_id}`, `{tenant, from_id}`, `{tenant, to_id}`, time bounds.
---
## 5) Documentation changes (create/update)
1. **`/docs/sbom/graph-explorer-overview.md`**
* Concepts, node/edge taxonomy, lenses, overlays, roles, limitations.
2. **`/docs/sbom/graph-using-the-console.md`**
* Walkthroughs: search, navigate, impact, diff, export; screenshots and keyboard cheatsheet.
3. **`/docs/sbom/graph-query-language.md`**
* JSON schema, examples, constraints, cost/budget rules.
4. **`/docs/sbom/graph-api.md`**
* REST endpoints, request/response examples, streaming and pagination.
5. **`/docs/sbom/graph-cli.md`**
* CLI command reference and example pipelines.
6. **`/docs/policy/graph-overlays.md`**
* How policy versions render in Graph; explain sampling; AOC guardrails.
7. **`/docs/vex/graph-integration.md`**
* How VEX suppressions appear and how to validate product scoping.
8. **`/docs/advisories/graph-integration.md`**
* Advisory linkage and severity normalization by policy.
9. **`/docs/architecture/graph-services.md`**
* Graph Indexer, Graph API, storage choices, failure modes.
10. **`/docs/observability/graph-telemetry.md`**
* Metrics, logs, tracing, dashboards.
11. **`/docs/runbooks/graph-incidents.md`**
* Handling runaway queries, cache poisoning, degraded render.
12. **`/docs/security/graph-rbac.md`**
* Permissions matrix, multi‑tenant boundaries.
Every doc should end with a “Compliance checklist.”
**Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 6) Tasks
### 6.1 Backend: Graph Indexer
* [ ] Define node/edge schemas and attribute dictionaries for each type.
* [ ] Implement event consumers for SBOM ingests, Conseiller updates, Excitator updates.
* [ ] Build ingestion pipeline that populates nodes/edges and maintains `valid_from/valid_to`.
* [ ] Implement aggregate counters and degree metrics.
* [ ] Implement clustering job and persist cluster ids per node.
* [ ] Implement snapshot materialization per SBOM and lineage tracking.
* [ ] Unit tests for each node/edge builder; property‑based tests for identity stability.
### 6.2 Backend: Graph API
* [ ] Implement `/graph/search` with prefix and exact match across node types.
* [ ] Implement `/graph/query` with validation, planning, cost estimation, and streaming tile results.
* [ ] Implement `/graph/paths` with constraints and depth limits; shortest path heuristic.
* [ ] Implement `/graph/diff` computing adds/removes/changed versions; stream results.
* [ ] Implement overlays: advisory join, VEX join, policy materialization and explain sampling.
* [ ] Implement exports: GraphML, CSV edge list, NDJSON findings, PNG/SVG snapshots.
* [ ] RBAC middleware integration; multi‑tenant scoping.
* [ ] Load tests with synthetic large SBOMs; define default budgets.
### 6.3 Policy Engine integration
* [ ] Add endpoint to fetch explain traces for specific node ids in batch.
* [ ] Add materialization export that Graph API can cache per policy version.
### 6.4 Console (Web UI)
* [ ] Create `graph-explorer` module with routes `/graph`, `/graph/diff`, `/graph/q/:id`.
* [ ] Implement WebGL canvas with LOD, culling, edge bundling, deterministic layout seed.
* [ ] Build search, filter, lens, and overlay toolbars.
* [ ] Side panels: details, evidence tabs, explain viewer.
* [ ] Diff mode: split/overlay toggles and color legend.
* [ ] Saved queries: create, update, run; deep links.
* [ ] Export UI: formats, server round‑trip, progress indicators.
* [ ] a11y audit and keyboard‑only flow.
### 6.5 CLI
* [ ] Implement `stella sbom graph *` subcommands with JSON IO and piping support.
* [ ] Document examples and stable output schemas for CI consumption.
### 6.6 Observability & Ops
* [ ] Dashboards for tile latency, query denials, cache hit rate, memory.
* [ ] Alerting on query error spikes, OOM risk, cache churn.
* [ ] Runbooks in `/docs/runbooks/graph-incidents.md`.
### 6.7 Docs
* [ ] Author all docs in section 5, link from Console contextual help.
* [ ] Add end‑to‑end tutorial: “Investigate GHSA‑XXXX across prod artifacts.”
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Acceptance criteria
* Console renders large SBOM graphs with semantic zoom, overlays, and responsive interactions.
* Users can run impact and path queries with bounded depth and get results within budget.
* VEX suppressions and advisory severities appear correctly and are consistent with policy.
* Diff view clearly shows added/removed/changed nodes/edges between two SBOMs.
* Saved queries and deep links reproduce the same view deterministically (given same data).
* Exports produce valid GraphML/CSV/NDJSON and image snapshots.
* CLI supports search, query, paths, impacted, diff, and export with stable schemas.
* AOC guardrails: explorer never mutates facts; overlays reflect Policy Engine decisions.
* RBAC enforced; all actions logged and observable.
---
## 8) Risks & mitigations
* **Graph explosion on large monorepos** → tiling, clustering, budgets, and strict depth limits.
* **Inconsistent identities across tools** → canonicalize purls/digests; property‑based tests for identity stability.
* **Policy overlay latency** → precompute materializations for hot policy versions; sample explains only on focus.
* **User confusion** → strong lens defaults, deterministic layouts, legends, in‑context help.
---
## 9) Test plan
* **Unit**: node/edge builders, identity normalization, cost estimator.
* **Integration**: ingest SBOM + advisories + VEX, verify overlays and counts.
* **E2E**: Playwright flows for search→impact→diff→export; deep link determinism.
* **Performance**: simulate 500k nodes/2M edges; measure tile latency and memory.
* **Security**: RBAC matrix; tenant isolation tests; query validation fuzzing.
* **Determinism**: snapshot round‑trip: same query and seed produce identical layout and stats.
---
## 10) Feature flags
* `graph.explorer` (UI feature module)
* `graph.paths` (advanced path queries)
* `graph.diff` (SBOM diff mode)
* `graph.overlays.policy` (policy overlay + explain sampling)
* `graph.export` (exports enabled)
Documented in `/docs/observability/graph-telemetry.md`.
---
## 11) Non‑goals (this epic)
* Real‑time process/runtime call graphs.
* Full substitution for text reports; Explorer complements Reports.
* Cross‑tenant graphs; all queries are tenant‑scoped.
---
## 12) Philosophy
* **See the system**: security and license risk are structural. If you cannot see structure, you will miss risk.
* **Evidence over assertion**: every colored node corresponds to raw facts and explainable determinations.
* **Bounded interactivity**: fast, partial answers beat slow “complete” ones.
* **Immutability**: graphs mirror SBOM snapshots and are never rewritten; we add context, not edits.
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**

650
EPIC_6.md Normal file
View File

@@ -0,0 +1,650 @@
Below is the expanded, “maximum documentation” package for Epic 6. It is paste‑ready for your repo and deliberately formal so engineering, docs, and audit folks can work from the same source of truth.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
# Epic 6: Vulnerability Explorer (policy‑aware)
**Short name:** `Vuln Explorer`
**Services touched:** Conseiller (Feedser), Excitator (Vexer), SBOM Service, Policy Engine, Findings Ledger (new), Web API Gateway, Authority (authN/Z), Workers/Scheduler, Telemetry/Analytics, Console (Web UI), CLI
**AOC ground rule:** Conseiller/Excitator aggregate but never merge or rewrite source documents. The Explorer only renders **effective** results as decided by Policy Engine and records human workflow as immutable ledger events.
---
## 1) What it is
The Vulnerability Explorer is the API and UI for operational triage, investigation, and reporting of vulnerabilities across all artifacts tracked by StellaOps. It correlates SBOM inventory with advisory and VEX evidence, then displays the **effective** status and obligations per the currently selected **Policy version**. It never edits source evidence. It provides:
* Policy‑aware lists, pivots, and detail views
* Triage workflow with immutable audit ledger
* Risk acceptance with expiry and evidence
* SLA tracking by severity and business tier
* Simulation against other policy versions
* Exports and cryptographically signed “evidence bundles”
**Identity principle:** one **Finding** per tuple `(artifact_id, purl, version, advisory_key)`, with links to every contributing AdvisoryEvidence and VexEvidence.
---
## 2) Why (concise)
* Prioritization must reflect **policy and VEX**, not raw feeds.
* Audit requires complete, reproducible lineage: what changed, why, and who decided.
* Operators need consistent APIs, CLI, and a UI that explain determinations, not just list CVEs.
---
## 3) How it should work (maximum detail)
### 3.1 Domain model and contracts
**Evidence (immutable)**
* `AdvisoryEvidence` (from Conseiller)
```json
{
"id": "adv_evd:...uuid...",
"tenant": "acme",
"source": "ghsa|nvd|vendor|ossindex|... ",
"source_id": "GHSA-xxxx",
"schema": "GHSA|CVE|CSAF-ADV",
"advisory_key": "CVE-2024-12345",
"affected": [{"ecosystem":"npm","purl":"pkg:npm/lodash","ranges":[{"type":"semver","events":[{"introduced":"0.0.0"},{"fixed":"4.17.21"}]}]}],
"cvss": {"version":"3.1","baseScore":7.5,"vectorString":"AV:N/AC:L/..."},
"severity": "HIGH",
"urls": ["https://..."],
"published": "2024-06-10T12:00:00Z",
"withdrawn": null,
"ingested_at": "2024-06-11T08:43:21Z"
}
```
* `VexEvidence` (from Excitator)
```json
{
"id": "vex_evd:...uuid...",
"tenant": "acme",
"source": "vendor-csaf",
"schema": "CSAF-VEX",
"advisory_key": "CVE-2024-12345",
"product_scope": [{"purl":"pkg:npm/lodash@4.17.20"}],
"status": "not_affected|affected|fixed|under_investigation",
"justification": "component_not_present|vulnerable_code_not_in_execute_path|... ",
"impact_statement": "Not reachable in Acme Payment Service.",
"timestamp": "2024-06-12T10:00:00Z",
"ingested_at": "2024-06-12T10:10:02Z"
}
```
* `InventoryEvidence` (from SBOM Service)
```json
{
"id": "inv_evd:...uuid...",
"tenant": "acme",
"artifact_id": "svc:payments@1.14.0",
"sbom_id": "sbom:sha256:...",
"purl": "pkg:npm/lodash@4.17.20",
"scope": "runtime|dev|test",
"runtime_flag": true,
"paths": [["root", "pkg:npm/a", "pkg:npm/lodash"]],
"discovered_at": "2024-06-12T11:00:00Z"
}
```
**PolicyDetermination** (read‑only from Policy Engine)
```json
{
"key": {
"artifact_id": "svc:payments@1.14.0",
"purl": "pkg:npm/lodash",
"version": "4.17.20",
"advisory_key": "CVE-2024-12345",
"policy_version": "1.3.0"
},
"applicable": true,
"effective_severity": "HIGH",
"exploitability": "ACTIVE|LIKELY|UNKNOWN|UNLIKELY",
"signals": {"epss": 0.86, "kev": true, "maturity": "weaponized"},
"suppression_state": "none|policy|vex",
"obligations": [{"type":"fix_by","due":"2024-07-15"},{"type":"document_risk"}],
"sla": {"due":"2024-07-15","tier":"gold"},
"rationale": [
{"rule":"sev.base=nvd","detail":"NVD base:7.5"},
{"rule":"exploit.upsell.kev","detail":"KEV flag → HIGH"},
{"rule":"env.weight.prod","detail":"Env=prod no downgrade"}
]
}
```
**Finding identity**
```
finding_id = hash(tenant, artifact_id, purl, version, advisory_key)
```
**Ledger event** (append‑only, tamper‑evident)
```json
{
"event_id": "led:...uuid...",
"finding_id": "f:...hash...",
"tenant": "acme",
"type": "assign|comment|attach|change_state|accept_risk|set_target_fix|verify_fix|reopen",
"payload": {"to":"team:platform","reason":"oncall triage"},
"actor": {"user_id":"u:42","display":"Dana S."},
"ts": "2024-06-12T14:01:02Z",
"prev_event_hash": "sha256:...",
"event_hash": "sha256:sha256(canonical_json(event) + prev_event_hash)"
}
```
**Projection** (materialized current state for fast lists)
```json
{
"finding_id":"f:...hash...",
"tenant":"acme",
"artifact_id":"svc:payments@1.14.0",
"purl":"pkg:npm/lodash",
"version":"4.17.20",
"advisory_key":"CVE-2024-12345",
"effective_severity":"HIGH",
"exploitability":"ACTIVE",
"suppression_state":"none",
"status":"UNDER_INVESTIGATION|REMEDIATING|ACCEPTED|RESOLVED|NEW|SUPPRESSED",
"sla_due":"2024-07-15",
"owner":"team:platform",
"kev":true,
"epss":0.86,
"new_since":"2024-06-12",
"has_fix":true,
"envs":["prod"],
"runtime_flag":true,
"updated_at":"2024-06-12T14:01:02Z"
}
```
**Advisory key normalization**
* Input identifiers: `CVE-*`, `GHSA-*`, vendor IDs.
* Preference order: CVE, then GHSA, else vendor id prefixed with namespace.
* Canonicalization: uppercase, trim, map withdrawn to same key but mark `withdrawn=true` in evidence.
* Conseiller must publish `links: [all source ids]` for provenance.
### 3.2 Resolver algorithm (candidate findings)
**Goal:** produce tuples `(artifact_id, purl, version, advisory_key)` where inventory intersects affected ranges and policy deems the path relevant.
**Pseudocode**
```
for each artifact sbom S:
inv = inventory(S)
for each advisory evidence A:
for each affected package spec in A.affected:
for each inv_item in inv where inv_item.purl package == spec.purl package:
if version_in_ranges(inv_item.version, spec.ranges, ecosystem):
if policy.path_scope_allows(inv_item.scope, inv_item.runtime_flag, inv_item.paths):
yield candidate (artifact_id, inv_item.purl, inv_item.version, A.advisory_key)
```
**Version semantics per ecosystem**
* npm: semver, pre-release excluded unless explicitly in range.
* Maven: Maven version rules, handle `-SNAPSHOT`, use maven‑resolver semantics.
* PyPI: PEP 440 versioning.
* Go: semver with `+incompatible` handling.
* OS packages: RPM/DEB epoch:version‑release ordering.
**Edge cases**
* Multiple paths: store shortest path and count.
* Dev/test scope: policy may exclude or downgrade.
* Withdrawn advisories: keep as evidence; Policy can set severity to `NONE`.
### 3.3 VEX precedence and scoping
* If any matching **VEX** says `not_affected` scoped to the artifact product/component per CSAF product tree, set `suppression_state="vex"` and `applicable=false`.
* If VEX says `fixed` and inventory version is >= fixed version, mark as **Resolved (verified)** after SBOM recrawl confirms.
* If VEX `under_investigation`, no suppression; may add a policy grace period obligation.
### 3.4 Policy evaluation
* Inputs: candidate tuple + context (artifact env, business tier, ownership, signals, fix availability).
* Determinations: applicability, effective severity, exploitability, obligations, SLA, suppression.
* Suppression by policy examples: test‑scope only; path through optional deps; package vendored but not linked at runtime.
* Simulation: identical input, alternate `policy_version`; returns determinations without side effects.
### 3.5 API surface (authoritative)
**List**
```
GET /vuln/findings?policy=1.3.0&sev=high,critical&group_by=artifact&exploit=kev&env=prod&page=1&page_size=100
```
Response
```json
{
"page": 1,
"page_size": 100,
"total": 740,
"group_by": "artifact",
"results": [
{"group":"svc:payments","counts":{"CRITICAL":3,"HIGH":12,"MEDIUM":8},"sla_breaches":2},
...
]
}
```
**Query (complex filters)**
```
POST /vuln/findings/query
```
```json
{
"policy": "1.3.0",
"filter": {
"severity": [">=MEDIUM"],
"exploit": ["kev", "epss>=0.8"],
"artifact": ["svc:payments", "svc:checkouts"],
"status": ["NEW","UNDER_INVESTIGATION"],
"env": ["prod"]
},
"sort": [{"field":"effective_severity","dir":"desc"},{"field":"epss","dir":"desc"}],
"page": {"number":1,"size":200}
}
```
**Detail**
```
GET /vuln/findings/{finding_id}?policy=1.3.0
```
Returns projection, evidence links, policy rationale, paths, history summary.
**Workflow**
```
POST /vuln/findings/{id}/assign { "to": "team:platform" }
POST /vuln/findings/{id}/comment { "text": "triage notes..." }
POST /vuln/findings/{id}/accept-risk { "until":"2025-06-30","reason":"vendor patch pending","evidence":["url|upload_id"] }
POST /vuln/findings/{id}/verify-fix { "sbom_id": "sbom:sha256:..." }
POST /vuln/findings/{id}/target-fix { "version": "4.17.21" }
```
**Simulation**
```
POST /vuln/simulate
{
"policy_from": "1.3.0",
"policy_to": "1.4.0",
"query": { "severity":[">=MEDIUM"], "env":["prod"] }
}
```
Response includes per‑finding delta `{before, after, diff}`.
**Export**
```
POST /vuln/export { "format":"ndjson","scope":{"query":{...}} }
```
Returns a signed bundle (see §3.10).
**Errors**
* `400` validation, `403` RBAC, `404` not found, `409` state conflict (idempotency), `429` rate limited, `5xx` server.
### 3.6 Console (Web UI)
**Routes**
* `/vuln` list with saved views
* `/vuln/:id` detail drawer state
* `/vuln/simulate/:policyVersion` diff mode
**State shape (client)**
```ts
interface VulnListState {
policyVersion: string;
filters: {...};
sort: [...];
columns: string[];
viewId?: string;
page: {number: number; size: number};
}
```
**UX**
* Virtualized grid with server paging; column chooser; density toggle.
* Quick filters: severity, exploit signals, status, env, owner, fix availability.
* Detail tabs: Summary, Evidence (raw docs with provenance), Policy (rationale chain), Paths (deep link to Graph Explorer), Fixes, History.
* Simulation bar shows delta chips: `+21 HIGH`, `-9 Suppressed by VEX` etc.
* Evidence bundle dialog previews scope and size.
* a11y: ARIA roles on grid, keyboard shortcuts: `A` assign, `C` comment, `R` accept risk, `V` verify fix.
### 3.7 CLI
Commands
```
stella vuln list --policy 1.3.0 --sev high,critical --group-by artifact --env prod --json
stella vuln show --id <finding-id> --policy 1.3.0
stella vuln simulate --from 1.3.0 --to 1.4.0 --sev '>=medium' --delta --json
stella vuln assign --filter 'advisory:CVE-2024-12345 artifact:payments' --to team:platform
stella vuln accept-risk --id <finding-id> --until 2025-06-30 --reason "vendor patch pending" --evidence url:https://ticket/123
stella vuln verify-fix --id <finding-id> --sbom <sbom-id>
```
Return codes: `0 ok`, `2 invalid args`, `3 budget exceeded`, `4 not found`, `5 denied`.
### 3.8 Storage schema (illustrative)
**Tables**
```sql
-- Evidence
CREATE TABLE evidence_advisory (...);
CREATE INDEX ea_tenant_key ON evidence_advisory(tenant, advisory_key);
CREATE TABLE evidence_vex (...);
CREATE INDEX ev_tenant_key ON evidence_vex(tenant, advisory_key);
CREATE TABLE evidence_inventory (...);
CREATE INDEX ei_artifact_purl ON evidence_inventory(tenant, artifact_id, purl);
-- Ledger
CREATE TABLE findings_ledger_events (
event_id uuid PRIMARY KEY,
finding_id bytea NOT NULL,
tenant text NOT NULL,
type text NOT NULL,
payload jsonb NOT NULL,
actor jsonb NOT NULL,
ts timestamptz NOT NULL,
prev_event_hash bytea,
event_hash bytea NOT NULL
);
CREATE INDEX fle_find_ts ON findings_ledger_events(tenant, finding_id, ts);
-- Projection
CREATE TABLE findings_projection (
finding_id bytea PRIMARY KEY,
tenant text NOT NULL,
artifact_id text NOT NULL,
purl text NOT NULL,
version text NOT NULL,
advisory_key text NOT NULL,
policy_version text NOT NULL,
effective_severity text NOT NULL,
exploitability text,
suppression_state text,
status text NOT NULL,
sla_due date,
owner text,
kev boolean,
epss double precision,
envs text[],
runtime_flag boolean,
updated_at timestamptz NOT NULL
);
CREATE INDEX fp_query ON findings_projection(tenant, policy_version, effective_severity, status);
```
**Tamper‑evidence**
* Ledger events use chained SHA‑256 hashes over canonical JSON + previous hash.
* Daily Merkle root of all event hashes is anchored to the audit store (and optionally external timestamping service).
### 3.9 Performance and scaling
* P95 list endpoint under 600 ms for 100‑row pages at 5M findings/tenant.
* Projections denormalize heavy joins; background projector uses idempotent jobs keyed by `(tenant,finding_id,policy_version)`.
* Rate limits per tenant and per API key; backpressure on export jobs; exponential retry for projector.
### 3.10 Evidence bundle format
* **Container:** ZIP with `manifest.json`, `findings.ndjson`, `advisory_evidence.ndjson`, `vex_evidence.ndjson`, `inventory_evidence.ndjson`, `policy_version.json`, `ledger_events.ndjson`, `CHECKSUMS`.
* **Signing:** Detached signature `bundle.sig` using tenant’s org key (Ed25519).
* **Manifest**
```json
{"generated_at":"2024-06-12T15:00:00Z","tenant":"acme","policy_version":"1.3.0","scope":{"query":{...}},"counts":{"findings":421}}
```
### 3.11 Observability
* Metrics (OpenTelemetry):
* `vuln_findings_list_latency_ms` (histogram)
* `vuln_projection_lag_seconds` (gauge)
* `vuln_new_findings_total` (counter)
* `vuln_sla_breaches_total` (counter by sev, owner)
* `vuln_simulation_latency_ms` (histogram)
* Logs: structured JSON with `tenant`, `policy_version`, `query_hash`, `result_count`.
* Traces: spans for resolver, policy calls, projection builds, export assembly.
* PII: redact comments in logs; store attachments encrypted at rest (KMS).
### 3.12 Security & RBAC
**Roles**
* Viewer: GET list/detail/export read scope.
* Investigator: Viewer + workflow actions except risk acceptance.
* Operator: Investigator + risk acceptance, verify fix, bulk actions.
* Auditor: Viewer + evidence bundles and ledger integrity checks.
**ABAC**
* Attribute constraints: by `artifact.owner`, `env`, and `business_tier`.
* CSRF protection for Console; all POST require anti‑forgery tokens.
* Attachments stored with envelope encryption; signed URLs for limited time access.
### 3.13 Rollout and migrations
* Feature flags: `vuln.explorer.ui`, `vuln.explorer.simulation`, `vuln.explorer.bulk_actions`, `vuln.explorer.evidence_bundle`.
* Phase 1: dark launch API and projections.
* Phase 2: UI read‑only list and detail.
* Phase 3: workflow actions and exports.
* Data backfill: replay advisory/VEX/SBOM events to seed projections.
* Compatibility: maintain projection v1 schema for two releases; migration scripts in `/migrations/vuln/`.
---
## 4) Implementation plan
### 4.1 Services
* **Findings Ledger (new)**
* Append‑only event store with projector to `findings_projection`.
* Event validation and canonicalization; hashing and Merkle root anchoring.
* **Vuln Explorer API (new)**
* Query/filter engine with policy parameterization and grouping.
* Simulation endpoint.
* Export job orchestrator.
* **Conseiller / Excitator (updates)**
* Guarantee canonical `advisory_key` and publish `links[]`.
* No merges; maintain raw payload snapshots.
* **Policy Engine (updates)**
* Batch evaluation endpoint `POST /policy/eval/batch` with `simulate` support.
* Return rationale chain with rule IDs.
* **SBOM Service (updates)**
* Publish inventory deltas; include `scope`, `runtime_flag`, `paths`.
* Nearest safe version hints.
* **Workers/Scheduler**
* Resolver job keyed by `(tenant, artifact_id, sbom_id)`; emits candidate tuples.
* Recompute on policy activation and evidence changes.
### 4.2 Code structure
```
/src/StellaOps.Findings.Ledger
/api
/projector
/storage
/src/StellaOps.VulnExplorer.Api
/routes
/query
/simulation
/export
/packages/console/features/vuln-explorer
/components
/pages
/state
/src/StellaOps.Cli
```
### 4.3 Performance tasks
* Projection indexes and covering queries; explain plans in `/docs/vuln/perf-notes.md`.
* Cache hot groupings per tenant with TTL and invalidation on ledger projector tick.
---
## 5) Documentation changes (create/update)
1. `/docs/vuln/explorer-overview.md`
Conceptual model, identities, evidence vs determinations, AOC guarantees.
2. `/docs/vuln/explorer-using-console.md`
Workflows with screenshots, keyboard shortcuts, saved views, deep links.
3. `/docs/vuln/explorer-api.md`
Endpoint specs, query language, grouping, pagination, errors, rate limits.
4. `/docs/vuln/explorer-cli.md`
Commands, flags, examples, exit codes.
5. `/docs/vuln/findings-ledger.md`
Event schema, state machine, hashing, Merkle roots, integrity checks.
6. `/docs/policy/vuln-determinations.md`
Inputs, outputs, precedence rules, simulation semantics.
7. `/docs/vex/explorer-integration.md`
CSAF mapping, scoping to product tree, precedence.
8. `/docs/advisories/explorer-integration.md`
Advisory key normalization, provenance, withdrawn handling.
9. `/docs/sbom/vuln-resolution.md`
Ecosystem version semantics, path sensitivity, scope rules.
10. `/docs/observability/vuln-telemetry.md`
Metrics, logs, traces, dashboards, SLOs.
11. `/docs/security/vuln-rbac.md`
Role mapping, ABAC, attachment encryption, CSRF.
12. `/docs/runbooks/vuln-ops.md`
Recompute storms, projector lag, policy activation drains, export failures.
13. `/docs/install/containers.md`
Add `findings-ledger`, `vuln-explorer-api` images, compose/k8s manifests, resource sizing, health checks.
> Each doc ends with: **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 6) Engineering tasks
### Backend: Findings & API
* [ ] Define evidence and ledger schemas; migrations.
* [ ] Implement resolver for npm, Maven, PyPI, Go, OS packages with property‑based tests for version comparisons.
* [ ] Implement ledger API and projector with idempotency and hashing.
* [ ] Implement list/detail/grouping endpoints with server‑side paging.
* [ ] Implement simulation and export (bundle assembly, signing).
* [ ] Integrate Policy Engine batch eval with rationale traces.
* [ ] RBAC via Authority with ABAC filters.
* [ ] Load tests at 5M findings/tenant; tune indexes.
### Conseiller/Excitator
* [ ] Normalize `advisory_key` and persist `links[]`.
* [ ] Ensure raw payload snapshots are retrievable by Explorer for Evidence tab.
### SBOM Service
* [ ] Emit `scope`, `runtime_flag`, `paths`; safe version hints.
* [ ] Inventory delta events to trigger resolver.
### Console
* [ ] Build grid with virtualization, saved views, deep link serializer.
* [ ] Implement detail tabs and path deep‑links to Graph Explorer.
* [ ] Add simulation bar and delta chips.
* [ ] Evidence bundle dialog.
* [ ] a11y keyboard flow and ARIA labeling; unit and E2E tests.
### CLI
* [ ] `stella vuln list|show|simulate|assign|accept-risk|verify-fix` with `--json` and CSV export.
* [ ] Stable output schemas; pipe‑friendly defaults.
### Observability/Ops
* [ ] Dashboards for list latency, projection lag, new/reopened, SLA breaches.
* [ ] Alerts on projector backlog, API 5xx spikes, export failures.
* [ ] Runbooks in `/docs/runbooks/vuln-ops.md`.
### Docs
* [ ] Author files listed in §5 with cross‑links to Policy Studio and SBOM Graph Explorer.
* [ ] Update `/docs/install/containers.md` with new images and compose/k8s snippets.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Acceptance criteria
* List and detail views reflect **effective** policy outcomes and update instantly when switching policy versions.
* Evidence tab shows all raw advisory/VEX documents with provenance; no source merging.
* Resolver respects ecosystem semantics, scope, and paths; path tab round‑trips to Graph Explorer.
* Ledger events are immutable and reconstruct historical list states accurately.
* Simulation returns diffs without side effects and matches Policy Engine outputs.
* CLI/API support paging, grouping, export, simulation; contracts stable and documented.
* RBAC and tenant isolation validated by tests; attachments encrypted.
* P95 performance budgets met; dashboards green for SLOs.
---
## 8) Risks and mitigations
* **Advisory identity collisions** → strict canonicalization; preserve `links[]`; never merge raw docs.
* **Projection lag** → backpressure, worker autoscaling, health checks; alerting on lag.
* **Resolver false positives** → path evidence required; dev/test scope rules explicit; ecosystem‑specific tests.
* **User confusion over suppression** → explicit badges; Policy tab with rationale and “why changed.”
* **Export size** → NDJSON streaming, size estimator in UI, scope previews.
---
## 9) Test plan
* **Unit:** version comparators, resolver per ecosystem, policy mapping, ledger state machine.
* **Integration:** SBOM + advisories + VEX ingestion, candidate generation, policy application, suppression precedence.
* **E2E Console:** triage, bulk assign, simulation, evidence bundle download; keyboard‑only flow.
* **Performance:** list/grouping at target scale; projector rebuild; export assembly.
* **Security:** RBAC matrix, ABAC filters, CSRF, signed URL lifetimes, tamper‑evidence verification.
* **Determinism:** time‑travel snapshots reproduce prior states byte‑for‑byte.
---
## 10) Philosophy
* **Facts first, decisions second.** Evidence is immutable; decisions and workflow sit on top in a ledger.
* **Policy is the lens.** The same facts can imply different obligations; the system must make that explicit and reproducible.
* **Audit > convenience.** Every state change is justified, signed, and verifiable.
* **No hidden magic.** If anything is suppressed, the UI shows the rule or VEX that did it, with documents attached.
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**

545
EPIC_7.md Normal file
View File

@@ -0,0 +1,545 @@
Below is the expanded, “maximum documentation” package for Epic 7. It is paste‑ready for your repo and deliberately formal so engineering, docs, governance, and audit can work from the same source of truth.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
# Epic 7: VEX Consensus Lens
**Short name:** `VEX Lens`
**Services touched:** Excitator (Vexer), Conseiller (Feedser), SBOM Service, Policy Engine, Findings Ledger, Web API Gateway, Authority (authN/Z), Console (Web UI), CLI, Telemetry/Analytics
**AOC ground rule:** Excitator and Conseiller aggregate but never merge or rewrite source documents. The Lens only computes a **derived, reproducible** consensus view while preserving all raw evidence and provenance.
---
## 1) What it is
The VEX Consensus Lens is a deterministic computation and presentation layer that ingests all available VEX statements about a given advisory key (e.g., CVE) and product/context, then produces an **explicit consensus state** for each `(artifact_id, purl, version, advisory_key)` tuple. It answers: “Given multiple, possibly conflicting VEX statements from different issuers, what is the most trustworthy, policy‑consistent interpretation for this artifact and version?”
It never edits or merges VEX documents. It normalizes them, aligns their **product scope** to SBOM inventory, scores issuers via a tenant‑configurable **trust model**, applies time and version scoping, and outputs:
* `consensus_state`: `NOT_AFFECTED | AFFECTED | FIXED | UNDER_INVESTIGATION | INCONCLUSIVE | DISPUTED`
* `confidence`: 0.0–1.0 numeric score
* `rationale`: structured explanation of evidence, weights, and rules applied
* `quorum`: weighted vote breakdown by issuer, timestamp, justification, and version applicability
The Lens plugs into the Policy Engine. Policy remains the source of truth for applicability, severity, and obligations; the Lens supplies structured signals and provenance that policy can use to suppress findings, downgrade risk, or set SLAs.
Key properties:
* **Reproducible:** Same inputs and policy yield same outputs.
* **Explainable:** Every output carries a machine‑readable rationale chain.
* **Scoped:** Product trees and version ranges are resolved against actual SBOM inventory.
* **Immutable evidence:** All raw VEX docs remain intact, retrievable, and linkable.
---
## 2) Why (concise)
Real ecosystems have overlapping VEX statements from upstream maintainers, vendors, distros, and third parties. Operators need a single, defensible view without losing provenance. The Lens reduces noise, flags conflicts, and turns fragmented VEX evidence into auditable signals that policy and humans can act on.
---
## 3) How it should work (maximum detail)
### 3.1 Input model
**From Excitator (Vexer):**
* Raw `VexEvidence` documents (CSAF VEX, OpenVEX, CycloneDX VEX) with:
* `advisory_key` (canonicalized)
* product tree or component coordinates (purl, CPE, vendor IDs)
* status (`not_affected`, `affected`, `fixed`, `under_investigation`)
* justifications (CSAF, OpenVEX enumerations)
* version ranges or fixed versions if available
* timestamps (issued, last_updated), source URLs
* cryptographic metadata (signature, issuer, certificate chain) if present
**From SBOM Service:**
* Inventory for each artifact: purls, versions, dependency paths, scopes, env flags.
**From Issuer Directory (new)**
* Directory of known VEX issuers:
* identity, organization, domain, CSAF publisher metadata
* public keys and trust anchors (Ed25519/X.509/PKIX/PKCS7/DSSE)
* default trust weight
* tenancy‑specific overrides
**From Policy Engine:**
* Policy version and options relevant to VEX evaluation:
* trust model parameters
* confidence thresholds per environment/tier
* justification whitelist/blacklist
* recency requirements and expiry windows
* precedence rules for status conflicts
### 3.2 Normalization
1. **Canonical advisory key:** CVE preferred; GHSA mapped; vendor IDs namespaced.
2. **Status mapping:** Normalize all encodings to `NOT_AFFECTED | AFFECTED | FIXED | UNDER_INVESTIGATION`.
3. **Product scope alignment:** Convert CSAF product tree/CPE to purl variants using deterministic mappings; store mapping evidence.
4. **Version scoping:** For each evidence, compute `applies_to(version)` using ecosystem comparators (npm semver, Maven, PEP 440, Go semver, RPM/DEB EVR).
5. **Signature verification:** If signed, verify against Issuer Directory trust anchors; attach `sig_verified=true/false` and chain metadata.
6. **Temporal fields:** Compute `effective_at` (issued), `observed_at` (ingested), and staleness.
### 3.3 Trust model
Each `VexEvidence` gets a base weight `w_base` from issuer type and verification:
* Maintainer/Upstream signed: 0.9
* Maintainer/Upstream unsigned: 0.7
* Distro/SIG signed: 0.8
* Vendor of downstream product signed: 0.8
* Third‑party scanner VEX: 0.4
* Unknown/unsigned with weak provenance: 0.2
Weights are then adjusted:
```
w = w_base
* f_signature(sig_verified) // e.g., 1.1 if verified, 0.8 if unverifiable
* f_recency(age_days) // decay after T days (policy)
* f_justification(type) // e.g., "component_not_present" lower if SBOM shows present
* f_scope_match(score) // quality of product match: exact purl > family > CPE wildcard
* f_env(app_env) // optional env-specific multipliers
```
All `f_*` are bounded to [0.5, 1.2] by default to prevent runaway effects. Tenants can override per Policy Studio.
### 3.4 Consensus algorithm
For a tuple `(artifact_id, purl, version, advisory_key)`:
1. **Collect** all normalized `VexEvidence` whose product scope maps to `purl` and `applies_to(version)==true`.
2. **Bucket** by normalized status; compute `W(status) = Σ w(evidence)` per status.
3. **Apply precedence** rules from policy:
* If `W(NOT_AFFECTED)` exceeds threshold `T_na` and there is no `FIXED` evidence contradictory for the same version, propose `NOT_AFFECTED`.
* If any `FIXED` applies and inventory version ≥ fixed version, propose `FIXED`.
* If `W(AFFECTED)` ≥ `T_aff` and no dominating `NOT_AFFECTED`, propose `AFFECTED`.
* If `W(UNDER_INVESTIGATION)` dominates and others below thresholds, propose `UNDER_INVESTIGATION`.
* If both `AFFECTED` and `NOT_AFFECTED` exceed thresholds within a small margin `δ`, mark `DISPUTED`.
* Otherwise `INCONCLUSIVE`.
4. **Confidence score:**
```
confidence = W(winning_status) / (W(AFFECTED)+W(NOT_AFFECTED)+W(FIXED)+W(UNDER_INVESTIGATION) + ε)
```
Clip to [0.0, 1.0].
5. **Rationale chain:**
* Winning status, thresholds used, top 5 contributing evidences with weights and reasons, product mapping quality, version scoping evidence, and policy knobs that influenced the result.
6. **Quorum summary:**
* List issuers and their votes, signature state, timestamps, and justifications.
### 3.5 Policy interaction
* The Lens returns `consensus_state`, `confidence`, and structured `signals`.
* Policy Engine consumes these and may:
* Suppress findings automatically on `NOT_AFFECTED` confidence ≥ `P_na`.
* Downgrade severity or extend SLA when `UNDER_INVESTIGATION` from high‑trust issuers.
* Require human approval when `DISPUTED`.
* Treat `FIXED` as resolved only when SBOM crawl verifies the fixed version or a `verify_fix` ledger event exists.
**Simulation:** Policy Studio can simulate different trust weights and thresholds and see consensus deltas without side effects.
### 3.6 Data contracts
**ConsensusRecord (materialized)**
```json
{
"id": "cons:sha256(tenant|artifact|purl|version|advisory|policy)",
"tenant": "acme",
"artifact_id": "svc:payments@1.14.0",
"purl": "pkg:npm/lodash@4.17.20",
"advisory_key": "CVE-2024-12345",
"policy_version": "1.3.0",
"consensus_state": "NOT_AFFECTED",
"confidence": 0.87,
"weights": {"AFFECTED":0.31,"NOT_AFFECTED":1.25,"FIXED":0.00,"UNDER_INVESTIGATION":0.12},
"top_evidence": ["vex_evd:...","vex_evd:..."],
"quorum": [
{"issuer":"lodash-maintainers","status":"NOT_AFFECTED","w":0.9,"sig_verified":true,"just":"vulnerable_code_not_present","issued":"2024-06-08"},
{"issuer":"vendorX-distro","status":"AFFECTED","w":0.25,"sig_verified":false,"just":"generic_advisory","issued":"2024-06-07"}
],
"rationale": [
{"rule":"trust.weight.issuer","detail":"maintainer signed evidence 0.9"},
{"rule":"scope.match.exact","detail":"exact purl match"},
{"rule":"justification.vcnp","detail":"supported by SBOM callgraph hint"}
],
"updated_at": "2024-06-12T10:00:00Z"
}
```
**Issuer Directory entry**
```json
{
"issuer_id": "iss:lodash",
"org": "Lodash Maintainers",
"domains": ["lodash.com"],
"keys": [{"type":"ed25519","pub":"...","expires":"2026-12-31"}],
"default_weight": 0.9,
"metadata": {"csaf_publisher": true}
}
```
### 3.7 APIs
**Compute/Query**
```
GET /vex/consensus?artifact=svc:payments@1.14.0&purl=pkg:npm/lodash@4.17.20&advisory=CVE-2024-12345&policy=1.3.0
POST /vex/consensus/query { "policy":"1.3.0", "filter": { "state":["DISPUTED","INCONCLUSIVE"], "confidence":"<0.6", "env":["prod"] }, "page":{...} }
GET /vex/consensus/{id} // full record
POST /vex/consensus/simulate // override trust knobs and thresholds for what-if
```
**Issuer Directory**
```
GET /vex/issuers
POST /vex/issuers (admin)
POST /vex/issuers/{id}/keys (admin)
```
**Exports**
```
POST /vex/consensus/export { "format":"ndjson","scope":{"filter":{...}} }
```
**Errors**
* `400` invalid mapping, `403` RBAC, `404` not found, `409` conflict, `429` rate limit.
### 3.8 Console (Web UI)
Routes:
* `/vex/consensus` overview with filters: state, confidence, issuer, advisory, artifact, env.
* `/vex/consensus/:id` detail: Evidence pane, Quorum graph, Policy impact, SBOM path links.
UX elements:
* **Quorum bar:** stacked bar showing weights per status; hover reveals issuer contributions.
* **Confidence chip:** numeric and qualitative band (Low/Med/High).
* **Evidence table:** paged list of VEX docs with signature icon, scope match quality tag, justification tag, issued/updated timestamps.
* **Conflict view:** for `DISPUTED`, show side-by-side issuer rationales and suggested next steps.
* **Deep link** into Vulnerability Explorer detail, preselecting the Policy version used for the consensus.
A11y:
* ARIA roles on grid and bars; keyboard shortcuts `S` switch policy, `T` trust presets, `E` export.
### 3.9 CLI
Commands:
```
stella vex consensus list --policy 1.3.0 --state disputed,inconclusive --confidence '<0.6' --artifact payments --json
stella vex consensus show --id <cons-id> --policy 1.3.0
stella vex simulate --policy 1.3.0 --trust 'issuer:lodash=1.0,issuer:vendorX=0.5' --thresholds 'na=1.0,aff=0.6' --json
stella vex issuers list
stella vex export --filter 'artifact:payments advisory:CVE-2024-12345' --out vex-consensus.ndjson
```
Exit codes: `0` ok, `2` invalid args, `4` not found, `5` denied.
### 3.10 Storage schema (illustrative)
```sql
-- Normalized VEX (reference Excitator id; do not alter raw)
CREATE TABLE vex_normalized (
id uuid PRIMARY KEY,
tenant text NOT NULL,
evidence_id text NOT NULL, -- link to Excitator
advisory_key text NOT NULL,
issuer_id text,
status text NOT NULL, -- NOT_AFFECTED|AFFECTED|FIXED|UNDER_INVESTIGATION
justification text,
purl text, -- normalized mapping target
version_range jsonb, -- ecosystem-specific encoding
sig_verified boolean,
scope_score real, -- 0..1 quality of mapping
issued timestamptz,
updated timestamptz,
w_base real,
UNIQUE (tenant, evidence_id)
);
-- Issuer Directory
CREATE TABLE vex_issuers (
issuer_id text PRIMARY KEY,
tenant text NOT NULL,
org text NOT NULL,
default_weight real NOT NULL,
metadata jsonb,
UNIQUE(tenant, org)
);
CREATE TABLE vex_issuer_keys (
id uuid PRIMARY KEY,
issuer_id text NOT NULL REFERENCES vex_issuers(issuer_id),
key_type text NOT NULL,
pubkey text NOT NULL,
expires timestamptz
);
-- Consensus projection
CREATE TABLE vex_consensus (
id bytea PRIMARY KEY,
tenant text NOT NULL,
artifact_id text NOT NULL,
purl text NOT NULL,
version text NOT NULL,
advisory_key text NOT NULL,
policy_version text NOT NULL,
consensus_state text NOT NULL,
confidence real NOT NULL,
weights jsonb NOT NULL,
top_evidence text[] NOT NULL,
updated_at timestamptz NOT NULL
);
CREATE INDEX vc_query ON vex_consensus(tenant, policy_version, consensus_state, confidence);
```
### 3.11 Integration with Findings Ledger and Vuln Explorer
* The Vuln Explorer reads `vex_consensus` for each finding and renders:
* Consensus chip, confidence, and a link to full quorum.
* For `NOT_AFFECTED` with confidence ≥ policy threshold, show “Suppressed by VEX (Consensus)” badge.
* For `DISPUTED`, open a triage banner prompting manual review and optional ledger comment/assignment.
* Ledger receives no new event type from lens computation itself. Human actions triggered by lens views produce standard events (`comment`, `assign`, `change_state`).
### 3.12 Security & RBAC
Roles:
* Viewer: query consensus read‑only.
* Investigator: Viewer + export.
* Operator: Investigator + trust simulation.
* Admin: manage Issuer Directory entries and keys.
CSRF for Console; ABAC scoping by artifact ownership and environment.
### 3.13 Observability
Metrics:
* `vex_consensus_compute_latency_ms` (histogram)
* `vex_consensus_records_total` (counter)
* `vex_consensus_disputed_total` (counter by issuer combinations)
* `vex_consensus_staleness_seconds` (gauge)
* `vex_signature_verification_rate` (gauge)
Logs: structured events with `tenant`, `policy_version`, `advisory_key`, `quorum_summary`.
Traces: spans for normalization, mapping, trust weighting, consensus decision, DB writes.
### 3.14 Performance & scaling
Targets:
* P95 query under 500 ms for 100‑row pages at 10M consensus records/tenant.
* Projection jobs are idempotent and keyed by `(tenant, artifact, purl, version, advisory, policy)`; backpressure with work queues.
* Cache popular queries with tenant‑scoped TTL; invalidate on Excitator or policy changes.
### 3.15 Edge cases
* **Ambiguous product mapping:** mark low `scope_score`, cap weight, surface warning in UI.
* **VEX “not present” vs SBOM shows present:** down‑weight with `f_justification`, require manual check.
* **Withdrawn or superseded VEX:** decay to near zero; keep provenance.
* **Partial fixes:** if fixed version applies to subset of platforms, map to env or arch dimension when available.
* **Time travel:** consensus recalculated as of a timestamp using only evidence ≤ `as_of` and the corresponding policy version.
---
## 4) Implementation plan
### 4.1 Services
* **VEX Lens Service (new)**
* Normalization pipeline, trust weighting, consensus computation, and projections.
* Batch recompute on policy activation and Excitator deltas.
* **Excitator (updates)**
* Ensure all VEX evidence carries issuer hints and raw signature blobs when present.
* Publish product trees and original coordinates intact.
* **Policy Engine (updates)**
* Add VEX trust knobs, thresholds, recency decay, and status precedence.
* Batch eval endpoint accepts `consensus inputs` where needed.
* **Issuer Directory (new)**
* Manage issuer metadata and keys; tenant overrides; audit logs.
### 4.2 Code structure
```
/src/StellaOps.VexLens
/normalizer
/mapping # CPE/purl translators
/trust # weighting functions
/consensus # algorithm and projections
/api
/src/StellaOps.Excititor # updates
/src/StellaOps.Policy # updates
/src/StellaOps.IssuerDirectory
/packages/console/features/vex-consensus
/src/StellaOps.Cli
```
### 4.3 Rollout
* Phase 1: API read‑only with basic trust model, Console list/detail, no simulation.
* Phase 2: Policy Studio integrations and simulation.
* Phase 3: Issuer Directory admin flows, exports, and advanced mapping diagnostics.
---
## 5) Documentation changes (create/update)
1. `/docs/vex/consensus-overview.md`
Purpose, scope, terminology, evidence vs derived view, AOC guarantees.
2. `/docs/vex/consensus-algorithm.md`
Normalization, mapping, weighting, thresholds, precedence, formulas, examples.
3. `/docs/vex/issuer-directory.md`
Managing issuers, keys, trust overrides, security model.
4. `/docs/vex/consensus-api.md`
Endpoints, request/response schemas, errors, pagination, rate limits.
5. `/docs/vex/consensus-console.md`
Screens, filters, conflict workflows, a11y, deep links.
6. `/docs/policy/vex-trust-model.md`
Policy knobs, thresholds, decay, simulation.
7. `/docs/sbom/vex-mapping.md`
Product tree mapping to purl/version, ecosystem comparators, edge cases.
8. `/docs/security/vex-signatures.md`
Signature verification flows, key management, auditing.
9. `/docs/runbooks/vex-ops.md`
Recompute storms, mapping failures, signature errors, lag, quotas.
All docs end with the imposed rule statement.
---
## 6) Engineering tasks
### Backend core
* [ ] Implement normalization for CSAF VEX, OpenVEX, CycloneDX VEX.
* [ ] Build product mapping library (CPE→purl, vendor tokens→purl families).
* [ ] Implement signature verification (Ed25519/PKIX/DSSE) using Issuer Directory keys.
* [ ] Implement trust weighting functions and configurable parameters.
* [ ] Implement consensus algorithm with unit tests and property tests.
* [ ] Materialize `vex_consensus` projection with indexes and idempotent workers.
* [ ] Batch recompute on policy activation and Excitator deltas.
### APIs & Integrations
* [ ] `/vex/consensus` query, detail, simulate, export.
* [ ] Policy Engine: consume consensus signals; add thresholds and precedence.
* [ ] Vuln Explorer: show consensus chip and triage banners; deep link to Lens.
### Issuer Directory
* [ ] CRUD for issuers and keys, audit logs, RBAC.
* [ ] Import common CSAF publishers; seed with sane defaults.
### Console
* [ ] Build list grid with filters and saved views.
* [ ] Quorum bar and Evidence table with signature icons and scope quality tags.
* [ ] Conflict view for `DISPUTED`.
* [ ] Simulation drawer integrated with Policy Studio.
### CLI
* [ ] `stella vex consensus list|show|simulate|export` with JSON/CSV.
* [ ] Stable schemas; tests for piping and scripting.
### Observability/Perf
* [ ] Metrics, logs, traces as specified; dashboards.
* [ ] Load tests at 10M consensus records/tenant; optimize indexes and caches.
### Docs
* [ ] Author and cross‑link all docs listed in §5.
* [ ] Add examples and screenshots to Console doc.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Acceptance criteria
* Normalization supports CSAF VEX, OpenVEX, CycloneDX VEX with product tree mapping to purls across npm, Maven, PyPI, Go, RPM/DEB.
* Signature verification works and affects weights; unverifiable signatures do not crash flows.
* Consensus outputs are reproducible, explainable, and queryable at scale.
* Vuln Explorer displays consensus state and affects policy outcomes per thresholds.
* Simulation reflects policy trust changes without side effects and returns rationale deltas.
* CLI/API feature parity; evidence and quorum are exportable.
* P95 performance budgets met; dashboards reflect health.
---
## 8) Risks and mitigations
* **Mapping errors (CPE→purl):** use conservative scope scores, cap weights, surface warnings, manual override hooks in Policy Studio.
* **Malicious or mistaken issuer:** signature verification plus trust weighting and tenancy overrides.
* **Evidence storms:** debounce recompute; batch and shard workers; backpressure and prioritization.
* **User confusion with conflicting VEX:** clear conflict UI, rationale chains, suggested actions, and policy banners.
* **Stale statements:** recency decay and expiry windows in policy.
---
## 9) Test plan
* **Unit:** status mapping, comparators per ecosystem, trust weighting, threshold math.
* **Property tests:** invariants such as monotonicity with added supporting evidence and idempotent recompute.
* **Integration:** Excitator→Lens→Policy→Vuln Explorer pipeline with signed/unsigned, conflicting, and stale evidence.
* **E2E Console:** list filters, detail with quorum, conflict workflows, export, simulation.
* **Security:** RBAC on Issuer Directory, CSRF for Console, signature verification path traversal guards.
* **Performance:** cold/hot query latencies, recompute throughput, cache hit ratios.
* **Determinism:** time‑travel snapshots reproduce prior consensus states.
---
## 10) Philosophy
* **Consensus, not replacement.** The Lens summarizes without erasing dissent.
* **Trust is contextual.** Tenants must tune weights and thresholds to their environments.
* **Proof over prose.** Every decision comes with math, provenance, and a rationale chain.
* **Safety by design.** When in doubt, surface conflicts instead of silently suppressing.
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**

440
EPIC_8.md Normal file
View File

@@ -0,0 +1,440 @@
Below is the “maximum documentation” bundle for Epic 8. It’s engineered to be pasted into your repo without turning into yet another unread wiki tomb. Slight sarcasm included to keep blood flowing.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
# Epic 8: Advisory AI Assistant (summaries, conflict explain, remediation hints)
**Short name:** `Advisory AI`
**Services touched:** Conseiller (Feedser), Excitator (Vexer), VEX Lens, SBOM Service, Policy Engine, Findings Ledger, Web API Gateway, Authority (authN/Z), Console (Web UI), CLI, Telemetry/Analytics
**AOC ground rule:** Conseiller and Excitator aggregate but never merge or mutate source docs. Advisory AI produces derived summaries and plans with strict provenance and citations. No silent rewriting of evidence. Ever.
---
## 1) What it is
Advisory AI is a tenant‑scoped, retrieval‑augmented assistant that turns noisy security advisories and VEX statements into three consumable artifacts:
1. **Advisory Summary**
Condenses one or more advisories (CSAF, OSV, GHSA, vendor PDFs, distro notices) into a concise brief with key facts: affected ranges, exploit status, impact, known workarounds, fixed versions, and links. Always cites the exact sources and sections used.
2. **Conflict Explain**
Explains why VEX statements or advisories disagree for a specific artifact and version. Uses the VEX Consensus Lens outputs and issuer trust model to produce a human‑readable, step‑by‑step explanation: who said what, where the product scoping diverges, and what policy thresholds caused the final state.
3. **Remediation Hints**
Suggests practical next steps: upgrade paths compatible with your dependency graph, backports, config toggles, temporary policy suppressions, or compensating controls. Every hint is grounded in SBOM, environment, and policy. It ships as structured JSON plus a human summary, ready to paste into a ticket.
It lives in the Console as a side panel, in the CLI for batch runs, and via APIs for automation. It does not change scanner results or consensus on its own. Humans remain in charge. The machine does the skimming and the math so humans can keep the judgment and the coffee.
---
## 2) Why (brief)
Advisories are long, inconsistent, and sometimes contradictory. Teams waste cycles reconciling PDFs with package manifests. The assistant eliminates that sludge: fast summaries, explicit conflict explanations, and remediation hints that are actually applicable to your software, not to an imaginary ideal project from 2013.
---
## 3) How it should work (maximum detail)
### 3.1 Capabilities
* **Summaries**
* Input: one advisory or a bundle linked by the same advisory key (CVE, GHSA, vendor‑ID), product scope, and environment.
* Output:
* 150 to 300 words summary
* `AdvisorySummary JSON` (schema below)
* Citations with paragraph anchors
* Confidence label and coverage score (how much of the advisory set is represented)
* **Conflict Explain**
* Input: `(artifact_id, purl, version, advisory_key)` tuple.
* Output: narrative plus a structured breakdown of consensus math, issuer votes, product mapping mismatches, and the exact policy knobs that tipped the result.
* **Remediation Hints**
* Input: same tuple plus SBOM context and environment.
* Output: ranked list of remediation options with feasibility score, blast radius estimate (derived from dependency paths), effort class, and links to fixed versions. Includes “do nothing” when the VEX consensus is not affected.
### 3.2 System design
**Architecture diagram in words (because ASCII art is a crime):**
1. **Retrievers**
* Structured retriever over Conseiller’s normalized advisory fields.
* Vector retriever over advisory text chunks with paragraph anchors.
* VEX retriever over Excitator evidence and VEX Lens consensus.
* SBOM retriever for purl, version, dependency paths, env flags.
2. **Deterministic resolvers**
* Version comparators per ecosystem.
* Range satisfaction checks.
* Dependency path scorers and blast radius estimator.
3. **Orchestrator**
* Task‑specific prompt templates for Summary, Conflict, Remediation.
* Tool calls to deterministics (version check, graph crawl) with results injected into the prompt as structured context.
* Strict token budgets and truncation rules to avoid model babble.
4. **Models**
* Default: on‑prem inference container with mid‑sized model.
* Optional: tenant‑enabled remote inference. Disabled by default.
* Temperature locked low for summary and conflict. Slightly higher for remediation narrative phrasing. No creativity in facts.
5. **Guardrails**
* Prompt injection defense by stripping or quarantining advisory text that tries to instruct the model.
* Fact boundary tagger. The assistant must only state facts that appear in structured inputs or cited chunks.
* Redaction of secrets before prompts.
* Output validator checks: required JSON fields, numeric ranges, valid version strings.
### 3.3 Data contracts
**AdvisorySummary JSON**
```json
{
"advisory_key": "CVE-2025-12345",
"sources": [
{"id":"csaf:vendorA:2025-001","uri":"...","sections":["2.1","3.4"]},
{"id":"osv:pkg:npm/lodash","uri":"...","sections":["affected","references"]}
],
"affected_ranges": [
{"ecosystem":"npm","purl_family":"pkg:npm/lodash","introduced":"<4.17.15","fixed": "4.17.21"}
],
"exploit_status": "no_known_exploit | poc_public | exploited_in_the_wild | n/a",
"impact": {"cvss":[{"vector":"CVSS:3.1/AV:N/...","score":7.5}], "cwes":["CWE-79"]},
"workarounds": ["Disable feature X", "Set flag Y=false"],
"fixed_versions": ["4.17.21"],
"notes": "Vendor states not affected on platform Z due to build option W.",
"coverage_score": 0.86,
"generated_at": "2025-10-25T12:00:00Z"
}
```
**ConflictExplanation JSON**
```json
{
"tuple": {"artifact_id":"svc:checkout@1.9.0","purl":"pkg:npm/lodash@4.17.20","advisory_key":"CVE-2025-12345"},
"consensus": {"state":"NOT_AFFECTED","confidence":0.82},
"quorum": [
{"issuer":"lodash-maintainers","status":"NOT_AFFECTED","weight":0.9,"sig":true,"justification":"component_not_present"},
{"issuer":"vendorX-distro","status":"AFFECTED","weight":0.25,"sig":false,"justification":"generic"}
],
"policy_factors": {"na_threshold":1.0,"aff_threshold":0.6,"recency_decay_days":90},
"mapping_issues": [{"kind":"cpe_to_purl","score":0.6,"detail":"CPE wildcard matched multiple purls"}],
"explanation_steps": [
"Exact purl match found for maintainer VEX; weight 0.9",
"Distro advisory generic; scope score 0.5; effective weight 0.25",
"NA threshold met. Result set to NOT_AFFECTED"
]
}
```
**RemediationPlan JSON**
```json
{
"tuple": {"artifact_id":"svc:checkout@1.9.0","purl":"pkg:npm/lodash@4.17.20","advisory_key":"CVE-2025-12345"},
"options": [
{
"kind": "upgrade",
"target_version": "4.17.21",
"feasibility": 0.92,
"blast_radius": {"direct_callers":3,"transitive_depth":2},
"effort": "low | medium | high",
"rationale": "Semver patch, no breaking APIs in release notes",
"links": ["release_notes_uri"]
},
{
"kind": "workaround",
"action": "Set SAFE_MODE=true",
"feasibility": 0.6,
"blast_radius": {"feature_flags":["SAFE_MODE"]},
"effort": "low",
"rationale": "Vendor states mitigation reduces attack surface on feature X"
}
],
"preferred": 0,
"policy_effects": {"sla_days": 7, "severity_override": "medium_if_not_fixed"},
"generated_at": "2025-10-25T12:00:00Z"
}
```
### 3.4 APIs
```
POST /advisory/ai/summary
{
"advisory_key":"CVE-2025-12345",
"artifact_id":"svc:checkout@1.9.0",
"purl":"pkg:npm/lodash@4.17.20",
"sources":["csaf:*","osv:*"], // optional filters
"policy_version":"1.3.0",
"lang":"en"
}
-> 200 { "summary_text":"...", "summary": {AdvisorySummary}, "citations":[...] }
POST /advisory/ai/conflict
{
"artifact_id":"svc:checkout@1.9.0",
"purl":"pkg:npm/lodash@4.17.20",
"advisory_key":"CVE-2025-12345",
"policy_version":"1.3.0"
}
-> 200 { "explanation_text":"...", "explanation": {ConflictExplanation} }
POST /advisory/ai/remediation
{
"artifact_id":"svc:checkout@1.9.0",
"purl":"pkg:npm/lodash@4.17.20",
"advisory_key":"CVE-2025-12345",
"policy_version":"1.3.0",
"max_options":5,
"strategy_preference":["upgrade","backport","workaround"]
}
-> 200 { "plan_text":"...", "plan": {RemediationPlan} }
POST /advisory/ai/batch
{
"items":[ {tuple}, {tuple}, ... ],
"task":"summary | conflict | remediation",
"policy_version":"1.3.0"
}
-> 207 multi-status
```
Status codes: `400` invalid, `403` RBAC, `404` missing evidence, `409` conflict lock, `422` output validation failed, `429` rate limit.
### 3.5 Console (Web UI)
* Surfaces:
* Vuln Explorer detail: “Advisory AI” side panel with 3 tabs: Summary, Conflict, Remediation.
* Consensus Lens detail: prominent “Explain conflict” button.
* Policy Studio sim: “Show effect on assistant output” preview.
* UX details:
* Citations are footnotes with hover to show source paragraph.
* “Copy as ticket” produces Markdown and JSON.
* Plan options show feasibility bar, blast radius chips, and required approvals per policy.
* Injection warnings appear if advisory text included unsafe instructions.
* A11y: ARIA tags for tabs, keyboard shortcuts `G` to generate, `R` to refresh, `C` to copy JSON.
### 3.6 CLI
```
stella advise summarize --advisory CVE-2025-12345 --artifact svc:checkout@1.9.0 --purl pkg:npm/lodash@4.17.20 --policy 1.3.0 --json
stella advise explain --advisory CVE-2025-12345 --artifact svc:checkout@1.9.0 --purl pkg:npm/lodash@4.17.20 --policy 1.3.0
stella advise remediate --advisory CVE-2025-12345 --artifact svc:checkout@1.9.0 --purl pkg:npm/lodash@4.17.20 --policy 1.3.0 --strategy upgrade,workaround --out plan.json
stella advise batch --file tuples.json --task remediation --policy 1.3.0
```
Exit codes: `0` ok, `2` invalid args, `4` not found, `5` denied, `7` validation fail.
### 3.7 RBAC and security
* Roles:
* Viewer can run summaries and read explanations.
* Operator can run remediation and export plans.
* Admin can toggle model endpoints and guardrail settings.
* Defaults:
* Remote model calls disabled.
* Redaction on.
* Prompt logging anonymized.
* Outputs stored as derived artifacts with TTL (default 30 days) unless pinned to a ticket.
### 3.8 Observability
* Metrics:
* `advisory_ai_latency_ms` by task type.
* `advisory_ai_guardrail_blocks_total`.
* `advisory_ai_output_validation_fail_total`.
* `advisory_ai_citation_coverage` gauge.
* Traces: retriever spans, tool calls, model inference, validator.
* Logs: include tuple key, token usage, truncation events, and guardrail outcomes.
### 3.9 Performance
* Targets:
* P95 under 1.5 s for Summary and Conflict with warm caches.
* P95 under 2.5 s for Remediation on medium SBOMs (1000 packages).
* Batch throughput 10 tuples per second per worker.
### 3.10 Edge cases
* Advisory missing fixed versions: produce workaround‑only plan and mark feasibility low.
* Conflicts with near‑tie weights: declare “DISPUTED” and require human approval, no auto plan preferred.
* Exotic version schemes: fallback to string compare with warning and feasibility cap.
* Private packages: no public release notes. Prefer internal changelog links if attached to artifact metadata.
* Multi‑env differences: render per‑env deltas when policy knobs differ (dev vs prod).
---
## 4) Implementation plan
### 4.1 Services and components
* **New:** `src/StellaOps.AdvisoryAI`
* `retriever/` wrappers for Conseiller, Excitator, VEX Lens, SBOM.
* `deterministic/` version and path analyzers.
* `orchestrator/` task routers and prompt builders.
* `guardrails/` injection, redaction, output validator.
* `api/` REST endpoints and schema enforcement.
* **Updates:**
* Conseiller: expose paragraph‑level anchors for advisories.
* Excitator: expose justifications and product trees in normalized form.
* VEX Lens: stable API for quorum and rationale.
* SBOM Service: efficient path queries and versions timeline per purl.
### 4.2 Packaging
* Container images:
* `stella/advisory-ai:<<version>>`
* `stella/inference:<<version>>` (if using on‑prem model)
* Helm values to toggle remote inference and GPU.
### 4.3 Rollout
* Phase 1: Summary and Conflict read‑only.
* Phase 2: Remediation with “Copy as ticket”.
* Phase 3: Batch APIs, CLI, and Policy Studio simulation hooks.
---
## 5) Documentation changes
Create or update the following files. Each doc ends with the imposed rule statement.
1. `/docs/advisory-ai/overview.md`
What it is, capabilities, guardrails, AOC alignment, RBAC.
2. `/docs/advisory-ai/architecture.md`
RAG design, retrievers, orchestrator, deterministics, models, caching.
3. `/docs/advisory-ai/api.md`
Endpoint specs, payload schemas, error codes, examples.
4. `/docs/advisory-ai/console.md`
Screens, actions, a11y, how citations work, copy‑as‑ticket.
5. `/docs/advisory-ai/cli.md`
Command usage, exit codes, piping examples.
6. `/docs/policy/assistant-parameters.md`
Temperature, max tokens, plan ranking weights, TTLs.
7. `/docs/security/assistant-guardrails.md`
Redaction rules, injection defense, output validation, logging.
8. `/docs/sbom/remediation-heuristics.md`
Feasibility scoring, blast radius, effort classes.
9. `/docs/runbooks/assistant-ops.md`
Warmup, cache priming, model outages, scaling, on‑call steps.
---
## 6) Engineering tasks
### Backend core
* [ ] Implement structured and vector retrievers with paragraph anchors from Conseiller.
* [ ] Implement VEX retriever using Lens APIs with caching.
* [ ] Build deterministics: ecosystem comparators, range checks, dependency path scorer.
* [ ] Implement orchestrator with task‑specific templates and tool call pipeline.
* [ ] Implement guardrails and validators with hard failure on invalid JSON.
* [ ] Add RBAC to endpoints and anonymized prompt logging.
* [ ] Add caching layer with tuple‑keyed entries and policy version scoping.
### Integrations
* [ ] Conseiller: expose advisory chunk API and metadata needed for citations.
* [ ] Excitator: ensure justifications and product trees are queryable.
* [ ] VEX Lens: add “policy factors” endpoint for explanation rendering.
* [ ] SBOM Service: implement `GET /sbom/paths?purl=...` and version timeline.
### Console
* [ ] Build Advisory AI panel with 3 tabs and citation tooltips.
* [ ] Implement “Copy as ticket” (Markdown + JSON) and download.
* [ ] Add injection warning banner when triggered.
* [ ] Respect a11y requirements and shortcuts.
### CLI
* [ ] `stella advise summarize|explain|remediate|batch` with JSON output.
* [ ] Add `--out` option to save plans and summaries.
* [ ] Tests for piping and jq workflows.
### Observability
* [ ] Emit metrics and traces listed in §3.8.
* [ ] Dashboards: latency, guardrail blocks, validation fails, coverage.
### Docs
* [ ] Write all files in §5 with examples and screenshots.
* [ ] Cross‑link to VEX Lens and Vulnerability Explorer docs.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Acceptance criteria
* Summaries cite specific source sections and reflect affected ranges and fixed versions correctly for at least 95% of a validation set.
* Conflict explanations enumerate issuers, weights, justifications, mapping issues, and policy thresholds that caused the consensus state.
* Remediation plans output at least one feasible option when a fixed version exists and correctly flag “no public fix” cases.
* JSON schemas validate for all outputs.
* Console shows the panel with citations, copy‑as‑ticket, and a11y passes.
* CLI produces identical JSON to API responses.
* Guardrails block injection attempts and redact secrets in prompts.
* P95 latency targets are met with warm caches.
* No mutation of raw advisory or VEX evidence occurs anywhere in the pipeline.
---
## 8) Risks and mitigations
* **Prompt injection in advisory text.** Strip instructions, sandbox chunks, and highlight to user when removed.
* **Hallucinated facts.** Hard validation requires facts to appear in structured inputs or cited text. Fail closed if not provable.
* **Mapping errors produce bad hints.** Depend on SBOM Graph and VEX Lens scope scores; cap feasibility when scope is weak.
* **Model outage.** Degrade to deterministic summaries (shorter, but accurate).
* **Privacy concerns.** Default on‑prem inference, remote endpoints opt‑in with clear flags and audit logs.
---
## 9) Test plan
* **Unit:** version comparators, range checks, feasibility scoring, output validators.
* **Golden files:** advisory sets mapped to expected summaries and plans; diff on each build.
* **Injection tests:** adversarial advisories with “ignore prior instructions” payloads must be neutralized.
* **Integration:** Conseiller→Advisory AI→Console loop, with VEX Lens conflicts and SBOM graph lookups.
* **E2E:** generate summary, explanation, and plan for representative ecosystems (npm, Maven, PyPI, Go, RPM/DEB).
* **Perf:** soak tests with 5k tuples batch; observe cache hit ratios and P95.
* **A11y:** keyboard navigation and screen reader labels.
---
## 10) Philosophy
* **Facts first.** If it is not in structured inputs or citations, it does not exist.
* **Explain everything.** Humans should see exactly why the tool said what it said.
* **Helpful by default.** Plans must consider the real dependency graph and environment, not fantasy.
* **No silent merges.** Evidence is sacred. Summaries and plans are separate, auditable derivatives.
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**

523
EPIC_9.md Normal file
View File

@@ -0,0 +1,523 @@
Below is the “maximum documentation” bundle for Epic 9. Paste it into your repo and pretend the ingestion chaos was always under control.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
# Epic 9: Source & Job Orchestrator Dashboard
**Short name:** `Orchestrator Dashboard`
**Primary service:** `orchestrator` (scheduler, queues, rate‑limits, job state)
**Surfaces:** Console (Web UI), CLI, Web API
**Touches:** Conseiller (Feedser), Excitator (Vexer), VEX Consensus Lens, SBOM Service, Policy Engine, Findings Ledger, Authority (authN/Z), Telemetry/Analytics, Object Storage, Relational DB, Message Bus
**AOC ground rule:** Conseiller and Excitator aggregate but never merge. The orchestrator schedules, tracks and recovers jobs; it does not transform evidence beyond transport and storage. No “smart” merging in flight.
---
## 1) What it is
The Source & Job Orchestrator Dashboard is the control surface for every data source and pipeline run across StellaOps. It gives operators:
* Live health of all advisory/VEX/SBOM sources and derived jobs.
* End‑to‑end pipeline visibility as DAGs and timelines.
* Controls for pausing, backfilling, replaying, throttling and retrying.
* Error pattern analysis, rate‑limit observability and backpressure insights.
* Provenance and audit trails from initial fetch through parse, normalize, index and policy evaluation.
The dashboard sits over the `orchestrator` service, which maintains job state, schedules runs, enforces quotas and rate limits, and collects metrics from worker pools embedded in Conseiller, Excitator, SBOM and related services.
---
## 2) Why (brief)
Ingestion breaks quietly and then loudly. Without a unified control plane, you learn about it from angry users or empty indexes. This dashboard shortens incident MTTR, enables safe backfills, and makes compliance reviewers stop sending emails with twelve attachments and one emoji.
---
## 3) How it should work (maximum detail)
### 3.1 Capabilities
* **Source registry**
* Register, tag and version connectors (OSV, GHSA, CSAF endpoints, vendor PDF scrapers, distro feeds, RSS, S3 drops, internal registries).
* Store connection details, secrets (via KMS), rate‑limit policy, schedules, and ownership metadata.
* Validate and “test connection” safely.
* **Job orchestration**
* Create DAGs composed of job types: `fetch`, `parse`, `normalize`, `dedupe`, `index`, `consensus_compute`, `policy_eval`, `crosslink`, `sbom_ingest`, `sbom_index`.
* Priorities, queues, concurrency caps, exponential backoff, circuit breakers.
* Idempotency keys and output artifact hashing to avoid duplicate work.
* Event‑time watermarks for backfills without double counting.
* **Observability & control**
* Gantt timeline and real‑time DAG view with critical path highlighting.
* Backpressure and queue depth heatmaps.
* Error clustering by class (HTTP 429, TLS, schema mismatch, parse failure, upstream 5xx).
* Per‑source SLOs and SLA budgets with burn‑rate alerts.
* One‑click actions: retry, replay range, pause/resume, throttle/unthrottle, reroute to canary workers.
* **Provenance & audit**
* Immutable run ledger linking input artifact → every job → output artifact.
* Schema version tracking and drift detection.
* Operator actions recorded with reason and ticket reference.
* **Safety**
* Secret redaction everywhere.
* Tenant isolation at API, queue and storage layers.
* AOC: no in‑flight merges of advisory or VEX content.
### 3.2 Core architecture
* **orchestrator (service)**
* Maintains job state in Postgres (`sources`, `runs`, `jobs`, `artifacts`, `dag_edges`, `quotas`, `schedules`).
* Publishes work to a message bus (e.g., `topic.jobs.ready.<queue>`).
* Distributed token‑bucket rate limiter per source/tenant/host.
* Watchdog for stuck jobs and circuit breakers for flapping sources.
* Watermark manager for backfills (event‑time windows).
* **worker SDK**
* Lightweight library embedded in Conseiller/Excitator/SBOM workers to:
* Claim work, heartbeat, update progress, report metrics.
* Emit artifact metadata and checksums.
* Enforce idempotency via orchestrator‑supplied key.
* **object store**
* Raw payloads and intermediate artifacts organized by schema and hash:
* `advisory/raw/<source_id>/<event_time>/<sha256>.json|pdf`
* `advisory/normalized/<schema_ver>/<hash>.json`
* `vex/raw|normalized/...`
* `sbom/raw|graph/...`
* **web API**
* CRUD for sources, runs, jobs, schedules, quotas.
* Control actions (retry, cancel, pause, backfill).
* Streaming updates via WebSocket/SSE for the Console.
* **console**
* React app consuming Orchestrator APIs, rendering DAGs, timelines, health charts and action panels with RBAC.
### 3.3 Data model (selected tables)
* `sources`
* `id`, `kind` (`advisory|vex|sbom|internal`), `subtype` (e.g., `osv`, `ghsa`, `csaf`, `vendor_pdf`), `display_name`, `owner_team`, `schedule_cron`, `rate_policy`, `enabled`, `secrets_ref`, `tags`, `schema_hint`, `created_at`, `updated_at`.
* `runs`
* `id`, `source_id`, `trigger` (`schedule|manual|event|backfill`), `window_start`, `window_end`, `state`, `started_at`, `finished_at`, `stats_json`.
* `jobs`
* `id`, `run_id`, `type`, `queue`, `priority`, `state` (`pending|running|succeeded|failed|canceled|deadletter`), `attempt`, `max_attempt`, `idempotency_key`, `input_artifact_id`, `output_artifact_id`, `worker_id`, `created_at`, `started_at`, `finished_at`, `error_class`, `error_message`, `metrics_json`.
* `dag_edges`
* `from_job_id`, `to_job_id`, `edge_kind` (`success_only|always`).
* `artifacts`
* `id`, `kind` (`raw|normalized|index|consensus`), `schema_ver`, `hash`, `uri`, `bytes`, `meta_json`, `created_at`.
* `quotas`
* `tenant_id`, `resource` (`requests_per_min`, `concurrent_jobs`), `limit`, `window_sec`.
* `schedules`
* Per‑source cron plus jitter, timezone, blackout windows.
### 3.4 Job lifecycle
1. **Plan**
Scheduler creates a `run` for a source and plans a DAG: e.g., `fetch → parse → normalize → dedupe → index → policy_eval` (advisory) or `fetch → parse → normalize → consensus_compute` (VEX).
2. **Enqueue**
Ready nodes become `jobs` with queue, priority, idempotency key and optional rate‑limit tokens reserved.
3. **Execute**
Worker claims job, heartbeats every N seconds. Output artifacts are stored and linked. Failures are classified and retried with exponential backoff and jitter, up to `max_attempt`.
4. **Complete**
Downstream nodes unblock. On run completion, orchestrator computes SLO deltas and emits run summary.
5. **Dead‑letter**
Jobs exceeding attempts move to a DLQ with structured context and suggested remediation.
### 3.5 Scheduling, backpressure, rate‑limits
* **Token bucket** per `{tenant, source.host}` with adaptive refill if upstream 429/503 seen.
* **Concurrency caps** per source and per job type to avoid thundering herd.
* **Backpressure signals** from queue depth, worker CPU, and upstream error rates; scheduler reduces inflight issuance accordingly.
* **Backfills** use event‑time windows with immutable watermarks to avoid re‑processing.
* **Blackout windows** for vendor maintenance periods.
### 3.6 APIs
```
POST /orchestrator/sources
GET /orchestrator/sources?kind=&tag=&q=
GET /orchestrator/sources/{id}
PATCH /orchestrator/sources/{id}
POST /orchestrator/sources/{id}/actions:test|pause|resume|sync-now
POST /orchestrator/sources/{id}/backfill { "from":"2024-01-01", "to":"2024-03-01" }
GET /orchestrator/runs?source_id=&state=&from=&to=
GET /orchestrator/runs/{run_id}
GET /orchestrator/runs/{run_id}/dag
POST /orchestrator/runs/{run_id}/cancel
GET /orchestrator/jobs?state=&type=&queue=&source_id=
GET /orchestrator/jobs/{job_id}
POST /orchestrator/jobs/{job_id}/actions:retry|cancel|prioritize
GET /orchestrator/metrics/overview
GET /orchestrator/errors/top?window=1h
GET /orchestrator/quotas
PATCH /orchestrator/quotas/{tenant_id}
WS /orchestrator/streams/updates
```
### 3.7 Console (Web UI)
* **Overview**
* KPI tiles: sources healthy, runs in progress, queue depth, error rate, burn‑rate to SLO.
* Heatmap of source health by last 24h success ratio.
* **Sources**
* Grid with filters, inline status (active, paused, throttled), next run eta, last error class.
* Detail panel: config, secrets status (redacted), schedule, rate limits, ownership, run history, action buttons.
* **Runs**
* Timeline (Gantt) with critical path, duration distribution, and per‑stage breakdown.
* Run detail: DAG view with node metrics, artifacts, logs, action menu (cancel).
* **Jobs**
* Live table with state filters and “tail” view.
* Job detail: payload preview (redacted), worker, attempts, stack traces, linked artifacts.
* **Errors**
* Clusters by class and signature, suggested remediations (pause source, lower concurrency, patch parser).
* **Queues & Backpressure**
* Per‑queue depth, service rate, inflight, age percentiles.
* Rate‑limit tokens graphs per source host.
* **Controls**
* Backfill wizard with event‑time preview and safety checks.
* Canary routing: route 5% of next 100 runs to a new worker pool.
* **A11y**
* Keyboard nav, ARIA roles for DAG nodes, live regions for updates, color‑blind friendly graphs.
### 3.8 CLI
```
stella orch sources list --kind advisory --tag prod
stella orch sources add --file source.yaml
stella orch sources test <source-id>
stella orch sources pause <source-id> # or resume
stella orch sources sync-now <source-id>
stella orch sources backfill <source-id> --from 2024-01-01 --to 2024-03-01
stella orch runs list --source <id> --state running
stella orch runs show <run-id> --dag
stella orch runs cancel <run-id>
stella orch jobs list --state failed --type parse --limit 100
stella orch jobs retry <job-id>
stella orch jobs cancel <job-id>
stella orch jobs tail --queue normalize --follow
stella orch quotas get --tenant default
stella orch quotas set --tenant default --concurrent-jobs 50 --rpm 1200
```
Exit codes: `0` success, `2` invalid args, `4` not found, `5` denied, `7` precondition failed, `8` rate‑limited.
### 3.9 RBAC & security
* **Roles**
* `Orch.Viewer`: read‑only sources/runs/jobs/metrics.
* `Orch.Operator`: perform actions on sources and jobs, launch backfills.
* `Orch.Admin`: manage quotas, schedules, connector versions, and delete sources.
* **Secrets**
* Stored only as references to your KMS; never persisted in cleartext.
* Console shows redact badges and last rotated timestamp.
* **Tenancy**
* Source, run, job rows scoped by tenant id.
* Queue names and token buckets namespaced per tenant.
* **Compliance**
* Full audit log for every operator action with “reason” and optional ticket link.
* Exportable run ledger for audits.
### 3.10 Observability
* **Metrics (examples)**
* `orch_jobs_inflight{type,queue}`
* `orch_jobs_latency_ms{type,percentile}`
* `orch_rate_tokens_available{source}`
* `orch_error_rate{source,error_class}`
* `orch_slo_burn_rate{source,slo}`
* `orch_deadletter_total{source,type}`
* **Traces**
* Span per job with baggage: `run_id`, `source_id`, `artifact_id`.
* Links across services to Conseiller/Excitator/SBOM workers.
* **Logs**
* Structured JSON with correlation ids, attempt numbers and redacted payload previews.
### 3.11 Performance targets
* Job dispatch P95 < 150 ms after dependency satisfied.
* Scheduler loop P95 < 500 ms for 10k pending jobs.
* Console live updates sub‑second at 1k events/sec per tenant.
* Backfill throughput ≥ 200 jobs/sec per worker pool with zero dupes.
### 3.12 Edge cases & behaviors
* **Upstream 429 storms:** auto‑throttle, pause optional, recommend extended jitter.
* **Schema drift:** parser moves job to DLQ with `error_class=schema_mismatch` and opens a change ticket via webhook.
* **Flapping source:** circuit breaker opens after N consecutive failures; requires human “resume”.
* **Clock skew:** watermark logic uses upstream event time; large skews flagged.
* **Idempotency collisions:** new attempt yields no‑op if artifact hash already exists.
---
## 4) Implementation plan
### 4.1 Modules (new and updated)
* New service: `src/StellaOps.Orchestrator`
* `api/` REST + WS handlers
* `scheduler/` run planner, DAG builder, watermark/backfill logic
* `queues/` publisher and consumer abstractions
* `ratelimit/` token bucket and adaptive controller
* `state/` Postgres repositories and migrations
* `audit/` action logging and export
* `metrics/` Prometheus exporters
* `security/` tenant scoping, KMS client, secret refs
* Worker SDKs:
* `src/StellaOps.Orchestrator.WorkerSdk.Go` and `src/StellaOps.Orchestrator.WorkerSdk.Python` with job claim, heartbeat, progress, artifact publish, and structured error reporting.
* Console:
* `console/apps/orch/` pages: Overview, Sources, Runs, Jobs, Errors, Queues.
* `components/dag-view/`, `components/gantt/`, `components/health-heatmap/`.
* Updates to existing services:
* Conseiller/Excitator/SBOM workers adopt SDK and emit artifacts with schema/version/fingerprint.
* VEX Lens exposes `consensus_compute` as a jobable operation.
* Policy Engine exposes `policy_eval` as a job type for scheduled recalcs.
### 4.2 Packaging & deployment
* Containers:
* `stella/orchestrator:<ver>`
* `stella/worker-sdk-examples:<ver>` for canary pools
* Helm values:
* Queues/topics, per‑tenant concurrency, rate‑limit defaults, WS replica count.
* KMS integration secrets.
* Migrations:
* Flyway/Goose migrations for new tables and indexes.
### 4.3 Rollout strategy
* Phase 1: Read‑only dashboard fed by existing job tables; no controls.
* Phase 2: Control actions enabled for non‑prod tenants.
* Phase 3: Backfills and quota management, then GA.
---
## 5) Documentation changes
Create/update the following, each ending with the imposed rule statement.
1. `/docs/orchestrator/overview.md`
Concepts, roles, responsibilities, AOC alignment.
2. `/docs/orchestrator/architecture.md`
Scheduler, DAGs, watermarks, queues, rate‑limits, data model.
3. `/docs/orchestrator/api.md`
Endpoints, WebSocket events, error codes, examples.
4. `/docs/orchestrator/console.md`
Screens, actions, a11y, live updates.
5. `/docs/orchestrator/cli.md`
Commands, examples, exit codes, scripting patterns.
6. `/docs/orchestrator/run‑ledger.md`
Provenance and audit export format.
7. `/docs/security/secrets‑handling.md`
KMS references, redaction rules, operator hygiene.
8. `/docs/operations/orchestrator‑runbook.md`
Common failures, backfill guide, circuit breakers, tuning.
9. `/docs/schemas/artifacts.md`
Artifact kinds, schema versions, hashing, storage layout.
10. `/docs/slo/orchestrator‑slo.md`
SLO definitions, measurement, alerting.
---
## 6) Engineering tasks
### Backend (orchestrator)
* [ ] Stand up Postgres schemas and indices for sources, runs, jobs, dag_edges, artifacts, quotas, schedules.
* [ ] Implement scheduler: DAG planner, dependency resolver, critical path computation.
* [ ] Implement rate limiter with adaptive behavior on 429/503 and per‑tenant tokens.
* [ ] Implement watermark/backfill manager with event‑time windows and idempotency keys.
* [ ] Implement API endpoints + OpenAPI spec + request validation.
* [ ] Implement WebSocket/SSE event stream for live updates.
* [ ] Implement audit logging and export.
* [ ] Implement dead‑letter store and replay.
### Worker SDKs and integrations
* [ ] Build Go/Python SDKs with claim/heartbeat/progress API.
* [ ] Integrate SDK into Conseiller, Excitator, SBOM workers; ensure artifact emission with schema ver.
* [ ] Add `consensus_compute` and `policy_eval` as job types with deterministic inputs/outputs.
### Console
* [ ] Overview tiles and health heatmap.
* [ ] Source list/detail with actions and config view.
* [ ] Runs timeline (Gantt) and DAG visualization with node inspector.
* [ ] Jobs “tail” with live updates and filters.
* [ ] Errors clustering and suggested remediations.
* [ ] Queues/backpressure dashboard.
* [ ] Backfill wizard with safety checks.
### Observability
* [ ] Emit metrics listed in §3.10 and wire traces across services.
* [ ] Dashboards: health, queue depth, error classes, burn‑rate, dispatch latency.
* [ ] Alerts for SLO burn and circuit breaker opens.
### Security & RBAC
* [ ] Enforce tenant scoping on all endpoints; test leakage.
* [ ] Wire KMS for secret refs and redact everywhere.
* [ ] Implement `Orch.Viewer|Operator|Admin` roles and check in Console and API.
### Docs
* [ ] Author all files in §5 with examples and screenshots.
* [ ] Cross‑link from Conseiller/Excitator/SBOM pages to the dashboard docs.
* [ ] Append imposed rule to each page.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Acceptance criteria
* Operators can: pause/resume a source, run “sync‑now,” initiate a backfill for a date range, and retry/cancel individual jobs from Console and CLI.
* DAG and timeline reflect reality within 1 second of job state changes at P95.
* Backfills do not create duplicate artifacts; idempotency proven by hash equality.
* Rate limiter reduces 429s by ≥80% under simulated throttle tests.
* Audit log includes who/when/why for every operator action.
* Provenance ledger exports a complete chain for any artifact.
* RBAC prevents non‑admins from quota changes; tenancy isolation proven via automated tests.
* SLO dashboard shows burn‑rate and triggers alerts under injected failure.
---
## 8) Risks & mitigations
* **Orchestrator becomes a single bottleneck.**
Horizontal scale stateless workers; DB indexes tuned; job state updates batched; cache hot paths.
* **Secret spillage.**
Only KMS references stored; aggressive redaction; log scrubbing in SDK.
* **Over‑eager backfills overwhelm upstream.**
Enforce per‑source quotas and sandbox previews; dry‑run backfills first.
* **Schema drift silently corrupts normalization.**
Hard‑fail on mismatch; DLQ with clear signatures; schema registry gating.
* **Flapping sources cause alert fatigue.**
Circuit breaker with cool‑down and deduped alerts; error budget policy.
---
## 9) Test plan
* **Unit**
Scheduler DAG building, topological sort, backoff math, token bucket, watermark math.
* **Integration**
Orchestrator ↔ worker SDK, artifact store wiring, DLQ replay, audit pipeline.
* **Chaos**
Inject 429 storms, packet loss, worker crashes; verify throttling and recovery.
* **Backfill**
Simulate overlapping windows and verify idempotency and watermark correctness.
* **Perf**
10k concurrent jobs: dispatch latency, DB contention, WebSocket fan‑out.
* **Security**
Multi‑tenant isolation tests; KMS mock tests for secret access; RBAC matrix.
* **UX/A11y**
Screen reader labels on DAG, keyboard navigation, live region updates.
---
## 10) Philosophy
* **Make the invisible visible.** Pipelines should be legible at a glance.
* **Prefer reproducibility to heroics.** Idempotency and provenance over “we think it ran.”
* **Safeguards before speed.** Throttle first, retry thoughtfully, never melt upstreams.
* **No silent merges.** Evidence remains immutable; transformations are explicit, logged and reversible.
> Final reminder: **Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.**

1170
EXECPLAN.md
View File

File diff suppressed because it is too large Load Diff

7
Read SPRINTs.md Normal file
View File

@@ -0,0 +1,7 @@
Read SPRINTs.md
Here follow a new EPIC you need to outline on the SPRINTS/TASKS.
Do not forget to read appropriate parts of the project to see what kind of tasks you need to edit/delete or create to forfill the goals of the sprint. Do not shy to create new projects to concentrate library or new webservice or plugin, but check the src/ for projects to make sure there is no already one that could do it.
Do not shy to edit existing tasks - if they needs to be adjusted
Do not shy to delete of existing task - if they do not make sense anymore for the new EPIC.
But most importantly create - detailed tasks to forfill the EPIC goals.

1279
SPRINTS.md
View File

File diff suppressed because it is too large Load Diff

2
SPRINTS_PRIOR_20251021.md
View File

@@ -32,7 +32,7 @@ This file describe implementation of Stella Ops (docs/README.md). Implementation
| Sprint 9 | Policy Foundations | src/StellaOps.Policy/TASKS.md | DONE (2025-10-19) | Policy Guild | POLICY-CORE-09-005 | Scoring/quiet engine – compute score, enforce VEX-only quiet rules, emit inputs and provenance. |
| Sprint 9 | Policy Foundations | src/StellaOps.Policy/TASKS.md | DONE (2025-10-19) | Policy Guild | POLICY-CORE-09-006 | Unknown state & confidence decay – deterministic bands surfaced in policy outputs. |
| Sprint 9 | Docs & Governance | docs/TASKS.md | DONE (2025-10-21) | Platform Events Guild | PLATFORM-EVENTS-09-401 | Embed canonical event samples into contract/integration tests and ensure CI validates payloads against published schemas. |
| Sprint 10 | Benchmarks | bench/TASKS.md | DONE (2025-10-21) | Bench Guild, Language Analyzer Guild | BENCH-SCANNER-10-002 | Wire real language analyzers into bench harness & refresh baselines post-implementation. |
| Sprint 10 | Benchmarks | src/StellaOps.Bench/TASKS.md | DONE (2025-10-21) | Bench Guild, Language Analyzer Guild | BENCH-SCANNER-10-002 | Wire real language analyzers into bench harness & refresh baselines post-implementation. |
| Sprint 10 | Scanner Analyzers & SBOM | src/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-302 | Node analyzer handling workspaces/symlinks emitting `pkg:npm`. |
| Sprint 10 | Scanner Analyzers & SBOM | src/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-21) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-303 | Python analyzer reading `*.dist-info`, RECORD hashes, entry points. |
| Sprint 10 | Scanner Analyzers & SBOM | src/StellaOps.Scanner.Analyzers.Lang/TASKS.md | DONE (2025-10-22) | Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-304 | Go analyzer leveraging buildinfo for `pkg:golang` components. |

34
SPRINTS_PRIOR_20251025.md Normal file
View File

@@ -0,0 +1,34 @@
This file describe implementation of Stella Ops (docs/README.md). Implementation must respect rules from AGENTS.md (read if you have not).
| Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description |
| --- | --- | --- | --- | --- | --- | --- |
| Sprint 11 | Signing Chain Bring-up | src/StellaOps.Attestor/TASKS.md | DONE (2025-10-19) | Attestor Guild | ATTESTOR-API-11-201 | `/rekor/entries` submission pipeline with dedupe, proof acquisition, and persistence. |
| Sprint 11 | Signing Chain Bring-up | src/StellaOps.Attestor/TASKS.md | DONE (2025-10-19) | Attestor Guild | ATTESTOR-VERIFY-11-202 | `/rekor/verify` + retrieval endpoints validating signatures and Merkle proofs. |
| Sprint 11 | Signing Chain Bring-up | src/StellaOps.Attestor/TASKS.md | DONE (2025-10-19) | Attestor Guild | ATTESTOR-OBS-11-203 | Telemetry, alerting, mTLS hardening, and archive workflow for Attestor. |
| Sprint 11 | Storage Platform Hardening | src/StellaOps.Scanner.Storage/TASKS.md | DONE (2025-10-23) | Scanner Storage Guild | SCANNER-STORAGE-11-401 | Migrate scanner object storage integration from MinIO to RustFS with data migration plan. |
| Sprint 11 | UI Integration | src/StellaOps.UI/TASKS.md | DONE (2025-10-23) | UI Guild | UI-ATTEST-11-005 | Attestation visibility (Rekor id, status) on Scan Detail. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Core/TASKS.md | DONE (2025-10-23) | Zastava Core Guild | ZASTAVA-CORE-12-201 | Define runtime event/admission DTOs, hashing helpers, and versioning strategy. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Core/TASKS.md | DONE (2025-10-23) | Zastava Core Guild | ZASTAVA-CORE-12-202 | Provide configuration/logging/metrics utilities shared by Observer/Webhook. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Core/TASKS.md | DONE (2025-10-23) | Zastava Core Guild | ZASTAVA-CORE-12-203 | Authority client helpers, OpTok caching, and security guardrails for runtime services. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Core/TASKS.md | DONE (2025-10-23) | Zastava Core Guild | ZASTAVA-OPS-12-204 | Operational runbooks, alert rules, and dashboard exports for runtime plane. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Observer/TASKS.md | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-001 | Container lifecycle watcher emitting deterministic runtime events with buffering. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Observer/TASKS.md | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-002 | Capture entrypoint traces + loaded libraries, hashing binaries and linking to baseline SBOM. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Observer/TASKS.md | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-003 | Posture checks for signatures/SBOM/attestation with offline caching. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Observer/TASKS.md | DONE (2025-10-24) | Zastava Observer Guild | ZASTAVA-OBS-12-004 | Batch `/runtime/events` submissions with disk-backed buffer and rate limits. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Webhook/TASKS.md | DONE (2025-10-24) | Zastava Webhook Guild | ZASTAVA-WEBHOOK-12-101 | Admission controller host with TLS bootstrap and Authority auth. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Webhook/TASKS.md | DONE (2025-10-24) | Zastava Webhook Guild | ZASTAVA-WEBHOOK-12-102 | Query Scanner `/policy/runtime`, resolve digests, enforce verdicts. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Webhook/TASKS.md | DONE (2025-10-24) | Zastava Webhook Guild | ZASTAVA-WEBHOOK-12-103 | Caching, fail-open/closed toggles, metrics/logging for admission decisions. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Zastava.Webhook/TASKS.md | DONE (2025-10-24) | Zastava Webhook Guild | ZASTAVA-WEBHOOK-12-104 | Wire `/admission` endpoint to runtime policy client and emit allow/deny envelopes. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-302 | `/policy/runtime` endpoint joining SBOM baseline + policy verdict, returning admission guidance. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-303 | Align `/policy/runtime` verdicts with canonical policy evaluation (Feedser/Vexer). |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-304 | Integrate attestation verification into runtime policy metadata. |
| Sprint 12 | Runtime Guardrails | src/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-24) | Scanner WebService Guild | SCANNER-RUNTIME-12-305 | Deliver shared fixtures + e2e validation with Zastava/CLI teams. |
| Sprint 13 | UX & CLI Experience | src/StellaOps.UI/TASKS.md | DONE (2025-10-23) | UI Guild | UI-AUTH-13-001 | Integrate Authority OIDC + DPoP flows with session management. |
| Sprint 13 | UX & CLI Experience | src/StellaOps.UI/TASKS.md | DONE (2025-10-25) | UI Guild | UI-NOTIFY-13-006 | Notify panel: channels/rules CRUD, deliveries view, test send. |
| Sprint 13 | Platform Reliability | ops/devops/TASKS.md | DONE (2025-10-25) | DevOps Guild, Platform Leads | DEVOPS-NUGET-13-001 | Wire up .NET 10 preview feeds/local mirrors so `dotnet restore` succeeds offline; document updated NuGet bootstrap. |
| Sprint 15 | Notify Foundations | src/StellaOps.Notify.Queue/TASKS.md | DONE (2025-10-23) | Notify Queue Guild | NOTIFY-QUEUE-15-401 | Bus abstraction + Redis Streams adapter with ordering/idempotency. |
| Sprint 15 | Notify Foundations | src/StellaOps.Notify.Queue/TASKS.md | DONE (2025-10-23) | Notify Queue Guild | NOTIFY-QUEUE-15-402 | NATS JetStream adapter with health probes and failover. |
| Sprint 15 | Notify Foundations | src/StellaOps.Notify.Queue/TASKS.md | DONE (2025-10-23) | Notify Queue Guild | NOTIFY-QUEUE-15-403 | Delivery queue with retry/dead-letter + metrics. |
| Sprint 15 | Notify Foundations | src/StellaOps.Notify.Worker/TASKS.md | DONE (2025-10-23) | Notify Worker Guild | NOTIFY-WORKER-15-201 | Bus subscription + leasing loop with backoff. |
| Sprint 17 | Symbol Intelligence & Forensics | src/StellaOps.Zastava.Observer/TASKS.md | DONE (2025-10-25) | Zastava Observer Guild | ZASTAVA-OBS-17-005 | Collect GNU build-id during runtime observation and attach it to emitted events. |
| Sprint 17 | Symbol Intelligence & Forensics | src/StellaOps.Scanner.WebService/TASKS.md | DONE (2025-10-25) | Scanner WebService Guild | SCANNER-RUNTIME-17-401 | Persist runtime build-id observations and expose them for debug-symbol correlation. |

8
bench/TASKS.md
View File

@@ -1,8 +0,0 @@
# Benchmarks Task Board
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| BENCH-SCANNER-10-001 | DONE | Bench Guild, Scanner Team | SCANNER-ANALYZERS-LANG-10-303 | Analyzer microbench harness (node_modules, site-packages) + baseline CSV. | Harness committed under `bench/Scanner.Analyzers`; baseline CSV recorded; CI job publishes results. |
| BENCH-SCANNER-10-002 | DONE (2025-10-21) | Bench Guild, Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-301..309 | Wire real language analyzers into bench harness & refresh baselines post-implementation. | Harness executes analyzer assemblies end-to-end; updated baseline committed; CI trend doc linked. |
| BENCH-IMPACT-16-001 | TODO | Bench Guild, Scheduler Team | SCHED-IMPACT-16-301 | ImpactIndex throughput bench (resolve 10k productKeys) + RAM profile. | Benchmark script ready; baseline metrics recorded; alert thresholds defined. |
| BENCH-NOTIFY-15-001 | TODO | Bench Guild, Notify Team | NOTIFY-ENGINE-15-301 | Notify dispatch throughput bench (vary rule density) with results CSV. | Bench executed; results stored; regression alert configured. |

0
dep_tmp.txt
View File

326
docs/12_PERFORMANCE_WORKBOOK.md
View File

@@ -1,170 +1,170 @@
# 12 - Performance Workbook
*Purpose* – define **repeatable, data‑driven** benchmarks that guard Stella Ops’ core pledge:
> *“P95 vulnerability feedback in ≤ 5 seconds.”*
---
## 0 Benchmark Scope
| Area | Included | Excluded |
|------------------|----------------------------------|---------------------------|
| SBOM‑first scan | Trivy engine w/ warmed DB | Full image unpack ≥ 300 MB |
| Delta SBOM ⭑ | Missing‑layer lookup & merge | Multi‑arch images |
| Policy eval ⭑ | YAML → JSON → rule match | Rego (until GA) |
| Feed merge | NVD JSON 2023–2025 | GHSA GraphQL (plugin) |
| Quota wait‑path | 5 s soft‑wait, 60 s hard‑wait behaviour | Paid tiers (unlimited) |
| API latency | REST `/scan`, `/layers/missing` | UI SPA calls |
⭑ = new in July 2025.
---
## 1 Hardware Baseline (Reference Rig)
| Element | Spec |
|-------------|------------------------------------|
| CPU | 8 vCPU (Intel Ice‑Lake equiv.) |
| Memory | 16 GiB |
| Disk | NVMe SSD, 3 GB/s R/W |
| Network | 1 Gbit virt. switch |
| Container | Docker 25.0 + overlay2 |
| OS | Ubuntu 22.04 LTS (kernel 6.8) |
*All P95 targets assume a **single‑node** deployment on this rig unless stated.*
---
## 2 Phase Targets & Gates
| Phase (ID) | Target P95 | Gate (CI) | Rationale |
|-----------------------|-----------:|-----------|----------------------------------------|
| **SBOM_FIRST** | ≤ 5 s | `hard` | Core UX promise. |
| **IMAGE_UNPACK** | ≤ 10 s | `soft` | Fallback path for legacy flows. |
| **DELTA_SBOM** ⭑ | ≤ 1 s | `hard` | Needed to stay sub‑5 s for big bases. |
| **POLICY_EVAL** ⭑ | ≤ 50 ms | `hard` | Keeps gate latency invisible to users. |
| **QUOTA_WAIT** ⭑ | *soft* ≤ 5 s<br>*hard* ≤ 60 s | `hard` | Ensures graceful Free‑tier throttling. |
| **SCHED_RESCAN** | ≤ 30 s | `soft` | Nightly batch – not user‑facing. |
| **FEED_MERGE** | ≤ 60 s | `soft` | Off‑peak cron @ 01:00. |
| **API_P95** | ≤ 200 ms | `hard` | UI snappiness. |
*Gate* legend — `hard`: break CI if regression > 3 × target,
`soft`: raise warning & issue ticket.
---
## 3 Test Harness
# 12 - Performance Workbook
*Purpose* – define **repeatable, data‑driven** benchmarks that guard Stella Ops’ core pledge:
> *“P95 vulnerability feedback in ≤ 5 seconds.”*
---
## 0 Benchmark Scope
| Area | Included | Excluded |
|------------------|----------------------------------|---------------------------|
| SBOM‑first scan | Trivy engine w/ warmed DB | Full image unpack ≥ 300 MB |
| Delta SBOM ⭑ | Missing‑layer lookup & merge | Multi‑arch images |
| Policy eval ⭑ | YAML → JSON → rule match | Rego (until GA) |
| Feed merge | NVD JSON 2023–2025 | GHSA GraphQL (plugin) |
| Quota wait‑path | 5 s soft‑wait, 60 s hard‑wait behaviour | Paid tiers (unlimited) |
| API latency | REST `/scan`, `/layers/missing` | UI SPA calls |
⭑ = new in July 2025.
---
## 1 Hardware Baseline (Reference Rig)
| Element | Spec |
|-------------|------------------------------------|
| CPU | 8 vCPU (Intel Ice‑Lake equiv.) |
| Memory | 16 GiB |
| Disk | NVMe SSD, 3 GB/s R/W |
| Network | 1 Gbit virt. switch |
| Container | Docker 25.0 + overlay2 |
| OS | Ubuntu 22.04 LTS (kernel 6.8) |
*All P95 targets assume a **single‑node** deployment on this rig unless stated.*
---
## 2 Phase Targets & Gates
| Phase (ID) | Target P95 | Gate (CI) | Rationale |
|-----------------------|-----------:|-----------|----------------------------------------|
| **SBOM_FIRST** | ≤ 5 s | `hard` | Core UX promise. |
| **IMAGE_UNPACK** | ≤ 10 s | `soft` | Fallback path for legacy flows. |
| **DELTA_SBOM** ⭑ | ≤ 1 s | `hard` | Needed to stay sub‑5 s for big bases. |
| **POLICY_EVAL** ⭑ | ≤ 50 ms | `hard` | Keeps gate latency invisible to users. |
| **QUOTA_WAIT** ⭑ | *soft* ≤ 5 s<br>*hard* ≤ 60 s | `hard` | Ensures graceful Free‑tier throttling. |
| **SCHED_RESCAN** | ≤ 30 s | `soft` | Nightly batch – not user‑facing. |
| **FEED_MERGE** | ≤ 60 s | `soft` | Off‑peak cron @ 01:00. |
| **API_P95** | ≤ 200 ms | `hard` | UI snappiness. |
*Gate* legend — `hard`: break CI if regression > 3 × target,
`soft`: raise warning & issue ticket.
---
## 3 Test Harness
* **Runner** – `perf/run.sh`, accepts `--phase` and `--samples`.
* **Language analyzers microbench** – `dotnet run --project bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj -- --repo-root . --out bench/Scanner.Analyzers/baseline.csv --json out/bench/scanner-analyzers/latest.json --prom out/bench/scanner-analyzers/latest.prom --commit $(git rev-parse HEAD)` produces CSV + JSON + Prometheus gauges for analyzer scenarios. Runs fail if `max_ms` regresses ≥ 20 % against `baseline.csv` or if thresholds are exceeded.
* **Language analyzers microbench** – `dotnet run --project src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj -- --repo-root . --out src/StellaOps.Bench/Scanner.Analyzers/baseline.csv --json out/bench/scanner-analyzers/latest.json --prom out/bench/scanner-analyzers/latest.prom --commit $(git rev-parse HEAD)` produces CSV + JSON + Prometheus gauges for analyzer scenarios. Runs fail if `max_ms` regresses ≥ 20 % against `baseline.csv` or if thresholds are exceeded.
* **Metrics** – Prometheus + `jq` extracts; aggregated via `scripts/aggregate.ts`.
* **CI** – GitLab CI job *benchmark* publishes JSON to `bench‑artifacts/`.
* **Visualisation** – Grafana dashboard *Stella‑Perf* (provisioned JSON).
> **Note** – harness mounts `/var/cache/trivy` tmpfs to avoid disk noise.
---
## 4 Current Results (July 2025)
| Phase | Samples | Mean (s) | P95 (s) | Target OK? |
|---------------|--------:|---------:|--------:|-----------:|
| SBOM_FIRST | 100 | 3.7 | 4.9 | ✅ |
| IMAGE_UNPACK | 50 | 6.4 | 9.2 | ✅ |
| **DELTA_SBOM**| 100 | 0.46 | 0.83 | ✅ |
| **POLICY_EVAL** | 1 000 | 0.021 | 0.041 | ✅ |
| **QUOTA_WAIT** | 80 | 4.0* | 4.9* | ✅ |
| SCHED_RESCAN | 10 | 18.3 | 24.9 | ✅ |
| FEED_MERGE | 3 | 38.1 | 41.0 | ✅ |
| API_P95 | 20 000 | 0.087 | 0.143 | ✅ |
*Data files:* `bench-artifacts/2025‑07‑14/phase‑stats.json`.
---
## 5 Δ‑SBOM Micro‑Benchmark Detail
### 5.1 Scenario
1. Base image `python:3.12-slim` already scanned (all layers cached).
2. Application layer (`COPY . /app`) triggers new digest.
3. `Stella CLI` lists **7** layers, backend replies *6 hit*, *1 miss*.
4. Builder scans **only 1 layer** (~9 MiB, 217 files) & uploads delta.
### 5.2 Key Timings
| Step | Time (ms) |
|---------------------|----------:|
| `/layers/missing` | 13 |
| Trivy single layer | 655 |
| Upload delta blob | 88 |
| Backend merge + CVE | 74 |
| **Total wall‑time** | **830 ms** |
---
## 6 Quota Wait‑Path Benchmark Detail
### 6.1 Scenario
1. Free‑tier token reaches **scan #200** – dashboard shows yellow banner.
### 6.2 Key Timings
| Step | Time (ms) |
|------------------------------------|----------:|
| `/quota/check` Redis LUA INCR | 0.8 |
| Soft wait sleep (server) | 5 000 |
| Hard wait sleep (server) | 60 000 |
| End‑to‑end wall‑time (soft‑hit) | 5 003 |
| End‑to‑end wall‑time (hard‑hit) | 60 004 |
---
## 7 Policy Eval Bench
### 7.1 Setup
* Policy YAML: **28** rules, mix severity & package conditions.
* Input: scan result JSON with **1 026** findings.
* Evaluator: custom rules engine (Go structs → map look‑ups).
### 7.2 Latency Histogram
```
0‑10 ms ▇▇▇▇▇▇▇▇▇▇ 38 %
10‑20 ms ▇▇▇▇▇▇▇▇▇▇ 42 %
20‑40 ms ▇▇▇▇▇▇ 17 %
40‑50 ms ▇ 3 %
```
P99 = 48 ms. Meets 50 ms gate.
---
## 8 Trend Snapshot
> **Note** – harness mounts `/var/cache/trivy` tmpfs to avoid disk noise.
---
## 4 Current Results (July 2025)
| Phase | Samples | Mean (s) | P95 (s) | Target OK? |
|---------------|--------:|---------:|--------:|-----------:|
| SBOM_FIRST | 100 | 3.7 | 4.9 | ✅ |
| IMAGE_UNPACK | 50 | 6.4 | 9.2 | ✅ |
| **DELTA_SBOM**| 100 | 0.46 | 0.83 | ✅ |
| **POLICY_EVAL** | 1 000 | 0.021 | 0.041 | ✅ |
| **QUOTA_WAIT** | 80 | 4.0* | 4.9* | ✅ |
| SCHED_RESCAN | 10 | 18.3 | 24.9 | ✅ |
| FEED_MERGE | 3 | 38.1 | 41.0 | ✅ |
| API_P95 | 20 000 | 0.087 | 0.143 | ✅ |
*Data files:* `bench-artifacts/2025‑07‑14/phase‑stats.json`.
---
## 5 Δ‑SBOM Micro‑Benchmark Detail
### 5.1 Scenario
1. Base image `python:3.12-slim` already scanned (all layers cached).
2. Application layer (`COPY . /app`) triggers new digest.
3. `Stella CLI` lists **7** layers, backend replies *6 hit*, *1 miss*.
4. Builder scans **only 1 layer** (~9 MiB, 217 files) & uploads delta.
### 5.2 Key Timings
| Step | Time (ms) |
|---------------------|----------:|
| `/layers/missing` | 13 |
| Trivy single layer | 655 |
| Upload delta blob | 88 |
| Backend merge + CVE | 74 |
| **Total wall‑time** | **830 ms** |
---
## 6 Quota Wait‑Path Benchmark Detail
### 6.1 Scenario
1. Free‑tier token reaches **scan #200** – dashboard shows yellow banner.
### 6.2 Key Timings
| Step | Time (ms) |
|------------------------------------|----------:|
| `/quota/check` Redis LUA INCR | 0.8 |
| Soft wait sleep (server) | 5 000 |
| Hard wait sleep (server) | 60 000 |
| End‑to‑end wall‑time (soft‑hit) | 5 003 |
| End‑to‑end wall‑time (hard‑hit) | 60 004 |
---
## 7 Policy Eval Bench
### 7.1 Setup
* Policy YAML: **28** rules, mix severity & package conditions.
* Input: scan result JSON with **1 026** findings.
* Evaluator: custom rules engine (Go structs → map look‑ups).
### 7.2 Latency Histogram
```
0‑10 ms ▇▇▇▇▇▇▇▇▇▇ 38 %
10‑20 ms ▇▇▇▇▇▇▇▇▇▇ 42 %
20‑40 ms ▇▇▇▇▇▇ 17 %
40‑50 ms ▇ 3 %
```
P99 = 48 ms. Meets 50 ms gate.
---
## 8 Trend Snapshot
![Perf trend spark‑line placeholder](perf‑trend.svg)
> **Grafana/Alerting** – Import `docs/ops/scanner-analyzers-grafana-dashboard.json` and point it at the Prometheus datasource storing `scanner_analyzer_bench_*` metrics. Configure an alert on `scanner_analyzer_bench_regression_ratio` ≥ 1.20 (default limit); the bundled Stat panel surfaces breached scenarios (non-zero values). On-call runbook: `docs/ops/scanner-analyzers-operations.md`.
_Plot generated weekly by `scripts/update‑trend.py`; shows last 12 weeks P95 per phase._
---
## 9 Action Items
1. **Image Unpack** – Evaluate zstd for layer decompress; aim to shave 1 s.
2. **Feed Merge** – Parallelise regional XML feed parse (plugin) once stable.
3. **Rego Support** – Prototype OPA side‑car; target ≤ 100 ms eval.
4. **Concurrency** – Stress‑test 100 rps on 4‑node Redis cluster (Q4‑2025).
---
## 10 Change Log
| Date | Note |
|------------|-------------------------------------------------------------------------|
| 2025‑07‑14 | Added Δ‑SBOM & Policy Eval phases; updated targets & current results. |
| 2025‑07‑12 | First public workbook (SBOM‑first, image‑unpack, feed merge). |
---
_Plot generated weekly by `scripts/update‑trend.py`; shows last 12 weeks P95 per phase._
---
## 9 Action Items
1. **Image Unpack** – Evaluate zstd for layer decompress; aim to shave 1 s.
2. **Feed Merge** – Parallelise regional XML feed parse (plugin) once stable.
3. **Rego Support** – Prototype OPA side‑car; target ≤ 100 ms eval.
4. **Concurrency** – Stress‑test 100 rps on 4‑node Redis cluster (Q4‑2025).
---
## 10 Change Log
| Date | Note |
|------------|-------------------------------------------------------------------------|
| 2025‑07‑14 | Added Δ‑SBOM & Policy Eval phases; updated targets & current results. |
| 2025‑07‑12 | First public workbook (SBOM‑first, image‑unpack, feed merge). |
---

144
docs/README.md
View File

@@ -1,36 +1,36 @@
# Stella Ops
> **Self‑hosted, SBOM‑first DevSecOps platform – offline‑friendly, AGPL‑3.0, free up to {{ quota_token }} scans per UTC day (soft delay only, never blocks).**
Stella Ops lets you discover container vulnerabilities in **< 5 s** without sending a single byte outside your network.
Everything here is open‑source and versioned — when you check out a git tag, the docs match the code you are running.
---
## 🚀 Start here (first 60 minutes)
| Step | What you will learn | Doc |
|------|--------------------|-----|
| 1 ️⃣ | 90‑second elevator pitch & pillars | **[What Is Stella Ops?](01_WHAT_IS_IT.md)** |
| 2 ️⃣ | Pain points it solves | **[Why Does It Exist?](02_WHY.md)** |
| 3 ️⃣ | Install & run a scan in 10 min | **[Install Guide](21_INSTALL_GUIDE.md)** |
| 4 ️⃣ | Components & data‑flow | **[High‑Level Architecture](07_HIGH_LEVEL_ARCHITECTURE.md)** |
| 5 ️⃣ | Integrate the CLI / REST API | **[API & CLI Reference](09_API_CLI_REFERENCE.md)** |
| 6 ️⃣ | Vocabulary used throughout the docs | **[Glossary](14_GLOSSARY_OF_TERMS.md)** |
---
## 📚 Complete Table of Contents
<details>
<summary>Click to expand the full docs index</summary>
### Overview
- **01 – [What Is Stella Ops?](01_WHAT_IS_IT.md)**
- **02 – [Why Does It Exist?](02_WHY.md)**
- **03 – [Vision & Road‑map](03_VISION.md)**
- **04 – [Feature Matrix](04_FEATURE_MATRIX.md)**
# Stella Ops
> **Self‑hosted, SBOM‑first DevSecOps platform – offline‑friendly, AGPL‑3.0, free up to {{ quota_token }} scans per UTC day (soft delay only, never blocks).**
Stella Ops lets you discover container vulnerabilities in **< 5 s** without sending a single byte outside your network.
Everything here is open‑source and versioned — when you check out a git tag, the docs match the code you are running.
---
## 🚀 Start here (first 60 minutes)
| Step | What you will learn | Doc |
|------|--------------------|-----|
| 1 ️⃣ | 90‑second elevator pitch & pillars | **[What Is Stella Ops?](01_WHAT_IS_IT.md)** |
| 2 ️⃣ | Pain points it solves | **[Why Does It Exist?](02_WHY.md)** |
| 3 ️⃣ | Install & run a scan in 10 min | **[Install Guide](21_INSTALL_GUIDE.md)** |
| 4 ️⃣ | Components & data‑flow | **[High‑Level Architecture](07_HIGH_LEVEL_ARCHITECTURE.md)** |
| 5 ️⃣ | Integrate the CLI / REST API | **[API & CLI Reference](09_API_CLI_REFERENCE.md)** |
| 6 ️⃣ | Vocabulary used throughout the docs | **[Glossary](14_GLOSSARY_OF_TERMS.md)** |
---
## 📚 Complete Table of Contents
<details>
<summary>Click to expand the full docs index</summary>
### Overview
- **01 – [What Is Stella Ops?](01_WHAT_IS_IT.md)**
- **02 – [Why Does It Exist?](02_WHY.md)**
- **03 – [Vision & Road‑map](03_VISION.md)**
- **04 – [Feature Matrix](04_FEATURE_MATRIX.md)**
### Reference & concepts
- **05 – [System Requirements Specification](05_SYSTEM_REQUIREMENTS_SPEC.md)**
- **07 – [High‑Level Architecture](07_HIGH_LEVEL_ARCHITECTURE.md)**
@@ -40,38 +40,38 @@ Everything here is open‑source and versioned — when you check out a git ta
- [Concelier](ARCHITECTURE_CONCELIER.md)
- [Excititor](ARCHITECTURE_EXCITITOR.md)
- [Excititor Mirrors](ARCHITECTURE_EXCITITOR_MIRRORS.md)
- [Signer](ARCHITECTURE_SIGNER.md)
- [Attestor](ARCHITECTURE_ATTESTOR.md)
- [Authority](ARCHITECTURE_AUTHORITY.md)
- [Notify](ARCHITECTURE_NOTIFY.md)
- [Scheduler](ARCHITECTURE_SCHEDULER.md)
- [CLI](ARCHITECTURE_CLI.md)
- [Web UI](ARCHITECTURE_UI.md)
- [Zastava Runtime](ARCHITECTURE_ZASTAVA.md)
- [Release & Operations](ARCHITECTURE_DEVOPS.md)
- **09 – [API & CLI Reference](09_API_CLI_REFERENCE.md)**
- [Signer](ARCHITECTURE_SIGNER.md)
- [Attestor](ARCHITECTURE_ATTESTOR.md)
- [Authority](ARCHITECTURE_AUTHORITY.md)
- [Notify](ARCHITECTURE_NOTIFY.md)
- [Scheduler](ARCHITECTURE_SCHEDULER.md)
- [CLI](ARCHITECTURE_CLI.md)
- [Web UI](ARCHITECTURE_UI.md)
- [Zastava Runtime](ARCHITECTURE_ZASTAVA.md)
- [Release & Operations](ARCHITECTURE_DEVOPS.md)
- **09 – [API & CLI Reference](09_API_CLI_REFERENCE.md)**
- **10 – [Plug‑in SDK Guide](10_PLUGIN_SDK_GUIDE.md)**
- **10 – [Concelier CLI Quickstart](10_CONCELIER_CLI_QUICKSTART.md)**
- **10 – [BuildX Generator Quickstart](dev/BUILDX_PLUGIN_QUICKSTART.md)**
- **10 – [Scanner Cache Configuration](dev/SCANNER_CACHE_CONFIGURATION.md)**
- **30 – [Excititor Connector Packaging Guide](dev/30_EXCITITOR_CONNECTOR_GUIDE.md)**
- **30 – Developer Templates**
- [Excititor Connector Skeleton](dev/templates/excititor-connector/)
- **11 – [Authority Service](11_AUTHORITY.md)**
- **11 – [Data Schemas](11_DATA_SCHEMAS.md)**
- **12 – [Performance Workbook](12_PERFORMANCE_WORKBOOK.md)**
- **13 – [Release‑Engineering Playbook](13_RELEASE_ENGINEERING_PLAYBOOK.md)**
- **30 – [Fixture Maintenance](dev/fixtures.md)**
### User & operator guides
- **14 – [Glossary](14_GLOSSARY_OF_TERMS.md)**
- **15 – [UI Guide](15_UI_GUIDE.md)**
- **17 – [Security Hardening Guide](17_SECURITY_HARDENING_GUIDE.md)**
- **18 – [Coding Standards](18_CODING_STANDARDS.md)**
- **19 – [Test‑Suite Overview](19_TEST_SUITE_OVERVIEW.md)**
- **21 – [Install Guide](21_INSTALL_GUIDE.md)**
- **22 – [CI/CD Recipes Library](ci/20_CI_RECIPES.md)**
- **23 – [FAQ](23_FAQ_MATRIX.md)**
- **30 – [Excititor Connector Packaging Guide](dev/30_EXCITITOR_CONNECTOR_GUIDE.md)**
- **30 – Developer Templates**
- [Excititor Connector Skeleton](dev/templates/excititor-connector/)
- **11 – [Authority Service](11_AUTHORITY.md)**
- **11 – [Data Schemas](11_DATA_SCHEMAS.md)**
- **12 – [Performance Workbook](12_PERFORMANCE_WORKBOOK.md)**
- **13 – [Release‑Engineering Playbook](13_RELEASE_ENGINEERING_PLAYBOOK.md)**
- **30 – [Fixture Maintenance](dev/fixtures.md)**
### User & operator guides
- **14 – [Glossary](14_GLOSSARY_OF_TERMS.md)**
- **15 – [UI Guide](15_UI_GUIDE.md)**
- **17 – [Security Hardening Guide](17_SECURITY_HARDENING_GUIDE.md)**
- **18 – [Coding Standards](18_CODING_STANDARDS.md)**
- **19 – [Test‑Suite Overview](19_TEST_SUITE_OVERVIEW.md)**
- **21 – [Install Guide](21_INSTALL_GUIDE.md)**
- **22 – [CI/CD Recipes Library](ci/20_CI_RECIPES.md)**
- **23 – [FAQ](23_FAQ_MATRIX.md)**
- **24 – [Offline Update Kit Admin Guide](24_OFFLINE_KIT.md)**
- **25 – [Mirror Operations Runbook](ops/concelier-mirror-operations.md)**
- **26 – [Concelier Apple Connector Operations](ops/concelier-apple-operations.md)**
@@ -86,9 +86,19 @@ Everything here is open‑source and versioned — when you check out a git ta
### Legal & licence
- **32 – [Legal & Quota FAQ](29_LEGAL_FAQ_QUOTA.md)**
</details>
---
© 2025 Stella Ops contributors – licensed AGPL‑3.0‑or‑later
</details>
---
## 🧹 Backlog hygiene
> Imposed rule: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
- **Aggregation-Only Contract (AOC).** Ingestion services aggregate and link facts only—derived precedence, severity, and safe-fix hints live in Policy overlays and dedicated explorers. Review [`../AGENTS.md`](../AGENTS.md) and the AOC guardrails in [`aoc/aoc-guardrails.md`](aoc/aoc-guardrails.md).
- **Cartographer owns graphs.** SBOM Service emits projections/events; Cartographer (`CARTO-GRAPH-21-00x`) builds graph storage, overlays, and tiles. See `ARCHITECTURE_CONCELIER.md` (Cartographer handshake section) for handoff boundaries.
- **Notifier replaces legacy Notify.** Sprint‑15 `StellaOps.Notify.*` tasks are frozen; use the Notifications Studio/Notifier backlogs (`NOTIFY-SVC-38..40`, `WEB-NOTIFY-3x-00x`, `CLI-NOTIFY-3x-00x`).
- **Dedicated services for Vuln & Policy.** Vuln Explorer work flows through `src/StellaOps.VulnExplorer.Api`/Console/CLI (Sprint 29); gateway routes proxy only. Policy Engine remains the sole source for precedence/suppression overlays.
- **Cleanup log.** The backlog consolidation summary lives in [`backlog/2025-10-cleanup.md`](backlog/2025-10-cleanup.md).
© 2025 Stella Ops contributors – licensed AGPL‑3.0‑or‑later

323
docs/TASKS.md
View File

@@ -17,7 +17,330 @@
| RUNTIME-GUILD-09-402 | DONE (2025-10-19) | Runtime Guild | SCANNER-POLICY-09-107 | Confirm Scanner WebService surfaces `quietedFindingCount` and progress hints to runtime consumers; document readiness checklist. | Runtime verification run captures enriched payload; checklist/doc updates merged; stakeholders acknowledge availability. |
| DOCS-CONCELIER-07-201 | DONE (2025-10-22) | Docs Guild, Concelier WebService | FEEDWEB-DOCS-01-001 | Final editorial review and publish pass for Concelier authority toggle documentation (Quickstart + operator guide). | Review feedback resolved, publish PR merged, release notes updated with documentation pointer. |
| DOCS-RUNTIME-17-004 | TODO | Docs Guild, Runtime Guild | SCANNER-EMIT-17-701, ZASTAVA-OBS-17-005, DEVOPS-REL-17-002 | Document build-id workflows: SBOM exposure, runtime event payloads (`process.buildId`), Scanner `/policy/runtime` response (`buildIds` list), debug-store layout, and operator guidance for symbol retrieval. | Architecture + operator docs updated with build-id sections (Observer, Scanner, CLI), examples show `readelf` output + debuginfod usage, references linked from Offline Kit/Release guides + CLI help. |
| DOCS-OBS-50-001 | TODO | Docs Guild, Observability Guild | TELEMETRY-OBS-50-001 | Publish `/docs/observability/overview.md` introducing scope, imposed rule banner, architecture diagram, and tenant guarantees. | Doc merged with imposed rule banner; diagram committed; cross-links to telemetry stack + evidence locker docs. |
| DOCS-OBS-50-002 | TODO | Docs Guild, Security Guild | TELEMETRY-OBS-50-002 | Author `/docs/observability/telemetry-standards.md` detailing common fields, scrubbing policy, sampling defaults, and redaction override procedure. | Doc merged; imposed rule banner present; examples validated with telemetry fixtures; security review sign-off captured. |
| DOCS-OBS-50-003 | TODO | Docs Guild, Observability Guild | TELEMETRY-OBS-50-001 | Create `/docs/observability/logging.md` covering structured log schema, dos/don'ts, tenant isolation, and copyable examples. | Doc merged with banner; sample logs redacted; lint passes; linked from coding standards. |
| DOCS-OBS-50-004 | TODO | Docs Guild, Observability Guild | TELEMETRY-OBS-50-002 | Draft `/docs/observability/tracing.md` explaining context propagation, async linking, CLI header usage, and sampling strategies. | Doc merged; imposed rule banner included; diagrams updated; references to CLI/Console features added. |
| DOCS-OBS-51-001 | TODO | Docs Guild, DevOps Guild | WEB-OBS-51-001, DEVOPS-OBS-51-001 | Publish `/docs/observability/metrics-and-slos.md` cataloging metrics, SLO targets, burn rate policies, and alert runbooks. | Doc merged with banner; SLO tables verified; alert workflows linked to incident runbook. |
| DOCS-SEC-OBS-50-001 | TODO | Docs Guild, Security Guild | TELEMETRY-OBS-51-002 | Update `/docs/security/redaction-and-privacy.md` to cover telemetry privacy controls, tenant opt-in debug, and imposed rule reminder. | Doc merged; redaction matrix updated; banner present; security sign-off recorded. |
| DOCS-INSTALL-50-001 | TODO | Docs Guild, DevOps Guild | DEVOPS-OBS-50-003 | Add `/docs/install/telemetry-stack.md` with collector deployment, exporter options, offline kit notes, and imposed rule banner. | Doc merged; install steps verified on air-gapped profile; banner present; screenshots attached. |
| DOCS-FORENSICS-53-001 | TODO | Docs Guild, Evidence Locker Guild | EVID-OBS-53-003 | Publish `/docs/forensics/evidence-locker.md` describing bundle formats, WORM options, retention, legal hold, and imposed rule banner. | Doc merged; manifest examples validated; banner present; legal hold steps aligned with API. |
| DOCS-FORENSICS-53-002 | TODO | Docs Guild, Provenance Guild | PROV-OBS-54-001 | Release `/docs/forensics/provenance-attestation.md` covering DSSE schema, signing process, verification workflow, and imposed rule banner. | Doc merged; sample statements reference fixtures; banner included; verification steps tested. |
| DOCS-FORENSICS-53-003 | TODO | Docs Guild, Timeline Indexer Guild | TIMELINE-OBS-52-003 | Publish `/docs/forensics/timeline.md` with schema, event kinds, filters, query examples, and imposed rule banner. | Doc merged; query examples validated; banner present; linked from Console/CLI docs. |
| DOCS-CONSOLE-OBS-52-001 | TODO | Docs Guild, Console Guild | CONSOLE-OBS-51-001 | Document `/docs/console/observability.md` showcasing Observability Hub widgets, trace/log search, imposed rule banner, and accessibility tips. | Doc merged; screenshots updated; banner present; navigation steps verified. |
| DOCS-CONSOLE-OBS-52-002 | TODO | Docs Guild, Console Guild | CONSOLE-OBS-52-002, CONSOLE-OBS-53-001 | Publish `/docs/console/forensics.md` covering timeline explorer, evidence viewer, attestation verifier, imposed rule banner, and troubleshooting. | Doc merged; banner included; workflows validated via Playwright capture; troubleshooting section populated. |
| DOCS-CLI-OBS-52-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-OBS-52-001 | Create `/docs/cli/observability.md` detailing `stella obs` commands, examples, exit codes, imposed rule banner, and scripting tips. | Doc merged; examples tested; banner included; CLI parity matrix updated. |
| DOCS-CLI-FORENSICS-53-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-FORENSICS-54-001 | Publish `/docs/cli/forensics.md` for snapshot/verify/attest commands with sample outputs, imposed rule banner, and offline workflows. | Doc merged; sample bundles verified; banner present; offline notes cross-linked. |
| DOCS-RUNBOOK-55-001 | TODO | Docs Guild, Ops Guild | DEVOPS-OBS-55-001, WEB-OBS-55-001 | Author `/docs/runbooks/incidents.md` describing incident mode activation, escalation steps, retention impact, verification checklist, and imposed rule banner. | Doc merged; runbook rehearsed; banner included; linked from alerts. |
| DOCS-AOC-19-001 | TODO | Docs Guild, Concelier Guild | CONCELIER-WEB-AOC-19-001, EXCITITOR-WEB-AOC-19-001 | Author `/docs/ingestion/aggregation-only-contract.md` covering philosophy, invariants, schemas, error codes, migration, observability, and security checklist. | New doc published with compliance checklist; cross-links from existing docs added. |
| DOCS-AOC-19-002 | TODO | Docs Guild, Architecture Guild | DOCS-AOC-19-001 | Update `/docs/architecture/overview.md` to include AOC boundary, raw stores, and sequence diagram (fetch → guard → raw insert → policy evaluation). | Overview doc updated with diagrams/text; lint passes; stakeholders sign off. |
| DOCS-AOC-19-003 | TODO | Docs Guild, Policy Guild | POLICY-AOC-19-003 | Refresh `/docs/architecture/policy-engine.md` clarifying ingestion boundary, raw inputs, and policy-only derived data. | Doc highlights raw-only ingestion contract, updated diagrams merge, compliance checklist added. |
| DOCS-AOC-19-004 | TODO | Docs Guild, UI Guild | UI-AOC-19-001 | Extend `/docs/ui/console.md` with Sources dashboard tiles, violation drill-down workflow, and verification action. | UI doc updated with screenshots/flow descriptions, compliance checklist appended. |
| DOCS-AOC-19-005 | TODO | Docs Guild, CLI Guild | CLI-AOC-19-003 | Update `/docs/cli/cli-reference.md` with `stella sources ingest --dry-run` and `stella aoc verify` usage, exit codes, and offline notes. | CLI reference + quickstart sections updated; examples validated; compliance checklist added. |
| DOCS-AOC-19-006 | TODO | Docs Guild, Observability Guild | CONCELIER-WEB-AOC-19-002, EXCITITOR-WEB-AOC-19-002 | Document new metrics/traces/log keys in `/docs/observability/observability.md`. | Observability doc lists new metrics/traces/log fields; dashboards referenced; compliance checklist appended. |
| DOCS-AOC-19-007 | TODO | Docs Guild, Authority Core | AUTH-AOC-19-001 | Update `/docs/security/authority-scopes.md` with new ingestion scopes and tenancy enforcement notes. | Doc reflects new scopes, sample policies updated, compliance checklist added. |
| DOCS-AOC-19-008 | TODO | Docs Guild, DevOps Guild | DEVOPS-AOC-19-002 | Refresh `/docs/deploy/containers.md` to cover validator enablement, guard env flags, and read-only verify user. | Deploy doc updated; offline kit section mentions validator scripts; compliance checklist appended. |
## Air-Gapped Mode (Epic 16)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-AIRGAP-56-001 | TODO | Docs Guild, AirGap Controller Guild | AIRGAP-CTL-56-002 | Publish `/docs/airgap/overview.md` outlining modes, lifecycle, responsibilities, and imposed rule banner. | Doc merged; banner present; diagrams included. |
| DOCS-AIRGAP-56-002 | TODO | Docs Guild, DevOps Guild | DEVOPS-AIRGAP-56-001 | Author `/docs/airgap/sealing-and-egress.md` covering network policies, EgressPolicy facade usage, and verification steps. | Doc merged; examples validated; banner included. |
| DOCS-AIRGAP-56-003 | TODO | Docs Guild, Exporter Guild | EXPORT-AIRGAP-56-001 | Create `/docs/airgap/mirror-bundles.md` describing bundle format, DSSE/TUF/Merkle validation, creation/import workflows. | Doc merged; sample commands verified; banner present. |
| DOCS-AIRGAP-56-004 | TODO | Docs Guild, Deployment Guild | DEVOPS-AIRGAP-56-003 | Publish `/docs/airgap/bootstrap.md` detailing Bootstrap Pack creation, validation, and install procedures. | Doc merged; checklist appended; screenshots verified. |
| DOCS-AIRGAP-57-001 | TODO | Docs Guild, AirGap Time Guild | AIRGAP-TIME-58-001 | Write `/docs/airgap/staleness-and-time.md` explaining time anchors, drift policies, staleness budgets, and UI indicators. | Doc merged; math checked; banner included. |
| DOCS-AIRGAP-57-002 | TODO | Docs Guild, Console Guild | CONSOLE-AIRGAP-57-001 | Publish `/docs/console/airgap.md` covering sealed badge, import wizard, staleness dashboards. | Doc merged; screenshots captured; banner present. |
| DOCS-AIRGAP-57-003 | TODO | Docs Guild, CLI Guild | CLI-AIRGAP-57-001 | Publish `/docs/cli/airgap.md` documenting commands, examples, exit codes. | Doc merged; examples validated; banner present. |
| DOCS-AIRGAP-57-004 | TODO | Docs Guild, Ops Guild | DEVOPS-AIRGAP-56-002 | Create `/docs/airgap/operations.md` with runbooks for imports, failure recovery, and auditing. | Doc merged; runbooks rehearsed; banner included. |
| DOCS-AIRGAP-58-001 | TODO | Docs Guild, Product Guild | CONSOLE-AIRGAP-58-002 | Provide `/docs/airgap/degradation-matrix.md` enumerating feature availability, fallbacks, remediation. | Doc merged; matrix reviewed; banner included. |
| DOCS-AIRGAP-58-002 | TODO | Docs Guild, Security Guild | PROV-OBS-54-001 | Update `/docs/security/trust-and-signing.md` with DSSE/TUF roots, rotation, and signed time tokens. | Doc merged; security sign-off recorded; banner present. |
| DOCS-AIRGAP-58-003 | TODO | Docs Guild, DevEx Guild | AIRGAP-POL-56-001 | Publish `/docs/dev/airgap-contracts.md` describing EgressPolicy usage, sealed-mode tests, linting. | Doc merged; sample code validated; banner included. |
| DOCS-AIRGAP-58-004 | TODO | Docs Guild, Evidence Locker Guild | EVID-OBS-55-001 | Document `/docs/airgap/portable-evidence.md` for exporting/importing portable evidence bundles across enclaves. | Doc merged; verification steps tested; banner present. |
## SDKs & OpenAPI (Epic 17)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-OAS-61-001 | TODO | Docs Guild, API Contracts Guild | OAS-61-002 | Publish `/docs/api/overview.md` covering auth, tenancy, pagination, idempotency, rate limits with banner. | Doc merged; examples validated; banner present. |
| DOCS-OAS-61-002 | TODO | Docs Guild, API Governance Guild | APIGOV-61-001 | Author `/docs/api/conventions.md` capturing naming, errors, filters, sorting, examples. | Doc merged; lint passes; banner included. |
| DOCS-OAS-61-003 | TODO | Docs Guild, API Governance Guild | APIGOV-63-001 | Publish `/docs/api/versioning.md` describing SemVer, deprecation headers, migration playbooks. | Doc merged; example headers validated; banner present. |
| DOCS-OAS-62-001 | TODO | Docs Guild, Developer Portal Guild | DEVPORT-62-002 | Stand up `/docs/api/reference/` auto-generated site; integrate with portal nav. | Reference site builds; search works; banner included. |
| DOCS-SDK-62-001 | TODO | Docs Guild, SDK Generator Guild | SDKGEN-63-001 | Publish `/docs/sdks/overview.md` plus language guides (`typescript.md`, `python.md`, `go.md`, `java.md`). | Docs merged; code samples pulled from tested examples; banner present. |
| DOCS-DEVPORT-62-001 | TODO | Docs Guild, Developer Portal Guild | DEVPORT-62-001 | Document `/docs/devportal/publishing.md` for build pipeline, offline bundle steps. | Doc merged; cross-links validated; banner included. |
| DOCS-CONTRIB-62-001 | TODO | Docs Guild, API Governance Guild | APIGOV-61-001 | Publish `/docs/contributing/api-contracts.md` detailing how to edit OAS, lint rules, compatibility checks. | Doc merged; banner present; examples validated. |
| DOCS-TEST-62-001 | TODO | Docs Guild, Contract Testing Guild | CONTR-62-001 | Author `/docs/testing/contract-testing.md` covering mock server, replay tests, golden fixtures. | Doc merged; references to tooling validated; banner present. |
| DOCS-SEC-62-001 | TODO | Docs Guild, Authority Core | AUTH-AIRGAP-56-001 | Update `/docs/security/auth-scopes.md` with OAuth2/PAT scopes, tenancy header usage. | Doc merged; scope tables verified; banner included. |
| DOCS-AIRGAP-DEVPORT-64-001 | TODO | Docs Guild, DevPortal Offline Guild | DVOFF-64-001 | Create `/docs/airgap/devportal-offline.md` describing offline bundle usage and verification. | Doc merged; verification steps tested; banner present. |
## Risk Profiles (Epic 18)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-RISK-66-001 | TODO | Docs Guild, Risk Profile Schema Guild | POLICY-RISK-66-001 | Publish `/docs/risk/overview.md` covering concepts and glossary. | Doc merged with banner; terminology reviewed. |
| DOCS-RISK-66-002 | TODO | Docs Guild, Policy Guild | POLICY-RISK-66-003 | Author `/docs/risk/profiles.md` (authoring, versioning, scope). | Doc merged; schema examples validated; banner present. |
| DOCS-RISK-66-003 | TODO | Docs Guild, Risk Engine Guild | RISK-ENGINE-67-001 | Publish `/docs/risk/factors.md` cataloging signals, transforms, reducers, TTLs. | Document merged; tables verified; banner included. |
| DOCS-RISK-66-004 | TODO | Docs Guild, Risk Engine Guild | RISK-ENGINE-66-002 | Create `/docs/risk/formulas.md` detailing math, normalization, gating, severity. | Doc merged; equations rendered; banner present. |
| DOCS-RISK-67-001 | TODO | Docs Guild, Risk Engine Guild | RISK-ENGINE-68-001 | Publish `/docs/risk/explainability.md` showing artifact schema and UI screenshots. | Doc merged; CLI examples validated; banner included. |
| DOCS-RISK-67-002 | TODO | Docs Guild, API Guild | POLICY-RISK-67-002 | Produce `/docs/risk/api.md` with endpoint reference/examples. | Doc merged; OAS examples synced; banner present. |
| DOCS-RISK-67-003 | TODO | Docs Guild, Console Guild | CONSOLE-RISK-66-001 | Document `/docs/console/risk-ui.md` for authoring, simulation, dashboards. | Doc merged; screenshots updated; banner included. |
| DOCS-RISK-67-004 | TODO | Docs Guild, CLI Guild | CLI-RISK-66-001 | Publish `/docs/cli/risk.md` covering CLI workflows. | Doc merged; command examples validated; banner present. |
| DOCS-RISK-68-001 | TODO | Docs Guild, Export Guild | RISK-BUNDLE-69-001 | Add `/docs/airgap/risk-bundles.md` for offline factor bundles. | Doc merged; verification steps confirmed; banner included. |
| DOCS-RISK-68-002 | TODO | Docs Guild, Security Guild | POLICY-RISK-66-003 | Update `/docs/security/aoc-invariants.md` with risk scoring provenance guarantees. | Doc merged; audit references updated; banner present. |
## Attestor Console (Epic 19)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-ATTEST-73-001 | TODO | Docs Guild, Attestor Service Guild | ATTEST-TYPES-73-001 | Publish `/docs/attestor/overview.md` with imposed rule banner. | Doc merged; terminology validated. |
| DOCS-ATTEST-73-002 | TODO | Docs Guild, Attestation Payloads Guild | ATTEST-TYPES-73-002 | Write `/docs/attestor/payloads.md` with schemas/examples. | Doc merged; examples validated via tests. |
| DOCS-ATTEST-73-003 | TODO | Docs Guild, Policy Guild | POLICY-ATTEST-73-002 | Publish `/docs/attestor/policies.md` covering verification policies. | Doc merged; policy examples validated. |
| DOCS-ATTEST-73-004 | TODO | Docs Guild, Attestor Service Guild | ATTESTOR-73-002 | Add `/docs/attestor/workflows.md` detailing ingest, verify, bulk operations. | Doc merged; workflows tested. |
| DOCS-ATTEST-74-001 | TODO | Docs Guild, KMS Guild | KMS-73-001 | Publish `/docs/attestor/keys-and-issuers.md`. | Doc merged; rotation guidance verified. |
| DOCS-ATTEST-74-002 | TODO | Docs Guild, Transparency Guild | TRANSP-74-001 | Document `/docs/attestor/transparency.md` with witness usage/offline validation. | Doc merged; proofs validated. |
| DOCS-ATTEST-74-003 | TODO | Docs Guild, Attestor Console Guild | CONSOLE-ATTEST-73-001 | Write `/docs/console/attestor-ui.md` with screenshots/workflows. | Doc merged; screenshots captured; banner present. |
| DOCS-ATTEST-74-004 | TODO | Docs Guild, CLI Attestor Guild | CLI-ATTEST-73-001 | Publish `/docs/cli/attest.md` covering CLI usage. | Doc merged; commands validated. |
| DOCS-ATTEST-75-001 | TODO | Docs Guild, Export Attestation Guild | EXPORT-ATTEST-75-002 | Add `/docs/attestor/airgap.md` for attestation bundles. | Doc merged; verification steps confirmed. |
| DOCS-ATTEST-75-002 | TODO | Docs Guild, Security Guild | ATTESTOR-73-002 | Update `/docs/security/aoc-invariants.md` with attestation invariants. | Doc merged; invariants detailed. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-POLICY-20-001 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-20-000 | Author `/docs/policy/overview.md` covering concepts, inputs/outputs, determinism, and compliance checklist. | Doc published with diagrams + glossary; lint passes; checklist included. |
| DOCS-POLICY-20-002 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-20-001 | Write `/docs/policy/dsl.md` with grammar, built-ins, examples, anti-patterns. | DSL doc includes grammar tables, examples, compliance checklist; validated against parser tests. |
| DOCS-POLICY-20-003 | TODO | Docs Guild, Authority Core | AUTH-POLICY-20-001 | Publish `/docs/policy/lifecycle.md` describing draft→approve workflow, roles, audit, compliance list. | Lifecycle doc linked from UI/CLI help; approvals roles documented; checklist appended. |
| DOCS-POLICY-20-004 | TODO | Docs Guild, Scheduler Guild | SCHED-MODELS-20-001 | Create `/docs/policy/runs.md` detailing run modes, incremental mechanics, cursors, replay. | Run doc includes sequence diagrams + compliance checklist; cross-links to scheduler docs. |
| DOCS-POLICY-20-005 | TODO | Docs Guild, BE-Base Platform Guild | WEB-POLICY-20-001 | Draft `/docs/api/policy.md` describing endpoints, schemas, error codes. | API doc validated against OpenAPI; examples included; checklist appended. |
| DOCS-POLICY-20-006 | TODO | Docs Guild, DevEx/CLI Guild | CLI-POLICY-20-002 | Produce `/docs/cli/policy.md` with command usage, exit codes, JSON output contracts. | CLI doc includes examples, exit codes, compliance checklist. |
| DOCS-POLICY-20-007 | TODO | Docs Guild, UI Guild | UI-POLICY-20-001 | Document `/docs/ui/policy-editor.md` covering editor, simulation, diff workflows, approvals. | UI doc includes screenshots/placeholders, accessibility notes, compliance checklist. |
| DOCS-POLICY-20-008 | TODO | Docs Guild, Architecture Guild | POLICY-ENGINE-20-003 | Write `/docs/architecture/policy-engine.md` (new epic content) with sequence diagrams, selection strategy, schema. | Architecture doc merged with diagrams; compliance checklist appended; references updated. |
| DOCS-POLICY-20-009 | TODO | Docs Guild, Observability Guild | POLICY-ENGINE-20-007 | Add `/docs/observability/policy.md` for metrics/traces/logs, sample dashboards. | Observability doc includes metrics tables, dashboard screenshots, checklist. |
| DOCS-POLICY-20-010 | TODO | Docs Guild, Security Guild | AUTH-POLICY-20-002 | Publish `/docs/security/policy-governance.md` covering scopes, approvals, tenancy, least privilege. | Security doc merged; compliance checklist appended; reviewed by Security Guild. |
| DOCS-POLICY-20-011 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-20-001 | Populate `/docs/examples/policies/` with baseline/serverless/internal-only samples and commentary. | Example policies committed with explanations; lint passes; compliance checklist per file. |
| DOCS-POLICY-20-012 | TODO | Docs Guild, Support Guild | WEB-POLICY-20-003 | Draft `/docs/faq/policy-faq.md` addressing common pitfalls, VEX conflicts, determinism issues. | FAQ published with Q/A entries, cross-links, compliance checklist. |
## Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-GRAPH-21-001 | TODO | Docs Guild, Cartographer Guild | CARTO-GRAPH-21-001..006 | Author `/docs/graph/overview.md` covering concepts, snapshot lifecycle, overlays, and compliance checklist. | Doc merged with diagrams; lint passes; checklist appended. |
| DOCS-GRAPH-21-002 | TODO | Docs Guild | CARTO-GRAPH-21-001 | Write `/docs/graph/schema.md` describing node/edge/overlay schemas, indexes, sharding strategy, and sample docs. | Schema doc validated against fixtures; compliance checklist included. |
| DOCS-GRAPH-21-003 | TODO | Docs Guild, BE-Base Platform Guild | WEB-GRAPH-21-001..004 | Produce `/docs/graph/api.md` with endpoint specs, parameters, pagination, errors, and curl examples. | API doc aligns with OpenAPI; examples verified; checklist appended. |
| DOCS-GRAPH-21-004 | TODO | Docs Guild, UI Guild | UI-GRAPH-21-001..006 | Document `/docs/ui/graph-explorer.md` (screens, filters, paths, diff, accessibility). | UI doc published with screenshots/placeholders; accessibility checklist satisfied. |
| DOCS-GRAPH-21-005 | TODO | Docs Guild, DevEx/CLI Guild | CLI-GRAPH-21-001..003 | Create `/docs/cli/graph.md` detailing CLI commands, exit codes, JSON schemas. | CLI doc merged; examples validated; checklist appended. |
| DOCS-GRAPH-21-006 | TODO | Docs Guild, Architecture Guild | CARTO-GRAPH-21-002..007 | Draft `/docs/architecture/cartographer.md` covering build pipeline, layout tiling, overlay patching, sequence diagrams. | Architecture doc merged with diagrams; compliance checklist included. |
| DOCS-GRAPH-21-007 | TODO | Docs Guild, Observability Guild | CARTO-GRAPH-21-008, DEVOPS-GRAPH-21-001 | Publish `/docs/observability/graph.md` (metrics/traces/logs, dashboards, alerts). | Observability doc live; dashboards linked; checklist appended. |
| DOCS-GRAPH-21-008 | TODO | Docs Guild, Security Guild | AUTH-GRAPH-21-001..003 | Write `/docs/security/graph-access.md` describing RBAC, tenancy, scopes, service identities. | Security doc merged; reviewer checklist completed. |
| DOCS-GRAPH-21-009 | TODO | Docs Guild, Cartographer Guild | CARTO-GRAPH-21-006 | Document `/docs/examples/graph/` sample SBOMs, screenshots, exports with reviewer checklist. | Example docs + assets committed; lint passes; checklist appended. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-LNM-22-001 | TODO | Docs Guild, Concelier Guild | CONCELIER-LNM-21-001..003 | Author `/docs/advisories/aggregation.md` covering observation vs linkset, conflict handling, AOC requirements, and reviewer checklist. | Doc merged with examples + checklist; lint passes. |
| DOCS-LNM-22-002 | TODO | Docs Guild, Excititor Guild | EXCITITOR-LNM-21-001..003 | Publish `/docs/vex/aggregation.md` describing VEX observation/linkset model, product matching, conflicts. | Doc merged with fixtures; checklist appended. |
| DOCS-LNM-22-003 | TODO | Docs Guild, BE-Base Platform Guild | WEB-LNM-21-001..003 | Update `/docs/api/advisories.md` and `/docs/api/vex.md` for new endpoints, parameters, errors, exports. | API docs aligned with OpenAPI; examples validated. |
| DOCS-LNM-22-004 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-40-001 | Create `/docs/policy/effective-severity.md` detailing severity selection strategies from multiple sources. | Doc merged with policy examples; checklist included. |
| DOCS-LNM-22-005 | TODO | Docs Guild, UI Guild | UI-LNM-22-001..003 | Document `/docs/ui/evidence-panel.md` with screenshots, conflict badges, accessibility guidance. | UI doc merged; accessibility checklist completed. |
## StellaOps Console (Sprint 23)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-CONSOLE-23-001 | TODO | Docs Guild, Console Guild | CONSOLE-CORE-23-004 | Publish `/docs/ui/console-overview.md` covering IA, tenant model, global filters, and AOC alignment with compliance checklist. | Doc merged with diagrams + overview tables; checklist appended; Console Guild sign-off. |
| DOCS-CONSOLE-23-002 | TODO | Docs Guild, Console Guild | DOCS-CONSOLE-23-001 | Author `/docs/ui/navigation.md` detailing routes, breadcrumbs, keyboard shortcuts, deep links, and tenant context switching. | Navigation doc merged with shortcut tables and screenshots; accessibility checklist satisfied. |
| DOCS-CONSOLE-23-003 | TODO | Docs Guild, SBOM Service Guild, Console Guild | SBOM-CONSOLE-23-001, CONSOLE-FEAT-23-102 | Document `/docs/ui/sbom-explorer.md` (catalog, detail, graph overlays, exports) including compliance checklist and performance tips. | Doc merged with annotated screenshots, export instructions, and overlay examples; checklist appended. |
| DOCS-CONSOLE-23-004 | TODO | Docs Guild, Concelier Guild, Excititor Guild | CONCELIER-CONSOLE-23-001, EXCITITOR-CONSOLE-23-001 | Produce `/docs/ui/advisories-and-vex.md` explaining aggregation-not-merge, conflict indicators, raw viewers, and provenance banners. | Doc merged; raw JSON examples included; compliance checklist complete. |
| DOCS-CONSOLE-23-005 | TODO | Docs Guild, Policy Guild | POLICY-CONSOLE-23-001, CONSOLE-FEAT-23-104 | Write `/docs/ui/findings.md` describing filters, saved views, explain drawer, exports, and CLI parity callouts. | Doc merged with filter matrix + explain walkthrough; checklist appended. |
| DOCS-CONSOLE-23-006 | TODO | Docs Guild, Policy Guild, Product Ops | POLICY-CONSOLE-23-002, CONSOLE-FEAT-23-105 | Publish `/docs/ui/policies.md` with editor, simulation, approvals, compliance checklist, and RBAC mapping. | Doc merged; Monaco screenshots + simulation diff examples included; approval flow described; checklist appended. |
| DOCS-CONSOLE-23-007 | TODO | Docs Guild, Scheduler Guild | SCHED-CONSOLE-23-001, CONSOLE-FEAT-23-106 | Document `/docs/ui/runs.md` covering queues, live progress, diffs, retries, evidence downloads, and troubleshooting. | Doc merged with SSE troubleshooting, metrics references, compliance checklist. |
| DOCS-CONSOLE-23-008 | TODO | Docs Guild, Authority Guild | AUTH-CONSOLE-23-002, CONSOLE-FEAT-23-108 | Draft `/docs/ui/admin.md` describing users/roles, tenants, tokens, integrations, fresh-auth prompts, and RBAC mapping. | Doc merged with tables for scopes vs roles, screenshots, compliance checklist. |
| DOCS-CONSOLE-23-009 | TODO | Docs Guild, DevOps Guild | DOWNLOADS-CONSOLE-23-001, CONSOLE-FEAT-23-109 | Publish `/docs/ui/downloads.md` listing product images, commands, offline instructions, parity with CLI, and compliance checklist. | Doc merged; manifest sample included; copy-to-clipboard guidance documented; checklist complete. |
| DOCS-CONSOLE-23-010 | TODO | Docs Guild, Deployment Guild, Console Guild | DEVOPS-CONSOLE-23-002, CONSOLE-REL-23-301 | Write `/docs/deploy/console.md` (Helm, ingress, TLS, CSP, env vars, health checks) with compliance checklist. | Deploy doc merged; templates validated; CSP guidance included; checklist appended. |
| DOCS-CONSOLE-23-011 | TODO | Docs Guild, Deployment Guild | DOCS-CONSOLE-23-010 | Update `/docs/install/docker.md` to cover Console image, Compose/Helm usage, offline tarballs, parity with CLI. | Doc updated with new sections; commands validated; compliance checklist appended. |
| DOCS-CONSOLE-23-012 | TODO | Docs Guild, Security Guild | AUTH-CONSOLE-23-003, WEB-CONSOLE-23-002 | Publish `/docs/security/console-security.md` detailing OIDC flows, scopes, CSP, fresh-auth, evidence handling, and compliance checklist. | Security doc merged; threat model notes included; checklist appended. |
| DOCS-CONSOLE-23-013 | TODO | Docs Guild, Observability Guild | TELEMETRY-CONSOLE-23-001, CONSOLE-QA-23-403 | Write `/docs/observability/ui-telemetry.md` cataloguing metrics/logs/traces, dashboards, alerts, and feature flags. | Doc merged with instrumentation tables, dashboard screenshots, checklist appended. |
| DOCS-CONSOLE-23-014 | TODO | Docs Guild, Console Guild, CLI Guild | CONSOLE-DOC-23-502 | Maintain `/docs/cli-vs-ui-parity.md` matrix and integrate CI check guidance. | Matrix published with parity status, CI workflow documented, compliance checklist appended. |
| DOCS-CONSOLE-23-015 | TODO | Docs Guild, Architecture Guild | CONSOLE-CORE-23-001, WEB-CONSOLE-23-001 | Produce `/docs/architecture/console.md` describing frontend packages, data flow diagrams, SSE design, performance budgets. | Architecture doc merged with diagrams + compliance checklist; reviewers approve. |
| DOCS-CONSOLE-23-016 | TODO | Docs Guild, Accessibility Guild | CONSOLE-QA-23-402, CONSOLE-FEAT-23-102 | Refresh `/docs/accessibility.md` with Console-specific keyboard flows, color tokens, testing tools, and compliance checklist updates. | Accessibility doc updated; audits referenced; checklist appended. |
| DOCS-CONSOLE-23-017 | TODO | Docs Guild, Console Guild | CONSOLE-FEAT-23-101..109 | Create `/docs/examples/ui-tours.md` providing triage, audit, policy rollout walkthroughs with annotated screenshots and GIFs. | UI tours doc merged; media assets stored; compliance checklist appended. |
| DOCS-LNM-22-006 | TODO | Docs Guild, Architecture Guild | CONCELIER-LNM-21-001..005, EXCITITOR-LNM-21-001..005 | Refresh `/docs/architecture/conseiller.md` and `/docs/architecture/excitator.md` describing observation/linkset pipelines and event contracts. | Architecture docs updated with diagrams; checklist appended. |
| DOCS-LNM-22-007 | TODO | Docs Guild, Observability Guild | CONCELIER-LNM-21-005, EXCITITOR-LNM-21-005, DEVOPS-LNM-22-002 | Publish `/docs/observability/aggregation.md` with metrics/traces/logs/SLOs. | Observability doc merged; dashboards referenced; checklist appended. |
| DOCS-LNM-22-008 | TODO | Docs Guild, DevOps Guild | MERGE-LNM-21-001, CONCELIER-LNM-21-102 | Write `/docs/migration/no-merge.md` describing migration plan, backfill steps, rollback, feature flags. | Migration doc approved by stakeholders; checklist appended. |
## Policy Engine + Editor v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-POLICY-23-001 | TODO | Docs Guild, Policy Guild | POLICY-SPL-23-001..003 | Author `/docs/policy/overview.md` describing SPL philosophy, layering, and glossary with reviewer checklist. | Doc merged; lint passes; checklist appended. |
| DOCS-POLICY-23-002 | TODO | Docs Guild, Policy Guild | POLICY-SPL-23-001 | Write `/docs/policy/spl-v1.md` (language reference, JSON Schema, examples). | Reference published with schema snippets; checklist completed. |
| DOCS-POLICY-23-003 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-50-001..004 | Produce `/docs/policy/runtime.md` covering compiler, evaluator, caching, events, SLOs. | Runtime doc merged with diagrams; observability references included. |
| DOCS-POLICY-23-004 | TODO | Docs Guild, UI Guild | UI-POLICY-23-001..006 | Document `/docs/policy/editor.md` (UI walkthrough, validation, simulation, approvals). | Editor doc merged with screenshots; accessibility checklist satisfied. |
| DOCS-POLICY-23-005 | TODO | Docs Guild, Security Guild | AUTH-POLICY-23-001..002 | Publish `/docs/policy/governance.md` (roles, scopes, approvals, signing, exceptions). | Governance doc merged; checklist appended. |
| DOCS-POLICY-23-006 | TODO | Docs Guild, BE-Base Platform Guild | WEB-POLICY-23-001..004 | Update `/docs/api/policy.md` with new endpoints, schemas, errors, pagination. | API doc aligns with OpenAPI; examples validated; checklist included. |
| DOCS-POLICY-23-007 | TODO | Docs Guild, DevEx/CLI Guild | CLI-POLICY-23-004..006 | Update `/docs/cli/policy.md` for lint/simulate/activate/history commands, exit codes. | CLI doc updated; samples verified; checklist appended. |
| DOCS-POLICY-23-008 | TODO | Docs Guild, Architecture Guild | POLICY-ENGINE-50-005..006 | Refresh `/docs/architecture/policy-engine.md` with data model, sequence diagrams, event flows. | Architecture doc merged with diagrams; checklist appended. |
| DOCS-POLICY-23-009 | TODO | Docs Guild, DevOps Guild | MERGE-LNM-21-001, DEVOPS-LNM-22-001 | Create `/docs/migration/policy-parity.md` covering dual-run parity plan and rollback. | Migration doc approved; checklist appended. |
| DOCS-POLICY-23-010 | TODO | Docs Guild, UI Guild | UI-POLICY-23-006 | Write `/docs/ui/explainers.md` showing explain trees, evidence overlays, interpretation guidance. | Doc merged with annotated screenshots; checklist appended. |
## Graph & Vuln Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-GRAPH-24-001 | TODO | Docs Guild, UI Guild | UI-GRAPH-24-001..006 | Author `/docs/ui/sbom-graph-explorer.md` detailing overlays, filters, saved views, accessibility, and AOC visibility. | Doc merged; screenshots included; checklist appended. |
| DOCS-GRAPH-24-002 | TODO | Docs Guild, UI Guild | UI-GRAPH-24-005 | Publish `/docs/ui/vulnerability-explorer.md` covering table usage, grouping, fix suggestions, Why drawer. | Doc merged with annotated images; accessibility checklist satisfied. |
| DOCS-GRAPH-24-003 | TODO | Docs Guild, SBOM Service Guild | SBOM-GRAPH-24-001..003 | Create `/docs/architecture/graph-index.md` describing data model, ingestion pipeline, caches, events. | Architecture doc merged with diagrams; checklist appended. |
| DOCS-GRAPH-24-004 | TODO | Docs Guild, BE-Base Platform Guild | WEB-GRAPH-24-001..003 | Document `/docs/api/graph.md` and `/docs/api/vuln.md` avec endpoints, parameters, errors, RBAC. | API docs aligned with OpenAPI; examples validated; checklist appended. |
| DOCS-GRAPH-24-005 | TODO | Docs Guild, DevEx/CLI Guild | CLI-GRAPH-24-001..003 | Update `/docs/cli/graph-and-vuln.md` covering new CLI commands, exit codes, scripting. | CLI doc merged; examples tested; checklist appended. |
| DOCS-GRAPH-24-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-60-001..002 | Write `/docs/policy/ui-integration.md` explaining overlays, cache usage, simulator contracts. | Doc merged; references cross-linked; checklist appended. |
| DOCS-GRAPH-24-007 | TODO | Docs Guild, DevOps Guild | DEVOPS-GRAPH-24-001..003 | Produce `/docs/migration/graph-parity.md` with rollout plan, parity checks, fallback guidance. | Migration doc approved; checklist appended. |
## Exceptions v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-EXC-25-001 | TODO | Docs Guild, Governance Guild | WEB-EXC-25-001 | Author `/docs/governance/exceptions.md` covering lifecycle, scope patterns, examples, compliance checklist. | Doc merged; reviewers sign off; checklist included. |
| DOCS-EXC-25-002 | TODO | Docs Guild, Authority Core | AUTH-EXC-25-001 | Publish `/docs/governance/approvals-and-routing.md` detailing roles, routing matrix, MFA rules, audit trails. | Doc merged; routing examples validated; checklist appended. |
| DOCS-EXC-25-003 | TODO | Docs Guild, BE-Base Platform Guild | WEB-EXC-25-001..003 | Create `/docs/api/exceptions.md` with endpoints, payloads, errors, idempotency notes. | API doc aligned with OpenAPI; examples tested; checklist appended. |
| DOCS-EXC-25-004 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-70-001 | Document `/docs/policy/exception-effects.md` explaining evaluation order, conflicts, simulation. | Doc merged; tests cross-referenced; checklist appended. |
| DOCS-EXC-25-005 | TODO | Docs Guild, UI Guild | UI-EXC-25-001..004 | Write `/docs/ui/exception-center.md` with UI walkthrough, badges, accessibility, shortcuts. | Doc merged with screenshots; accessibility checklist completed. |
| DOCS-EXC-25-006 | TODO | Docs Guild, DevEx/CLI Guild | CLI-EXC-25-001..002 | Update `/docs/cli/exceptions.md` covering command usage and exit codes. | CLI doc updated; examples validated; checklist appended. |
| DOCS-EXC-25-007 | TODO | Docs Guild, DevOps Guild | SCHED-WORKER-25-101, DEVOPS-GRAPH-24-003 | Publish `/docs/migration/exception-governance.md` describing cutover from legacy suppressions, notifications, rollback. | Migration doc approved; checklist included. |
> Update statuses (TODO/DOING/REVIEW/DONE/BLOCKED) as progress changes. Keep guides in sync with configuration samples under `etc/`.
> Remark (2025-10-13, DOC4.AUTH-PDG): Rate limit guide published (`docs/security/rate-limits.md`) and handed to plugin docs team for diagram uplift once PLG6.DIAGRAM lands.
## Orchestrator Dashboard (Epic 9)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-ORCH-32-001 | TODO | Docs Guild | ORCH-SVC-32-001, AUTH-ORCH-32-001 | Author `/docs/orchestrator/overview.md` covering mission, roles, AOC alignment, governance, with imposed rule reminder. | Doc merged with diagrams; imposed rule statement included; entry linked from docs index. |
| DOCS-ORCH-32-002 | TODO | Docs Guild | ORCH-SVC-32-002 | Author `/docs/orchestrator/architecture.md` detailing scheduler, DAGs, rate limits, data model, message bus, storage layout, restating imposed rule. | Architecture doc merged; diagrams reviewed; imposed rule noted. |
| DOCS-ORCH-33-001 | TODO | Docs Guild | ORCH-SVC-33-001..004, WEB-ORCH-33-001 | Publish `/docs/orchestrator/api.md` (REST/WebSocket endpoints, payloads, error codes) with imposed rule note. | API doc merged; examples validated; imposed rule appended. |
| DOCS-ORCH-33-002 | TODO | Docs Guild | CONSOLE-ORCH-32-002, CONSOLE-ORCH-33-001..002 | Publish `/docs/orchestrator/console.md` covering screens, a11y, live updates, control actions, reiterating imposed rule. | Console doc merged with screenshots; accessibility checklist done; imposed rule statement present. |
| DOCS-ORCH-33-003 | TODO | Docs Guild | CLI-ORCH-33-001 | Publish `/docs/orchestrator/cli.md` documenting commands, options, exit codes, streaming output, offline usage, and imposed rule. | CLI doc merged; examples tested; imposed rule appended. |
| DOCS-ORCH-34-001 | TODO | Docs Guild | ORCH-SVC-34-002, LEDGER-34-101 | Author `/docs/orchestrator/run-ledger.md` covering ledger schema, provenance chain, audit workflows, with imposed rule reminder. | Run-ledger doc merged; payload samples validated; imposed rule included; cross-links added. |
| DOCS-ORCH-34-002 | TODO | Docs Guild | AUTH-ORCH-32-001, AUTH-ORCH-34-001 | Update `/docs/security/secrets-handling.md` for orchestrator KMS refs, redaction badges, operator hygiene, reiterating imposed rule. | Security doc merged; checklists updated; imposed rule restated; references from Console/CLI docs added. |
| DOCS-ORCH-34-003 | TODO | Docs Guild | ORCH-SVC-33-003, ORCH-SVC-34-001, DEVOPS-ORCH-34-001 | Publish `/docs/operations/orchestrator-runbook.md` (incident playbook, backfill guide, circuit breakers, throttling) with imposed rule statement. | Runbook merged; steps validated with DevOps; imposed rule included; runbook linked from ops index. |
| DOCS-ORCH-34-004 | TODO | Docs Guild | ORCH-SVC-32-005, WORKER-GO-33-001, WORKER-PY-33-001 | Document `/docs/schemas/artifacts.md` describing artifact kinds, schema versions, hashing, storage layout, restating imposed rule. | Schema doc merged; JSON schema provided; imposed rule included; sample payload validated. |
| DOCS-ORCH-34-005 | TODO | Docs Guild | ORCH-SVC-34-001, DEVOPS-ORCH-34-001 | Author `/docs/slo/orchestrator-slo.md` defining SLOs, burn alerts, measurement, and reiterating imposed rule. | SLO doc merged; dashboard screenshots embedded; imposed rule appended; alerts documented. |
## Export Center (Epic 10)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-EXPORT-35-001 | TODO | Docs Guild | EXPORT-SVC-35-001..006 | Author `/docs/export-center/overview.md` covering purpose, profiles, security, AOC alignment, surfaces, ending with imposed rule statement. | Doc merged with diagrams/examples; imposed rule line present; index updated. |
| DOCS-EXPORT-35-002 | TODO | Docs Guild | EXPORT-SVC-35-002..005 | Publish `/docs/export-center/architecture.md` describing planner, adapters, manifests, signing, distribution flows, restating imposed rule. | Architecture doc merged; sequence diagrams included; rule statement appended. |
| DOCS-EXPORT-35-003 | TODO | Docs Guild | EXPORT-SVC-35-003..004 | Publish `/docs/export-center/profiles.md` detailing schema fields, examples, compatibility, and imposed rule reminder. | Profiles doc merged; JSON schemas linked; imposed rule noted. |
| DOCS-EXPORT-36-004 | TODO | Docs Guild | EXPORT-SVC-36-001..004, WEB-EXPORT-36-001 | Publish `/docs/export-center/api.md` covering endpoints, payloads, errors, and mention imposed rule. | API doc merged; examples validated; rule included. |
| DOCS-EXPORT-36-005 | TODO | Docs Guild | CLI-EXPORT-35-001, CLI-EXPORT-36-001 | Publish `/docs/export-center/cli.md` with command reference, CI scripts, verification steps, restating imposed rule. | CLI doc merged; script snippets tested; rule appended. |
| DOCS-EXPORT-36-006 | TODO | Docs Guild | EXPORT-SVC-36-001, DEVOPS-EXPORT-36-001 | Publish `/docs/export-center/trivy-adapter.md` covering field mappings, compatibility matrix, and imposed rule reminder. | Doc merged; mapping tables validated; rule included. |
| DOCS-EXPORT-37-001 | TODO | Docs Guild | EXPORT-SVC-37-001, DEVOPS-EXPORT-37-001 | Publish `/docs/export-center/mirror-bundles.md` describing filesystem/OCI layouts, delta/encryption, import guide, ending with imposed rule. | Doc merged; diagrams provided; verification steps tested; rule stated. |
| DOCS-EXPORT-37-002 | TODO | Docs Guild | EXPORT-SVC-35-005, EXPORT-SVC-37-002 | Publish `/docs/export-center/provenance-and-signing.md` detailing manifests, attestation flow, verification, reiterating imposed rule. | Doc merged; signature examples validated; rule appended. |
| DOCS-EXPORT-37-003 | TODO | Docs Guild | DEVOPS-EXPORT-37-001 | Publish `/docs/operations/export-runbook.md` covering failures, tuning, capacity planning, with imposed rule reminder. | Runbook merged; procedures validated; rule included. |
| DOCS-EXPORT-37-004 | TODO | Docs Guild | AUTH-EXPORT-37-001, EXPORT-SVC-37-002 | Publish `/docs/security/export-hardening.md` outlining RBAC, tenancy, encryption, redaction, restating imposed rule. | Security doc merged; checklist updated; rule appended. |
## Reachability v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-SIG-26-001 | TODO | Docs Guild, Signals Guild | SIGNALS-24-004 | Write `/docs/signals/reachability.md` covering states, scores, provenance, retention. | Doc merged with diagrams/examples; checklist appended. |
| DOCS-SIG-26-002 | TODO | Docs Guild, Signals Guild | SIGNALS-24-002 | Publish `/docs/signals/callgraph-formats.md` with schemas and validation errors. | Doc merged; examples tested; checklist included. |
| DOCS-SIG-26-003 | TODO | Docs Guild, Runtime Guild | SIGNALS-24-003 | Create `/docs/signals/runtime-facts.md` detailing agent capabilities, privacy safeguards, opt-in flags. | Doc merged; privacy review done; checklist appended. |
| DOCS-SIG-26-004 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-80-001 | Document `/docs/policy/signals-weighting.md` for SPL predicates and weighting strategies. | Doc merged; sample policies validated; checklist appended. |
| DOCS-SIG-26-005 | TODO | Docs Guild, UI Guild | UI-SIG-26-001..003 | Draft `/docs/ui/reachability-overlays.md` with badges, timelines, shortcuts. | Doc merged with screenshots; accessibility checklist completed. |
| DOCS-SIG-26-006 | TODO | Docs Guild, DevEx/CLI Guild | CLI-SIG-26-001..002 | Update `/docs/cli/reachability.md` for new commands and automation recipes. | Doc merged; examples verified; checklist appended. |
| DOCS-SIG-26-007 | TODO | Docs Guild, BE-Base Platform Guild | WEB-SIG-26-001..003 | Publish `/docs/api/signals.md` covering endpoints, payloads, ETags, errors. | API doc aligned with OpenAPI; examples tested; checklist appended. |
| DOCS-SIG-26-008 | TODO | Docs Guild, DevOps Guild | DEVOPS-SIG-26-001..002 | Write `/docs/migration/enable-reachability.md` guiding rollout, fallbacks, monitoring. | Migration doc approved; checklist appended. |
## Policy Studio (Sprint 27)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-POLICY-27-001 | TODO | Docs Guild, Policy Guild | REGISTRY-API-27-001, POLICY-ENGINE-27-001 | Publish `/docs/policy/studio-overview.md` covering lifecycle, roles, glossary, and compliance checklist. | Doc merged with diagrams + lifecycle table; checklist appended; stakeholders sign off. |
| DOCS-POLICY-27-002 | TODO | Docs Guild, Console Guild | CONSOLE-STUDIO-27-001 | Write `/docs/policy/authoring.md` detailing workspace templates, snippets, lint rules, IDE shortcuts, and best practices. | Authoring doc includes annotated screenshots, snippet catalog, compliance checklist. |
| DOCS-POLICY-27-003 | TODO | Docs Guild, Policy Registry Guild | REGISTRY-API-27-007 | Document `/docs/policy/versioning-and-publishing.md` (semver rules, attestations, rollback) with compliance checklist. | Doc merged with flow diagrams; attestation steps documented; checklist appended. |
| DOCS-POLICY-27-004 | TODO | Docs Guild, Scheduler Guild | REGISTRY-API-27-005, SCHED-WORKER-27-301 | Write `/docs/policy/simulation.md` covering quick vs batch sim, thresholds, evidence bundles, CLI examples. | Simulation doc includes charts, sample manifests, checklist appended. |
| DOCS-POLICY-27-005 | TODO | Docs Guild, Product Ops | REGISTRY-API-27-006 | Publish `/docs/policy/review-and-approval.md` with approver requirements, comments, webhooks, audit trail guidance. | Doc merged with role matrix + webhook schema; checklist appended. |
| DOCS-POLICY-27-006 | TODO | Docs Guild, Policy Guild | REGISTRY-API-27-008 | Author `/docs/policy/promotion.md` covering environments, canary, rollback, and monitoring steps. | Promotion doc includes examples + checklist; verified by Policy Ops. |
| DOCS-POLICY-27-007 | TODO | Docs Guild, DevEx/CLI Guild | CLI-POLICY-27-001..004 | Update `/docs/policy/cli.md` with new commands, JSON schemas, CI usage, and compliance checklist. | CLI doc merged with transcripts; schema references validated; checklist appended. |
| DOCS-POLICY-27-008 | TODO | Docs Guild, Policy Registry Guild | REGISTRY-API-27-001..008 | Publish `/docs/policy/api.md` describing Registry endpoints, request/response schemas, errors, and feature flags. | API doc aligned with OpenAPI; examples validated; checklist appended. |
| DOCS-POLICY-27-009 | TODO | Docs Guild, Security Guild | AUTH-POLICY-27-002 | Create `/docs/security/policy-attestations.md` covering signing, verification, key rotation, and compliance checklist. | Security doc approved by Security Guild; verifier steps documented; checklist appended. |
| DOCS-POLICY-27-010 | TODO | Docs Guild, Architecture Guild | REGISTRY-API-27-001, SCHED-WORKER-27-301 | Author `/docs/architecture/policy-registry.md` (service design, schemas, queues, failure modes) with diagrams and checklist. | Architecture doc merged; diagrams committed; checklist appended. |
| DOCS-POLICY-27-011 | TODO | Docs Guild, Observability Guild | DEVOPS-POLICY-27-004 | Publish `/docs/observability/policy-telemetry.md` with metrics/log tables, dashboards, alerts, and compliance checklist. | Observability doc merged; dashboards linked; checklist appended. |
| DOCS-POLICY-27-012 | TODO | Docs Guild, Ops Guild | DEPLOY-POLICY-27-002 | Write `/docs/runbooks/policy-incident.md` detailing rollback, freeze, forensic steps, notifications. | Runbook merged; rehearsal recorded; checklist appended. |
| DOCS-POLICY-27-013 | TODO | Docs Guild, Policy Guild | CONSOLE-STUDIO-27-001, REGISTRY-API-27-002 | Update `/docs/examples/policy-templates.md` with new templates, snippets, and sample policies. | Examples committed with commentary; lint passes; checklist appended. |
| DOCS-POLICY-27-014 | TODO | Docs Guild, Policy Registry Guild | REGISTRY-API-27-003, WEB-POLICY-27-001 | Refresh `/docs/aoc/aoc-guardrails.md` to include Studio-specific guardrails and validation scenarios. | Doc updated with Studio guardrails; compliance checklist appended. |
## Vulnerability Explorer (Sprint 29)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-VULN-29-001 | TODO | Docs Guild, Vuln Explorer Guild | VULN-API-29-001 | Publish `/docs/vuln/explorer-overview.md` covering domain model, identities, AOC guarantees, workflow summary. | Doc merged with diagrams/table; compliance checklist appended. |
| DOCS-VULN-29-002 | TODO | Docs Guild, Console Guild | CONSOLE-VULN-29-001..006 | Write `/docs/vuln/explorer-using-console.md` with workflows, screenshots, keyboard shortcuts, saved views, deep links. | Doc merged; images stored; WCAG notes included; checklist appended. |
| DOCS-VULN-29-003 | TODO | Docs Guild, Vuln Explorer API Guild | VULN-API-29-001..009 | Author `/docs/vuln/explorer-api.md` (endpoints, query schema, grouping, errors, rate limits). | Doc aligned with OpenAPI; examples validated; checklist appended. |
| DOCS-VULN-29-004 | TODO | Docs Guild, DevEx/CLI Guild | CLI-VULN-29-001..005 | Publish `/docs/vuln/explorer-cli.md` with command reference, samples, exit codes, CI snippets. | CLI doc merged; transcripts/JSON outputs validated; checklist appended. |
| DOCS-VULN-29-005 | TODO | Docs Guild, Findings Ledger Guild | LEDGER-29-001..009 | Write `/docs/vuln/findings-ledger.md` detailing event schema, hashing, Merkle roots, replay tooling. | Doc merged; compliance checklist appended; audit team sign-off. |
| DOCS-VULN-29-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-29-001..003 | Update `/docs/policy/vuln-determinations.md` for new rationale, signals, simulation semantics. | Doc updated; examples validated; checklist appended. |
| DOCS-VULN-29-007 | TODO | Docs Guild, Excititor Guild | EXCITITOR-VULN-29-001..004 | Publish `/docs/vex/explorer-integration.md` covering CSAF mapping, suppression precedence, status semantics. | Doc merged; compliance checklist appended. |
| DOCS-VULN-29-008 | TODO | Docs Guild, Concelier Guild | CONCELIER-VULN-29-001..004 | Publish `/docs/advisories/explorer-integration.md` covering key normalization, withdrawn handling, provenance. | Doc merged; checklist appended. |
| DOCS-VULN-29-009 | TODO | Docs Guild, SBOM Service Guild | SBOM-VULN-29-001..002 | Author `/docs/sbom/vuln-resolution.md` detailing version semantics, scope, paths, safe version hints. | Doc merged; ecosystem tables validated; checklist appended. |
| DOCS-VULN-29-010 | TODO | Docs Guild, Observability Guild | VULN-API-29-009, DEVOPS-VULN-29-002 | Publish `/docs/observability/vuln-telemetry.md` (metrics, logs, tracing, dashboards, SLOs). | Doc merged; dashboards linked; checklist appended. |
| DOCS-VULN-29-011 | TODO | Docs Guild, Security Guild | AUTH-VULN-29-001..003 | Create `/docs/security/vuln-rbac.md` for roles, ABAC policies, attachment encryption, CSRF. | Security doc approved; checklist appended. |
| DOCS-VULN-29-012 | TODO | Docs Guild, Ops Guild | DEVOPS-VULN-29-002, SCHED-WORKER-29-003 | Write `/docs/runbooks/vuln-ops.md` (projector lag, resolver storms, export failures, policy activation). | Runbook merged; rehearsal recorded; checklist appended. |
| DOCS-VULN-29-013 | TODO | Docs Guild, Deployment Guild | DEPLOY-VULN-29-001..002 | Update `/docs/install/containers.md` with Findings Ledger & Vuln Explorer API images, manifests, resource sizing, health checks. | Install doc updated; validation commands included; checklist appended. |
## VEX Lens (Sprint 30)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-VEX-30-001 | TODO | Docs Guild, VEX Lens Guild | VEXLENS-30-005 | Publish `/docs/vex/consensus-overview.md` describing purpose, scope, AOC guarantees. | Doc merged with diagrams/terminology tables; compliance checklist appended. |
| DOCS-VEX-30-002 | TODO | Docs Guild, VEX Lens Guild | VEXLENS-30-005 | Author `/docs/vex/consensus-algorithm.md` covering normalization, weighting, thresholds, examples. | Doc merged; math reviewed by Policy; checklist appended. |
| DOCS-VEX-30-003 | TODO | Docs Guild, Issuer Directory Guild | ISSUER-30-001..003 | Document `/docs/vex/issuer-directory.md` (issuer management, keys, trust overrides, audit). | Doc merged; security review done; checklist appended. |
| DOCS-VEX-30-004 | TODO | Docs Guild, VEX Lens Guild | VEXLENS-30-007 | Publish `/docs/vex/consensus-api.md` with endpoint specs, query params, rate limits. | API doc aligned with OpenAPI; examples validated; checklist appended. |
| DOCS-VEX-30-005 | TODO | Docs Guild, Console Guild | CONSOLE-VEX-30-001 | Write `/docs/vex/consensus-console.md` covering UI workflows, filters, conflicts, accessibility. | Doc merged; screenshots added; checklist appended. |
| DOCS-VEX-30-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-29-001, VEXLENS-30-004 | Add `/docs/policy/vex-trust-model.md` detailing policy knobs, thresholds, simulation. | Doc merged; policy review completed; checklist appended. |
| DOCS-VEX-30-007 | TODO | Docs Guild, SBOM Service Guild | VEXLENS-30-002 | Publish `/docs/sbom/vex-mapping.md` (CPE→purl strategy, edge cases, overrides). | Doc merged; mapping tables validated; checklist appended. |
| DOCS-VEX-30-008 | TODO | Docs Guild, Security Guild | ISSUER-30-002, VEXLENS-30-003 | Deliver `/docs/security/vex-signatures.md` (verification flow, key rotation, audit). | Doc approved by Security; checklist appended. |
| DOCS-VEX-30-009 | TODO | Docs Guild, DevOps Guild | VEXLENS-30-009, DEVOPS-VEX-30-001 | Create `/docs/runbooks/vex-ops.md` for recompute storms, mapping failures, signature errors. | Runbook merged; rehearsal logged; checklist appended. |
## Advisory AI (Sprint 31)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-AIAI-31-001 | TODO | Docs Guild, Advisory AI Guild | AIAI-31-006 | Publish `/docs/advisory-ai/overview.md` covering capabilities, guardrails, RBAC. | Doc merged with diagrams; compliance checklist appended. |
| DOCS-AIAI-31-002 | TODO | Docs Guild, Advisory AI Guild | AIAI-31-004 | Author `/docs/advisory-ai/architecture.md` detailing RAG pipeline, deterministics, caching, model options. | Doc merged; architecture review done; checklist appended. |
| DOCS-AIAI-31-003 | TODO | Docs Guild, Advisory AI Guild | AIAI-31-006 | Write `/docs/advisory-ai/api.md` describing endpoints, schemas, errors, rate limits. | API doc aligned with OpenAPI; examples validated; checklist appended. |
| DOCS-AIAI-31-004 | TODO | Docs Guild, Console Guild | CONSOLE-VULN-29-001, CONSOLE-VEX-30-001 | Create `/docs/advisory-ai/console.md` with screenshots, a11y notes, copy-as-ticket instructions. | Doc merged; images stored; checklist appended. |
| DOCS-AIAI-31-005 | TODO | Docs Guild, DevEx/CLI Guild | CLI-VULN-29-001, CLI-VEX-30-001 | Publish `/docs/advisory-ai/cli.md` covering commands, exit codes, scripting patterns. | Doc merged; examples tested; checklist appended. |
| DOCS-AIAI-31-006 | TODO | Docs Guild, Policy Guild | POLICY-ENGINE-31-001 | Update `/docs/policy/assistant-parameters.md` covering temperature, token limits, ranking weights, TTLs. | Doc merged; policy review done; checklist appended. |
| DOCS-AIAI-31-007 | TODO | Docs Guild, Security Guild | AIAI-31-005 | Write `/docs/security/assistant-guardrails.md` detailing redaction, injection defense, logging. | Doc approved by Security; checklist appended. |
| DOCS-AIAI-31-008 | TODO | Docs Guild, SBOM Service Guild | SBOM-AIAI-31-001 | Publish `/docs/sbom/remediation-heuristics.md` (feasibility scoring, blast radius). | Doc merged; heuristics reviewed; checklist appended. |
| DOCS-AIAI-31-009 | TODO | Docs Guild, DevOps Guild | DEVOPS-AIAI-31-001 | Create `/docs/runbooks/assistant-ops.md` for warmup, cache priming, model outages, scaling. | Runbook merged; rehearsal logged; checklist appended. |
## Notifications Studio
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-NOTIFY-38-001 | TODO | Docs Guild, Notifications Service Guild | NOTIFY-SVC-38-001..004 | Publish `/docs/notifications/overview.md` and `/docs/notifications/architecture.md`, each ending with imposed rule reminder. | Docs merged; diagrams verified; imposed rule appended. |
| DOCS-NOTIFY-39-002 | TODO | Docs Guild, Notifications Service Guild | NOTIFY-SVC-39-001..004 | Publish `/docs/notifications/rules.md`, `/docs/notifications/templates.md`, `/docs/notifications/digests.md` with examples and imposed rule line. | Docs merged; examples validated; imposed rule appended. |
| DOCS-NOTIFY-40-001 | TODO | Docs Guild, Security Guild | AUTH-NOTIFY-38-001, NOTIFY-SVC-40-001..004 | Publish `/docs/notifications/channels.md`, `/docs/notifications/escalations.md`, `/docs/notifications/api.md`, `/docs/operations/notifier-runbook.md`, `/docs/security/notifications-hardening.md`; each ends with imposed rule line. | Docs merged; accessibility checks passed; imposed rule appended. |
## CLI Parity & Task Packs
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-CLI-41-001 | TODO | Docs Guild, DevEx/CLI Guild | CLI-CORE-41-001 | Publish `/docs/cli/overview.md`, `/docs/cli/configuration.md`, `/docs/cli/output-and-exit-codes.md` with imposed rule statements. | Docs merged; examples verified; imposed rule appended. |
| DOCS-CLI-42-001 | TODO | Docs Guild | DOCS-CLI-41-001, CLI-PARITY-41-001 | Publish `/docs/cli/parity-matrix.md` and command guides under `/docs/cli/commands/*.md` (policy, sbom, vuln, vex, advisory, export, orchestrator, notify, aoc, auth). | Guides merged; parity automation documented; imposed rule appended. |
| DOCS-PACKS-43-001 | TODO | Docs Guild, Task Runner Guild | PACKS-REG-42-001, TASKRUN-42-001 | Publish `/docs/task-packs/spec.md`, `/docs/task-packs/authoring-guide.md`, `/docs/task-packs/registry.md`, `/docs/task-packs/runbook.md`, `/docs/security/pack-signing-and-rbac.md`, `/docs/operations/cli-release-and-packaging.md` with imposed rule statements. | Docs merged; tutorials validated; imposed rule appended; cross-links added. |
## Containerized Distribution (Epic 13)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-INSTALL-44-001 | TODO | Docs Guild, Deployment Guild | COMPOSE-44-001 | Publish `/docs/install/overview.md` and `/docs/install/compose-quickstart.md` with imposed rule line and copy-ready commands. | Docs merged; screenshots/commands verified; imposed rule appended. |
| DOCS-INSTALL-45-001 | TODO | Docs Guild, Deployment Guild | HELM-45-001 | Publish `/docs/install/helm-prod.md` and `/docs/install/configuration-reference.md` with values tables and imposed rule reminder. | Docs merged; configuration matrix verified; imposed rule appended. |
| DOCS-INSTALL-46-001 | TODO | Docs Guild, Security Guild | DEPLOY-PACKS-43-001, CLI-PACKS-43-001 | Publish `/docs/install/airgap.md`, `/docs/security/supply-chain.md`, `/docs/operations/health-and-readiness.md`, `/docs/release/image-catalog.md`, `/docs/console/onboarding.md` (each with imposed rule). | Docs merged; checksum/signature sections validated; imposed rule appended. |
## Authority-Backed Scopes & Tenancy (Epic 14)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCS-TEN-47-001 | TODO | Docs Guild, Authority Core | AUTH-TEN-47-001 | Publish `/docs/security/tenancy-overview.md` and `/docs/security/scopes-and-roles.md` outlining scope grammar, tenant model, imposed rule reminder. | Docs merged; diagrams included; imposed rule appended. |
| DOCS-TEN-48-001 | TODO | Docs Guild, Platform Ops | WEB-TEN-48-001 | Publish `/docs/operations/multi-tenancy.md`, `/docs/operations/rls-and-data-isolation.md`, `/docs/console/admin-tenants.md`. | Docs merged; examples validated; imposed rule appended. |
| DOCS-TEN-49-001 | TODO | Docs & DevEx Guilds | CLI-TEN-47-001, AUTH-TEN-49-001 | Publish `/docs/cli/authentication.md`, `/docs/api/authentication.md`, `/docs/policy/examples/abac-overlays.md`, update `/docs/install/configuration-reference.md` with new env vars, all ending with imposed rule line. | Docs merged; command examples verified; imposed rule appended. |

429
docs/airgap/EPIC_16_AIRGAP_MODE.md Normal file
View File

@@ -0,0 +1,429 @@
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
# Epic 16: Air‑Gapped Mode
**Short name:** Air‑Gapped Mode
**Primary components:** Web Services API, Console, CLI, Orchestrator, Task Runner, Conseiller (Feedser), Excitator (VEXer), Policy Engine, Findings Ledger, Export Center, Authority & Tenancy, Notifications, Observability & Forensics
**Surfaces:** offline bootstrap, update ingestion via mirror bundles, sealed egress, deterministic jobs, offline advisories/VEX, offline policy packs, offline notifications, evidence exports
**Dependencies:** Export Center, Containerized Distribution, Authority‑Backed Scopes & Tenancy, Observability & Forensics, Policy Studio
**AOC ground rule reminder:** Conseiller and Excitator aggregate and link advisories/VEX. They never merge or mutate source records. Air‑Gapped Mode must preserve this invariant even when mirroring and importing updates.
---
## 1) What it is
A fully supported operating profile where StellaOps runs in a disconnected environment with:
* **Zero external egress** from platform services and jobs.
* **Deterministic inputs** provided via signed, offline **Mirror Bundles** (advisories, VEX, policy packs, vendor feeds, Stella metadata, container images, dashboards).
* **Offline bootstrap** for images and charts, plus reproducible configuration and cryptographically verifiable updates.
* **Graceful feature degradation** with explicit UX: features that require external connectivity are either backed by local artifacts or clearly disabled with an explanation.
* **Auditable import/export** including provenance attestations, evidence bundles, and chain‑of‑custody for all offline exchanges.
Air‑Gapped Mode is selectable at install time and enforceable at runtime. When enabled, all components operate under an “egress sealed” policy and only consume data from local stores.
---
## 2) Why
Many users operate in classified, regulated, or high‑sensitivity networks where egress is prohibited. They still need SBOM analysis, policy evaluation, advisory/VEX mapping, and reporting. Air‑Gapped Mode provides the same core outcomes with verifiable offline inputs and explicit operational guardrails.
---
## 3) How it should work
### 3.1 Modes and lifecycle
* **Connected Mode:** normal operation; can create Mirror Bundles on a staging host.
* **Sealed Air‑Gapped Mode:** platform enforces no egress. Only local resources are allowed.
* **Transition flow:**
1. Prepare an offline **Bootstrap Pack** with all container images, Helm/compose charts, seed database, and initial Mirror Bundle.
2. Install in the air‑gapped enclave and **seal** egress.
3. Periodically import new **Mirror Bundles** via removable media.
4. Export evidence/reports as needed.
### 3.2 Egress sealing
* **Static guardrails:**
* Platform flag `STELLA_AIRGAP=sealed` and database feature flag `env.mode='sealed'`.
* NetworkPolicy/iptables/eBPF deny‑all egress for namespaces/pods except loopback and the internal object store.
* Outbound DNS blocked.
* HTTP clients in code use a single `EgressPolicy` facade. When sealed, it panics on direct network calls and returns a typed error with remediation (“import a Mirror Bundle”).
* **Verification:** `GET /system/airgap/status` returns `sealed: true|false`, current policy hash, and last import timestamp. CLI prints warning if not sealed in declared air‑gapped install.
### 3.3 Trusted time
* Air‑gapped systems cannot NTP. Each Mirror Bundle includes a **signed time token** (Roughtime‑style or RFC 3161) from a trusted authority. On import, platform stores `time_anchor` for drift calculations and staleness checks.
* If time drift exceeds policy threshold, UI shows “stale view” badges and some jobs are blocked until a new bundle provides a fresh anchor.
### 3.4 Mirror Bundles (offline updates)
* **Content types:**
* Public advisories (OSV, GHSA, vendor advisories), NVD mappings, CPE/Package metadata.
* VEX statements from vendors/communities.
* Policy packs (templates, baselines, versioned rule sets).
* StellaOps engine metadata and schema migrations.
* Optional: **OCI image set** for platform and recommended runners.
* Optional: dashboards and alert rule packs.
* **Format:** a TUF‑like layout:
```
root.json, snapshot.json, timestamp.json, targets/
advisories/*.jsonl.zst
vex/*.jsonl.zst
policy/*.tar.zst
images/* (OCI layout or oci-archive)
meta/engine/*.tgz
meta/time-anchor.json (signed)
```
* **Integrity & trust:**
* DSSE‑signed target manifests.
* Root of trust rotated via `root.json` within strict policy; rotation requires manual dual approval in sealed mode.
* Each content artifact has a content digest and a **Merkle root** for the overall bundle.
* **Creation:** in connected networks, `stella mirror create --content advisories,vex,policy,images --since 2025-01-01 --out bundle.tgz`.
* **Import:** in air‑gap, `stella airgap import bundle.tgz`. The importer verifies DSSE, TUF metadata, Merkle root, then writes to local object store and updates catalog tables.
* **Idempotence:** imports are content‑addressed; re‑imports deduplicate.
### 3.5 Deterministic jobs and sources
* **Allowed sources:** filesystem, internal object store, tenant private registry, and pre‑approved connectors that don’t require external egress.
* **Disallowed in sealed mode:** remote package registries, web scrapers, outbound webhooks, cloud KMS unless on the enclave network.
* **Runner policy:** the Task Runner verifies job descriptors contain no network calls unless marked `internal:` with allow‑listed destinations. Violations fail at plan time with an explainable error.
### 3.6 Conseiller and Excitator in air‑gap
* **Conseiller (Feedser):** ingests advisories only from imported bundles or tenant local feeds. It preserves source identities and never merges. Linkage uses bundle‑provided cross‑refs and local heuristics.
* **Excitator (VEXer):** imports VEX records as‑is, links them to components and advisories, and records the origin bundle and statement digests. Consensus Lens (Epic 7) operates offline across the imported sources.
### 3.7 Policy Engine and Studio
* Policy packs are versioned and imported via bundles.
* Simulation and authoring work locally. Exports of new or updated policies can be packaged as **Policy Sub‑Bundles** for transfer back to connected environments if needed.
* Engine shows which rules depend on external evidence and how they degrade in sealed mode (e.g., “No external EPSS; using cached percentile from last bundle.”).
### 3.8 Notifications in sealed mode
* Default to **local delivery** only: SMTP relay inside enclave, syslog, file sink.
* External webhooks are disabled.
* Notification templates show “air‑gap compliant channel” tags to avoid misconfiguration.
### 3.9 Observability & Forensics
* Traces, logs, metrics remain local.
* Evidence Locker supports **portable evidence packages** for cross‑domain transfer: `stella forensic snapshot create --portable`.
* Importing an evidence bundle in another enclave verifies signatures and maintains chain‑of‑custody.
### 3.10 Console and CLI behavior
* Console shows a prominent **Air‑Gapped: Sealed** badge with last import time and staleness indicators for advisories, VEX, and policy packs.
* CLI commands gain `--sealed` awareness: any operation that would egress prints a refusal with remediation suggesting the appropriate import.
### 3.11 Multi‑tenant and scope
* Tenancy works unchanged. Bundle imports can target:
* `--tenant-global`: shared catalogs (advisories, VEX, policy baselines).
* `--tenant=<id>`: tenant‑specific content (e.g., private advisories).
* Authority scopes gain `airgap:import`, `airgap:status:read`, `airgap:seal` (admin‑only).
### 3.12 Feature degradation matrix
* **AI Assistant:** offline variants use local models if installed; otherwise feature is disabled with a message.
* **External reputation feeds (e.g., EPSS‑like):** replaced by cached values from the bundle.
* **Container base image lookups:** rely on imported metadata or tenant private registry.
---
## 4) Architecture
### 4.1 New modules
* `airgap/controller`
* Sealing state machine; status API; guardrails wiring into HTTP clients and runner.
* `airgap/importer`
* TUF/DSSE verification, Merkle validation, object store loader, catalog updater.
* `mirror/creator`
* Connected‑side builder for bundles; content plug‑ins for advisories/VEX/policy/images.
* `airgap/policy`
* Enforcement library exposing `EgressPolicy` facade and job plan validators.
* `airgap/time`
* Time anchor parser, drift checks, staleness annotations.
* `console/airgap`
* Sealed badge, import UI, staleness dashboards, degradation notices.
* `cli/airgap`
* `stella airgap seal|status|import|verify` commands; `stella mirror create|verify`.
### 4.2 Data model additions
* `airgap_state(id, sealed BOOLEAN, policy_hash TEXT, last_import_at TIMESTAMP, time_anchor JSONB)`
* `bundle_catalog(id, kind ENUM, merkle_root TEXT, dsse_signer TEXT, created_at TIMESTAMP, imported_at TIMESTAMP, scope ENUM('global','tenant'), tenant_id NULLABLE, labels JSONB)`
* `bundle_items(bundle_id, path TEXT, sha256 TEXT, size BIGINT, type TEXT, meta JSONB)`
* `import_audit(id, bundle_id, actor, tenant_scope, verify_result, trace_id, created_at)`
RLS: tenant‑scoped rows when `scope='tenant'`; global rows readable only with `stella:airgap:status:read`.
### 4.3 Storage layout
Object store paths:
```
tenants/_global/mirror/<bundle_id>/targets/...
tenants/<tenant>/mirror/<bundle_id>/targets/...
tenants/_global/images/<digest>/...
```
Evidence locker remains separate. Imported images use **OCI layout** for local registry sync.
### 4.4 Message topics
* `stella.<tenant>.airgap.imported` with bundle metadata.
* `stella.<tenant>.airgap.staleness` periodic events emitted for UX.
* `stella.<tenant>.policy.degraded` when rules fall back due to sealed mode.
---
## 5) APIs and contracts
### 5.1 Status and control
* `GET /system/airgap/status` → `{ sealed, policy_hash, last_import_at, time_anchor, drift_seconds, staleness: { advisories_days, vex_days, policy_days } }`
* `POST /system/airgap/seal` → seals environment; requires `stella:airgap:seal#tenant/<id or global>`.
* `POST /system/airgap/unseal` → only allowed if installed mode is not declared “permanently sealed” at bootstrap. Typically disabled.
### 5.2 Import & verify
* `POST /airgap/import` multipart or file reference → runs verify, writes catalog, returns bundle summary and warnings.
* `POST /airgap/verify` dry‑run verification returning DSSE/TUF and Merkle results.
* `GET /airgap/bundles` list imported bundles with filters.
### 5.3 Conseiller/Excitator sources
* `POST /feeds/register` supports `kind=mirror` with `bundle_id` and paths; disallowed to point to external URLs in sealed mode.
* `GET /feeds/status` shows per‑source staleness and last artifact version.
### 5.4 Errors
Standardized sealed‑mode error:
```
{
"code": "AIRGAP_EGRESS_BLOCKED",
"message": "Egress is sealed. Import a Mirror Bundle with advisories.",
"remediation": "Run: stella airgap import bundle.tgz",
"trace_id": "..."
}
```
---
## 6) Documentation changes
Create or update:
1. `/docs/airgap/overview.md`
* Modes, lifecycle, responsibilities, threat model, what degrades.
2. `/docs/airgap/bootstrap.md`
* Offline Bootstrap Pack creation, validation, install steps for Helm/compose, local registry seeding.
3. `/docs/airgap/mirror-bundles.md`
* Bundle format, DSSE/TUF/Merkle, signed time, creation on connected host, import in sealed environment, rotation of roots.
4. `/docs/airgap/sealing-and-egress.md`
* Network policies, EgressPolicy facade, runner validation, verifying sealed status.
5. `/docs/airgap/staleness-and-time.md`
* Time anchor, drift, staleness budgets and UI behavior.
6. `/docs/airgap/operations.md`
* Periodic update cadence, runbooks, failure scenarios, disaster recovery.
7. `/docs/airgap/degradation-matrix.md`
* Feature map: available, degraded, disabled; with remediation.
8. `/docs/console/airgap.md`
* Status badges, import wizard, staleness indicators.
9. `/docs/cli/airgap.md`
* Commands, examples, exit codes.
10. `/docs/security/trust-and-signing.md`
* Roots of trust, key rotation, DSSE, TUF model.
11. `/docs/dev/airgap-contracts.md`
* EgressPolicy usage, testing patterns, sealed‑mode CI gates.
Add the banner at the top of each page:
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Implementation plan
### Phase 1 — Foundations
* Add `airgap/controller` with sealed state and status API.
* Integrate `EgressPolicy` facade in all outbound network call sites.
* Provide default NetworkPolicy/iptables templates and Helm values to block egress.
* Console shows sealed badge and status.
### Phase 2 — Mirror Bundles
* Implement `mirror/creator` in connected mode with content plug‑ins.
* Implement `airgap/importer` with DSSE/TUF/Merkle verification and catalog updates.
* Export Center gains **Mirror bundle** build and verify commands (connected side).
### Phase 3 — Deterministic jobs
* Add job plan validation in the Task Runner.
* Restrict sources in sealed mode.
* Conseiller/Excitator add “mirror source” adapters.
### Phase 4 — Staleness and time
* Parse time anchors; enforce staleness budgets; add UI indicators and task refusal when budgets exceeded.
* Notifications for expiring anchors.
### Phase 5 — Degradation matrix and UX
* Wire feature flags and fallbacks in Console and APIs.
* Improve error messages with remediation guidance.
### Phase 6 — Evidence portability
* Portable evidence packages: export/import with full verification.
* Document cross‑domain workflows.
---
## 8) Engineering tasks
**Air‑gap controller and sealing**
* [ ] Implement `airgap/controller` with persistent state and RBAC.
* [ ] Add `GET /system/airgap/status`, `POST /system/airgap/seal`.
* [ ] Provide cluster egress templates for Kubernetes and for docker‑compose.
* [ ] Instrument startup checks to refuse running in sealed mode if egress rules aren’t applied.
**EgressPolicy integration**
* [ ] Create `pkg/egress` facade and replace all direct HTTP client constructions in services.
* [ ] Add linter rule and CI check forbidding raw `http.NewClient` in server code.
* [ ] Add unit tests for sealed and unsealed behavior.
**Mirror bundles**
* [ ] Implement TUF/DSSE verifiers and Merkle root builder.
* [ ] Build content plug‑ins: advisories, VEX, policy packs, images.
* [ ] Write `bundle_catalog` and `bundle_items` tables with RLS.
* [ ] CLI: `stella mirror create|verify`, `stella airgap import|verify`.
**Conseiller/Excitator**
* [ ] Add mirror adapters for read‑only ingestion from bundle paths.
* [ ] Persist source digests and bundle IDs on each linked record.
* [ ] Unit tests to ensure no merge behavior is introduced by bundle ingestion.
**Policy Engine & Studio**
* [ ] Accept policy packs from bundles; track `policy_version` and `bundle_id`.
* [ ] Add degradation notices for rules requiring external reputation; provide cached fallbacks.
**Task Runner & Orchestrator**
* [ ] Plan‑time validation against network calls; add `internal:` allow‑list mapping.
* [ ] Emit sealed‑mode violations to Timeline with remediation text.
**Console**
* [ ] Status panel: sealed badge, last import, staleness meters.
* [ ] Import wizard with verify results and catalog diff preview.
* [ ] Degradation matrix UI and contextual tooltips.
**Observability & Forensics**
* [ ] Mark sealed mode in telemetry attributes.
* [ ] Add portable evidence package export/import; verify on read.
**Authority & Tenancy**
* [ ] New scopes: `airgap:seal`, `airgap:import`, `airgap:status:read`.
* [ ] Audit import actions with actor and trace ID.
**Docs**
* [ ] Author all pages listed in section 6, include signed‑time workflow diagrams.
* [ ] Insert banner statement in each page.
**Testing**
* [ ] Sealed‑mode e2e: attempt egress; ensure refusal and remediation.
* [ ] Bundle import e2e: corrupt DSSE, wrong root, tampered artifact → rejected.
* [ ] Performance: large advisory bundle import within target time (see Acceptance).
* [ ] Time drift scenarios and staleness budget enforcement.
* [ ] Regression: ensure AOC rules unchanged in sealed mode.
---
## 9) Feature changes required in other components
* **Export Center:** add mirror bundle export profile, signed‑time token inclusion, and portable evidence packages.
* **Notifications:** remove external webhooks by default in sealed mode; add local SMTP/syslog sinks.
* **CLI Parity:** ensure all admin and import operations are exposed; add sealed‑mode safety prompts.
* **Containerized Distribution:** ship **Bootstrap Pack** that includes all images and charts in a single oci‑archive set with index manifest.
* **Observability:** disable remote exporters; include local dashboards; mark sealed mode in UI.
* **Policy Studio:** enable offline authoring and export of policy sub‑bundles.
* **VEX Consensus Lens:** ensure it operates solely on imported VEX statements; highlight coverage vs. stale.
> **Imposed rule reminder:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 10) Acceptance criteria
* Environment can be **sealed** and verified via API, CLI, and network policies.
* Import of a valid Mirror Bundle succeeds; DSSE, TUF, and Merkle validations recorded in `import_audit`.
* Conseiller and Excitator operate only on imported sources; linkage reflects original source identities.
* Policy packs are importable and versioned; rules that depend on external evidence show clear degradation.
* Large bundle (e.g., 8–12 GB with images) imports in under 20 minutes on SSD storage and indexes advisories in under 5 minutes on a 4‑core node.
* Console displays sealed badge, last import, staleness, and degradation matrix.
* Attempted egress in sealed mode fails with `AIRGAP_EGRESS_BLOCKED` and remediation.
* Portable evidence packages export and verify across separate enclaves.
* All changes documented with the banner statement.
---
## 11) Risks and mitigations
* **Key management complexity:** rotate TUF roots with dual‑control workflow and explicit docs; fail‑safe to previous root if rotation bundle absent.
* **Staleness risk:** enforce budgets and block risk‑critical jobs when expired; provide monitoring and notifications for impending staleness.
* **Operator error during import:** dry‑run verification, diff preview of catalog changes, and ability to roll back via content address.
* **Hidden egress paths:** CI lints and runtime guardrails; network policies enforced at cluster layer.
* **Bundle size bloat:** Zstandard compression, delta bundles, and selective content flags for creation.
---
## 12) Philosophy
* **Predictable over perfect:** deterministic, explainable results beat unknown “live” results in sensitive networks.
* **Trust is earned:** every offline exchange is signed, verifiable, and auditable.
* **Degrade transparently:** when features reduce capability, explain it and guide remediation.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

11
docs/aoc/aoc-guardrails.md Normal file
View File

@@ -0,0 +1,11 @@
# Aggregation-Only Contract (AOC) Guardrails
The Aggregation-Only Contract keeps ingestion services deterministic and policy-neutral. Use these checkpoints whenever you add or modify backlog items:
1. **Ingestion writes raw facts only.** Concelier and Excititor append immutable observations/linksets. No precedence, severity, suppression, or "safe fix" hints may be computed at ingest time.
2. **Derived semantics live elsewhere.** Policy Engine overlays, Vuln Explorer composition, and downstream reporting layers attach severity, precedence, policy verdicts, and UI hints.
3. **Provenance is mandatory.** Every ingestion write must include original source metadata, digests, and signing/provenance evidence when available. Reject writes lacking provenance.
4. **Deterministic outputs.** Given the same inputs, ingestion must produce identical documents, hashes, and event payloads across reruns.
5. **Guardrails everywhere.** Roslyn analyzers, schema validators, and CI smoke tests should fail builds that attempt forbidden writes.
For detailed roles and ownership boundaries, see `AGENTS.md` at the repo root and the module-specific `ARCHITECTURE_*.md` dossiers.

356
docs/api/EPIC_17_SDKS_OPENAPI.md Normal file
View File

@@ -0,0 +1,356 @@
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
# Epic 17: SDKs and OpenAPI Docs
**Short name:** SDKs & OpenAPI
**Primary components:** API Gateway, Web Services, Policy Engine, Conseiller (Feedser), Excitator (VEXer), Orchestrator, Findings Ledger, Export Center, Authority & Tenancy, Console, CLI
**Surfaces:** OpenAPI 3.1 contracts, language SDKs (TS/Node, Python, Go, Java, C#), dev portal, examples, mock server, conformance tests, changelogs, versioning, deprecations
**Dependencies:** Authority‑Backed Scopes & Tenancy, CLI Parity, Export Center, Notifications Studio, Air‑Gapped Mode, Observability
**AOC ground rule reminder:** Conseiller and Excitator aggregate and link advisories/VEX. They never merge or mutate source records. SDKs must preserve this invariant and expose source identity in all models.
---
## 1) What it is
A contract‑first program that standardizes StellaOps’ APIs with OpenAPI 3.1 and ships official, versioned SDKs for popular languages. It includes:
* A single **source‑of‑truth OpenAPI** for each service and a canonical aggregate spec.
* **Generated SDKs** with idiomatic ergonomics, retries, auth helpers, pagination cursors, streaming downloads, and typed error envelopes.
* A **developer portal** with interactive reference, runnable examples, and “copy‑curl” snippets.
* **Mock server & conformance tests** so changes are validated against the contract before code ships.
* **Versioning & deprecation policy**, automated changelogs, and notification hooks.
* **Air‑gapped bundles** of docs and SDKs for disconnected environments.
Net result: partners and internal teams integrate quickly without reverse‑engineering request bodies from error logs.
---
## 2) Why
* Reduce friction and support load with a single, accurate contract.
* Make the platform extensible: third parties can build automation, dashboards, and policy pipelines without trawling source code.
* Enforce stability: contract linting and backwards‑compat checks prevent accidental breakage.
* Bring CLI and Console parity to programmatic users through first‑class clients.
---
## 3) How it should work
### 3.1 Source of truth and layout
* Each service owns a **module‑scoped OAS** file: `src/StellaOps.Api.OpenApi/<service>/openapi.yaml`.
* An aggregate spec `src/StellaOps.Api.OpenApi/stella.yaml` is produced by build tooling that composes per‑service specs, resolves `$ref`s, and validates cross‑service schemas.
* JSON Schema dialect: 2020‑12 (OpenAPI 3.1). No vendor‑specific features for core models.
* Every response and error has at least one **validated example**.
### 3.2 API conventions (normative)
* **Paths:** `/v1/{resource}`, plural nouns. Subresources use `/v1/resources/{id}/subresources`.
* **Identifiers:** `id` fields are ULID/UUIDv7 as strings.
* **Pagination:** cursor‑based: `?cursor=<token>&limit=<n>`, response envelope includes `next_cursor`.
* **Sorting/filtering:** `?sort=field:asc|desc`, `?filter[field]=op:value` with documented operators.
* **Idempotency:** POST operations that create or mutate accept `Idempotency-Key`.
* **Errors:** single envelope:
```json
{
"error": {
"code": "STRING_CODE",
"message": "human friendly",
"details": { "field": "value" },
"trace_id": "..."
}
}
```
Standard codes include `AIRGAP_EGRESS_BLOCKED`, `POLICY_VIOLATION`, `NOT_FOUND`, `RATE_LIMITED`.
* **Auth:** OAuth2 client credentials and PAT. Scopes are explicit (see 14: Authority‑Backed Scopes). Tenancy via claims; optional override header: `X-Stella-Scope: tenant/<id>` if the token permits delegation.
* **Content negotiation:** JSON only for request/response unless endpoint is a stream or file download (`application/octet-stream`).
* **Long‑running operations:** either webhooks (if enabled) or polling via `operation_id` resource.
### 3.3 Versioning and deprecation
* **SemVer** for the aggregate API: `v1`, `v2` in base path.
* Backwards‑compatible changes allowed in minor versions (add fields, new optional params).
* Breaking changes require new major version and coexistence for a **deprecation window** (min 12 months) with:
* Deprecation headers: `Deprecation: true`, `Sunset: <rfc1123 date>`, `Link: <doc rel="deprecation">`.
* Portal banners and Notifications Studio broadcast.
### 3.4 Governance and linting
* Enforce naming, pagination, error envelope, and example requirements via an OAS linter.
* CI gate: no PR merges if OAS validation fails or coverage < 100% for operation examples.
* **Compatibility check:** diff new OAS vs previous release, fail on breaking changes unless explicitly flagged.
### 3.5 SDK generation
* Initial languages: **TypeScript/Node**, **Python**, **Go**, **Java**. C# and Rust are follow‑ups.
* Generated via a stable, reproducible toolchain. Post‑generation patches are applied by templates, not hand edits.
* **Capabilities:**
* Auth helpers: PAT and OAuth2.
* Retries with decorrelated jitter and `Retry‑After` respect.
* Pluggable HTTP transport for proxies and air‑gapped environments.
* Binary download helpers and upload helpers for multipart endpoints.
* Paginators that yield items and handle `next_cursor`.
* Rich error types mapping `error.code` to language enums.
* Telemetry hooks (before/after request callbacks).
* **Packaging:**
* TS: npm package with ESM and CJS builds, types included.
* Python: PyPI package, Pydantic‑friendly models, type hints.
* Go: module with context‑aware methods and `io.Reader` streaming.
* Java: Maven coordinates, builder pattern, OkHttp/HTTP client provider.
* **Versioning:** SDK major matches API major. Minor/patch track generator changes only.
### 3.6 Dev portal and artifacts
* **Reference docs** auto‑built from the aggregate OAS with searchable nav, schema diagrams, and example blocks.
* **Try‑it** panel wired to the sandbox environment (disabled in air‑gap).
* **Download center:** links to SDKs, changelogs, and Postman/HTTP collection exports.
* **.well‑known discovery:** `GET /.well-known/openapi` returns the canonical spec.
### 3.7 Conformance testing
* **Mock server** generated from OAS for contract tests.
* **Replay tests**: real services are validated against the OAS via request/response capture; deviations fail CI.
* **Golden examples**: every endpoint has recorded examples exercised in tests.
### 3.8 Air‑Gapped support
* Export Center can build a **Docs & SDKs bundle**: `stella export devportal --offline`, including HTML docs, specs, and packages.
* SDKs avoid network discovery and accept explicit base URLs; no auto‑updates.
### 3.9 Domain‑specific notes
* **Conseiller/Excitator:** models expose `source_id`, `source_type`, `source_digest`. SDKs never hide source multiplicity.
* **Policy Engine:** policy documents are versioned; SDK supports dry‑run/simulate endpoints with structured explanations.
* **Findings Ledger:** paginated listing includes stable, filterable fields for evidence export.
---
## 4) Architecture
### 4.1 New modules
* `src/StellaOps.Api.OpenApi/*` per service and aggregate composer
* `src/StellaOps.Api.Governance` OAS linter rules and compatibility checker
* `src/StellaOps.Sdk.Generator` codegen drivers, post‑processing templates, smoke tests
* `src/StellaOps.Sdk.Release` packaging, signing, publishing
* `src/StellaOps.DevPortal.Site` static generator and assets
* `test/contract` mock server config, golden examples
* `src/StellaOps.ExportCenter.DevPortalOffline` bundler
### 4.2 Build flow
1. Validate per‑service specs → compose aggregate → lint → compatibility diff.
2. Generate SDKs → build → run language‑level tests → publish to internal registry.
3. Build dev portal and publish.
4. Optionally build offline bundle.
### 4.3 Runtime contracts
* `GET /.well-known/openapi` per service and at the gateway.
* All services embed `x-stella-service` and `x-stella-version` extensions for traceability.
---
## 5) APIs and contracts (select)
* **Discovery**: `GET /.well-known/openapi` → JSON or YAML.
* **Errors**: standard envelope (see 3.2).
* **Rate limits**: expose `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`.
* **Operations**: long‑running ops expose `operation_id` and `status` via `GET /v1/operations/{id}`.
---
## 6) Documentation changes
Create or update:
1. `/docs/api/overview.md`
* API surface, auth, tenancy, pagination, idempotency, rate limits.
2. `/docs/api/conventions.md`
* Path, naming, errors, filters, sorting, examples.
3. `/docs/api/versioning.md`
* SemVer policy, deprecation windows, headers, migration playbooks.
4. `/docs/api/reference/`
* Auto‑generated OAS site; link into service pages.
5. `/docs/sdks/overview.md`
* Supported languages, install, hello‑world, retry/auth patterns.
6. `/docs/sdks/typescript.md`, `/python.md`, `/go.md`, `/java.md`
* Language‑specific guides, snippets, paginator usage, streaming.
7. `/docs/devportal/publishing.md`
* Build pipeline, offline bundle steps.
8. `/docs/contributing/api-contracts.md`
* How to edit OAS, lint rules, compatibility checks, examples.
9. `/docs/testing/contract-testing.md`
* Mock server, golden examples, replay tests.
10. `/docs/security/auth-scopes.md`
* OAuth2, PAT, scope mapping, tenancy header.
11. `/docs/airgap/devportal-offline.md`
* Air‑gapped docs and SDK bundle usage.
Add the banner at the top of each page:
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 7) Implementation plan
### Phase 1 — Foundations
* Establish per‑service OAS skeletons and the aggregate composer.
* Introduce linting and compatibility checks in CI.
* Define the standard error envelope and migrate services.
### Phase 2 — Reference & discovery
* Implement `/.well-known/openapi` for gateway and services.
* Build the dev portal with search, schema diagrams, and examples.
### Phase 3 — SDKs (TS, Python, Go, Java)
* Implement generator drivers and templates.
* Publish alpha packages internally; integrate in CLI and Console integration tests.
* Add paginators, retries, auth helpers, and streaming.
### Phase 4 — Conformance & examples
* Wire mock server into PR CI.
* Record golden example fixtures and replay tests against staging.
* Automate example extraction into docs.
### Phase 5 — Release automation & deprecation
* Automate changelogs from OAS diffs.
* Notifications Studio integration for API deprecations.
* Offline dev portal bundle through Export Center.
### Phase 6 — Follow‑ups
* C# and Rust SDKs, Postman/HTTP collections, sample apps repo.
---
## 8) Engineering tasks
**OAS & governance**
* [ ] Create `src/StellaOps.Api.OpenApi/<service>/openapi.yaml` for all services with minimal paths and shared components.
* [ ] Implement aggregate composer and `$ref` resolver.
* [ ] Add CI job: lint, validate, compatibility diff; block merges on failure.
* [ ] Migrate all endpoints to standard error envelope and provide examples.
**Discovery & portal**
* [ ] Implement `GET /.well-known/openapi` at service and gateway.
* [ ] Build dev portal: nav, search, schema viewer, try‑it (non‑prod), copy‑curl.
* [ ] Add version selector for v1/v2 specs.
**SDKs**
* [ ] Generator driver with pinned templates; forbid manual edits in generated folders.
* [ ] TS SDK: ESM/CJS build, tree‑shaking, paginator, middleware hooks.
* [ ] Python SDK: async and sync clients, type hints, file upload/download helpers.
* [ ] Go SDK: context‑first API, streaming, error type mapping.
* [ ] Java SDK: builder pattern, HTTP client provider abstraction.
* [ ] Common: retries, `Retry‑After` handling, idempotency key helper, auth helpers, telemetry hooks.
* [ ] Language‑specific tests and smoke examples.
**Conformance**
* [ ] Mock server config with operation examples.
* [ ] Replay tests against staging; fail on schema drift.
* [ ] Golden example extraction pipeline.
**Air‑Gapped**
* [ ] Export Center job: `devportal --offline` producing HTML docs, specs, and package artifacts.
* [ ] SDKs accept explicit base URLs; disable online discovery.
**Authority & Tenancy**
* [ ] Document scopes per endpoint in OAS (`securitySchemes` + `security` blocks).
* [ ] Implement optional `X-Stella-Scope` override with validation.
**Release automation**
* [ ] Version bump tooling for OAS and SDKs; SemVer aligned.
* [ ] Auto‑generate `CHANGELOG.md` from OAS diffs.
* [ ] Publish to registries with signed artifacts and provenance.
**Docs**
* [ ] Author all pages listed in section 6; embed code snippets pulled from tested examples.
* [ ] Insert banner statement in each page.
**Testing**
* [ ] Contract tests in PR CI; 100% operation coverage with at least one example.
* [ ] Language SDK integration tests against mock server and staging.
* [ ] Backwards‑compat test suite comparing last N releases.
---
## 9) Feature changes required in other components
* **Web Services:** unify on error envelope, pagination, idempotency handling, and deprecation headers.
* **CLI:** consume the official TS or Go SDK instead of bespoke HTTP calls; this enforces parity.
* **Console:** use SDKs for backend calls where appropriate; helps dogfood the clients.
* **Export Center:** add `devportal --offline` and package signing.
* **Observability:** include `x-stella-service` and API version attributes in spans; trace IDs mirrored in error responses.
* **Notifications Studio:** templates for API deprecations and SDK updates.
* **Air‑Gapped Mode:** ship offline dev portal and SDKs bundle; console disables try‑it.
> **Imposed rule reminder:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
## 10) Acceptance criteria
* Aggregate OpenAPI validates, lints cleanly, and covers 100% of public endpoints with examples.
* `/.well-known/openapi` available at gateway and service level.
* Dev portal builds with search, example blocks, and version selector.
* TS/Python/Go/Java SDKs publish successfully; each has paginators, retries, auth helpers, streaming, and typed errors.
* CLI integrations pass using SDKs.
* Contract tests run in PR CI; schema drift causes failures.
* Deprecation headers and Notifications Studio flow proven in a staged deprecation.
* Offline dev portal bundle exports and renders in a sealed environment.
---
## 11) Risks and mitigations
* **Spec drift vs code reality:** mock‑first development and replay tests keep services aligned with OAS.
* **Generator churn:** pin generator and templates; only update via planned minor releases.
* **Breaking changes under pressure:** enforce compatibility gate and documented exception process.
* **SDK ergonomics mismatch:** run language‑native design reviews with maintainers before GA.
* **Air‑gapped constraints:** prebuild full offline bundles; avoid dynamic CDN assets in docs.
---
## 12) Philosophy
* **Contract first, code second.** The spec is the product; servers and SDKs are implementations.
* **Stability over cleverness.** Boring, predictable APIs beat “magical” behavior.
* **Truth preservation.** Never hide or merge advisory/VEX sources; surface provenance everywhere.
* **Automation everywhere.** Humans shouldn’t manually edit generated code or publish packages.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

135
docs/attestor/EPIC_19_ATTESTOR_CONSOLE.md Normal file
View File

@@ -0,0 +1,135 @@
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
# Epic 19: Attestor Console
**Short name:** Attestor Console
**Primary components:** Attestation Service, Policy Studio, Authority & Tenancy, Export Center, Observability, CLI, StellaOps Console
**Surfaces:** Web UI, REST APIs, CLI, SDKs, docs
**AOC ground rule reminder:** Conseiller and Excitator only aggregate and link. They never merge sources. Attestations must reference the exact subject digests and their source provenance. No collapsing of distinct advisories or VEX statements.
---
## 1) What it is
Attestor Console centralizes creation, verification, browsing, and operationalization of software supply chain attestations within StellaOps. It covers build provenance, SBOM attestations, VEX statements, scan results, policy evaluations, risk profile evidence, and custom facts—all signed and verifiable end-to-end.
---
## 2) Why
Trustworthy pipelines need signed evidence, repeatable verification, explainability, and policy enforcement. Attestor Console weaves attestations into the rest of Stella (AOC, Policy Studio gates, Risk Profiles, Notifications, Export Center) to reduce blind spots and accelerate compliance.
---
## 3) How it should work
### 3.1 Roles, identities, scopes
* Subjects: immutable digests for artifacts (images, packages, SBOMs, findings).
* Issuers: identities that sign attestations (builders, scanners, policy engines).
* Scopes: tenant/project/environment context enforced by Authority & Tenancy.
### 3.2 Payload types
Supports DSSE envelopes for BuildProvenance v1, SBOMAttestation v1, VEXAttestation v1, ScanResults v1, PolicyEvaluation v1, RiskProfileEvidence v1, and CustomEvidence v1. All payloads include subject, issuer, scope, materials, provenance, policy context, and versioned schemas.
### 3.3 Envelope & signature model
DSSE envelopes with multi-signature support, Ed25519/ECDSA keys, KMS/HSM/FIDO2 drivers, transparency log witnesses, and detached payload storage. Identity policies ensure least privilege and traceability.
### 3.4 Verification pipeline
Runs at ingest, policy gates, and interactively. Steps: resolve subject, fetch envelopes + witness proofs, validate DSSE structure/signatures, evaluate issuer trust and policies, produce cached verification reports.
### 3.5 Verification policies
Policy Studio authored rules covering required evidence types, allowed issuers, freshness, transparency requirements, signature counts, and waivers. Supports scoped overrides and defaults.
### 3.6 UI workflows
Evidence Browser, Verification Reports, Chain of Custody Graph, Key & Issuer Management, Attestation Workbench, and Bulk Verification views in Console.
### 3.7 CLI & SDK
Commands: `stella attest sign`, `verify`, `list`, `fetch`, `key create/import/rotate/revoke`. SDKs expose Sign/Verify/List/Fetch APIs.
### 3.8 Data model
Tables for attestations, issuers, verification reports, transparency index, key store. Indexed by subject digest, type, issuer, and timestamps.
### 3.9 Storage & air gap
Store envelopes in CAS object storage; optionally mirror transparency logs. `stella export attestation-bundle` enables offline transfer. Policies can relax witness requirements for sealed environments while logging the gap.
### 3.10 Observability & security
Spans, metrics, logs for signing and verification. Private keys never leave KMS/HSM. Revocation/rotation supported. Verification rejects mismatched subjects and ensures AOC invariants for scan/VEX evidence.
### 3.11 Performance
Use compressed JSON payloads, cached verification results, batched operations, and concurrency controls. P95 target: 1k envelopes/min on a single worker.
---
## 4) Architecture
New services (`src/StellaOps.Attestor/`), libraries (`src/StellaOps.Attestor.Envelope/`, `src/StellaOps.Attestor.Types/`, `src/StellaOps.Attestor.Verify/`), CLI (`src/StellaOps.Cli/`), export tooling (`src/StellaOps.ExportCenter.AttestationBundles/`), and shared KMS providers (`src/StellaOps.Cryptography.Kms/`). REST endpoints documented in OpenAPI.
---
## 5) Documentation changes
Requires new pages for overview, payloads, policies, workflows, key management, transparency, air gap, console UI, CLI, and updated security invariants, all with the imposed rule banner.
---
## 6) Implementation plan
Six phases: Foundations; Policies & UI; Scan & VEX support; Transparency & keys; Bulk & air gap; Performance & hardening.
---
## 7) Engineering tasks
Detailed tasks across envelope/crypto, payload schemas, APIs, Policy Studio integration, Console UI, CLI, transparency, bulk operations, observability, security, docs, and testing.
---
## 8) Feature changes required elsewhere
Policy Studio (VerificationPolicy), Export Center (attestation bundles), Authority & Tenancy, SBOM/Vulnerability explorers, Notifications, Observability. All must inherit the imposed rule and update docs accordingly.
---
## 9) Acceptance criteria
Signing and verification for all payloads, policy enforcement, Console views, bulk verification, export/import for air gap, observability coverage, AOC invariants respected, documented OpenAPI endpoints. Tests and performance targets met.
---
## 10) Risks & mitigations
Key compromise, parsing bugs, policy complexity, transparency outages addressed via rotation workflows, fuzz tests, curated starter policies, and fallback/mirroring strategies.
---
## 11) Philosophy
Evidence first, scoped identities, cheap verification, portable attestations, truth preservation.
---
## 12) Examples
Includes abbreviated SBOM attestation and verification report JSON samples illustrating required fields and outcomes.
---
## 13) Cross-epic documentation updates
Cross-link Attestor docs from Policy Studio, Export Center, Air-Gapped, Observability, Risk Profiles, SBOM Graph, and Vulnerability Explorer pages, maintaining the imposed rule banner.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

17
docs/backlog/2025-10-cleanup.md Normal file
View File

@@ -0,0 +1,17 @@
# Backlog Cleanup — 26 October 2025
This note captures the Sprint backlog hygiene pass applied on 26 October 2025. The goal was to eliminate legacy tasks that violated the aggregation-only contract (AOC), duplicated scope, or conflicted with the current module ownership map.
## Summary
- **Console replaces legacy Angular UI.** Sprint 13 UI tasks (`UI-SCANS-13-002`, `UI-VEX-13-003`, `UI-ADMIN-13-004`, `UI-SCHED-13-005`) are retired. Console Sprint 23 (`CONSOLE-CORE-23-001..005`, `CONSOLE-FEAT-23-101..109`, `CONSOLE-REL-23-301..303`) owns the experience.
- **Policy CLI runtime verbs consolidated.** `CLI-RUNTIME-13-005` is superseded by `CLI-POLICY-20-002` and Policy Studio flows (`CLI-POLICY-27-00x`).
- **Notifier supersedes legacy Notify.* modules.** All Sprint 15 `StellaOps.Notify.*` tasks are archived. Replacement work lives in Notifications Studio / Notifier Sprints 38–40 (`NOTIFY-SVC-38-00x`, `NOTIFY-SVC-39-00x`, `NOTIFY-SVC-40-00x`, plus `WEB/CLI-NOTIFY-3x-00x`).
- **Cartographer owns graph construction.** `SBOM-GRAPH-24-00{1..4}` tasks are deleted from SBOM Service; Cartographer backlog (`CARTO-GRAPH-21-001..009`) covers graph storage, overlays, and tiling.
- **Dedicated Vuln Explorer service.** Gateway/UI/CLI entries that attempted to inline Vuln Explorer logic (`WEB-GRAPH-24-003`, `UI-GRAPH-24-005`, `CLI-VULN-24-003`) now defer to Sprint 29 Vuln Explorer (`VULN-API-29-00x`, `CONSOLE-VULN-29-00x`, `CLI-VULN-29-00x`).
- **AOC enforcement.** Ingestion-layer tasks attempting to compute derived severity/safe-fix metadata (`CONCELIER-VULN-29-003`, `EXCITITOR-VULN-29-003`) were removed; the Policy Engine overlay backlog (`POLICY-ENGINE-29-001..003`) is the canonical home.
- **CI/Offline adjustments.** `DEVOPS-UI-13-006` and `DEVOPS-OFFLINE-18-003` moved under Console release tasks (`CONSOLE-QA-23-401`, `DEVOPS-CONSOLE-23-001`, `CONSOLE-REL-23-302`).
## Follow-up
- Update module task boards only under their active backlogs (`src/StellaOps.Notifier`, Cartographer, Vuln Explorer).
- Ensure future ingestion tasks reference AOC guardrails and avoid derived semantics.
- Cross-check `SPRINTS.md` after adding new tasks to keep tables consistent with module `TASKS.md` files.

4
docs/ops/scanner-analyzers-operations.md
View File

@@ -9,10 +9,10 @@ Keep the language analyzer microbench under the < 5 s SBOM pledge. CI emits
1. CI (or engineers running locally) execute:
```bash
dotnet run \
--project bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj \
--project src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj \
-- \
--repo-root . \
--out bench/Scanner.Analyzers/baseline.csv \
--out src/StellaOps.Bench/Scanner.Analyzers/baseline.csv \
--json out/bench/scanner-analyzers/latest.json \
--prom out/bench/scanner-analyzers/latest.prom \
--commit "$(git rev-parse HEAD)" \

260
docs/risk/EPIC_18_RISK_PROFILES.md Normal file
View File

@@ -0,0 +1,260 @@
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
---
# Epic 18: Risk Scoring Profiles
**Short name:** Risk Profiles
**Primary components:** Policy Engine, Findings Ledger, Conseiller (Feedser), Excitator (VEXer), StellaOps Console, Policy Studio, CLI, Export Center, Authority & Tenancy, Observability
**Surfaces:** policy documents, scoring engine, factor providers, explainability artifacts, APIs, CLI, UI
**AOC ground rule reminder:** Conseiller and Excitator aggregate and link advisories/VEX. They never merge or mutate source records. Risk scoring consumes linked items and computes a contextual score per finding and per asset without collapsing sources; provenance is preserved and shown.
---
## 1) What it is
Risk Scoring Profiles let users define, version and apply customizable formulas that turn raw signals (CVSS, EPSS‑like exploit likelihood, KEV‑style exploited lists, VEX status, reachability, runtime evidence, fix availability, asset criticality, provenance trust, etc.) into a single normalized risk score from 0 to 100 with severity buckets. Profiles are authored in Policy Studio, attached to scopes/tenants/projects, simulated against inventories and SBOMs, and executed by a scoring engine that outputs:
* A final score and severity.
* A factor‑by‑factor contribution breakdown with math.
* Gating decisions (e.g., VEX “not affected” forces score to 0).
* Audit and provenance for every signal used.
Profiles can differ by environment: “Exploit‑aware prod,” “Compliance‑focused,” “Safety‑critical,” “Dev velocity,” and so on. The engine is pluggable: new signals can be added without breaking existing profiles.
---
## 2) Why
* One size doesn’t fit anyone. Different orgs weigh exploitability vs business criticality differently.
* Reduce noise and accelerate triage by aligning scores with how teams actually make decisions.
* Make risk explainable. If a score says 86, show why.
* Enable policy‑aware flows elsewhere: gates, notifications, dashboards, remediation queues.
---
## 3) How it should work
### 3.1 Core model
A **RiskProfile** defines:
* **Metadata:** name, version, description, owner, scope selector, status (draft/published/deprecated).
* **Signals:** named inputs with source bindings and transforms.
* **Formula:** a composition of weighted terms, caps, gates, and overrides producing a 0‑100 score.
* **Severity mapping:** score→{Critical, High, Medium, Low, None}.
* **Gates:** hard conditions that short‑circuit scoring (e.g., VEX Not Affected → 0).
* **Overrides:** explicit per‑package/per‑CVE/per‑asset adjustments with audit.
* **Explainability:** must compute contribution of each term and include raw values.
* **Versioning:** immutable content hash, `profile_id@version`. Inheritance supported via `extends`.
### 3.2 Signals (factor) catalog
Initial signals supported out of the box:
| Signal | Description | Expected range | Default transform |
| ---------------------- | ------------------------------------------------ | -------------- | -------------------- |
| `cvss_base` | CVSS base score from each advisory | 0..10 | linear: `x/10` |
| `epss_like` | Exploit likelihood (0..1) | 0..1 | identity |
| `kev_flag` | Known exploited in the wild (boolean) | {0,1} | step: 0 or 1 |
| `vex_status` | VEX: affected, not_affected, under_investigation | enum | gate + multiplier |
| `reachability` | Static reachability to vulnerable code path | 0..1 | identity |
| `runtime_evidence` | Runtime evidence of vulnerable symbol/path | 0..1 | identity |
| `internet_exposed` | Asset externally reachable | {0,1} | multiplier |
| `asset_criticality` | Business criticality of asset | 1..5 | normalize: `(x-1)/4` |
| `fix_available` | Patch or upgrade exists | {0,1} | negative weight |
| `age_days` | Days since advisory published | 0..∞ | logistic decay |
| `privilege_escalation` | Elevation potential | {0,1} | positive bump |
| `rce_flag` | Remote code execution | {0,1} | positive bump |
| `provenance_trust` | Signature/provenance (SLSA‑ish) | 0..1 | inverse weight |
| `pkg_popularity` | Package ecosystem usage | 0..1 | mild bump |
| `source_consensus` | Count of agreeing sources (Conseiller‑linked) | 1..N | saturating transform |
Notes:
* Conseiller can link multiple advisories per CVE. Signals like `cvss_base` and `kev_flag` are aggregated via declared **reducers**: `max`, `mean`, or **consensus** (e.g., count of sources claiming exploited).
### 3.3 Formula template
Default formula (normalized result 0..1 before scaling to 0..100):
```
score =
gate(VEX_not_affected => 0) *
clamp01(
w1*cvss'
+ w2*epss'
+ w3*reachability'
+ w4*runtime_evidence'
+ w5*internet_exposed'
+ w6*asset_criticality'
+ w7*kev_flag'
+ w8*rce_flag'
+ w9*privilege_escalation'
+ w10*source_consensus'
+ w11*(1 - provenance_trust')
+ w12*(1 - fix_available')
+ w13*age_decay'
+ bias
)
```
* Each term is a transformed, normalized signal (denoted `'`).
* Weights default to reasonable values (e.g., cvss 0.25, epss 0.2, reachability 0.1, runtime 0.1, internet_exposed 0.08, asset_criticality 0.08, kev 0.07, rce 0.04, priv_esc 0.03, consensus 0.03, provenance inverse 0.01, fix inverse 0.005, age 0.005).
* **Severity mapping (default):**
* Critical ≥ 85
* High 70–84
* Medium 40–69
* Low 15–39
* None < 15
Profiles can override weights, gating, transforms and severity thresholds.
### 3.4 Reducers and provenance
For signals with multiple sources:
* `cvss_base`: default reducer `max`.
* `kev_flag`: reducer `any`.
* `epss_like`: reducer `max`.
* `vex_status`: **gate precedence:** if any linked VEX says `not_affected`, apply gate 0 unless an explicit policy disables that source; otherwise, most conservative status wins (`affected` > `under_investigation` > `unknown`).
* Every reduction lists contributing sources in the explanation with their digests.
### 3.5 Explainability artifact
For every scored item, produce a JSON object:
```json
{
"profile_id": "risk-default",
"profile_version": "1.2.0",
"input": { "asset_id": "...", "package": "openssl@1.1.1u", "cve": "CVE-XXXX-YYYY" },
"signals": {
"cvss_base": { "values": [{"source":"nvd","value":9.8}, {"source":"vendor","value":9.1}], "reducer":"max", "reduced":9.8, "normalized":0.98 },
"epss_like": { "value":0.72, "normalized":0.72 },
"vex_status": { "values":[{"source":"vendor","value":"affected"}], "decision":"affected" }
},
"formula": {
"weights": { "cvss":0.25, "epss":0.20 },
"gates": [{ "name":"VEX_not_affected", "applied": false }]
},
"contributions": [
{ "signal":"cvss_base", "weight":0.25, "value":0.98, "contribution":24.5 },
{ "signal":"epss_like", "weight":0.20, "value":0.72, "contribution":14.4 }
],
"score": 87.1,
"severity": "Critical",
"provenance": { "calculated_at":"2025-10-25T12:00:00Z", "engine":"risk-engine@v0.6.3", "trace_id":"..." }
}
```
### 3.6 Profile scoping and inheritance
* Profiles attach to scopes via Authority & Tenancy: org/tenant/project/environment.
* A scope resolves **one active profile** by precedence: project > environment > org default.
* Profiles may `extends` a base profile, overriding weights and thresholds. Resolve via immutable parent chain.
### 3.7 Execution path
1. New or updated findings arrive from Conseiller/Excitator into Findings Ledger.
2. A **Scoring Job** is enqueued per scope with a batch of items.
3. The engine pulls necessary signals via **Factor Providers** (reachability, runtime, KEV lists, etc.).
4. The formula executes; results are upserted to Findings Ledger with an explainability blob pointer.
5. Notifications Studio triggers based on severity deltas.
6. Console and CLI read scored findings; filters and charts operate on score and severity.
### 3.8 Factor Provider interface
```
interface FactorProvider {
id(): string;
requiredInputs(): string[];
fetch(ctx, inputs[]): Promise<Map<inputKey, FactorValue>>;
}
```
Providers must be deterministic and cacheable. Every factor has a TTL and a backfill policy.
### 3.9 Simulation
Policy Studio provides “Simulate with profile” functionality to test profiles against SBOMs or asset sets. Simulation outputs include distributions, severity shifts, and top movers, and can be exported.
### 3.10 Air‑gapped behavior
Profiles work offline; providers rely on bundled datasets produced by Export Center. Missing providers surface explicit gaps in explanations.
---
## 4) Architecture
### 4.1 New modules
* `src/StellaOps.RiskEngine/`
* `src/StellaOps.RiskEngine/providers/`
* `src/StellaOps.Policy.RiskProfile/`
* Database migrations for profiles/results/explanations
* `src/StellaOps.UI`
* `src/StellaOps.Cli`
* `src/StellaOps.ExportCenter.RiskBundles`
### 4.2 Data model
Tables for `risk_profiles`, `scoring_jobs`, `scoring_results`, `explanations` with indexes on finding keys, scope, severity, and timestamps.
---
## 5) APIs and contracts
Endpoints include profile CRUD, publish, simulate, job enqueue, results queries, explanation retrieval, and schema discovery. Authentication scopes: `risk.profile:*`, `risk.result:read`, `risk.job:write`.
---
## 6) Documentation changes
List of required docs with banner statements covering overview, profiles, factors, formulas, explainability, API, console UI, CLI commands, air-gapped bundles, and AOC invariants.
---
## 7) Implementation plan
Seven phases: foundations, storage/APIs, Console & Policy Studio, CLI & SDKs, expanded factors, air-gapped support, quality/performance.
---
## 8) Engineering tasks
Detailed task list spanning schema, engine, providers, APIs, ledger integration, console, CLI, export center, observability, docs, and testing.
---
## 9) Feature changes required in other components
Defines cross-team expectations for Conseiller, Excitator, Findings Ledger, Policy Studio, Vulnerability Explorer, SBOM Graph Explorer, Notifications, Authority, Export Center, CLI & SDKs.
---
## 10) Acceptance criteria
Coverage of authoring, simulation, scoring, UI, CLI, air-gapped support, AOC invariants, and performance.
---
## 11) Risks and mitigations
Addresses signal drift, weight overfitting, performance, VEX trust, and compliance differences.
---
## 12) Philosophy
Principles: context, explainability, truth preservation, portability, and loud failures.
---
## 13) Example profile
Contains an abbreviated YAML example demonstrating schema usage, weights, gates, severity mapping, and overrides.
> **Imposed rule:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

49
ops/deployment/TASKS.md
View File

@@ -1,5 +1,48 @@
# Deployment Task Board
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-OPS-14-003 | TODO | Deployment Guild | DEVOPS-REL-14-001 | Document and script upgrade/rollback flows, channel management, and compatibility matrices per architecture. | Helm/Compose guides updated with digest pinning, automated checks committed, rollback drill recorded. |
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-OPS-14-003 | TODO | Deployment Guild | DEVOPS-REL-14-001 | Document and script upgrade/rollback flows, channel management, and compatibility matrices per architecture. | Helm/Compose guides updated with digest pinning, automated checks committed, rollback drill recorded. |
| DOWNLOADS-CONSOLE-23-001 | TODO | Deployment Guild, DevOps Guild | DEVOPS-CONSOLE-23-002 | Maintain signed downloads manifest pipeline (images, Helm, offline bundles), publish JSON under `deploy/downloads/manifest.json`, and document sync cadence for Console + docs parity. | Pipeline generates signed manifest with checksums, automated PR updates manifest, docs updated with sync workflow, parity check in CI passes. |
| DEPLOY-POLICY-27-001 | TODO | Deployment Guild, Policy Registry Guild | REGISTRY-API-27-001, DEVOPS-POLICY-27-003 | Produce Helm/Compose overlays for Policy Registry + simulation workers, including Mongo migrations, object storage buckets, signing key secrets, and tenancy defaults. | Overlays committed with deterministic digests; install docs updated; smoke deploy validated in staging. |
| DEPLOY-POLICY-27-002 | TODO | Deployment Guild, Policy Guild | DEPLOY-POLICY-27-001, WEB-POLICY-27-004 | Document rollout/rollback playbooks for policy publish/promote (canary strategy, emergency freeze toggle, evidence retrieval) under `/docs/runbooks/policy-incident.md`. | Runbook published with decision tree; checklist appended; rehearsal recorded. |
| DEPLOY-VULN-29-001 | TODO | Deployment Guild, Findings Ledger Guild | LEDGER-29-009 | Produce Helm/Compose overlays for Findings Ledger + projector, including DB migrations, Merkle anchor jobs, and scaling guidance. | Overlays committed; migrations documented; smoke deploy executed; rollback steps recorded. |
| DEPLOY-VULN-29-002 | TODO | Deployment Guild, Vuln Explorer API Guild | VULN-API-29-011 | Package `stella-vuln-explorer-api` deployment manifests, health checks, autoscaling policies, and offline kit instructions with signed images. | Deployment docs merged; health checks validated; offline kit updated; change control recorded. |
| DEPLOY-VEX-30-001 | TODO | Deployment Guild, VEX Lens Guild | VEXLENS-30-011 | Provide Helm/Compose overlays, scaling defaults, and offline kit instructions for VEX Lens service. | Overlays committed; smoke deploy validated; offline kit includes initial config; docs updated. |
| DEPLOY-VEX-30-002 | TODO | Deployment Guild, Issuer Directory Guild | ISSUER-30-006 | Package Issuer Directory deployment manifests, backups, and security hardening guidance. | Deployment docs merged; backup tested; hardening checklist appended. |
| DEPLOY-AIAI-31-001 | TODO | Deployment Guild, Advisory AI Guild | AIAI-31-008 | Provide Helm/Compose manifests, GPU toggle, scaling/runbook, and offline kit instructions for Advisory AI service + inference container. | Deployment docs merged; smoke deploy executed; offline kit updated; runbooks published. |
| DEPLOY-ORCH-34-001 | TODO | Deployment Guild, Orchestrator Service Guild | ORCH-SVC-34-004 | Provide orchestrator Helm/Compose manifests, scaling defaults, secret templates, offline kit instructions, and GA rollout/rollback playbook. | Manifests committed with digests; scaling guidance documented; smoke deploy/rollback rehearsed; offline kit instructions updated. |
| DEPLOY-EXPORT-35-001 | TODO | Deployment Guild, Exporter Service Guild | EXPORT-SVC-35-001..006 | Package exporter service/worker Helm overlays (download-only), document rollout/rollback, and integrate signing KMS secrets. | Overlays committed; smoke deploy executed; rollback steps recorded; secrets templates provided. |
| DEPLOY-EXPORT-36-001 | TODO | Deployment Guild, Exporter Service Guild | DEPLOY-EXPORT-35-001, EXPORT-SVC-36-003 | Document OCI/object storage distribution workflows, registry credential automation, and monitoring hooks for exports. | Documentation merged; automation scripts validated; monitoring instructions added. |
## CLI Parity & Task Packs
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEPLOY-CLI-41-001 | TODO | Deployment Guild, DevEx/CLI Guild | CLI-CORE-41-001 | Package CLI release artifacts (tarballs per OS/arch, checksums, signatures, completions, container image) and publish distribution docs. | Artifacts built and uploaded; docs updated with installation steps; signatures verified. |
| DEPLOY-PACKS-42-001 | TODO | Deployment Guild, Packs Registry Guild | PACKS-REG-41-001 | Provide deployment manifests for packs-registry and task-runner services, including Helm/Compose overlays, scaling defaults, and secret templates. | Manifests committed; smoke deploy executed; rollback documented. |
| DEPLOY-PACKS-43-001 | TODO | Deployment Guild, Task Runner Guild | TASKRUN-42-001 | Ship remote Task Runner worker profiles, object storage bootstrap, approval workflow integration, and Offline Kit packaging instructions. | Deployment docs merged; offline kit updated; approvals tested; rollback steps recorded. |
## Containerized Distribution (Epic 13)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEPLOY-COMPOSE-44-001 | TODO | Deployment Guild | COMPOSE-44-001 | Finalize Quickstart scripts (`quickstart.sh`, `backup.sh`, `reset.sh`), seed data container, and publish README with imposed rule reminder. | Scripts run end-to-end; README merged; imposed rule appended. |
| DEPLOY-HELM-45-001 | TODO | Deployment Guild | HELM-45-001 | Publish Helm install guide and sample values for prod/airgap; integrate with docs site build. | Docs merged; values validated; helm lint/test passing. |
| DEPLOY-AIRGAP-46-001 | TODO | Deployment Guild, Offline Kit Guild | DEVOPS-CONTAINERS-46-001 | Provide instructions and scripts (`load.sh`) for importing air-gap bundle into private registry; update Offline Kit guide. | Scripts tested; docs updated; imposed rule appended. |
### Compose Quickstart (Epic 13)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| COMPOSE-44-001 | TODO | Deployment Guild, DevEx Guild | EXPORT-SVC-37-001 | Author `docker-compose.yml`, `.env.example`, and `quickstart.sh` with all core services + dependencies (postgres, redis, object-store, queue, otel). | `docker compose up` yields working stack with seed data; script handles preflight; imposed rule line applied in docs. |
| COMPOSE-44-002 | TODO | Deployment Guild | COMPOSE-44-001 | Implement `backup.sh` and `reset.sh` scripts with safety prompts and documentation. | Backup produces tarball with checksums; reset script requires confirm flag; docs updated. |
| COMPOSE-44-003 | TODO | Deployment Guild, Docs Guild | COMPOSE-44-001 | Package seed data container and onboarding wizard toggle (`QUICKSTART_MODE`), ensuring default creds randomized on first run. | Seed job loads demo SBOM/advisory/policy; credentials randomized and saved to .secrets; onboarding wizard triggers. |
### Helm Chart (Epic 13)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| HELM-45-001 | TODO | Deployment Guild | COMPOSE-44-001 | Scaffold `deploy/helm/stella` chart with values, component toggles, and pinned image digests for all services; include migration Job templates. | Chart installs in dev cluster; images pinned; lint/tests pass. |
| HELM-45-002 | TODO | Deployment Guild, Security Guild | HELM-45-001 | Add TLS/Ingress, NetworkPolicy, PodSecurityContexts, Secrets integration (external secrets), and document security posture. | Helm values support secure defaults; policies validated; docs updated. |
| HELM-45-003 | TODO | Deployment Guild, Observability Guild | HELM-45-001 | Implement HPA, PDB, readiness gates, Prometheus scraping annotations, OTel configuration hooks, and upgrade hooks. | Rolling upgrade succeeds in CI; observability wires confirmed; upgrade docs updated. |

149
ops/devops/TASKS.md
View File

@@ -1,23 +1,158 @@
# DevOps Task Board
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-HELM-09-001 | DONE | DevOps Guild | SCANNER-WEB-09-101 | Create Helm/Compose environment profiles (dev, staging, airgap) with deterministic digests. | Profiles committed under `deploy/`; docs updated; CI smoke deploy passes. |
# DevOps Task Board
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-HELM-09-001 | DONE | DevOps Guild | SCANNER-WEB-09-101 | Create Helm/Compose environment profiles (dev, staging, airgap) with deterministic digests. | Profiles committed under `deploy/`; docs updated; CI smoke deploy passes. |
| DEVOPS-SCANNER-09-204 | DONE (2025-10-21) | DevOps Guild, Scanner WebService Guild | SCANNER-EVENTS-15-201 | Surface `SCANNER__EVENTS__*` environment variables across docker-compose (dev/stage/airgap) and Helm values, defaulting to share the Redis queue DSN. | Compose/Helm configs ship enabled Redis event publishing with documented overrides; lint jobs updated; docs cross-link to new knobs. |
| DEVOPS-SCANNER-09-205 | DONE (2025-10-21) | DevOps Guild, Notify Guild | DEVOPS-SCANNER-09-204 | Add Notify smoke stage that tails the Redis stream and asserts `scanner.report.ready`/`scanner.scan.completed` reach Notify WebService in staging. | CI job reads Redis stream during scanner smoke deploy, confirms Notify ingestion via API, alerts on failure. |
| DEVOPS-PERF-10-001 | DONE | DevOps Guild | BENCH-SCANNER-10-001 | Add perf smoke job (SBOM compose <5 s target) to CI. | CI job runs sample build verifying <5 s; alerts configured. |
| DEVOPS-PERF-10-001 | DONE | DevOps Guild | BENCH-SCANNER-10-001 | Add perf smoke job (SBOM compose <5 s target) to CI. | CI job runs sample build verifying <5 s; alerts configured. |
| DEVOPS-PERF-10-002 | DONE (2025-10-23) | DevOps Guild | BENCH-SCANNER-10-002 | Publish analyzer bench metrics to Grafana/perf workbook and alarm on ≥20 % regressions. | CI exports JSON for dashboards; Grafana panel wired; Ops on-call doc updated with alert hook. |
| DEVOPS-AOC-19-001 | TODO | DevOps Guild, Platform Guild | WEB-AOC-19-003 | Integrate the AOC Roslyn analyzer and guard tests into CI, failing builds when ingestion projects attempt banned writes. | Analyzer runs in PR/CI pipelines, results surfaced in build summary, docs updated under `docs/ops/ci-aoc.md`. |
| DEVOPS-AOC-19-002 | TODO | DevOps Guild | CLI-AOC-19-002, CONCELIER-WEB-AOC-19-004, EXCITITOR-WEB-AOC-19-004 | Add pipeline stage executing `stella aoc verify --since` against seeded Mongo snapshots for Concelier + Excititor, publishing violation report artefacts. | Stage runs on main/nightly, fails on violations, artifacts retained, runbook documented. |
| DEVOPS-AOC-19-003 | TODO | DevOps Guild, QA Guild | CONCELIER-WEB-AOC-19-003, EXCITITOR-WEB-AOC-19-003 | Enforce unit test coverage thresholds for AOC guard suites and ensure coverage exported to dashboards. | Coverage report includes guard projects, threshold gate passes/fails as expected, dashboards refreshed with new metrics. |
| DEVOPS-OBS-50-001 | TODO | DevOps Guild, Observability Guild | TELEMETRY-OBS-50-001 | Deliver default OpenTelemetry collector deployment (Compose/Helm manifests), OTLP ingestion endpoints, and secure pipeline (authN, mTLS, tenant partitioning). Provide smoke test verifying traces/logs/metrics ingestion. | Collector manifests committed; smoke test green; docs updated; imposed rule banner reminder noted. |
| DEVOPS-OBS-50-002 | TODO | DevOps Guild, Security Guild | DEVOPS-OBS-50-001, TELEMETRY-OBS-51-002 | Stand up multi-tenant storage backends (Prometheus, Tempo/Jaeger, Loki) with retention policies, tenant isolation, and redaction guard rails. Integrate with Authority scopes for read paths. | Storage stack deployed with auth; retention configured; integration tests verify tenant isolation; runbook drafted. |
| DEVOPS-OBS-50-003 | TODO | DevOps Guild, Offline Kit Guild | DEVOPS-OBS-50-001 | Package telemetry stack configs for air-gapped installs (Offline Kit bundle, documented overrides, sample values) and automate checksum/signature generation. | Offline bundle includes collector+storage configs; checksums published; docs cross-linked; imposed rule annotation recorded. |
| DEVOPS-OBS-51-001 | TODO | DevOps Guild, Observability Guild | WEB-OBS-51-001, DEVOPS-OBS-50-001 | Implement SLO evaluator service (burn rate calculators, webhook emitters), Grafana dashboards, and alert routing to Notifier. Provide Terraform/Helm automation. | Dashboards live; evaluator emits webhooks; alert runbook referenced; staging alert fired in test. |
| DEVOPS-OBS-52-001 | TODO | DevOps Guild, Timeline Indexer Guild | TIMELINE-OBS-52-002 | Configure streaming pipeline (NATS/Redis/Kafka) with retention, partitioning, and backpressure tuning for timeline events; add CI validation of schema + rate caps. | Pipeline deployed; load test meets SLA; schema validation job passes; documentation updated. |
| DEVOPS-OBS-53-001 | TODO | DevOps Guild, Evidence Locker Guild | EVID-OBS-53-001 | Provision object storage with WORM/retention options (S3 Object Lock / MinIO immutability), legal hold automation, and backup/restore scripts for evidence locker. | Storage configured with WORM; legal hold script documented; backup test performed; runbook updated. |
| DEVOPS-OBS-54-001 | TODO | DevOps Guild, Security Guild | PROV-OBS-53-002, EVID-OBS-54-001 | Manage provenance signing infrastructure (KMS keys, rotation schedule, timestamp authority integration) and integrate verification jobs into CI. | Keys provisioned with rotation policy; timestamp authority configured; CI verifies sample bundles; audit trail stored. |
| DEVOPS-OBS-55-001 | TODO | DevOps Guild, Ops Guild | DEVOPS-OBS-51-001, WEB-OBS-55-001 | Implement incident mode automation: feature flag service, auto-activation via SLO burn-rate, retention override management, and post-incident reset job. | Incident mode toggles via API/CLI; automation tested in staging; reset job verified; runbook referenced. |
## Air-Gapped Mode (Epic 16)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-AIRGAP-56-001 | TODO | DevOps Guild | AIRGAP-CTL-56-001 | Ship deny-all egress policies for Kubernetes (NetworkPolicy/eBPF) and docker-compose firewall rules; provide verification script for sealed mode. | Policies committed with tests; verification script passes/fails as expected; docs cross-linked. |
| DEVOPS-AIRGAP-56-002 | TODO | DevOps Guild, AirGap Importer Guild | AIRGAP-IMP-57-002 | Provide import tooling for bundle staging: checksum validation, offline object-store loader scripts, removable media guidance. | Scripts documented; smoke tests validate import; runbook updated. |
| DEVOPS-AIRGAP-56-003 | TODO | DevOps Guild, Container Distribution Guild | EXPORT-AIRGAP-56-002 | Build Bootstrap Pack pipeline bundling images/charts, generating checksums, and publishing manifest for offline transfer. | Pipeline runs in connected env; pack verified in air-gap smoke test; manifest recorded. |
| DEVOPS-AIRGAP-57-001 | TODO | DevOps Guild, Mirror Creator Guild | MIRROR-CRT-56-002 | Automate Mirror Bundle creation jobs with dual-control approvals, artifact signing, and checksum publication. | Approval workflow enforced; CI artifact includes DSSE/TUF metadata; audit logs stored. |
| DEVOPS-AIRGAP-57-002 | TODO | DevOps Guild, Authority Guild | AUTH-OBS-50-001 | Configure sealed-mode CI tests that run services with sealed flag and ensure no egress occurs (iptables + mock DNS). | CI suite fails on attempted egress; reports remediation; documentation updated. |
| DEVOPS-AIRGAP-58-001 | TODO | DevOps Guild, Notifications Guild | NOTIFY-AIRGAP-56-002 | Provide local SMTP/syslog container templates and health checks for sealed environments; integrate into Bootstrap Pack. | Templates deployed successfully; health checks in CI; docs updated. |
| DEVOPS-AIRGAP-58-002 | TODO | DevOps Guild, Observability Guild | DEVOPS-AIRGAP-56-001, DEVOPS-OBS-51-001 | Ship sealed-mode observability stack (Prometheus/Grafana/Tempo/Loki) pre-configured with offline dashboards and no remote exporters. | Stack boots offline; dashboards available; verification script confirms zero egress. |
| DEVOPS-REL-14-001 | DOING (2025-10-23) | DevOps Guild | SIGNER-API-11-101, ATTESTOR-API-11-201 | Deterministic build/release pipeline with SBOM/provenance, signing, manifest generation. | CI pipeline produces signed images + SBOM/attestations, manifests published with verified hashes, docs updated. |
| DEVOPS-REL-14-004 | TODO | DevOps Guild, Scanner Guild | DEVOPS-REL-14-001, SCANNER-ANALYZERS-LANG-10-309P | Extend release/offline smoke jobs to exercise the Python analyzer plug-in (warm/cold scans, determinism, signature checks). | Release/Offline pipelines run Python analyzer smoke suite; alerts hooked; docs updated with new coverage matrix. |
| DEVOPS-REL-17-002 | TODO | DevOps Guild | DEVOPS-REL-14-001, SCANNER-EMIT-17-701 | Persist stripped-debug artifacts organised by GNU build-id and bundle them into release/offline kits with checksum manifests. | CI job writes `.debug` files under `artifacts/debug/.build-id/`, manifest + checksums published, offline kit includes cache, smoke job proves symbol lookup via build-id. |
| DEVOPS-MIRROR-08-001 | DONE (2025-10-19) | DevOps Guild | DEVOPS-REL-14-001 | Stand up managed mirror profiles for `*.stella-ops.org` (Concelier/Excititor), including Helm/Compose overlays, multi-tenant secrets, CDN caching, and sync documentation. | Infra overlays committed, CI smoke deploy hits mirror endpoints, runbooks published for downstream sync and quota management. |
| DEVOPS-SEC-10-301 | DONE (2025-10-20) | DevOps Guild | Wave 0A complete | Address NU1902/NU1903 advisories for `MongoDB.Driver` 2.12.0 and `SharpCompress` 0.23.0 surfaced during scanner cache and worker test runs. | Dependencies bumped to patched releases, audit logs free of NU1902/NU1903 warnings, regression tests green, change log documents upgrade guidance. |
| DEVOPS-CONSOLE-23-001 | TODO | DevOps Guild, Console Guild | CONSOLE-CORE-23-001 | Add console CI workflow (pnpm cache, lint, type-check, unit, Storybook a11y, Playwright, Lighthouse) with offline runners and artifact retention for screenshots/reports. | Workflow runs on PR & main, caches reduce install time, failing checks block merges, artifacts uploaded for triage, docs updated. |
| DEVOPS-CONSOLE-23-002 | TODO | DevOps Guild, Console Guild | DEVOPS-CONSOLE-23-001, CONSOLE-REL-23-301 | Produce `stella-console` container build + Helm chart overlays with deterministic digests, SBOM/provenance artefacts, and offline bundle packaging scripts. | Container published to registry mirror, Helm values committed, SBOM/attestations generated, offline kit job passes smoke test, docs updated. |
| DEVOPS-LAUNCH-18-100 | TODO | DevOps Guild | - | Finalise production environment footprint (clusters, secrets, network overlays) for full-platform go-live. | IaC/compose overlays committed, secrets placeholders documented, dry-run deploy succeeds in staging. |
| DEVOPS-LAUNCH-18-900 | TODO | DevOps Guild, Module Leads | Wave 0 completion | Collect “full implementation” sign-off from module owners and consolidate launch readiness checklist. | Sign-off record stored under `docs/ops/launch-readiness.md`; outstanding gaps triaged; checklist approved. |
| DEVOPS-LAUNCH-18-001 | TODO | DevOps Guild | DEVOPS-LAUNCH-18-100, DEVOPS-LAUNCH-18-900 | Production launch cutover rehearsal and runbook publication. | `docs/ops/launch-cutover.md` drafted, rehearsal executed with rollback drill, approvals captured. |
| DEVOPS-NUGET-13-001 | DONE (2025-10-25) | DevOps Guild, Platform Leads | DEVOPS-REL-14-001 | Add .NET 10 preview feeds / local mirrors so `Microsoft.Extensions.*` 10.0 preview packages restore offline; refresh restore docs. | NuGet.config maps preview feeds (or local mirrored packages), `dotnet restore` succeeds for Excititor/Concelier solutions without ad-hoc feed edits, docs updated for offline bootstrap. |
| DEVOPS-NUGET-13-002 | TODO | DevOps Guild | DEVOPS-NUGET-13-001 | Ensure all solutions/projects prefer `local-nuget` before public sources and document restore order validation. | `NuGet.config` and solution-level configs resolve from `local-nuget` first; automated check verifies priority; docs updated for restore ordering. |
| DEVOPS-NUGET-13-003 | TODO | DevOps Guild, Platform Leads | DEVOPS-NUGET-13-002 | Sweep `Microsoft.*` NuGet dependencies pinned to 8.* and upgrade to latest .NET 10 equivalents (or .NET 9 when 10 unavailable), updating restore guidance. | Dependency audit shows no 8.* `Microsoft.*` packages remaining; CI builds green; changelog/doc sections capture upgrade rationale. |
| DEVOPS-UI-13-006 | TODO | DevOps Guild, UI Guild | UI-AUTH-13-001 | Add Playwright-based UI auth smoke job to CI/offline pipelines, wiring sample `/config.json` provisioning and reporting. | CI + Offline Kit run Playwright auth smoke (headless Chromium) post-build; job reuses stub config artifact, exports junit + trace on failure, docs updated under `docs/ops/ui-auth-smoke.md`. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-POLICY-20-001 | TODO | DevOps Guild, Policy Guild | POLICY-ENGINE-20-001 | Integrate DSL linting in CI (parser/compile) to block invalid policies; add pipeline step compiling sample policies. | CI fails on syntax errors; lint logs surfaced; docs updated with pipeline instructions. |
| DEVOPS-POLICY-20-002 | TODO | DevOps Guild | DEVOPS-POLICY-20-001, POLICY-ENGINE-20-006 | Add `stella policy simulate` CI stage against golden SBOMs to detect delta explosions; publish diff artifacts. | Stage runs nightly/main; artifacts retained; alert thresholds configured. |
| DEVOPS-POLICY-20-003 | TODO | DevOps Guild, QA Guild | DEVOPS-POLICY-20-001, POLICY-ENGINE-20-005 | Determinism CI: run Policy Engine twice with identical inputs and diff outputs to guard non-determinism. | CI job compares outputs, fails on differences, logs stored; documentation updated. |
## Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-GRAPH-21-001 | TODO | DevOps Guild, Cartographer Guild | CARTO-GRAPH-21-006 | Add load/perf jobs hitting graph viewport/path/diff endpoints with synthetic 50k/100k graphs; emit dashboards/alerts for SLOs. | CI perf job introduced; Grafana panels live; alerts configured for latency/SLA breaches. |
| DEVOPS-GRAPH-21-002 | TODO | DevOps Guild, UI Guild | UI-GRAPH-21-001 | Capture golden screenshots (Playwright) and JSON exports for visual regressions; wire into CI/offline kit. | Visual regression suite runs in CI; artifacts stored; failure triage docs updated. |
| DEVOPS-GRAPH-21-003 | TODO | DevOps Guild | CARTO-GRAPH-21-009, SBOM-SERVICE-21-002 | Package Cartographer + SBOM Service into offline kit bundles with seeded data/layout caches; document deployment steps. | Offline kit includes graph seeds; docs updated; smoke scripts validate airgapped startup. |
## Orchestrator Dashboard
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-ORCH-32-001 | TODO | DevOps Guild, Orchestrator Service Guild | ORCH-SVC-32-001 | Provision orchestrator Postgres/message-bus infrastructure, add CI smoke deploy, seed Grafana dashboards (queue depth, inflight jobs), and document bootstrap. | Helm/Compose profiles committed; CI smoke deploy runs; dashboards live with metrics; runbook updated. |
| DEVOPS-ORCH-33-001 | TODO | DevOps Guild, Observability Guild | DEVOPS-ORCH-32-001, ORCH-SVC-33-001..003 | Publish Grafana dashboards/alerts for rate limiter, backpressure, error clustering, and DLQ depth; integrate with on-call rotations. | Dashboards and alerts configured; synthetic tests validate thresholds; on-call playbook updated. |
| DEVOPS-ORCH-34-001 | TODO | DevOps Guild, Orchestrator Service Guild | DEVOPS-ORCH-33-001, ORCH-SVC-34-001..003 | Harden production monitoring (synthetic probes, burn-rate alerts, replay smoke), document incident response, and prep GA readiness checklist. | Synthetic probes created; burn-rate alerts firing on test scenario; GA checklist approved; runbook linked. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-LNM-22-001 | TODO | DevOps Guild, Concelier Guild | CONCELIER-LNM-21-102 | Run migration/backfill pipelines for advisory observations/linksets in staging, validate counts/conflicts, and automate deployment steps. | Migration job scripted; staging validation report produced; rollback documented. |
| DEVOPS-LNM-22-002 | TODO | DevOps Guild, Excititor Guild | EXCITITOR-LNM-21-102 | Execute VEX observation/linkset backfill with monitoring; ensure NATS/Redis events integrated; document ops runbook. | Backfill completed in staging; monitoring dashboards updated; runbook published. |
| DEVOPS-LNM-22-003 | TODO | DevOps Guild, Observability Guild | CONCELIER-LNM-21-005, EXCITITOR-LNM-21-005 | Add CI/monitoring coverage for new metrics (`advisory_observations_total`, `linksets_total`, etc.) and alerts on ingest-to-API SLA breaches. | Metrics scraped into Grafana; alert thresholds set; CI job verifies metric emission. |
## Graph & Vuln Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-GRAPH-24-001 | TODO | DevOps Guild, SBOM Service Guild | SBOM-GRAPH-24-002 | Load test graph index/adjacency APIs with 40k-node assets; capture perf dashboards and alert thresholds. | Perf suite added; dashboards live; alerts configured. |
| DEVOPS-GRAPH-24-002 | TODO | DevOps Guild, UI Guild | UI-GRAPH-24-001..005 | Integrate synthetic UI perf runs (Playwright/WebGL metrics) for Graph/Vuln explorers; fail builds on regression. | CI job runs UI perf tests; baseline stored; documentation updated. |
| DEVOPS-GRAPH-24-003 | TODO | DevOps Guild | WEB-GRAPH-24-002 | Implement smoke job for simulation endpoints ensuring we stay within SLA (<3s upgrade) and log results. | Smoke job in CI; alerts when SLA breached; runbook documented. |
| DEVOPS-POLICY-27-001 | TODO | DevOps Guild, DevEx/CLI Guild | CLI-POLICY-27-001, REGISTRY-API-27-001 | Add CI pipeline stages to run `stella policy lint|compile|test` with secret scanning on policy sources for PRs touching `/policies/**`; publish diagnostics artifacts. | Pipeline executes on PR/main, failures block merges, secret scan summary uploaded, docs updated. |
| DEVOPS-POLICY-27-002 | TODO | DevOps Guild, Policy Registry Guild | REGISTRY-API-27-005, SCHED-WORKER-27-301 | Provide optional batch simulation CI job (staging inventory) that triggers Registry run, polls results, and posts markdown summary to PR; enforce drift thresholds. | Job configurable via label, summary comment generated, drift threshold gates merges, runbook documented. |
| DEVOPS-POLICY-27-003 | TODO | DevOps Guild, Security Guild | AUTH-POLICY-27-002, REGISTRY-API-27-007 | Manage signing key material for policy publish pipeline (OIDC workload identity + cosign), rotate keys, and document verification steps; integrate attestation verification stage. | Keys stored in secure vault, rotation procedure documented, CI verifies attestations, audit logs recorded. |
| DEVOPS-POLICY-27-004 | TODO | DevOps Guild, Observability Guild | WEB-POLICY-27-005, TELEMETRY-CONSOLE-27-001 | Create dashboards/alerts for policy compile latency, simulation queue depth, approval latency, and promotion outcomes; integrate with on-call playbooks. | Grafana dashboards live, alerts tuned, runbooks updated, observability tests verify metric ingestion. |
> Remark (2025-10-20): Repacked `Mongo2Go` local feed to require MongoDB.Driver 3.5.0 + SharpCompress 0.41.0; cache regression tests green and NU1902/NU1903 suppressed.
> Remark (2025-10-21): Compose/Helm profiles now surface `SCANNER__EVENTS__*` toggles with docs pointing at new `.env` placeholders.
## Reachability v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-SIG-26-001 | TODO | DevOps Guild, Signals Guild | SIGNALS-24-001 | Provision CI/CD pipelines, Helm/Compose manifests for Signals service, including artifact storage and Redis dependencies. | Pipelines ship Signals service; deployment docs updated; smoke tests green. |
| DEVOPS-SIG-26-002 | TODO | DevOps Guild, Observability Guild | SIGNALS-24-004 | Create dashboards/alerts for reachability scoring latency, cache hit rates, sensor staleness. | Dashboards live; alert thresholds configured; documentation updated. |
| DEVOPS-VULN-29-001 | TODO | DevOps Guild, Findings Ledger Guild | LEDGER-29-002..009 | Provision CI jobs for ledger projector (replay, determinism), set up backups, monitor Merkle anchoring, and automate verification. | CI job verifies hash chains; backups documented; alerts for anchoring failures configured. |
| DEVOPS-VULN-29-002 | TODO | DevOps Guild, Vuln Explorer API Guild | VULN-API-29-002..009 | Configure load/perf tests (5M findings/tenant), query budget enforcement, API SLO dashboards, and alerts for `vuln_list_latency` and `projection_lag`. | Perf suite integrated; dashboards live; alerts firing; runbooks updated. |
| DEVOPS-VULN-29-003 | TODO | DevOps Guild, Console Guild | WEB-VULN-29-004, CONSOLE-VULN-29-007 | Instrument analytics pipeline for Vuln Explorer (telemetry ingestion, query hashes), ensure compliance with privacy/PII guardrails, and update observability docs. | Telemetry pipeline operational; PII redaction verified; docs updated with checklist. |
| DEVOPS-VEX-30-001 | TODO | DevOps Guild, VEX Lens Guild | VEXLENS-30-009, ISSUER-30-005 | Provision CI, load tests, dashboards, alerts for VEX Lens and Issuer Directory (compute latency, disputed totals, signature verification rates). | CI/perf suites running; dashboards live; alerts configured; docs updated. |
| DEVOPS-AIAI-31-001 | TODO | DevOps Guild, Advisory AI Guild | AIAI-31-006..007 | Stand up CI pipelines, inference monitoring, privacy logging review, and perf dashboards for Advisory AI (summaries/conflicts/remediation). | CI covers golden outputs, telemetry dashboards live, privacy controls reviewed, alerts configured. |
## Export Center
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-EXPORT-35-001 | TODO | DevOps Guild, Exporter Service Guild | EXPORT-SVC-35-001..006 | Establish exporter CI pipeline (lint/test/perf smoke), configure object storage fixtures, seed Grafana dashboards, and document bootstrap steps. | CI pipeline running; smoke export job seeded; dashboards live; runbook updated. |
| DEVOPS-EXPORT-36-001 | TODO | DevOps Guild, Exporter Service Guild | DEVOPS-EXPORT-35-001, EXPORT-SVC-36-001..004 | Integrate Trivy compatibility validation, OCI push smoke tests, and throughput/error dashboards. | CI executes Trivy validation; OCI push smoke passes; dashboards/alerts configured. |
| DEVOPS-EXPORT-37-001 | TODO | DevOps Guild, Exporter Service Guild | DEVOPS-EXPORT-36-001, EXPORT-SVC-37-001..004 | Finalize exporter monitoring (failure alerts, verify metrics, retention jobs) and chaos/latency tests ahead of GA. | Alerts tuned; chaos tests documented; retention monitoring active; runbook updated. |
## CLI Parity & Task Packs
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-CLI-41-001 | TODO | DevOps Guild, DevEx/CLI Guild | CLI-CORE-41-001 | Establish CLI build pipeline (multi-platform binaries, SBOM, checksums), parity matrix CI enforcement, and release artifact signing. | Build pipeline operational; SBOM/checksums published; parity gate failing on drift; docs updated. |
| DEVOPS-CLI-42-001 | TODO | DevOps Guild | DEVOPS-CLI-41-001, CLI-PARITY-41-001 | Add CLI golden output tests, parity diff automation, pack run CI harness, and artifact cache for remote mode. | Golden tests running; parity diff automation in CI; pack run harness executes sample packs; documentation updated. |
| DEVOPS-CLI-43-001 | TODO | DevOps Guild | DEVOPS-CLI-42-001, TASKRUN-42-001 | Finalize multi-platform release automation, SBOM signing, parity gate enforcement, and Task Pack chaos tests. | Release automation verified; SBOM signed; parity gate enforced; chaos tests documented. |
## Containerized Distribution (Epic 13)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-CONTAINERS-44-001 | TODO | DevOps Guild | DOCKER-44-001..003 | Automate multi-arch image builds with buildx, SBOM generation, cosign signing, and signature verification in CI. | Pipeline builds amd64/arm64; SBOMs pushed as referrers; cosign verify job passes. |
| DEVOPS-CONTAINERS-45-001 | TODO | DevOps Guild | HELM-45-001 | Add Compose and Helm smoke tests (fresh VM + kind cluster) to CI; publish test artifacts and logs. | CI jobs running; failures block releases; documentation updated. |
| DEVOPS-CONTAINERS-46-001 | TODO | DevOps Guild | DEPLOY-PACKS-43-001 | Build air-gap bundle generator (`tools/make-airgap-bundle.sh`), produce signed bundle, and verify in CI using private registry. | Bundle artifact produced with signatures/checksums; verification job passes; instructions documented. |
### Container Images (Epic 13)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOCKER-44-001 | TODO | DevOps Guild, Service Owners | DEVOPS-CLI-41-001 | Author multi-stage Dockerfiles for all core services (API, Console, Orchestrator, Task Runner, Conseiller, Excitator, Policy, Notify, Export, AI) with non-root users, read-only file systems, and health scripts. | Dockerfiles committed; images build successfully; container security scans clean; health endpoints reachable. |
| DOCKER-44-002 | TODO | DevOps Guild | DOCKER-44-001 | Generate SBOMs and cosign attestations for each image and integrate verification into CI. | SBOMs attached as OCI artifacts; cosign signatures published; CI verifies signatures prior to release. |
| DOCKER-44-003 | TODO | DevOps Guild | DOCKER-44-001 | Implement `/health/liveness`, `/health/readiness`, `/version`, `/metrics`, and ensure capability endpoint returns `merge=false` for Conseiller/Excitator. | Endpoints available across services; automated tests confirm responses; documentation updated with imposed rule reminder. |
## Authority-Backed Scopes & Tenancy (Epic 14)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-TEN-47-001 | TODO | DevOps Guild | AUTH-TEN-47-001 | Add JWKS cache monitoring, signature verification regression tests, and token expiration chaos tests to CI. | CI verifies tokens using cached keys; chaos test for expired keys passes; documentation updated. |
| DEVOPS-TEN-48-001 | TODO | DevOps Guild | WEB-TEN-48-001 | Build integration tests to assert RLS enforcement, tenant-prefixed object storage, and audit event emission; set up lint to prevent raw SQL bypass. | Tests fail on cross-tenant access; lint enforced; dashboards capture audit events. |
| DEVOPS-TEN-49-001 | TODO | DevOps Guild | AUTH-TEN-49-001 | Deploy audit pipeline, scope usage metrics, JWKS outage chaos tests, and tenant load/perf benchmarks. | Audit pipeline live; metrics dashboards updated; chaos tests documented; perf benchmarks recorded. |
## SDKs & OpenAPI (Epic 17)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-OAS-61-001 | TODO | DevOps Guild, API Contracts Guild | OAS-61-002 | Add CI stages for OpenAPI linting, validation, and compatibility diff; enforce gating on PRs. | Pipeline active; merge blocked on failures; documentation updated. |
| DEVOPS-OAS-61-002 | TODO | DevOps Guild, Contract Testing Guild | CONTR-62-002 | Integrate mock server + contract test suite into PR and nightly workflows; publish artifacts. | Tests run in CI; artifacts stored; failures alert. |
| DEVOPS-SDK-63-001 | TODO | DevOps Guild, SDK Release Guild | SDKREL-63-001 | Provision registry credentials, signing keys, and secure storage for SDK publishing pipelines. | Keys stored/rotated; publish pipeline authenticated; audit logs recorded. |
| DEVOPS-DEVPORT-63-001 | TODO | DevOps Guild, Developer Portal Guild | DEVPORT-62-001 | Automate developer portal build pipeline with caching, link & accessibility checks, performance budgets. | Pipeline enforced; reports archived; failures gate merges. |
| DEVOPS-DEVPORT-64-001 | TODO | DevOps Guild, DevPortal Offline Guild | DVOFF-64-001 | Schedule `devportal --offline` nightly builds with checksum validation and artifact retention policies. | Nightly job running; checksums published; retention policy documented. |
## Attestor Console (Epic 19)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-ATTEST-73-001 | TODO | DevOps Guild, Attestor Service Guild | ATTESTOR-72-002 | Provision CI pipelines for attestor service (lint/test/security scan, seed data) and manage secrets for KMS drivers. | CI pipeline running; secrets stored securely; docs updated. |
| DEVOPS-ATTEST-73-002 | TODO | DevOps Guild, KMS Guild | KMS-72-001 | Establish secure storage for signing keys (vault integration, rotation schedule) and audit logging. | Key storage configured; rotation documented; audit logs verified. |
| DEVOPS-ATTEST-74-001 | TODO | DevOps Guild, Transparency Guild | TRANSP-74-001 | Deploy transparency log witness infrastructure and monitoring. | Witness service deployed; dashboards/alerts live. |
| DEVOPS-ATTEST-74-002 | TODO | DevOps Guild, Export Attestation Guild | EXPORT-ATTEST-74-001 | Integrate attestation bundle builds into release/offline pipelines with checksum verification. | Bundle job in CI; checksum verification passes; docs updated. |
| DEVOPS-ATTEST-75-001 | TODO | DevOps Guild, Observability Guild | ATTEST-VERIFY-74-001 | Add dashboards/alerts for signing latency, verification failures, key rotation events. | Dashboards live; alerts configured. |

5
ops/offline-kit/TASKS.md
View File

@@ -3,6 +3,9 @@
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DEVOPS-OFFLINE-14-002 | TODO | Offline Kit Guild | DEVOPS-REL-14-001 | Build offline kit packaging workflow (artifact bundling, manifest generation, signature verification). | Offline tarball generated with manifest + checksums + signatures; import script verifies integrity; docs updated. |
| DEVOPS-OFFLINE-18-003 | TODO | Offline Kit Guild, UX Specialist | DEVOPS-OFFLINE-14-002 | Capture Angular workspace npm cache + Chromium bundle in Offline Kit (`out/offline-kit/web/`) and document refresh cadence. | Web cache directory added to kit manifest; documentation updated with `npm run ci:install`/`verify:chromium` workflow; periodic refresh SOP recorded in Offline Kit guide. |
| DEVOPS-OFFLINE-18-004 | DONE (2025-10-22) | Offline Kit Guild, Scanner Guild | DEVOPS-OFFLINE-18-003, SCANNER-ANALYZERS-LANG-10-309G | Rebuild Offline Kit bundle with Go analyzer plug-in and updated manifest/signature set. | Kit tarball includes Go analyzer artifacts; manifest/signature refreshed; verification steps executed and logged; docs updated with new bundle version. |
| DEVOPS-OFFLINE-18-005 | TODO | Offline Kit Guild, Scanner Guild | DEVOPS-REL-14-004, SCANNER-ANALYZERS-LANG-10-309P | Repackage Offline Kit with Python analyzer plug-in artefacts and refreshed manifest/signature set. | Kit tarball includes Python analyzer DLL/PDB/manifest; signature + manifest updated; Offline Kit guide references Python coverage; smoke import validated. |
| DEVOPS-OFFLINE-34-006 | TODO | Offline Kit Guild, Orchestrator Service Guild | ORCH-SVC-34-004, DEPLOY-ORCH-34-001 | Bundle orchestrator service container, worker SDK samples, Postgres snapshot, and dashboards into Offline Kit with manifest/signature updates. | Offline kit contains orchestrator assets; manifest/signature validated; docs updated with air-gapped install steps; smoke import executed. |
| DEVOPS-OFFLINE-37-001 | TODO | Offline Kit Guild, Exporter Service Guild | EXPORT-SVC-37-001..004, DEPLOY-EXPORT-36-001 | Package Export Center tooling, sample mirror bundles, verification CLI, and docs into Offline Kit with manifest/signature refresh and air-gap import script. | Offline kit includes export bundles/tools; verification script passes; manifest/signature updated; docs detail import workflow. |
| CLI-PACKS-43-002 | TODO | Offline Kit Guild, Packs Registry Guild | PACKS-REG-42-001, DEPLOY-PACKS-43-001 | Bundle Task Pack samples, registry mirror seeds, Task Runner configs, and CLI binaries with checksums into Offline Kit. | Offline kit includes packs registry mirror, Task Runner configs, CLI binaries; manifest/signature updated; docs describe air-gapped execution. |
| OFFLINE-CONTAINERS-46-001 | TODO | Offline Kit Guild, Deployment Guild | DEVOPS-CONTAINERS-46-001, DEPLOY-AIRGAP-46-001 | Include container air-gap bundle, verification docs, and mirrored registry instructions inside Offline Kit. | Offline kit ships bundle + how-to; verification steps validated; manifest/signature updated; imposed rule noted. |

28
samples/TASKS.md
View File

@@ -4,3 +4,31 @@
|----|--------|----------|------------|-------------|---------------|
| SAMPLES-10-001 | DONE | Samples Guild, Scanner Team | SCANNER-EMIT-10-605 | Curate sample images (nginx, alpine+busybox, distroless+go, .NET AOT, python venv, npm monorepo) with expected SBOM/BOM-Index sidecars. | Samples committed under `samples/`; golden SBOM/BOM-Index files present; documented usage. |
| SAMPLES-13-004 | DONE (2025-10-23) | Samples Guild, Policy Guild | POLICY-CORE-09-006, UI-POLICY-13-007 | Add policy preview/report fixtures showing confidence bands and unknown-age tags. | Confidence sample (`samples/policy/policy-preview-unknown.json`) reviewed, documented usage in UI dev guide, ajv validation hook updated. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SAMPLES-POLICY-20-001 | TODO | Samples Guild, Policy Guild | POLICY-ENGINE-20-002, DOCS-POLICY-20-011 | Create sample policies (`baseline.pol`, `serverless.pol`, `internal-only.pol`) with annotated SBOM/advisory fixtures. | Samples stored under `samples/policy/`; README documents usage; tests validate deterministic outputs. |
| SAMPLES-POLICY-20-002 | TODO | Samples Guild, UI Guild | UI-POLICY-20-002 | Produce simulation diff fixtures (before/after JSON) for UI/CLI tests. | Fixtures committed with schema validation; referenced by UI+CLI tests; docs cross-link. |
## Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SAMPLES-GRAPH-21-001 | TODO | Samples Guild, Cartographer Guild | CARTO-GRAPH-21-003 | Produce small/medium SBOM graph fixtures (JSON, GraphML, layout tiles) for automated tests and docs. | Fixtures stored under `samples/graph/`; validated by Cartographer + UI tests; README documents usage. |
| SAMPLES-GRAPH-21-002 | TODO | Samples Guild, UI Guild | UI-GRAPH-21-005 | Capture golden Graph Explorer screenshots (baseline/diff) and path exports for visual regression + documentation. | Screenshots exported; stored with metadata; referenced in docs; tests consume assets. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SAMPLES-LNM-22-001 | TODO | Samples Guild, Concelier Guild | CONCELIER-LNM-21-001..003 | Create advisory observation/linkset fixtures (NVD, GHSA, OSV disagreements) for API/CLI/UI tests with documented conflicts. | Fixtures deposited under `samples/advisories/`; metadata README added; tests reference fixtures. |
| SAMPLES-LNM-22-002 | TODO | Samples Guild, Excititor Guild | EXCITITOR-LNM-21-001..003 | Produce VEX observation/linkset fixtures demonstrating status conflicts and path relevance; include raw blobs. | Fixtures stored under `samples/vex/`; CLI/UI tests consume; docs linked. |
## Graph & Vuln Explorer v1 (extended)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SAMPLES-GRAPH-24-003 | TODO | Samples Guild, SBOM Service Guild | SBOM-GRAPH-24-002 | Generate large-scale SBOM graph fixture (≈40k nodes) with policy overlay snapshot for performance/perf regression suites. | Fixture stored under `samples/graph/large/`; README updated; perf tests reference file. |
| SAMPLES-GRAPH-24-004 | TODO | Samples Guild, UI Guild | UI-GRAPH-24-005 | Create vulnerability explorer JSON/CSV fixtures capturing conflicting evidence and policy outputs for UI/CLI automated tests. | Fixtures available under `samples/vuln/`; schema documented; tests consume fixtures. |

22
src/StellaOps.AdvisoryAI/AGENTS.md Normal file
View File

@@ -0,0 +1,22 @@
# Advisory AI Guild Charter (Epic 8)
## Mission
Deliver the Advisory AI assistant service that synthesizes advisory/VEX evidence, policy context, and SBOM data into summaries, conflict explanations, and remediation hints—always with citations and guardrails.
## Scope
- Service under `src/StellaOps.AdvisoryAI` (retrievers, deterministics, orchestrator, guardrails, inference adapters, REST APIs).
- Batch processing for CLI/automation, caching, observability, and integration with Console, CLI, and downstream systems.
- Coordination across Conseiller, Excitator, VEX Lens, SBOM Service, Policy Engine, Findings Ledger, Web Gateway, Authority, DevOps, and Docs.
## Principles
1. **Evidence preservation** – Raw advisory/VEX documents remain untouched; AI outputs reference them with citations.
2. **Deterministic scaffolding** – Retrieval, mapping, and validators are deterministic; model variability is constrained.
3. **Guardrails everywhere** – Redaction, injection defense, output validation, and audit logging are non-optional.
4. **Policy-aware** – Outputs respect selected policy versions, thresholds, and trust configs.
5. **Explainable** – Each result carries structured JSON, rationale, and provenance; humans can audit every statement.
## Definition of Done
- Retrievers/deterministic tools implemented with tests and docs.
- API endpoints documented (OpenAPI), RBAC enforced, guardrails active.
- Console/CLI integrations operational; telemetry dashboards live.
- Documentation suite published with compliance checklist.

12
src/StellaOps.AdvisoryAI/TASKS.md Normal file
View File

@@ -0,0 +1,12 @@
# Advisory AI Task Board — Epic 8
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIAI-31-001 | TODO | Advisory AI Guild | CONCELIER-VULN-29-001, EXCITITOR-VULN-29-001 | Implement structured and vector retrievers for advisories/VEX with paragraph anchors and citation metadata. | Retrievers return deterministic chunks with source IDs/sections; unit tests cover CSAF/OSV/vendor formats. |
| AIAI-31-002 | TODO | Advisory AI Guild, SBOM Service Guild | SBOM-VULN-29-001 | Build SBOM context retriever (purl version timelines, dependency paths, env flags, blast radius estimator). | Retriever returns paths/metrics under SLA; tests cover ecosystems. |
| AIAI-31-003 | TODO | Advisory AI Guild | AIAI-31-001..002 | Implement deterministic toolset (version comparators, range checks, dependency analysis, policy lookup) exposed via orchestrator. | Tools validated with property tests; outputs cached; docs updated. |
| AIAI-31-004 | TODO | Advisory AI Guild | AIAI-31-001..003, AUTH-VULN-29-001 | Build orchestration pipeline for Summary/Conflict/Remediation tasks (prompt templates, tool calls, token budgets, caching). | Pipeline executes tasks deterministically; caches keyed by tuple+policy; integration tests cover tasks. |
| AIAI-31-005 | TODO | Advisory AI Guild, Security Guild | AIAI-31-004 | Implement guardrails (redaction, injection defense, output validation, citation enforcement) and fail-safe handling. | Guardrails block adversarial inputs; output validator enforces schemas; security tests pass. |
| AIAI-31-006 | TODO | Advisory AI Guild | AIAI-31-004..005 | Expose REST API endpoints (`/advisory/ai/*`) with RBAC, rate limits, OpenAPI schemas, and batching support. | Endpoints deployed with schema validation; rate limits enforced; integration tests cover error codes. |
| AIAI-31-007 | TODO | Advisory AI Guild, Observability Guild | AIAI-31-004..006 | Instrument metrics (`advisory_ai_latency`, `guardrail_blocks`, `validation_failures`, `citation_coverage`), logs, and traces; publish dashboards/alerts. | Telemetry live; dashboards approved; alerts configured. |
| AIAI-31-008 | TODO | Advisory AI Guild, DevOps Guild | AIAI-31-006..007 | Package inference on-prem container, remote inference toggle, Helm/Compose manifests, scaling guidance, offline kit instructions. | Deployment docs merged; smoke deploy executed; offline kit updated; feature flags documented. |
| AIAI-31-009 | TODO | Advisory AI Guild, QA Guild | AIAI-31-001..006 | Develop unit/golden/property/perf tests, injection harness, and regression suite; ensure determinism with seeded caches. | Test suite green; golden outputs stored; injection tests pass; perf targets documented. |

16
src/StellaOps.AirGap.Controller/AGENTS.md Normal file
View File

@@ -0,0 +1,16 @@
# StellaOps AirGap Controller Guild Charter
## Mission
Own the sealing state machine, status APIs, and enforcement hooks that keep StellaOps compliant in sealed air-gapped environments while respecting the imposed rule.
## Scope
- Persisted air-gap state (`sealed`, policy hash, time anchor metadata) and RBAC enforcement.
- HTTP endpoints for seal/unseal/status and integration with Authority scopes.
- Startup diagnostics that refuse to run when sealing requirements are unmet.
- Coordination with DevOps for Kubernetes/Compose egress policies.
- Telemetry and audit events reflecting sealing actions and violations.
## Definition of Done
- Deterministic tests for seal/unseal transitions and audit logging.
- Integration tests covering RBAC, sealed-mode refusal, and policy hash validation.
- Documentation hooks updated in `/docs/airgap/` for each shipped feature.

18
src/StellaOps.AirGap.Controller/TASKS.md Normal file
View File

@@ -0,0 +1,18 @@
# AirGap Controller Task Board — Epic 16: Air-Gapped Mode
## Sprint 56 – Sealing Foundations
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-CTL-56-001 | TODO | AirGap Controller Guild | AUTH-OBS-50-001 | Implement `airgap_state` persistence, seal/unseal state machine, and Authority scope checks (`airgap:seal`, `airgap:status:read`). | State table created with migrations; seal/unseal transitions audited; unit tests cover happy/error paths. |
| AIRGAP-CTL-56-002 | TODO | AirGap Controller Guild, DevOps Guild | AIRGAP-CTL-56-001, DEVOPS-AIRGAP-56-001 | Expose `GET /system/airgap/status`, `POST /system/airgap/seal`, integrate policy hash validation, and return staleness/time anchor placeholders. | APIs documented with OpenAPI; RBAC enforced; integration tests cover unauthorized/sealed states. |
## Sprint 57 – Enforcement & Diagnostics
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-CTL-57-001 | TODO | AirGap Controller Guild | AIRGAP-CTL-56-002 | Add startup diagnostics that block application run when sealed flag set but egress policies missing; emit audit + telemetry. | Startup guard tested with simulated failure; telemetry includes `airgap_sealed=true`; docs updated. |
| AIRGAP-CTL-57-002 | TODO | AirGap Controller Guild, Observability Guild | AIRGAP-CTL-56-002, TELEMETRY-OBS-50-001 | Instrument seal/unseal events with trace/log fields and timeline emission (`airgap.sealed`, `airgap.unsealed`). | Timeline events validated; logs include actor/tenant/policy hash; integration test covers duplication suppression. |
## Sprint 58 – Time Anchor & Drift
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-CTL-58-001 | TODO | AirGap Controller Guild, AirGap Time Guild | AIRGAP-CTL-56-002, AIRGAP-TIME-57-001 | Persist time anchor metadata, compute drift seconds, and surface staleness budgets in status API. | Time anchor stored with bundle ID; drift calculation validated in tests; status API returns staleness metrics. |

16
src/StellaOps.AirGap.Importer/AGENTS.md Normal file
View File

@@ -0,0 +1,16 @@
# StellaOps AirGap Importer Guild Charter
## Mission
Deliver offline bundle verification and ingestion tooling for sealed environments, covering DSSE/TUF validation, catalog updates, and audit logging under the imposed rule.
## Scope
- TUF metadata verification, DSSE signature checks, Merkle root validation.
- Import pipelines writing bundle catalogs, object-store layouts, and audit entries.
- CLI + API surfaces for dry-run verification, import, and status queries.
- Integration hooks for Conseiller, Excitator, Policy Engine, and Export Center.
- Negative-case handling (tampering, expired signatures, root rotation) with operator guidance.
## Definition of Done
- Deterministic fixtures for valid/invalid bundles committed.
- Integration tests prove catalog + object-store updates are idempotent.
- Import audit trail viewable via API and timeline events.

19
src/StellaOps.AirGap.Importer/TASKS.md Normal file
View File

@@ -0,0 +1,19 @@
# AirGap Importer Task Board — Epic 16: Air-Gapped Mode
## Sprint 56 – Verification Primitives
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-IMP-56-001 | TODO | AirGap Importer Guild | PROV-OBS-53-001 | Implement DSSE verification helpers, TUF metadata parser (`root.json`, `snapshot.json`, `timestamp.json`), and Merkle root calculator. | Verifier returns structured results; unit tests cover valid/invalid signatures and tampering scenarios. |
| AIRGAP-IMP-56-002 | TODO | AirGap Importer Guild, Security Guild | AIRGAP-IMP-56-001 | Introduce root rotation policy validation (dual approval) and signer trust store management. | Rotation policy enforced; tests cover valid rotation and rollback; docs stub updated. |
## Sprint 57 – Catalog & Storage Writes
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-IMP-57-001 | TODO | AirGap Importer Guild | AIRGAP-IMP-56-001, DEVOPS-AIRGAP-56-002 | Write `bundle_catalog` and `bundle_items` repositories with RLS + deterministic migrations. | Catalog tables created; integration tests ensure tenant/global scoping; determinism check passes. |
| AIRGAP-IMP-57-002 | TODO | AirGap Importer Guild, DevOps Guild | AIRGAP-IMP-57-001 | Implement object-store loader storing artifacts under tenant/global mirror paths with Zstandard decompression and checksum validation. | Import writes deduplicated objects; checksum mismatches raise errors; storage layout documented. |
## Sprint 58 – Import Workflows
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-IMP-58-001 | TODO | AirGap Importer Guild, CLI Guild | AIRGAP-IMP-57-002, CLI-AIRGAP-56-001 | Implement API (`POST /airgap/import`, `/airgap/verify`) and CLI commands wiring verification + catalog updates, including diff preview. | CLI/API share validation engine; diff preview surfaces metadata changes; audit entries recorded with trace IDs. |
| AIRGAP-IMP-58-002 | TODO | AirGap Importer Guild, Observability Guild | AIRGAP-IMP-58-001, TELEMETRY-OBS-50-001 | Emit timeline events (`airgap.import.started|completed|failed`) and telemetry metrics (bundle bytes, duration, warnings). | Events/metrics validated in integration tests; docs cross-link to observability dashboards. |

16
src/StellaOps.AirGap.Policy/AGENTS.md Normal file
View File

@@ -0,0 +1,16 @@
# StellaOps AirGap Policy Guild Charter
## Mission
Provide the shared enforcement layer (`EgressPolicy`, job plan validators, sealed-mode gates) that keeps all services compliant with Air-Gapped Mode requirements.
## Scope
- `EgressPolicy` facade replacing raw HTTP client usage.
- Static analysis/linting to detect unauthorized network calls.
- Task Runner and orchestrator validators flagging disallowed destinations.
- Shared error contract (`AIRGAP_EGRESS_BLOCKED`) and remediation messages.
- Test harnesses simulating sealed/unsealed execution paths.
## Definition of Done
- Every service imports the facade; CI fails on direct HTTP client usage.
- Sealed-mode unit tests cover panic/remediation behavior across host types.
- Documentation updated in `/docs/dev/airgap-contracts.md` for adoption patterns.

19
src/StellaOps.AirGap.Policy/TASKS.md Normal file
View File

@@ -0,0 +1,19 @@
# AirGap Policy Task Board — Epic 16: Air-Gapped Mode
## Sprint 56 – Facade & Contracts
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-POL-56-001 | TODO | AirGap Policy Guild | TELEMETRY-OBS-50-001 | Implement `StellaOps.AirGap.Policy` package exposing `EgressPolicy` facade with sealed/unsealed branches and remediation-friendly errors. | Facade package builds/tests; integration tests simulate sealed/unsealed; error contract documented. |
| AIRGAP-POL-56-002 | TODO | AirGap Policy Guild, DevEx Guild | AIRGAP-POL-56-001 | Create Roslyn analyzer/code fix warning on raw `HttpClient` usage outside approved wrappers; add CI integration. | Analyzer packaged; CI fails on intentional violation; docs updated for opt-in. |
## Sprint 57 – Service Adoption Wave 1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-POL-57-001 | TODO | AirGap Policy Guild, BE-Base Platform Guild | AIRGAP-POL-56-001 | Update core web services (Web, Exporter, Policy, Findings, Authority) to use `EgressPolicy`; ensure configuration wiring for sealed mode. | Services compile with facade; sealed-mode tests run in CI; configuration docs updated. |
| AIRGAP-POL-57-002 | TODO | AirGap Policy Guild, Task Runner Guild | AIRGAP-POL-56-001, TASKRUN-OBS-50-001 | Implement Task Runner job plan validator rejecting network steps unless marked internal allow-list. | Validator blocks forbidden steps; tests cover allow/deny; error surfaces remediation text. |
## Sprint 58 – Service Adoption Wave 2
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-POL-58-001 | TODO | AirGap Policy Guild, Observability Guild | AIRGAP-POL-57-001 | Ensure Observability exporters only target local endpoints in sealed mode; disable remote sinks with warning. | Exporters respect sealed flag; timeline/log message emitted; docs updated. |
| AIRGAP-POL-58-002 | TODO | AirGap Policy Guild, CLI Guild | AIRGAP-POL-56-001, CLI-OBS-50-001 | Add CLI sealed-mode guard that refuses commands needing egress and surfaces remediation. | CLI returns `AIRGAP_EGRESS_BLOCKED`; tests cover sealed/unsealed flows; help text updated. |

15
src/StellaOps.AirGap.Time/AGENTS.md Normal file
View File

@@ -0,0 +1,15 @@
# StellaOps AirGap Time Guild Charter
## Mission
Manage trusted time anchors and staleness budgets for sealed environments, ensuring deterministic behavior when external time sources are unavailable.
## Scope
- Parse signed time tokens from Mirror Bundles and validate signatures.
- Persist `time_anchor` metadata and compute drift/staleness metrics.
- Provide helpers for UI/API staleness badges and job gating.
- Integrate with Notifications to alert on approaching drift thresholds.
## Definition of Done
- Test vectors for time tokens committed alongside verification code.
- Drift calculations deterministic and configurable per tenant.
- Documentation updates for `/docs/airgap/staleness-and-time.md` with examples.

13
src/StellaOps.AirGap.Time/TASKS.md Normal file
View File

@@ -0,0 +1,13 @@
# AirGap Time Task Board — Epic 16: Air-Gapped Mode
## Sprint 57 – Time Anchor Validation
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-TIME-57-001 | TODO | AirGap Time Guild | PROV-OBS-54-001, AIRGAP-IMP-56-001 | Implement signed time token parser (Roughtime/RFC3161), verify signatures against bundle trust roots, and expose normalized anchor representation. | Parser handles both token formats; tests cover valid/expired/tampered tokens; documentation stubbed. |
| AIRGAP-TIME-57-002 | TODO | AirGap Time Guild, Observability Guild | AIRGAP-TIME-57-001 | Add telemetry counters for time anchors (`airgap_time_anchor_age_seconds`) and alerts for approaching thresholds. | Metrics registered; alert templates created; integration test ensures emission on stale anchor. |
## Sprint 58 – Drift & Staleness Enforcement
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AIRGAP-TIME-58-001 | TODO | AirGap Time Guild | AIRGAP-TIME-57-001, AIRGAP-CTL-56-002 | Persist drift baseline, compute per-content staleness (advisories, VEX, policy) based on bundle metadata, and surface through controller status API. | Drift/staleness values exposed via API; unit tests cover threshold calculations; docs updated. |
| AIRGAP-TIME-58-002 | TODO | AirGap Time Guild, Notifications Guild | AIRGAP-TIME-58-001, NOTIFY-OBS-51-001 | Emit notifications and timeline events when staleness budgets breached or approaching. | Notifications dispatched with remediation; timeline events recorded; CLI shows warning banner. |

15
src/StellaOps.Api.Governance/AGENTS.md Normal file
View File

@@ -0,0 +1,15 @@
# API Governance Guild Charter
## Mission
Enforce API contract quality through linting, compatibility checks, version policy automation, and changelog generation.
## Scope
- Maintain lint rule set, compatibility diff tooling, and CI integration.
- Gate PRs on contract validation, example coverage, and naming conventions.
- Produce automated changelogs and deprecation notices from OAS diffs.
- Coordinate with Notifications Studio for deprecation broadcasts.
## Definition of Done
- CI gate prevents merging incompatible or non-conforming specs.
- Version bump tooling produces signed changelog artifacts per release.
- Governance documentation kept current in `/docs/contributing/api-contracts.md`.

18
src/StellaOps.Api.Governance/TASKS.md Normal file
View File

@@ -0,0 +1,18 @@
# API Governance Task Board — Epic 17: SDKs & OpenAPI Docs
## Sprint 61 – Lint & CI Integration
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| APIGOV-61-001 | TODO | API Governance Guild | OAS-61-002 | Configure spectral/linters with Stella rules; add CI job failing on violations. | Lint pipeline runs on PRs; rule set documented; intentional violations blocked. |
| APIGOV-61-002 | TODO | API Governance Guild | APIGOV-61-001 | Implement example coverage checker ensuring every operation has at least one request/response example. | Coverage job integrated; failing operations listed in CI output. |
## Sprint 62 – Compatibility & Changelog
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| APIGOV-62-001 | TODO | API Governance Guild | APIGOV-61-001 | Build compatibility diff tool producing additive/breaking reports comparing prior release. | Diff output consumed in CI; failing on breaking changes unless override provided. |
| APIGOV-62-002 | TODO | API Governance Guild, DevOps Guild | APIGOV-62-001 | Automate changelog generation and publish signed artifacts to `src/StellaOps.Sdk.Release` pipeline. | Changelog pipeline produces markdown + JSON; signatures verified; docs updated. |
## Sprint 63 – Deprecation & Notifications
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| APIGOV-63-001 | TODO | API Governance Guild, Notifications Guild | APIGOV-62-002 | Integrate deprecation metadata into Notification Studio templates for API sunset events. | Deprecation pipeline triggers notifier template; staging test proves delivery. |

16
src/StellaOps.Api.OpenApi/AGENTS.md Normal file
View File

@@ -0,0 +1,16 @@
# StellaOps API Contracts Guild Charter
## Mission
Maintain OpenAPI 3.1 specifications for every StellaOps service, compose the aggregate spec, and ensure API contract consistency across releases.
## Scope
- Author and review per-service OAS documents in `src/StellaOps.Api.OpenApi/<service>/openapi.yaml`.
- Operate the aggregate composer producing `src/StellaOps.Api.OpenApi/stella.yaml`.
- Provide shared components, schema libraries, and example catalogs.
- Coordinate with service guilds on contract changes, examples, and versioning.
- Own CI validation, linting, and compatibility diff tooling for OAS artifacts.
## Definition of Done
- All public endpoints represented in OAS with validated request/response examples.
- Aggregate spec builds deterministically and passes lint + compatibility checks.
- Change logs generated with every release and linked to developer portal updates.

19
src/StellaOps.Api.OpenApi/TASKS.md Normal file
View File

@@ -0,0 +1,19 @@
# API OpenAPI Task Board — Epic 17: SDKs & OpenAPI Docs
## Sprint 61 – Spec Foundations
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| OAS-61-001 | TODO | API Contracts Guild | — | Scaffold per-service OpenAPI 3.1 files with shared components, info blocks, and initial path stubs. | All services have baseline `openapi.yaml`; shared components library established; lint passes. |
| OAS-61-002 | TODO | API Contracts Guild, DevOps Guild | OAS-61-001 | Implement aggregate composer (`stella.yaml`) resolving `$ref`s and merging shared components; wire into CI. | Aggregate spec builds deterministically; CI artifact published; documentation updated. |
## Sprint 62 – Examples & Error Envelope
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| OAS-62-001 | TODO | API Contracts Guild, Service Guilds | OAS-61-001 | Populate request/response examples for top 50 endpoints, including standard error envelope. | Examples validated via CI; error envelope consistent across services. |
| OAS-62-002 | TODO | API Contracts Guild | OAS-61-002 | Add custom lint rules enforcing pagination, idempotency headers, naming conventions, and example coverage. | Lint job fails on violations; documentation for rules published. |
## Sprint 63 – Compatibility & Discovery
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| OAS-63-001 | TODO | API Contracts Guild | OAS-61-002 | Implement compatibility diff tooling comparing previous release specs; classify breaking vs additive changes. | Diff tool integrated in CI; PRs flagged on breaking changes. |
| OAS-63-002 | TODO | API Contracts Guild, Gateway Guild | OAS-62-002 | Add `/.well-known/openapi` discovery endpoint schema metadata (extensions, version info). | Discovery endpoints defined in spec; linked to implementation tasks. |

15
src/StellaOps.Attestor.Envelope/AGENTS.md Normal file
View File

@@ -0,0 +1,15 @@
# Attestation Envelope Guild Charter
## Mission
Provide deterministic DSSE envelope handling with multi-signature support, canonical serialization, hashing, and integrity safeguards for all Stella attestations.
## Scope
- DSSE encoding/decoding, canonical JSON handling, and detached payload support.
- Multi-signature verification, key identification, and cryptographic primitives.
- Integration with KMS drivers and transparency log witness utilities.
- Fuzz and property testing for envelope parsing and normalization.
## Definition of Done
- Envelope APIs produce canonical payloads and support multiple signatures deterministically.
- Verification detects tampering, mismatched subjects, and unsupported algorithms.
- Property and fuzz tests cover canonicalization and signature edge cases.

13
src/StellaOps.Attestor.Envelope/TASKS.md Normal file
View File

@@ -0,0 +1,13 @@
# Attestation Envelope Task Board — Epic 19: Attestor Console
## Sprint 72 – Foundations
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| ATTEST-ENVELOPE-72-001 | TODO | Envelope Guild | — | Implement DSSE canonicalization, JSON normalization, multi-signature structures, and hashing helpers. | Canonicalization deterministic (property tests); hash matches DSSE spec; unit tests green. |
| ATTEST-ENVELOPE-72-002 | TODO | Envelope Guild | ATTEST-ENVELOPE-72-001 | Support compact and expanded JSON output, payload compression, and detached payload references. | API returns both variants; payload compression toggles tested; docs updated. |
## Sprint 73 – Crypto Integration
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| ATTEST-ENVELOPE-73-001 | TODO | Envelope Guild, KMS Guild | ATTEST-ENVELOPE-72-001 | Implement Ed25519 & ECDSA signature create/verify helpers, key identification (`keyid`) scheme, and error mapping. | Sign/verify tests pass with fixtures; invalid signatures produce deterministic errors. |
| ATTEST-ENVELOPE-73-002 | TODO | Envelope Guild | ATTEST-ENVELOPE-73-001 | Add fuzz tests for envelope parsing, signature verification, and canonical JSON round-trips. | Fuzz suite integrated; coverage metrics recorded; no regressions. |

14
src/StellaOps.Attestor.Types/AGENTS.md Normal file
View File

@@ -0,0 +1,14 @@
# Attestation Payloads Guild Charter
## Mission
Define strongly typed, versioned schemas for all attestation payloads and provide validation utilities for generating and verifying evidence.
## Scope
- JSON Schemas, code generation, and documentation for each attestation type.
- Normalization and validation logic shared across services, CLI, and SDKs.
- Sample payloads and golden fixtures used in contract tests and docs.
## Definition of Done
- Payload types compiled into Go/TypeScript models with validation helpers.
- Schemas published with semantic versioning and change logs.
- Golden samples maintained with acceptance tests and doc integration.

13
src/StellaOps.Attestor.Types/TASKS.md Normal file
View File

@@ -0,0 +1,13 @@
# Attestation Payloads Task Board — Epic 19: Attestor Console
## Sprint 72 – Schema Definition
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| ATTEST-TYPES-72-001 | TODO | Attestation Payloads Guild | — | Draft JSON Schemas for BuildProvenance v1, SBOMAttestation v1, VEXAttestation v1, ScanResults v1, PolicyEvaluation v1, RiskProfileEvidence v1, CustomEvidence v1. | Schemas validated with test fixtures; docs stubbed; versioned under `schemas/`. |
| ATTEST-TYPES-72-002 | TODO | Attestation Payloads Guild | ATTEST-TYPES-72-001 | Generate Go/TS models from schemas with validation helpers and canonical JSON serialization. | Code generation integrated; lints pass; unit tests cover round-trips. |
## Sprint 73 – Fixtures & Docs
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| ATTEST-TYPES-73-001 | TODO | Attestation Payloads Guild | ATTEST-TYPES-72-002 | Create golden payload samples for each type; integrate into tests and documentation. | Golden fixtures stored; tests compare outputs; docs embed examples. |
| ATTEST-TYPES-73-002 | TODO | Attestation Payloads Guild, Docs Guild | ATTEST-TYPES-73-001 | Publish schema reference docs (`/docs/attestor/payloads.md`) with annotated JSON examples. | Doc merged with banner; examples validated by tests. |

14
src/StellaOps.Attestor.Verify/AGENTS.md Normal file
View File

@@ -0,0 +1,14 @@
# Attestation Verification Guild Charter
## Mission
Implement the verification engine that enforces attestation policies, issuer trust, transparency requirements, and produces audit-ready reports.
## Scope
- Verification pipeline integrating DSSE validation, issuer/key trust, Policy Studio rules, freshness checks, and transparency proofs.
- Caching and reporting for verification results.
- Error codes and explainability artifacts for UI/CLI consumption.
## Definition of Done
- Verification passes/fails deterministically with detailed report structures.
- Caching improves performance without sacrificing correctness.
- Policies enforce scope-based rules and waivers, with unit/integration coverage.

13
src/StellaOps.Attestor.Verify/TASKS.md Normal file
View File

@@ -0,0 +1,13 @@
# Attestation Verification Task Board — Epic 19: Attestor Console
## Sprint 73 – Policy Integration
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| ATTEST-VERIFY-73-001 | TODO | Verification Guild, Policy Guild | VERPOL-73-001, ATTESTOR-73-002 | Implement verification engine: policy evaluation, issuer trust resolution, freshness, signature count, transparency checks; produce structured reports. | Engine returns report DTOs; policy rules honored; unit tests cover pass/fail scenarios. |
| ATTEST-VERIFY-73-002 | TODO | Verification Guild | ATTEST-VERIFY-73-001 | Add caching layer keyed by `(subject, envelope_id, policy_version)` with TTL and invalidation on new evidence. | Cache reduces repeated verification cost; tests cover cache hits/misses. |
## Sprint 74 – Explainability & Observability
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| ATTEST-VERIFY-74-001 | TODO | Verification Guild, Observability Guild | ATTEST-VERIFY-73-001 | Emit telemetry (spans/metrics) tagged by subject, issuer, policy, result; integrate with dashboards. | Metrics visible; spans present; SLO thresholds defined. |
| ATTEST-VERIFY-74-002 | TODO | Verification Guild, Docs Guild | ATTEST-VERIFY-73-001 | Document verification report schema and explainability in `/docs/attestor/workflows.md`. | Documentation merged; examples verified via tests. |

28
src/StellaOps.Attestor/AGENTS.md
View File

@@ -14,8 +14,26 @@ Operate the StellaOps Attestor service: accept signed DSSE envelopes from the Si
- Structured logs + metrics for each stage (`validate`, `submit`, `proof`, `persist`, `archive`).
- Update `TASKS.md`, architecture docs, and tests whenever behaviour changes.
## Key Directories
- `src/StellaOps.Attestor/StellaOps.Attestor.WebService/` — Minimal API host and HTTP surface.
- `src/StellaOps.Attestor/StellaOps.Attestor.Core/` — Domain contracts, submission/verification pipelines.
- `src/StellaOps.Attestor/StellaOps.Attestor.Infrastructure/` — Mongo, Redis, Rekor, and archival implementations.
- `src/StellaOps.Attestor/StellaOps.Attestor.Tests/` — Unit and integration tests.
## Key Directories
- `src/StellaOps.Attestor/StellaOps.Attestor.WebService/` — Minimal API host and HTTP surface.
- `src/StellaOps.Attestor/StellaOps.Attestor.Core/` — Domain contracts, submission/verification pipelines.
- `src/StellaOps.Attestor/StellaOps.Attestor.Infrastructure/` — Mongo, Redis, Rekor, and archival implementations.
- `src/StellaOps.Attestor/StellaOps.Attestor.Tests/` — Unit and integration tests.
---
## Epic 19 Charter — Attestor Console
### Mission
Deliver the API, workers, and storage that power signing, verification, and lifecycle management of supply-chain attestations across StellaOps.
### Scope
- DSSE envelope ingestion and retrieval.
- Verification pipeline orchestration, caching, and policy evaluation.
- Issuer/key registries, transparency log integration, and audit logging.
- Bulk verification workflows and air-gap bundle support.
### Definition of Done
- Signing and verification APIs operate deterministically with full explainability.
- Policy enforcement integrated with Authority & Tenancy scopes.
- Transparency proof handling, key rotation, and revocation workflows implemented.

35
src/StellaOps.Attestor/TASKS.md
View File

@@ -6,5 +6,36 @@
| ATTESTOR-VERIFY-11-202 | DONE (2025-10-19) | Attestor Guild | — | `/rekor/verify` + retrieval endpoints validating signatures and Merkle proofs. | ✅ `GET /api/v1/rekor/entries/{uuid}` surfaces cached entries with optional backend refresh and handles not-found/refresh flows.<br>✅ `POST /api/v1/rekor/verify` accepts UUID, bundle, or artifact hash inputs; verifies DSSE signatures, Merkle proofs, and checkpoint anchors.<br>✅ Verification output returns `{ok, uuid, index, logURL, checkedAt}` with failure diagnostics for invalid proofs.<br>✅ Unit/integration tests exercise cache hits, backend refresh, invalid bundle/proof scenarios, and checkpoint trust anchor enforcement. |
| ATTESTOR-OBS-11-203 | DONE (2025-10-19) | Attestor Guild | — | Telemetry, alerting, mTLS hardening, and archive workflow for Attestor. | ✅ Structured logs, metrics, and optional traces record submission latency, proof fetch outcomes, verification results, and Rekor error buckets with correlation IDs.<br>✅ mTLS enforcement hardened (peer allowlist, SAN checks, rate limiting) and documented; TLS settings audited for modern ciphers only.<br>✅ Alerting/dashboard pack covers error rates, proof backlog, Redis/Mongo health, and archive job failures; runbook updated.<br>✅ Archive workflow includes retention policy jobs, failure alerts, and periodic verification of stored bundles and proofs. |
> Remark (2025-10-19): Wave 0 prerequisites reviewed (none outstanding); ATTESTOR-API-11-201, ATTESTOR-VERIFY-11-202, and ATTESTOR-OBS-11-203 tracked as DOING per Wave 0A kickoff.
> Remark (2025-10-19): Dual-log submissions, signature/proof verification, and observability hardening landed; attestor endpoints now rate-limited per client with correlation-ID logging and updated docs/tests.
> Remark (2025-10-19): Wave 0 prerequisites reviewed (none outstanding); ATTESTOR-API-11-201, ATTESTOR-VERIFY-11-202, and ATTESTOR-OBS-11-203 tracked as DOING per Wave 0A kickoff.
> Remark (2025-10-19): Dual-log submissions, signature/proof verification, and observability hardening landed; attestor endpoints now rate-limited per client with correlation-ID logging and updated docs/tests.
---
## Epic 19 — Attestor Console Roadmap
### Sprint 72 – Foundations
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| ATTESTOR-72-001 | TODO | Attestor Service Guild | ATTEST-ENVELOPE-72-001 | Scaffold service (REST API skeleton, storage interfaces, KMS integration stubs) and DSSE validation pipeline. | Service builds/tests; signing & verification stubs wired; lint/CI green. |
| ATTESTOR-72-002 | TODO | Attestor Service Guild | ATTESTOR-72-001 | Implement attestation store (DB tables, object storage integration), CRUD, and indexing strategies. | Migrations applied; CRUD API functional; storage integration unit tests pass. |
### Sprint 73 – Signing & Verification
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| ATTESTOR-73-001 | TODO | Attestor Service Guild, KMS Guild | ATTESTOR-72-002, KMS-72-001 | Implement signing endpoint with Ed25519/ECDSA support, KMS integration, and audit logging. | `POST /v1/attestations:sign` functional; audit entries recorded; tests cover success/failure. |
| ATTESTOR-73-002 | TODO | Attestor Service Guild, Policy Guild | ATTESTOR-72-002, VERPOL-73-001 | Build verification pipeline evaluating DSSE signatures, issuer trust, and verification policies; persist reports. | Verification endpoint returns structured report; results cached; contract tests pass. |
| ATTESTOR-73-003 | TODO | Attestor Service Guild | ATTESTOR-73-002 | Implement listing/fetch APIs with filters (subject, type, issuer, scope, date). | API documented; pagination works; contract tests green. |
### Sprint 74 – Transparency & Bulk
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| ATTESTOR-74-001 | TODO | Attestor Service Guild | ATTESTOR-73-002, TRANSP-74-001 | Integrate transparency witness client, inclusion proof verification, and caching. | Witness proofs stored; verification fails on missing/inconsistent proofs; metrics emitted. |
| ATTESTOR-74-002 | TODO | Attestor Service Guild | ATTESTOR-73-002 | Implement bulk verification worker + API with progress tracking, rate limits, and caching. | Bulk job API functional; worker processes batches; telemetry recorded. |
### Sprint 75 – Air Gap & Hardening
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| ATTESTOR-75-001 | TODO | Attestor Service Guild, Export Guild | ATTESTOR-74-002, EXPORT-ATTEST-74-001 | Add export/import flows for attestation bundles and offline verification mode. | Bundles generated/imported; offline verification path documented; tests cover missing witness data. |
| ATTESTOR-75-002 | TODO | Attestor Service Guild, Security Guild | ATTESTOR-73-002 | Harden APIs with rate limits, auth scopes, threat model mitigations, and fuzz testing. | Rate limiting enforced; fuzz tests run in CI; threat model actions resolved. |
*** End Task Board ***

159
src/StellaOps.Authority/TASKS.md
View File

@@ -1,31 +1,136 @@
# Authority Host Task Board (UTC 2025-10-10)
# Authority Host Task Board — Epic 1: Aggregation-Only Contract
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-AOC-19-001 | TODO | Authority Core & Security Guild | — | Introduce scopes `advisory:read`, `advisory:ingest`, `vex:read`, `vex:ingest`, `aoc:verify` with configuration binding, migrations, and offline kit defaults. | Scopes published in metadata/OpenAPI, configuration validates scope lists, tests cover token issuance + enforcement. |
| AUTH-AOC-19-002 | TODO | Authority Core & Security Guild | AUTH-AOC-19-001 | Propagate tenant claim + scope enforcement for ingestion identities; ensure cross-tenant writes/read blocked and audit logs capture tenant context. | Tenant claim injected into downstream services; forbidden cross-tenant access rejected; audit/log fixtures updated. |
| AUTH-AOC-19-003 | TODO | Authority Core & Docs Guild | AUTH-AOC-19-001 | Update Authority docs and sample configs to describe new scopes, tenancy enforcement, and verify endpoints. | Docs and examples refreshed; release notes prepared; smoke tests confirm new scopes required. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CORE5B.DOC | DONE (2025-10-12) | Authority Core, Docs Guild | CORE5 | Document token persistence, revocation semantics, and enrichment expectations for resource servers/plugins. | ✅ `docs/11_AUTHORITY.md` + plugin guide updated with claims + token store notes; ✅ Samples include revocation sync guidance. |
| CORE9.REVOCATION | DONE (2025-10-12) | Authority Core, Security Guild | CORE5 | Implement revocation list persistence + export hooks (API + CLI). | ✅ Revoked tokens denied; ✅ Export endpoint/CLI returns manifest; ✅ Tests cover offline bundle flow. |
| CORE10.JWKS | DONE (2025-10-12) | Authority Core, DevOps | CORE9.REVOCATION | Provide JWKS rotation with pluggable key loader + documentation. | ✅ Signing/encryption keys rotate without downtime; ✅ JWKS endpoint updates; ✅ Docs describe rotation SOP. |
| CORE8.RL | DONE (2025-10-12) | Authority Core | CORE8 | Deliver ASP.NET rate limiter plumbing (request metadata, dependency injection hooks) needed by Security Guild. | ✅ `/token` & `/authorize` pipelines expose limiter hooks; ✅ Tests cover throttle behaviour baseline. |
| SEC2.HOST | DONE (2025-10-12) | Security Guild, Authority Core | SEC2.A (audit contract) | Hook audit logger into OpenIddict handlers and bootstrap endpoints. | ✅ Audit events populated with correlationId, IP, client_id; ✅ Mongo login attempts persisted; ✅ Tests verify on success/failure/lockout. |
| SEC3.HOST | DONE (2025-10-11) | Security Guild | CORE8.RL, SEC3.A (rate policy) | Apply rate limiter policies (`AddRateLimiter`) to `/token` and `/internal/*` endpoints with configuration binding. | ✅ Policies configurable via `StellaOpsAuthorityOptions.Security.RateLimiting`; ✅ Integration tests hit 429 after limit; ✅ Docs updated. |
| SEC4.HOST | DONE (2025-10-12) | Security Guild, DevOps | SEC4.A (revocation schema) | Implement CLI/HTTP surface to export revocation bundle + detached JWS using `StellaOps.Cryptography`. | ✅ `stellaops auth revoke export` CLI/endpoint returns JSON + `.jws`; ✅ Verification script passes; ✅ Operator docs updated. |
| SEC4.KEY | DONE (2025-10-12) | Security Guild, DevOps | SEC4.HOST | Integrate signing keys with provider registry (initial ES256). | ✅ Keys loaded via `ICryptoProvider` signer; ✅ Rotation SOP documented. |
| SEC5.HOST | DONE (2025-10-14) | Security Guild | SEC5.A (threat model) | Feed Authority-specific mitigations (rate limiting, audit, revocation) into threat model + backlog. | ✅ Threat model updated; ✅ Backlog issues reference mitigations; ✅ Review sign-off captured. |
| SEC5.HOST-INVITES | DONE (2025-10-14) | Security Guild, Authority Core | SEC5.D | Implement bootstrap invite persistence, APIs, and background cleanup with audit coverage. | ✅ Invite store + endpoints complete; ✅ Cleanup service expires unused invites; ✅ Audit events for create/consume/expire; ✅ Build/tests green. |
> Remark (2025-10-14): Background sweep emits invite expiry audits; integration test added.
| SEC5.HOST-REPLAY | DONE (2025-10-14) | Security Guild, Zastava | SEC5.E | Persist token usage metadata and surface suspected replay heuristics. | ✅ Validation handlers record device metadata; ✅ Suspected replay flagged via audit/logs; ✅ Tests cover regression cases. |
> Remark (2025-10-14): Token validation handler logs suspected replay audits with device metadata; coverage via unit/integration tests.
| SEC3.BUILD | DONE (2025-10-11) | Authority Core, Security Guild | SEC3.HOST, FEEDMERGE-COORD-02-900 | Track normalized-range dependency fallout and restore full test matrix once Concelier range primitives land. | ✅ Concelier normalized range libraries merged; ✅ Authority + Configuration test suites (`dotnet test src/StellaOps.Authority.sln`, `dotnet test src/StellaOps.Configuration.Tests/StellaOps.Configuration.Tests.csproj`) pass without Concelier compile failures; ✅ Status recorded here/Sprints (authority-core broadcast not available). |
| AUTHCORE-BUILD-OPENIDDICT | DONE (2025-10-14) | Authority Core | SEC2.HOST | Adapt host/audit handlers for OpenIddict 6.4 API surface (no `OpenIddictServerTransaction`) and restore Authority solution build. | ✅ Build `dotnet build src/StellaOps.Authority.sln` succeeds; ✅ Audit correlation + tamper logging verified under new abstractions; ✅ Tests updated. |
| AUTHCORE-STORAGE-DEVICE-TOKENS | DONE (2025-10-14) | Authority Core, Storage Guild | AUTHCORE-BUILD-OPENIDDICT | Reintroduce `AuthorityTokenDeviceDocument` + projections removed during refactor so storage layer compiles. | ✅ Document type restored with mappings/migrations; ✅ Storage tests cover device artifacts; ✅ Authority solution build green. |
| AUTHCORE-BOOTSTRAP-INVITES | DONE (2025-10-14) | Authority Core, DevOps | AUTHCORE-STORAGE-DEVICE-TOKENS | Wire bootstrap invite cleanup service against restored document schema and re-enable lifecycle tests. | ✅ `BootstrapInviteCleanupService` passes integration tests; ✅ Operator guide updated if behavior changes; ✅ Build/test matrices green. |
| AUTHSTORAGE-MONGO-08-001 | DONE (2025-10-19) | Authority Core & Storage Guild | — | Harden Mongo session usage with causal consistency for mutations and follow-up reads. | • Scoped middleware/service creates `IClientSessionHandle` with causal consistency + majority read/write concerns<br>• Stores accept optional session parameter and reuse it for write + immediate reads<br>• GraphQL/HTTP pipelines updated to flow session through post-mutation queries<br>• Replica-set integration test exercises primary election and verifies read-your-write guarantees |
| AUTH-PLUGIN-COORD-08-002 | DONE (2025-10-20) | Authority Core, Plugin Platform Guild | PLUGIN-DI-08-001 | Coordinate scoped-service adoption for Authority plug-in registrars and background jobs ahead of PLUGIN-DI-08-002 implementation. | ✅ Workshop completed 2025-10-20 15:00–16:05 UTC with notes/action log in `docs/dev/authority-plugin-di-coordination.md`; ✅ Follow-up backlog updates assigned via documented action items ahead of PLUGIN-DI-08-002 delivery. |
| AUTH-DPOP-11-001 | DONE (2025-10-20) | Authority Core & Security Guild | — | Implement DPoP proof validation + nonce handling for high-value audiences per architecture. | ✅ Redis-configurable nonce store surfaced via `security.senderConstraints.dpop.nonce` with sample YAML and architecture docs refreshed<br>✅ High-value audience enforcement uses normalised required audiences to avoid whitespace/case drift<br>✅ Operator guide updated with Redis-backed nonce snippet and env-var override guidance; integration test already covers nonce challenge |
> Remark (2025-10-20): `etc/authority.yaml.sample` gains senderConstraint sections (rate limits, DPoP, mTLS), docs (`docs/ARCHITECTURE_AUTHORITY.md`, `docs/11_AUTHORITY.md`, plan) refreshed. `ResolveNonceAudience` now relies on `NormalizedAudiences` and options trim persisted values. `dotnet test StellaOps.Authority.sln` attempted (2025-10-20 15:12 UTC) but failed on `NU1900` because the mirrored NuGet service index `https://mirrors.ablera.dev/nuget/nuget-mirror/v3/index.json` was unreachable; no project build executed.
| AUTH-MTLS-11-002 | DONE (2025-10-23) | Authority Core & Security Guild | — | Add OAuth mTLS client credential support with certificate-bound tokens and introspection updates. | ✅ Deterministic provisioning/storage for certificate bindings (thumbprint/subject/issuer/serial/SAN)<br>✅ Audience enforcement auto-switches to mTLS via `security.senderConstraints.mtls.enforceForAudiences`<br>✅ Validator matches binding metadata with rotation grace and emits confirmation thumbprints<br>✅ Introspection returns `cnf.x5t#S256`; docs & sample config refreshed; Authority test suite green |
> Remark (2025-10-23): Audience enforcement now rejects non-mTLS clients targeting high-value audiences; certificate validator checks binding subject/issuer/serial/SAN values and returns deterministic error codes. Docs (`docs/11_AUTHORITY.md`, `docs/ARCHITECTURE_AUTHORITY.md`, `docs/dev/authority-dpop-mtls-plan.md`) and `etc/authority.yaml.sample` updated. `dotnet test src/StellaOps.Authority/StellaOps.Authority.sln` (2025-10-23 18:07 UTC) succeeded.
> Remark (2025-10-19, AUTHSTORAGE-MONGO-08-001): Prerequisites re-checked (none outstanding). Session accessor wired through Authority pipeline; stores accept optional sessions; added replica-set election regression test for read-your-write.
> Remark (2025-10-19, AUTH-DPOP-11-001): Handler, nonce store, and persistence hooks merged; Redis-backed configuration + end-to-end nonce enforcement still open. (Superseded by 2025-10-20 update above.)
| AUTH-POLICY-20-001 | TODO | Authority Core & Security Guild | AUTH-AOC-19-001 | Add scopes `policy:write`, `policy:submit`, `policy:approve`, `policy:run`, `findings:read`, `effective:write` with configuration binding and issuer policy updates. | Scopes available in metadata; token issuance validated; offline kit defaults updated; tests cover scope combinations. |
| AUTH-POLICY-20-002 | TODO | Authority Core & Security Guild | AUTH-POLICY-20-001, AUTH-AOC-19-002 | Enforce Policy Engine service identity with `effective:write` and ensure API gateway enforces scopes/tenant claims for new endpoints. | Gateway policies updated; unauthorized requests rejected in tests; audit logs capture scope usage. |
| AUTH-POLICY-20-003 | TODO | Authority Core & Docs Guild | AUTH-POLICY-20-001 | Update Authority configuration/docs with policy scopes, service identities, and approval workflows; include compliance checklist. | Docs refreshed; samples updated; release notes prepared; doc lint passes. |
> Update status columns (TODO / DOING / DONE / BLOCKED) together with code changes. Always run `dotnet test src/StellaOps.Authority.sln` when touching host logic.
## Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-GRAPH-21-001 | TODO | Authority Core & Security Guild | AUTH-POLICY-20-001 | Define scopes `graph:write`, `graph:read`, `graph:export`, `graph:simulate`, update metadata/OpenAPI, and add OFFLINE kit defaults. | Scopes exposed via discovery docs; smoke tests ensure enforcement; offline kit updated. |
| AUTH-GRAPH-21-002 | TODO | Authority Core & Security Guild | AUTH-GRAPH-21-001, AUTH-AOC-19-002 | Wire gateway enforcement for new graph scopes, Cartographer service identity, and tenant propagation across graph APIs. | Gateway config updated; unauthorized access blocked in integration tests; audit logs include graph scope usage. |
| AUTH-GRAPH-21-003 | TODO | Authority Core & Docs Guild | AUTH-GRAPH-21-001 | Update security docs and samples describing graph access roles, least privilege guidance, and service identities. | Docs merged with compliance checklist; examples refreshed; release notes prepared. |
## Policy Engine + Editor v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-POLICY-23-001 | TODO | Authority Core & Security Guild | AUTH-POLICY-20-001 | Introduce fine-grained scopes `policy:read`, `policy:edit`, `policy:approve`, `policy:activate`, `policy:simulate`; update issuer templates and metadata. | Scopes exposed; integration tests confirm enforcement; offline kit updated. |
| AUTH-POLICY-23-002 | TODO | Authority Core & Security Guild | AUTH-POLICY-23-001 | Implement optional two-person rule for activation: require two distinct `policy:activate` approvals when configured; emit audit logs. | Activation endpoint enforces rule; audit logs contain approver IDs; tests cover 2-person path. |
| AUTH-POLICY-23-003 | TODO | Authority Core & Docs Guild | AUTH-POLICY-23-001 | Update documentation and sample configs for policy roles, approval workflow, and signing requirements. | Docs updated with reviewer checklist; configuration examples validated. |
## Graph & Vuln Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-GRAPH-24-001 | TODO | Authority Core & Security Guild | AUTH-GRAPH-21-001 | Extend scopes to include `vuln:read` and signed permalinks with scoped claims for Graph/Vuln Explorer; update metadata. | Scopes published; permalinks validated; integration tests cover RBAC. |
## Orchestrator Dashboard
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-ORCH-32-001 | TODO | Authority Core & Security Guild | — | Define `orch:read` scope, register `Orch.Viewer` role, update discovery metadata, and seed offline defaults. | Scope/role available in metadata; integration tests confirm read-only enforcement; offline kit updated. |
| AUTH-ORCH-33-001 | TODO | Authority Core & Security Guild | AUTH-ORCH-32-001 | Add `Orch.Operator` role/scopes for control actions, require reason/ticket attributes, and update issuer templates. | Operator tokens issued; action endpoints enforce scope + reason; audit logs capture operator info; docs refreshed. |
| AUTH-ORCH-34-001 | TODO | Authority Core & Security Guild | AUTH-ORCH-33-001 | Introduce `Orch.Admin` role with quota/backfill scopes, enforce audit reason on quota changes, and update offline defaults/docs. | Admin role available; quotas/backfills require scope + reason; tests confirm tenant isolation; documentation updated. |
## StellaOps Console (Sprint 23)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-CONSOLE-23-001 | TODO | Authority Core & Security Guild | AUTH-POLICY-20-001 | Register StellaOps Console confidential client with OIDC PKCE support, short-lived ID/access tokens, `console:*` audience claims, and SPA-friendly refresh (token exchange endpoint). Publish discovery metadata + offline kit defaults. | Client registration committed, configuration templates updated, integration tests validate PKCE + scope issuance, security review recorded. |
| AUTH-CONSOLE-23-002 | TODO | Authority Core & Security Guild | AUTH-CONSOLE-23-001, AUTH-AOC-19-002 | Expose tenant catalog, user profile, and token introspection endpoints required by Console (fresh-auth prompts, scope checks); enforce tenant header requirements and audit logging with correlation IDs. | Endpoints ship with RBAC enforcement, audit logs include tenant+scope, integration tests cover unauthorized/tenant-mismatch scenarios. |
| AUTH-CONSOLE-23-003 | TODO | Authority Core & Docs Guild | AUTH-CONSOLE-23-001, AUTH-CONSOLE-23-002 | Update security docs/config samples for Console flows (PKCE, tenant badge, fresh-auth for admin actions, session inactivity timeouts) with compliance checklist. | Docs merged, config samples validated, release notes updated, ops runbook references new flows. |
## Policy Studio (Sprint 27)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-POLICY-27-001 | TODO | Authority Core & Security Guild | AUTH-POLICY-20-001, AUTH-CONSOLE-23-001 | Define Policy Studio roles (`policy:author`, `policy:review`, `policy:approve`, `policy:operate`, `policy:audit`) with tenant-scoped claims, update issuer metadata, and seed offline kit defaults. | Scopes/roles exposed via discovery docs; tokens issued with correct claims; integration tests cover role combinations; docs updated. |
| AUTH-POLICY-27-002 | TODO | Authority Core & Security Guild | AUTH-POLICY-27-001, REGISTRY-API-27-007 | Provide attestation signing service bindings (OIDC token exchange, cosign integration) and enforce publish/promote scope checks, fresh-auth requirements, and audit logging. | Publish/promote requests require fresh auth + correct scopes; attestations signed with validated identity; audit logs enriched with digest + tenant; integration tests pass. |
| AUTH-POLICY-27-003 | TODO | Authority Core & Docs Guild | AUTH-POLICY-27-001, AUTH-POLICY-27-002 | Update Authority configuration/docs for Policy Studio roles, signing policies, approval workflows, and CLI integration; include compliance checklist. | Docs merged; samples validated; governance checklist appended; release notes updated. |
## Exceptions v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-EXC-25-001 | TODO | Authority Core & Security Guild | AUTH-POLICY-23-001 | Introduce exception scopes (`exceptions:read`, `exceptions:write`, `exceptions:approve`) and approval routing configuration with MFA gating. | Scopes published in metadata; routing matrix validated; integration tests enforce scope + MFA rules. |
| AUTH-EXC-25-002 | TODO | Authority Core & Docs Guild | AUTH-EXC-25-001 | Update documentation/samples for exception roles, routing matrix, MFA requirements, and audit trail references. | Docs merged with compliance checklist; samples verified. |
## Reachability v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-SIG-26-001 | TODO | Authority Core & Security Guild | AUTH-EXC-25-001 | Add `signals:read`, `signals:write`, `signals:admin` scopes, issue `SignalsUploader` role template, and enforce AOC for sensor identities. | Scopes exposed; configuration validated; integration tests ensure RBAC + AOC enforcement. |
## Vulnerability Explorer (Sprint 29)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-VULN-29-001 | TODO | Authority Core & Security Guild | AUTH-POLICY-27-001 | Define Vuln Explorer scopes/roles (`vuln:view`, `vuln:investigate`, `vuln:operate`, `vuln:audit`) with ABAC attributes (env, owner, business_tier) and update discovery metadata/offline kit defaults. | Roles/scopes published; issuer templates updated; integration tests cover ABAC filters; docs refreshed. |
| AUTH-VULN-29-002 | TODO | Authority Core & Security Guild | AUTH-VULN-29-001, LEDGER-29-002 | Enforce CSRF/anti-forgery tokens for workflow actions, sign attachment tokens, and record audit logs with ledger event hashes. | Workflow calls require valid tokens; audit logs include ledger references; security tests cover token expiry/abuse. |
| AUTH-VULN-29-003 | TODO | Authority Core & Docs Guild | AUTH-VULN-29-001..002 | Update security docs/config samples for Vuln Explorer roles, ABAC policies, attachment signing, and ledger verification guidance. | Docs merged with compliance checklist; configuration examples validated; release notes updated. |
## Advisory AI (Sprint 31)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-AIAI-31-001 | TODO | Authority Core & Security Guild | AUTH-VULN-29-001 | Define Advisory AI scopes (`advisory-ai:view`, `advisory-ai:operate`, `advisory-ai:admin`) and remote inference toggles; update discovery metadata/offline defaults. | Scopes/flags published; integration tests cover RBAC + opt-in settings; docs updated. |
| AUTH-AIAI-31-002 | TODO | Authority Core & Security Guild | AUTH-AIAI-31-001, AIAI-31-006 | Enforce anonymized prompt logging, tenant consent for remote inference, and audit logging of assistant tasks. | Logging/audit flows verified; privacy review passed; docs updated. |
## Export Center
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-EXPORT-35-001 | TODO | Authority Core & Security Guild | AUTH-AOC-19-001 | Introduce `Export.Viewer`, `Export.Operator`, `Export.Admin` scopes, configure issuer templates, and update discovery metadata/offline defaults. | Scopes available; metadata updated; tests ensure enforcement; offline kit defaults refreshed. |
| AUTH-EXPORT-37-001 | TODO | Authority Core & Security Guild | AUTH-EXPORT-35-001, WEB-EXPORT-37-001 | Enforce admin-only access for scheduling, retention, encryption key references, and verify endpoints with audit reason capture. | Admin scope required; audit logs include reason/ticket; integration tests cover denial cases; docs updated. |
## Notifications Studio
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-NOTIFY-38-001 | TODO | Authority Core & Security Guild | — | Define `Notify.Viewer`, `Notify.Operator`, `Notify.Admin` scopes/roles, update discovery metadata, offline defaults, and issuer templates. | Scopes available; metadata updated; tests ensure enforcement; offline kit defaults refreshed. |
| AUTH-NOTIFY-40-001 | TODO | Authority Core & Security Guild | AUTH-NOTIFY-38-001, WEB-NOTIFY-40-001 | Implement signed ack token key rotation, webhook allowlists, admin-only escalation settings, and audit logging of ack actions. | Ack tokens signed/rotated; webhook allowlists enforced; admin enforcement validated; audit logs capture ack/resolution. |
## CLI Parity & Task Packs
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-PACKS-41-001 | TODO | Authority Core & Security Guild | AUTH-AOC-19-001 | Define CLI SSO profiles and pack scopes (`Packs.Read`, `Packs.Write`, `Packs.Run`, `Packs.Approve`), update discovery metadata, offline defaults, and issuer templates. | Scopes available; metadata updated; tests ensure enforcement; offline kit templates refreshed. |
| AUTH-PACKS-43-001 | TODO | Authority Core & Security Guild | AUTH-PACKS-41-001, ORCH-SVC-42-101 | Enforce pack signing policies, approval RBAC checks, CLI CI token scopes, and audit logging for approvals. | Signing policies enforced; approvals require correct roles; CI token scope tests pass; audit logs recorded. |
## Authority-Backed Scopes & Tenancy (Epic 14)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-TEN-47-001 | TODO | Authority Core & Security Guild | AUTH-AOC-19-001 | Align Authority with OIDC/JWT claims (tenants, projects, scopes), implement JWKS caching/rotation, publish scope grammar, and enforce required claims on tokens. | Tokens include tenant/project claims; JWKS cache validated; docs updated; imposed rule noted. |
| AUTH-TEN-49-001 | TODO | Authority Core & Security Guild | AUTH-TEN-47-001 | Implement service accounts & delegation tokens (`act` chain), per-tenant quotas, audit stream of auth decisions, and revocation APIs. | Service tokens minted with scopes/TTL; delegation logged; quotas configurable; audit stream live; docs updated. |
## Observability & Forensics (Epic 15)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-OBS-50-001 | TODO | Authority Core & Security Guild | AUTH-AOC-19-001 | Introduce scopes `obs:read`, `timeline:read`, `timeline:write`, `evidence:create`, `evidence:read`, `evidence:hold`, `attest:read`, and `obs:incident` (all tenant-scoped). Update discovery metadata, offline defaults, and scope grammar docs. | Scopes exposed via metadata; issuer templates updated; offline kit seeded; integration tests cover new scopes. |
| AUTH-OBS-52-001 | TODO | Authority Core & Security Guild | AUTH-OBS-50-001, TIMELINE-OBS-52-003, EVID-OBS-53-003 | Configure resource server policies for Timeline Indexer, Evidence Locker, Exporter, and Observability APIs enforcing new scopes + tenant claims. Emit audit events including scope usage and trace IDs. | Policies deployed; unauthorized access blocked; audit logs prove scope usage; contract tests updated. |
| AUTH-OBS-55-001 | TODO | Authority Core & Security Guild, Ops Guild | AUTH-OBS-50-001, WEB-OBS-55-001 | Harden incident mode authorization: require `obs:incident` scope + fresh auth, log activation reason, and expose verification endpoint for auditors. Update docs/runbooks. | Incident activate/deactivate requires scope; audit entries logged; docs updated with imposed rule reminder. |
## Air-Gapped Mode (Epic 16)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-AIRGAP-56-001 | TODO | Authority Core & Security Guild | AIRGAP-CTL-56-001 | Provision new scopes (`airgap:seal`, `airgap:import`, `airgap:status:read`) in configuration metadata, offline kit defaults, and issuer templates. | Scopes exposed in discovery docs; offline kit updated; integration tests cover issuance. |
| AUTH-AIRGAP-56-002 | TODO | Authority Core & Security Guild | AUTH-AIRGAP-56-001, AIRGAP-IMP-58-001 | Audit import actions with actor, tenant, bundle ID, and trace ID; expose `/authority/audit/airgap` endpoint. | Audit records persisted; endpoint paginates results; tests cover RBAC + filtering. |
| AUTH-AIRGAP-57-001 | TODO | Authority Core & Security Guild, DevOps Guild | AUTH-AIRGAP-56-001, DEVOPS-AIRGAP-57-002 | Enforce sealed-mode CI gating by refusing token issuance when declared sealed install lacks sealing confirmation. | CI scenario validated; error surfaces remediation; docs updated. |
## SDKs & OpenAPI (Epic 17)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| AUTH-OAS-61-001 | TODO | Authority Core & Security Guild, API Contracts Guild | OAS-61-001 | Document Authority authentication/token endpoints in OAS with scopes, examples, and error envelopes. | Spec complete with security schemes; lint passes. |
| AUTH-OAS-61-002 | TODO | Authority Core & Security Guild | AUTH-OAS-61-001 | Implement `/.well-known/openapi` with scope metadata, supported grant types, and build version. | Endpoint deployed; contract tests cover discovery. |
| AUTH-OAS-62-001 | TODO | Authority Core & Security Guild, SDK Generator Guild | AUTH-OAS-61-001, SDKGEN-63-001 | Provide SDK helpers for OAuth2/PAT flows, tenancy override header; add integration tests. | SDKs expose auth helpers; tests cover token issuance; docs updated. |
| AUTH-OAS-63-001 | TODO | Authority Core & Security Guild, API Governance Guild | APIGOV-63-001 | Emit deprecation headers and notifications for legacy auth endpoints. | Headers emitted; notifications verified; migration guide published. |

12
bench/Scanner.Analyzers/README.md → src/StellaOps.Bench/Scanner.Analyzers/README.md
View File

@@ -1,7 +1,7 @@
# Scanner Analyzer Microbench Harness
The bench harness exercises the language analyzers against representative filesystem layouts so that regressions are caught before they ship.
# Scanner Analyzer Microbench Harness
The bench harness exercises the language analyzers against representative filesystem layouts so that regressions are caught before they ship.
## Layout
- `StellaOps.Bench.ScannerAnalyzers/` – .NET 10 console harness that executes the real language analyzers (and fallback metadata walks for ecosystems that are still underway).
- `config.json` – Declarative list of scenarios the harness executes. Each scenario points at a directory in `samples/`.
@@ -16,10 +16,10 @@ The bench harness exercises the language analyzers against representative filesy
```bash
dotnet run \
--project bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj \
--project src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj \
-- \
--repo-root . \
--out bench/Scanner.Analyzers/baseline.csv \
--out src/StellaOps.Bench/Scanner.Analyzers/baseline.csv \
--json out/bench/scanner-analyzers/latest.json \
--prom out/bench/scanner-analyzers/latest.prom \
--commit "$(git rev-parse HEAD)"

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers.Tests/BaselineLoaderTests.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers.Tests/BaselineLoaderTests.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers.Tests/BenchmarkJsonWriterTests.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers.Tests/BenchmarkJsonWriterTests.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers.Tests/BenchmarkScenarioReportTests.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers.Tests/BenchmarkScenarioReportTests.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers.Tests/PrometheusWriterTests.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers.Tests/PrometheusWriterTests.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers.Tests/StellaOps.Bench.ScannerAnalyzers.Tests.csproj → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers.Tests/StellaOps.Bench.ScannerAnalyzers.Tests.csproj
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Baseline/BaselineEntry.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Baseline/BaselineEntry.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Baseline/BaselineLoader.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Baseline/BaselineLoader.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/BenchmarkConfig.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/BenchmarkConfig.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Program.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Program.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Reporting/BenchmarkJsonWriter.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Reporting/BenchmarkJsonWriter.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Reporting/BenchmarkScenarioReport.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Reporting/BenchmarkScenarioReport.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Reporting/PrometheusWriter.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Reporting/PrometheusWriter.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/ScenarioResult.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/ScenarioResult.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/ScenarioRunners.cs → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/ScenarioRunners.cs
View File

0
bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj → src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj
View File

14
bench/Scanner.Analyzers/baseline.csv → src/StellaOps.Bench/Scanner.Analyzers/baseline.csv
View File

@@ -1,7 +1,7 @@
scenario,iterations,sample_count,mean_ms,p95_ms,max_ms
node_monorepo_walk,5,4,6.0975,21.7421,26.8537
java_demo_archive,5,1,6.2007,23.4837,29.1143
go_buildinfo_fixture,5,2,6.1949,22.6851,27.9196
dotnet_multirid_fixture,5,2,11.4884,37.7460,46.4850
python_site_packages_scan,5,3,5.6420,18.2943,22.3739
python_pip_cache_fixture,5,1,5.8598,13.2855,15.6256
scenario,iterations,sample_count,mean_ms,p95_ms,max_ms
node_monorepo_walk,5,4,6.0975,21.7421,26.8537
java_demo_archive,5,1,6.2007,23.4837,29.1143
go_buildinfo_fixture,5,2,6.1949,22.6851,27.9196
dotnet_multirid_fixture,5,2,11.4884,37.7460,46.4850
python_site_packages_scan,5,3,5.6420,18.2943,22.3739
python_pip_cache_fixture,5,1,5.8598,13.2855,15.6256
1 scenario iterations sample_count mean_ms p95_ms max_ms
2 node_monorepo_walk 5 4 6.0975 21.7421 26.8537
3 java_demo_archive 5 1 6.2007 23.4837 29.1143
4 go_buildinfo_fixture 5 2 6.1949 22.6851 27.9196
5 dotnet_multirid_fixture 5 2 11.4884 37.7460 46.4850
6 python_site_packages_scan 5 3 5.6420 18.2943 22.3739
7 python_pip_cache_fixture 5 1 5.8598 13.2855 15.6256

8
bench/Scanner.Analyzers/config.json → src/StellaOps.Bench/Scanner.Analyzers/config.json
View File

@@ -1,7 +1,7 @@
{
"thresholdMs": 5000,
"iterations": 5,
"scenarios": [
{
"thresholdMs": 5000,
"iterations": 5,
"scenarios": [
{
"id": "node_monorepo_walk",
"label": "Node.js analyzer on monorepo fixture",

14
bench/Scanner.Analyzers/lang/README.md → src/StellaOps.Bench/Scanner.Analyzers/lang/README.md
View File

@@ -1,8 +1,8 @@
# Scanner Language Analyzer Benchmarks
This directory will capture benchmark results for language analyzers (Node, Python, Go, .NET, Rust).
Pending tasks:
# Scanner Language Analyzer Benchmarks
This directory will capture benchmark results for language analyzers (Node, Python, Go, .NET, Rust).
Pending tasks:
- LA1: Node analyzer microbench CSV + flamegraph.
- LA2: Python hash throughput CSV.
- LA3: Go build info extraction benchmarks.
@@ -16,13 +16,13 @@ Results should be committed as deterministic CSV/JSON outputs with accompanying
- Scenario `go_buildinfo_fixture` captures our Go analyzer running against the basic build-info fixture. The Oct 23 baseline (`baseline.csv`) shows a mean duration of **35.03 ms** (p95 136.55 ms, max 170.16 ms) over 5 iterations on the current rig; earlier Oct 21 measurement recorded **4.02 ms** mean when the analyzer was profiled on the warm perf runner.
- Comparative run against Syft v1.29.1 on the same fixture (captured 2025-10-21) reported a mean of **5.18 ms** (p95 18.64 ms, max 23.51 ms); raw measurements live in `go/syft-comparison-20251021.csv`.
- Bench command (from repo root):\
`dotnet run --project bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj -- --config bench/Scanner.Analyzers/config.json --out bench/Scanner.Analyzers/baseline.csv`
`dotnet run --project src/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/StellaOps.Bench.ScannerAnalyzers.csproj -- --config src/StellaOps.Bench/Scanner.Analyzers/config.json --out src/StellaOps.Bench/Scanner.Analyzers/baseline.csv`
## Sprint LA4 — .NET Analyzer Benchmark Notes (2025-10-23)
- Scenario `dotnet_multirid_fixture` exercises the .NET analyzer against the multi-RID test fixture that merges two applications and four runtime identifiers. Latest baseline run (Release build, 5 iterations) records a mean duration of **29.19 ms** (p95 106.62 ms, max 132.30 ms) with a stable component count of 2.
- Syft v1.29.1 scanning the same fixture (`syft scan dir:…`) averaged **1 546 ms** (p95 ≈2 100 ms, max ≈2 100 ms) while also reporting duplicate packages; raw numbers captured in `dotnet/syft-comparison-20251023.csv`.
- The new scenario is declared in `bench/Scanner.Analyzers/config.json`; rerun the bench command above after rebuilding analyzers to refresh baselines and comparison data.
- The new scenario is declared in `src/StellaOps.Bench/Scanner.Analyzers/config.json`; rerun the bench command above after rebuilding analyzers to refresh baselines and comparison data.
## Sprint LA2 — Python Analyzer Benchmark Notes (2025-10-23)

0
bench/Scanner.Analyzers/lang/dotnet/syft-comparison-20251023.csv → src/StellaOps.Bench/Scanner.Analyzers/lang/dotnet/syft-comparison-20251023.csv
View File

0
bench/Scanner.Analyzers/lang/go/syft-comparison-20251021.csv → src/StellaOps.Bench/Scanner.Analyzers/lang/go/syft-comparison-20251021.csv
View File

0
bench/Scanner.Analyzers/lang/python/hash-throughput-20251023.csv → src/StellaOps.Bench/Scanner.Analyzers/lang/python/hash-throughput-20251023.csv
View File

43
src/StellaOps.Bench/TASKS.md Normal file
View File

@@ -0,0 +1,43 @@
# Benchmarks Task Board
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| BENCH-SCANNER-10-001 | DONE | Bench Guild, Scanner Team | SCANNER-ANALYZERS-LANG-10-303 | Analyzer microbench harness (node_modules, site-packages) + baseline CSV. | Harness committed under `src/StellaOps.Bench/Scanner.Analyzers`; baseline CSV recorded; CI job publishes results. |
| BENCH-SCANNER-10-002 | DONE (2025-10-21) | Bench Guild, Language Analyzer Guild | SCANNER-ANALYZERS-LANG-10-301..309 | Wire real language analyzers into bench harness & refresh baselines post-implementation. | Harness executes analyzer assemblies end-to-end; updated baseline committed; CI trend doc linked. |
| BENCH-IMPACT-16-001 | TODO | Bench Guild, Scheduler Team | SCHED-IMPACT-16-301 | ImpactIndex throughput bench (resolve 10k productKeys) + RAM profile. | Benchmark script ready; baseline metrics recorded; alert thresholds defined. |
| BENCH-NOTIFY-15-001 | TODO | Bench Guild, Notify Team | NOTIFY-ENGINE-15-301 | Notify dispatch throughput bench (vary rule density) with results CSV. | Bench executed; results stored; regression alert configured. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| BENCH-POLICY-20-001 | TODO | Bench Guild, Policy Guild | POLICY-ENGINE-20-002, POLICY-ENGINE-20-006 | Build policy evaluation benchmark suite (100k components, 1M advisories) capturing latency, throughput, memory. | Bench harness committed; baseline metrics recorded; ties into CI dashboards. |
| BENCH-POLICY-20-002 | TODO | Bench Guild, Policy Guild, Scheduler Guild | BENCH-POLICY-20-001, SCHED-WORKER-20-302 | Add incremental run benchmark measuring delta evaluation vs full; capture SLA compliance. | Incremental bench executed; results stored; regression alerts configured. |
## Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| BENCH-GRAPH-21-001 | TODO | Bench Guild, Cartographer Guild | CARTO-GRAPH-21-004, CARTO-GRAPH-21-006 | Build graph viewport/path benchmark harness simulating 50k/100k nodes; record latency, memory, tile cache hit rates. | Harness committed; baseline metrics logged; integrates with perf dashboards. |
| BENCH-GRAPH-21-002 | TODO | Bench Guild, UI Guild | BENCH-GRAPH-21-001, UI-GRAPH-21-001 | Add headless UI load benchmark (Playwright) for graph canvas interactions to track render times and FPS budgets. | Benchmark runs in CI; results exported; alert thresholds defined. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| BENCH-LNM-22-001 | TODO | Bench Guild, Concelier Guild | CONCELIER-LNM-21-002 | Create ingest benchmark simulating 500 advisory observations/sec, measuring correlator latency and Mongo throughput; publish baseline metrics. | Harness added; baseline stored; alerts wired for SLA breach. |
| BENCH-LNM-22-002 | TODO | Bench Guild, Excititor Guild | EXCITITOR-LNM-21-002 | Build VEX ingestion/correlation perf test focusing on alias/product matching and event emission latency. | Benchmark executed; metrics captured; CI integration established. |
## Graph & Vuln Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| BENCH-GRAPH-24-001 | TODO | Bench Guild, SBOM Service Guild | SBOM-GRAPH-24-002 | Develop SBOM graph performance benchmark measuring build time, memory, and cache warm latency for 40k-node assets. | Benchmark runs in CI; baseline metrics recorded; alerts configured. |
| BENCH-GRAPH-24-002 | TODO | Bench Guild, UI Guild | UI-GRAPH-24-001..005 | Implement UI interaction benchmarks (filter/zoom/table operations) citing p95 latency; integrate with perf dashboards. | UI perf metrics collected; thresholds enforced; documentation updated. |
## Reachability v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| BENCH-SIG-26-001 | TODO | Bench Guild, Signals Guild | SIGNALS-24-004 | Develop benchmark for reachability scoring pipeline (facts/sec, latency, memory) using synthetic callgraphs/runtime batches. | Benchmark runs in CI; baseline metrics recorded; alerts configured. |
| BENCH-SIG-26-002 | TODO | Bench Guild, Policy Guild | POLICY-ENGINE-80-001 | Measure policy evaluation overhead with reachability cache hot/cold; ensure ≤8 ms p95 added latency. | Benchmark integrated; results tracked in dashboards; regression alerts set. |

17
src/StellaOps.Cartographer/AGENTS.md Normal file
View File

@@ -0,0 +1,17 @@
# StellaOps.Cartographer — Agent Charter
## Mission
Build and operate the Cartographer service that materializes immutable SBOM property graphs, precomputes layout tiles, and hydrates policy/VEX overlays so other services (API, UI, CLI) can navigate and reason about dependency relationships with context.
## Responsibilities
- Ingest normalized SBOM projections (CycloneDX/SPDX) and generate versioned graph snapshots with tenant-aware storage.
- Maintain overlay workers that merge Policy Engine effective findings and VEX metadata onto graph nodes/edges, including path relevance computation.
- Serve graph APIs for viewport tiles, paths, filters, exports, simulation overlays, and diffing.
- Coordinate with Policy Engine, Scheduler, Conseiller, Excitator, and Authority to keep overlays current, respect RBAC, and uphold determinism guarantees.
- Deliver observability (metrics/traces/logs) and performance benchmarks for large graphs (≥50k nodes).
## Expectations
- Keep builds deterministic; snapshots are write-once and content-addressed.
- Tenancy and scope enforcement must match Authority policies (`graph:*`, `sbom:read`, `findings:read`).
- Update `TASKS.md`, `SPRINTS.md` when status changes.
- Provide fixtures and documentation so UI/CLI teams can simulate graphs offline.

17
src/StellaOps.Cartographer/Program.cs Normal file
View File

@@ -0,0 +1,17 @@
var builder = WebApplication.CreateBuilder(args);
builder.Configuration
.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
.AddEnvironmentVariables("CARTOGRAPHER_");
builder.Services.AddOptions();
builder.Services.AddLogging();
// TODO: register Cartographer graph builders, overlay workers, and Authority client once implementations land.
var app = builder.Build();
app.MapGet("/healthz", () => Results.Ok(new { status = "ok" }));
app.MapGet("/readyz", () => Results.Ok(new { status = "warming" }));
app.Run();

16
src/StellaOps.Cartographer/StellaOps.Cartographer.csproj Normal file
View File

@@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<LangVersion>preview</LangVersion>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<AspNetCoreHostingModel>InProcess</AspNetCoreHostingModel>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Configuration\StellaOps.Configuration.csproj" />
<ProjectReference Include="..\StellaOps.DependencyInjection\StellaOps.DependencyInjection.csproj" />
<ProjectReference Include="..\StellaOps.Policy.Engine\StellaOps.Policy.Engine.csproj" />
</ItemGroup>
</Project>

12
src/StellaOps.Cartographer/TASKS.md Normal file
View File

@@ -0,0 +1,12 @@
# Cartographer Task Board — Epic 3: Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CARTO-GRAPH-21-001 | TODO | Cartographer Guild | SBOM-SERVICE-21-001 | Define graph storage schema: snapshot catalog, node/edge collections, overlay tables, sharding strategy, indexes, and retention rules. Document migration/rollback. | Mongo collections + indexes created via bootstrapper; schema doc drafted; integration tests verify tenant isolation and sharding. |
| CARTO-GRAPH-21-002 | TODO | Cartographer Guild | CARTO-GRAPH-21-001 | Implement SBOM projection reader consuming normalized CycloneDX/SPDX from SBOM Service; validate fixtures and handle entrypoint tagging. | Reader ingests sample SBOMs (CycloneDX/SPDX); entrypoints resolved; unit tests cover malformed SBOM handling. |
| CARTO-GRAPH-21-003 | TODO | Cartographer Guild | CARTO-GRAPH-21-002 | Build graph constructor deduping PURLs, emitting typed nodes/edges with metadata (licenses, supplier, hashes, scope). | Graph snapshots generated deterministically for fixtures; duplicate suppression verified; stats recorded. |
| CARTO-GRAPH-21-004 | TODO | Cartographer Guild | CARTO-GRAPH-21-003 | Implement layout & tiling pipeline (global force layout + zoom tiles) and persist tiles to blob/object storage with manifests. | Layout tiles produced for fixtures; bbox metadata recorded; perf target (<10 min for 50k nodes) documented. |
| CARTO-GRAPH-21-005 | TODO | Cartographer Guild | CARTO-GRAPH-21-003, POLICY-ENGINE-30-001 | Overlay worker hydrating policy effective findings/VEX data, computing path relevance from entrypoints, and supporting simulation overlays. | Overlay documents created for selected policies; path relevance validated via unit tests; SLA metric recorded. |
| CARTO-GRAPH-21-006 | TODO | Cartographer Guild, BE-Base Platform Guild | CARTO-GRAPH-21-005 | Expose Cartographer APIs (versions, viewport tiles, node lookup, path queries, filters, diff, export, simulate) with pagination and streaming support. | API endpoints documented, tested, and OpenAPI published; large responses stream without timeouts; error codes mapped. |
| CARTO-GRAPH-21-007 | TODO | Cartographer Guild, Scheduler Guild | CARTO-GRAPH-21-005, SCHED-WORKER-21-201 | Build backfill + incremental overlay jobs listening to policy/SBOM change events; ensure eventual consistency SLA (<2 min). | Change stream consumers enqueue work; retries/backoff implemented; observability metrics emit overlay lag. |
| CARTO-GRAPH-21-008 | TODO | Cartographer Guild, QA Guild | CARTO-GRAPH-21-004..007 | Add unit/property/integration tests, synthetic 50k/100k perf suites, chaos scenarios (missing overlays, cyclic graphs), and determinism checks. | Test suite green; perf benchmarks logged; regression pipeline enforces determinism. |
| CARTO-GRAPH-21-009 | TODO | Cartographer Guild, DevOps Guild | CARTO-GRAPH-21-006 | Provide deployment artefacts (Helm/Compose), configuration docs, and Offline Kit bundle notes for Cartographer. | Deployment descriptors merged; docs updated; offline kit includes seeds/layout caches; smoke tests added. |

9
src/StellaOps.Cli/AGENTS.md
View File

@@ -23,5 +23,10 @@
## Reference Materials
- `docs/ARCHITECTURE_CONCELIER.md` for database operations surface area.
- Backend OpenAPI/contract docs (once available) for job triggers and scanner endpoints.
- Existing module AGENTS/TASKS files for style and coordination cues.
- `docs/09_API_CLI_REFERENCE.md` (section 3) for the user-facing synopsis of the CLI verbs and flags.
- Existing module AGENTS/TASKS files for style and coordination cues.
- `docs/09_API_CLI_REFERENCE.md` (section 3) for the user-facing synopsis of the CLI verbs and flags.
### Attestor Command Guild
- Owns the `stella attest` verb family (sign, verify, list, fetch) plus key lifecycle helpers (create, import, rotate, revoke).
- Ensures all attestation flows use the official SDK transport, support offline bundles, and surface JSON/table outputs for automation.
- Guards parity with attestor service policies (verification policies, explainability) and keeps fixtures/tests covering file-based and KMS-backed keys.

206
src/StellaOps.Cli/TASKS.md
View File

@@ -1,24 +1,182 @@
If you are working on this file you need to read docs/ARCHITECTURE_EXCITITOR.md and ./AGENTS.md).
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Bootstrap configuration fallback (env → appsettings{{.json/.yaml}})|DevEx/CLI|Core|**DONE** – CLI loads `API_KEY`/`STELLAOPS_BACKEND_URL` from environment or local settings, defaulting to empty strings when unset.|
|Introduce command host & routing skeleton|DevEx/CLI|Configuration|**DONE** – System.CommandLine (v2.0.0-beta5) router stitched with `scanner`, `scan`, `db`, and `config` verbs.|
|Scanner artifact download/install commands|Ops Integrator|Backend contracts|**DONE** – `scanner download` caches bundles, validates SHA-256 (plus optional RSA signature), installs via `docker load`, persists metadata, and retries with exponential backoff.|
|Scan execution & result upload workflow|Ops Integrator, QA|Scanner cmd|**DONE** – `scan run` drives container scans against directories, emits artefacts in `ResultsDirectory`, auto-uploads on success, and `scan upload` covers manual retries.|
|Concelier DB operations passthrough|DevEx/CLI|Backend, Concelier APIs|**DONE** – `db fetch|merge|export` trigger `/jobs/*` endpoints with parameter binding and consistent exit codes.|
|CLI observability & tests|QA|Command host|**DONE** – Added console logging defaults & configuration bootstrap tests; future metrics hooks tracked separately.|
|Authority auth commands|DevEx/CLI|Auth libraries|**DONE** – `auth login/logout/status` wrap the shared auth client, manage token cache, and surface status messages.|
|Document authority workflow in CLI help & quickstart|Docs/CLI|Authority auth commands|**DONE (2025-10-10)** – CLI help now surfaces Authority config fields and docs/09 + docs/10 describe env vars, auth login/status flow, and cache location.|
|Authority whoami command|DevEx/CLI|Authority auth commands|**DONE (2025-10-10)** – Added `auth whoami` verb that displays subject/audience/expiry from cached tokens and handles opaque tokens gracefully.|
|Expose auth client resilience settings|DevEx/CLI|Auth libraries LIB5|**DONE (2025-10-10)** – CLI options now bind resilience knobs, `AddStellaOpsAuthClient` honours them, and tests cover env overrides.|
|Document advanced Authority tuning|Docs/CLI|Expose auth client resilience settings|**DONE (2025-10-10)** – docs/09 and docs/10 describe retry/offline settings with env examples and point to the integration guide.|
|Surface password policy diagnostics in CLI output|DevEx/CLI, Security Guild|AUTHSEC-CRYPTO-02-004|**DONE (2025-10-15)** – CLI startup runs the Authority plug-in analyzer, logs weakened password policy warnings with manifest paths, added unit tests (`dotnet test src/StellaOps.Cli.Tests`) and updated docs/09 with remediation guidance.|
|EXCITITOR-CLI-01-001 – Add `excititor` command group|DevEx/CLI|EXCITITOR-WEB-01-001|DONE (2025-10-18) – Introduced `excititor` verbs (init/pull/resume/list-providers/export/verify/reconcile) with token-auth backend calls, provenance-friendly logging, and regression coverage.|
|EXCITITOR-CLI-01-002 – Export download & attestation UX|DevEx/CLI|EXCITITOR-CLI-01-001, EXCITITOR-EXPORT-01-001|DONE (2025-10-19) – CLI export prints digest/size/Rekor metadata, `--output` downloads with SHA-256 verification + cache reuse, and unit coverage validated via `dotnet test src/StellaOps.Cli.Tests`.|
|EXCITITOR-CLI-01-003 – CLI docs & examples for Excititor|Docs/CLI|EXCITITOR-CLI-01-001|**DOING (2025-10-19)** – Update docs/09_API_CLI_REFERENCE.md and quickstart snippets to cover Excititor verbs, offline guidance, and attestation verification workflow.|
|CLI-RUNTIME-13-005 – Runtime policy test verbs|DevEx/CLI|SCANNER-RUNTIME-12-302, ZASTAVA-WEBHOOK-12-102|**DONE (2025-10-19)** – Added `runtime policy test` command (stdin/file support, JSON output), backend client method + typed models, verdict table output, docs/tests updated (`dotnet test src/StellaOps.Cli.Tests`).|
|CLI-OFFLINE-13-006 – Offline kit workflows|DevEx/CLI|DEVOPS-OFFLINE-14-002|**DONE (2025-10-21)** – Added `offline kit pull/import/status` commands with resumable downloads, digest/metadata validation, metrics, docs updates, and regression coverage (`dotnet test src/StellaOps.Cli.Tests`).|
|CLI-PLUGIN-13-007 – Plugin packaging|DevEx/CLI|CLI-RUNTIME-13-005, CLI-OFFLINE-13-006|DONE (2025-10-22) – Packaged non-core verbs as restart-time plug-ins with manifest + loader updates and tests ensuring no hot reload.|
|CLI-RUNTIME-13-008 – Runtime policy contract sync|DevEx/CLI, Scanner WebService Guild|SCANNER-RUNTIME-12-302|**DONE (2025-10-19)** – CLI runtime table/JSON now align with SCANNER-RUNTIME-12-302 (SBOM referrers, quieted provenance, confidence, verified Rekor); docs/09 updated with joint sign-off note.|
|CLI-RUNTIME-13-009 – Runtime policy smoke fixture|DevEx/CLI, QA Guild|CLI-RUNTIME-13-005|**DONE (2025-10-19)** – Spectre console harness + regression tests cover table and `--json` output paths for `runtime policy test`, using stubbed backend and integrated into `dotnet test` suite.|
# CLI Task Board — Epic 1: Aggregation-Only Contract
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-AOC-19-001 | TODO | DevEx/CLI Guild | CONCELIER-WEB-AOC-19-001, EXCITITOR-WEB-AOC-19-001 | Implement `stella sources ingest --dry-run` printing would-write payloads with forbidden field scan results and guard status. | Command displays diff-safe JSON, highlights forbidden fields, exits non-zero on guard violation, and has unit tests. |
| CLI-AOC-19-002 | TODO | DevEx/CLI Guild | CLI-AOC-19-001 | Add `stella aoc verify` command supporting `--since`/`--limit`, mapping `ERR_AOC_00x` to exit codes, with JSON/table output. | Command integrates with both services, exit codes documented, regression tests green. |
| CLI-AOC-19-003 | TODO | Docs/CLI Guild | CLI-AOC-19-001, CLI-AOC-19-002 | Update CLI reference and quickstart docs to cover new commands, exit codes, and offline verification workflows. | Docs updated; examples recorded; release notes mention new commands. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-POLICY-20-001 | TODO | DevEx/CLI Guild | WEB-POLICY-20-001 | Add `stella policy new|edit|submit|approve` commands with local editor integration, version pinning, and approval workflow wiring. | Commands round-trip policy drafts with temp files; approval requires correct scopes; unit tests cover happy/error paths. |
| CLI-POLICY-20-002 | TODO | DevEx/CLI Guild | CLI-POLICY-20-001, WEB-POLICY-20-001, WEB-POLICY-20-002 | Implement `stella policy simulate` with SBOM/env arguments and diff output (table/JSON), handling exit codes for `ERR_POL_*`. | Simulation outputs deterministic diffs; JSON schema documented; tests validate exit codes + piping of env variables. |
| CLI-POLICY-20-003 | TODO | DevEx/CLI Guild, Docs Guild | CLI-POLICY-20-002, WEB-POLICY-20-003, DOCS-POLICY-20-006 | Extend `stella findings ls|get` commands for policy-filtered retrieval with pagination, severity filters, and explain output. | Commands stream paginated results; explain view renders rationale entries; docs/help updated; end-to-end tests cover filters. |
## Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-GRAPH-21-001 | TODO | DevEx/CLI Guild, Cartographer Guild | CARTO-GRAPH-21-006, WEB-GRAPH-21-001 | Implement `stella sbom graph build/export/query/diff` commands with RBAC-aware behavior, streaming outputs, and offline mode. | Commands execute against local fixtures; GraphML/JSONL exports match API; scope errors mapped to `ERR_Graph_*`; unit tests added. |
| CLI-GRAPH-21-002 | TODO | DevEx/CLI Guild | CLI-GRAPH-21-001, WEB-GRAPH-21-004 | Add path query and simulation options (k-shortest paths, policy selection, filters) with JSON output suitable for CI. | Path query returns expected JSON schema; simulation overlay toggles; regression tests cover CLI-to-API contract. |
| CLI-GRAPH-21-003 | TODO | DevEx/CLI Guild, Docs Guild | CLI-GRAPH-21-001, DOCS-GRAPH-21-005 | Document CLI usage and provide golden fixtures for CI; ensure exit codes align with `ERR_Graph_*`. | Docs updated; fixtures stored under `samples/graph/`; CI job runs CLI smoke; exit codes verified. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-LNM-22-001 | TODO | DevEx/CLI Guild | WEB-LNM-21-001 | Implement `stella advisory obs get/linkset show/export` commands with JSON/OSV output, pagination, and conflict display; ensure `ERR_AGG_*` mapping. | Commands fetch observation/linkset data; exports validated against fixtures; unit tests cover error handling. |
| CLI-LNM-22-002 | TODO | DevEx/CLI Guild | WEB-LNM-21-002 | Implement `stella vex obs get/linkset show` commands with product filters, status filters, and JSON output for CI usage. | Commands support filters + streaming; integration tests use sample linksets; docs updated. |
## Policy Engine + Editor v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-POLICY-23-004 | TODO | DevEx/CLI Guild | WEB-POLICY-23-001 | Add `stella policy lint` command validating SPL files with compiler diagnostics; support JSON output. | Command returns lint diagnostics; exit codes documented; tests cover error scenarios. |
| CLI-POLICY-23-005 | TODO | DevEx/CLI Guild | WEB-POLICY-23-002 | Implement `stella policy activate` with scheduling window, approval enforcement, and summary output. | Activation command integrates with API, handles 2-person rule failures; tests cover success/error. |
| CLI-POLICY-23-006 | TODO | DevEx/CLI Guild | WEB-POLICY-23-004 | Provide `stella policy history` and `stella policy explain` commands to pull run history and explanation trees. | Commands output JSON/table; integration tests with fixtures; docs updated. |
## Graph & Vuln Explorer v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-GRAPH-24-001 | TODO | DevEx/CLI Guild | WEB-GRAPH-24-001 | Add `stella graph show|search|diff` commands with JSON/table outputs, pagination, depth controls, and RBAC-aware error handling. | Commands return graph data; diff output validated with fixtures; unit tests cover pagination/error cases. |
| CLI-GRAPH-24-002 | TODO | DevEx/CLI Guild | WEB-GRAPH-24-002 | Implement `stella graph simulate upgrade|policy` commands including progress feedback, diff summary, and exit codes for failure/timeouts. | Simulation commands integrate with API; regression tests exercise upgrade/policy scenarios; docs updated. |
## Exceptions v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-EXC-25-001 | TODO | DevEx/CLI Guild | WEB-EXC-25-001 | Implement `stella exceptions list|draft|propose|approve|revoke` commands with JSON/table output, validation, and workflow exit codes. | Commands exercise end-to-end workflow; unit/integration tests cover errors; docs updated. |
| CLI-EXC-25-002 | TODO | DevEx/CLI Guild | WEB-EXC-25-002 | Extend `stella policy simulate` with `--with-exception`/`--without-exception` flags to preview exception impact. | Simulation handles overrides; regression tests cover presence/absence; help text updated. |
## Reachability v1
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-SIG-26-001 | TODO | DevEx/CLI Guild | WEB-SIG-26-001 | Implement `stella reachability upload-callgraph` and `stella reachability list/explain` commands with streaming upload, pagination, and exit codes. | Commands operate end-to-end; integration tests with fixtures; docs updated. |
| CLI-SIG-26-002 | TODO | DevEx/CLI Guild | WEB-SIG-26-003 | Extend `stella policy simulate` with reachability override flags (`--reachability-state`, `--reachability-score`). | Simulation command accepts overrides; regression tests cover adjustments; help text updated. |
## Policy Studio (Sprint 27)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-POLICY-27-001 | TODO | DevEx/CLI Guild | REGISTRY-API-27-001, WEB-POLICY-27-001 | Implement policy workspace commands (`stella policy init`, `edit`, `lint`, `compile`, `test`) with template selection, local cache, JSON output, and deterministic temp directories. | Commands operate offline with cached templates; diagnostics mirror API responses; unit tests cover happy/error paths; help text updated. |
| CLI-POLICY-27-002 | TODO | DevEx/CLI Guild | REGISTRY-API-27-006, WEB-POLICY-27-002 | Add submission/review workflow commands (`stella policy version bump`, `submit`, `review comment`, `approve`, `reject`) supporting reviewer assignment, changelog capture, and exit codes. | Workflow commands enforce required approvers; comments upload correctly; integration tests cover approval failure; docs updated. |
| CLI-POLICY-27-003 | TODO | DevEx/CLI Guild | REGISTRY-API-27-005, SCHED-CONSOLE-27-001 | Implement `stella policy simulate` enhancements (quick vs batch, SBOM selectors, heatmap summary, manifest download) with `--json` and Markdown report output for CI. | CLI can trigger batch sim, poll progress, download artifacts; outputs deterministic schemas; CI sample workflow documented; tests cover cancellation/timeouts. |
| CLI-POLICY-27-004 | TODO | DevEx/CLI Guild | REGISTRY-API-27-007, REGISTRY-API-27-008, AUTH-POLICY-27-002 | Add lifecycle commands for publish/promote/rollback/sign (`stella policy publish --sign`, `promote --env`, `rollback`) with attestation verification and canary arguments. | Commands enforce signing requirement, support dry-run, produce audit logs; integration tests cover promotion + rollback; documentation updated. |
| CLI-POLICY-27-005 | TODO | DevEx/CLI Guild, Docs Guild | DOCS-CONSOLE-27-007, DOCS-POLICY-27-007 | Update CLI reference and samples for Policy Studio including JSON schemas, exit codes, and CI snippets. | CLI docs merged with screenshots/transcripts; parity matrix updated; acceptance tests ensure `--help` examples compile. |
## Vulnerability Explorer (Sprint 29)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-VULN-29-001 | TODO | DevEx/CLI Guild | VULN-API-29-002, AUTH-VULN-29-001 | Implement `stella vuln list` with grouping, paging, filters, `--json/--csv`, and policy selection. | Command returns deterministic output; paging works; regression tests cover filters/grouping. |
| CLI-VULN-29-002 | TODO | DevEx/CLI Guild | VULN-API-29-003 | Implement `stella vuln show` displaying evidence, policy rationale, paths, ledger summary; support `--json` for automation. | Output matches schema; evidence rendered with provenance; tests cover missing data. |
| CLI-VULN-29-003 | TODO | DevEx/CLI Guild | VULN-API-29-004, LEDGER-29-005 | Add workflow commands (`assign`, `comment`, `accept-risk`, `verify-fix`, `target-fix`, `reopen`) with filter selection (`--filter`) and idempotent retries. | Commands create ledger events; exit codes documented; integration tests cover role enforcement. |
| CLI-VULN-29-004 | TODO | DevEx/CLI Guild | VULN-API-29-005 | Implement `stella vuln simulate` producing delta summaries and optional Markdown report for CI. | CLI simulation returns diff tables + JSON; tests verify diff correctness; docs updated. |
| CLI-VULN-29-005 | TODO | DevEx/CLI Guild | VULN-API-29-008 | Add `stella vuln export` and `stella vuln bundle verify` commands to trigger/download evidence bundles and verify signatures. | Export command streams to file; verify command checks signatures; tests cover success/failure. |
| CLI-VULN-29-006 | TODO | DevEx/CLI Guild, Docs Guild | DOCS-VULN-29-004, DOCS-VULN-29-005 | Update CLI docs/examples for Vulnerability Explorer with compliance checklist and CI snippets. | Docs merged; automated examples validated; compliance checklist appended. |
## VEX Lens (Sprint 30)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-VEX-30-001 | TODO | DevEx/CLI Guild | VEXLENS-30-007 | Implement `stella vex consensus list` with filters, paging, policy selection, `--json/--csv`. | Command returns deterministic output; regression tests cover filters/paging; docs updated. |
| CLI-VEX-30-002 | TODO | DevEx/CLI Guild | VEXLENS-30-007 | Implement `stella vex consensus show` displaying quorum, evidence, rationale, signature status. | Output matches schema; tests cover conflicting evidence; docs updated. |
| CLI-VEX-30-003 | TODO | DevEx/CLI Guild | VEXLENS-30-007 | Implement `stella vex simulate` for trust/threshold overrides with JSON diff output. | Simulation command returns diff summary; tests cover policy scenarios; docs updated. |
| CLI-VEX-30-004 | TODO | DevEx/CLI Guild | VEXLENS-30-007 | Implement `stella vex export` for consensus NDJSON bundles with signature verification helper. | Export & verify commands operational; tests cover file output; docs updated. |
## Advisory AI (Sprint 31)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-AIAI-31-001 | TODO | DevEx/CLI Guild | AIAI-31-006 | Implement `stella advise summarize` command with JSON/Markdown outputs and citation display. | Command returns summary + JSON; citations preserved; tests cover filters. |
| CLI-AIAI-31-002 | TODO | DevEx/CLI Guild | AIAI-31-006 | Implement `stella advise explain` showing conflict narrative and structured rationale. | Output matches schemas; tests cover disputed cases. |
| CLI-AIAI-31-003 | TODO | DevEx/CLI Guild | AIAI-31-006 | Implement `stella advise remediate` generating remediation plans with `--strategy` filters and file output. | Plans saved to file; exit codes documented; tests cover version mapping. |
| CLI-AIAI-31-004 | TODO | DevEx/CLI Guild | AIAI-31-006 | Implement `stella advise batch` for summaries/conflicts/remediation with progress + multi-status responses. | Batch command handles 207 responses; tests cover partial failures. |
## Export Center (Epic 10)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-EXPORT-35-001 | TODO | DevEx/CLI Guild | WEB-EXPORT-35-001, AUTH-EXPORT-35-001 | Implement `stella export profiles|runs` list/show, `run create`, `run status`, and resumable download commands with manifest/provenance retrieval. | Commands respect viewer/operator scopes; downloads resume via range requests; integration tests cover filters and offline mode. |
| CLI-EXPORT-36-001 | TODO | DevEx/CLI Guild | CLI-EXPORT-35-001, WEB-EXPORT-36-001 | Add distribution commands (`stella export distribute`, `run download --resume` enhancements) and improved status polling with progress bars. | Distribution commands push OCI/object storage; status polling handles SSE fallback; tests cover failure cases. |
| CLI-EXPORT-37-001 | TODO | DevEx/CLI Guild | CLI-EXPORT-36-001, WEB-EXPORT-37-001 | Provide scheduling (`stella export schedule`), retention, and `export verify` commands performing signature/hash validation. | Scheduling/retention commands enforce admin scopes; verify command checks signatures/hashes; examples documented; tests cover success/failure. |
## Orchestrator Dashboard (Epic 9)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-ORCH-32-001 | TODO | DevEx/CLI Guild | WEB-ORCH-32-001, AUTH-ORCH-32-001 | Implement `stella orch sources|runs|jobs` list/show commands with filters, pagination, table/JSON output, and deterministic exit codes. | Commands respect viewer scope; JSON schema documented; integration tests cover filters/paging/offline mode. |
| CLI-ORCH-33-001 | TODO | DevEx/CLI Guild | CLI-ORCH-32-001, WEB-ORCH-33-001, AUTH-ORCH-33-001 | Add action verbs (`sources test|pause|resume|sync-now`, `jobs retry|cancel|tail`) with streaming output, reason prompts, and retry/backoff handling. | Actions succeed with operator scope; streaming tail resilient to reconnect; tests cover permission failures and retries. |
| CLI-ORCH-34-001 | TODO | DevEx/CLI Guild | CLI-ORCH-33-001, WEB-ORCH-34-001, AUTH-ORCH-34-001 | Provide backfill wizard (`--from/--to --dry-run`), quota management (`quotas get|set`), and safety guardrails for orchestrator GA. | Backfill preview output matches API; quota updates require reason; CLI docs/help updated; regression tests cover dry-run + failure paths. |
## Notifications Studio (Epic 11)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-NOTIFY-38-001 | TODO | DevEx/CLI Guild | WEB-NOTIFY-38-001, AUTH-NOTIFY-38-001 | Implement `stella notify rules|templates|incidents` commands (list/create/update/test/ack) with file inputs, JSON output, and RBAC-aware flow. | Commands invoke notifier APIs successfully; rule test uses local events file; integration tests cover create/test/ack; help docs updated. |
| CLI-NOTIFY-39-001 | TODO | DevEx/CLI Guild | CLI-NOTIFY-38-001, WEB-NOTIFY-39-001 | Add simulation (`stella notify simulate`) and digest commands with diff output and schedule triggering, including dry-run mode. | Simulation command returns deterministic diff; digest command triggers run and polls status; tests cover filters and failures. |
| CLI-NOTIFY-40-001 | TODO | DevEx/CLI Guild | CLI-NOTIFY-39-001, WEB-NOTIFY-40-001 | Provide ack token redemption workflow, escalation management, localization previews, and channel health checks. | Ack redemption validates signed tokens; escalation commands manage schedules; localization preview shows variants; integration tests cover negative cases. |
## CLI Parity & Task Packs (Epic 12)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-CORE-41-001 | TODO | DevEx/CLI Guild | AUTH-PACKS-41-001 | Implement CLI core features: config precedence, profiles/contexts, auth flows, output renderer (json/yaml/table), error mapping, global flags, telemetry opt-in. | CLI loads config deterministically; auth works (device/PAT); outputs render correctly; tests cover precedence and exit codes. |
| CLI-PARITY-41-001 | TODO | DevEx/CLI Guild | CLI-CORE-41-001 | Deliver parity command groups (`policy`, `sbom`, `vuln`, `vex`, `advisory`, `export`, `orchestrator`) with `--explain`, deterministic outputs, and parity matrix entries. | Commands match Console behavior; parity matrix green for covered actions; integration tests cover major flows. |
| CLI-PARITY-41-002 | TODO | DevEx/CLI Guild | CLI-PARITY-41-001, WEB-NOTIFY-38-001 | Implement `notify`, `aoc`, `auth` command groups, idempotency keys, shell completions, config docs, and parity matrix export tooling. | Commands functional; completions generated; docs updated; parity matrix auto-exported; CI checks gating. |
| CLI-PACKS-42-001 | TODO | DevEx/CLI Guild | CLI-CORE-41-001, PACKS-REG-41-001, TASKRUN-41-001 | Implement Task Pack commands (`pack plan/run/push/pull/verify`) with schema validation, expression sandbox, plan/simulate engine, remote execution. | Pack commands operational; plan/sim produce accurate graph; remote run streams logs; schema validation enforced. |
| CLI-PACKS-43-001 | TODO | DevEx/CLI Guild | CLI-PACKS-42-001, TASKRUN-42-001 | Deliver advanced pack features (approvals pause/resume, secret injection, localization, man pages, offline cache). | Approvals handled; secrets redacted; localization supported; man pages built; offline cache documented; integration tests cover scenarios. |
## Authority-Backed Scopes & Tenancy (Epic 14)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-TEN-47-001 | TODO | DevEx/CLI Guild | AUTH-TEN-47-001 | Implement `stella login`, `whoami`, `tenants list`, persistent profiles, secure token storage, and `--tenant` override with validation. | Commands functional across platforms; tokens stored securely; tenancy header set on requests; integration tests cover login/tenant switch. |
| CLI-TEN-49-001 | TODO | DevEx/CLI Guild | CLI-TEN-47-001, AUTH-TEN-49-001 | Add service account token minting, delegation (`stella token delegate`), impersonation banner, and audit-friendly logging. | Service tokens minted with scopes/TTL; delegation recorded; CLI displays impersonation banner; docs updated. |
## Observability & Forensics (Epic 15)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-OBS-50-001 | TODO | DevEx/CLI Guild | TELEMETRY-OBS-50-002, WEB-OBS-50-001 | Ensure CLI HTTP client propagates `traceparent` headers for all commands, prints correlation IDs on failure, and records trace IDs in verbose logs (scrubbed). | Trace headers observed in integration tests; verbose logs include trace IDs; redaction guard verified. |
| CLI-OBS-51-001 | TODO | DevEx/CLI Guild | CLI-OBS-50-001, WEB-OBS-51-001 | Implement `stella obs top` command streaming service health metrics, SLO status, and burn-rate alerts with TUI view and JSON output. | Command streams metrics; JSON output documented; integration tests cover streaming and exit codes. |
| CLI-OBS-52-001 | TODO | DevEx/CLI Guild | CLI-OBS-51-001, TIMELINE-OBS-52-003 | Add `stella obs trace <trace_id>` and `stella obs logs --from/--to` commands that correlate timeline events, logs, and evidence links with pagination + guardrails. | Commands fetch timeline/log data; paging tokens handled; fixtures stored under `samples/obs/`; tests cover errors. |
| CLI-FORENSICS-53-001 | TODO | DevEx/CLI Guild, Evidence Locker Guild | CLI-OBS-52-001, EVID-OBS-53-003 | Implement `stella forensic snapshot create --case` and `snapshot list/show` commands invoking evidence locker APIs, surfacing manifest digests, and storing local cache metadata. | Snapshot commands functional; manifests displayed; cache metadata deterministic; docs/help updated. |
| CLI-FORENSICS-54-001 | TODO | DevEx/CLI Guild, Provenance Guild | CLI-FORENSICS-53-001, PROV-OBS-54-001 | Provide `stella forensic verify <bundle>` command validating checksums, DSSE signatures, and timeline chain-of-custody. Support JSON/pretty output and exit codes for CI. | Verification works with sample bundles; tests cover success/failure; docs updated. |
| CLI-FORENSICS-54-002 | TODO | DevEx/CLI Guild, Provenance Guild | CLI-FORENSICS-54-001 | Implement `stella forensic attest show <artifact>` listing attestation details (signer, timestamp, subjects) and verifying signatures. | Command prints attestation summary; verification errors flagged; tests cover offline mode. |
| CLI-OBS-55-001 | TODO | DevEx/CLI Guild, DevOps Guild | CLI-OBS-52-001, WEB-OBS-55-001, DEVOPS-OBS-55-001 | Add `stella obs incident-mode enable|disable|status` commands with confirmation guards, cooldown timers, and audit logging. | Commands manage incident mode; audit logs verified; tests cover permissions and cooldown. |
## Air-Gapped Mode (Epic 16)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-AIRGAP-56-001 | TODO | DevEx/CLI Guild | MIRROR-CRT-56-001, AIRGAP-IMP-56-001 | Implement `stella mirror create|verify` and `stella airgap verify` commands with DSSE/TUF results, dry-run mode, and deterministic manifests. | Commands produce deterministic bundles; verify outputs structured DSSE/TUF results; integration tests cover tampering scenarios. |
| CLI-AIRGAP-57-001 | TODO | DevEx/CLI Guild | CLI-AIRGAP-56-001, AIRGAP-IMP-58-001 | Add `stella airgap import` with diff preview, bundle scope selection (`--tenant`, `--global`), audit logging, and progress reporting. | Import updates catalog; diff preview rendered; audit entries include bundle ID + scope; tests cover idempotent re-import. |
| CLI-AIRGAP-57-002 | TODO | DevEx/CLI Guild | CLI-AIRGAP-56-001, AIRGAP-CTL-56-002 | Provide `stella airgap seal|status` commands surfacing sealing state, drift, staleness metrics, and remediation guidance with safe confirmation prompts. | Status command prints drift/staleness; seal requires confirmation + scope; integration tests cover RBAC denials. |
| CLI-AIRGAP-58-001 | TODO | DevEx/CLI Guild, Evidence Locker Guild | CLI-AIRGAP-57-001, CLI-FORENSICS-54-001 | Implement `stella airgap export evidence` helper for portable evidence packages, including checksum manifest and verification. | Command generates portable bundle; verification step validates signatures; docs/help updated with examples. |
## SDKs & OpenAPI (Epic 17)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-SDK-62-001 | TODO | DevEx/CLI Guild, SDK Generator Guild | SDKGEN-63-001 | Replace bespoke HTTP clients with official SDK (TS/Go) for all CLI commands; ensure modular transport for air-gapped mode. | CLI builds using SDK; regression suite passes; telemetry shows SDK version. |
| CLI-SDK-62-002 | TODO | DevEx/CLI Guild | CLI-SDK-62-001, APIGOV-61-001 | Update CLI error handling to surface standardized API error envelope with `error.code` and `trace_id`. | CLI displays envelope data; integration tests cover new output. |
| CLI-SDK-63-001 | TODO | DevEx/CLI Guild, API Governance Guild | OAS-61-002 | Expose `stella api spec download` command retrieving aggregate OAS and verifying checksum/ETag. | Command downloads + verifies spec; docs updated; tests cover failure cases. |
| CLI-SDK-64-001 | TODO | DevEx/CLI Guild, SDK Release Guild | SDKREL-63-001 | Add CLI subcommand `stella sdk update` to fetch latest SDK manifests/changelogs; integrate with Notifications for deprecations. | Command lists versions/changelogs; notifications triggered on updates. |
## Risk Profiles (Epic 18)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-RISK-66-001 | TODO | DevEx/CLI Guild, Policy Guild | POLICY-RISK-67-002 | Implement `stella risk profile list|get|create|publish` commands with schema validation and scope selectors. | Commands operate against API; validation errors surfaced; tests cover CRUD. |
| CLI-RISK-66-002 | TODO | DevEx/CLI Guild, Risk Engine Guild | RISK-ENGINE-69-001 | Ship `stella risk simulate` supporting SBOM/asset inputs, diff mode, and export to JSON/CSV. | Simulation runs via CLI; output tested; docs updated. |
| CLI-RISK-67-001 | TODO | DevEx/CLI Guild, Findings Ledger Guild | LEDGER-RISK-67-001 | Provide `stella risk results` with filtering, severity thresholds, explainability fetch. | Results command returns paginated data; explaination fetch command outputs artifact; tests pass. |
| CLI-RISK-68-001 | TODO | DevEx/CLI Guild, Export Guild | RISK-BUNDLE-70-001 | Add `stella risk bundle verify` and integrate with offline risk bundles. | Verification command validates signatures; integration tests cover tampered bundle. |
## Attestor Console (Epic 19)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| CLI-ATTEST-73-001 | TODO | CLI Attestor Guild | ATTESTOR-73-001, SDKGEN-63-001 | Implement `stella attest sign` (payload selection, subject digest, key reference, output format) using official SDK transport. | Command signs envelopes; tests cover file/KMS keys; docs updated. |
| CLI-ATTEST-73-002 | TODO | CLI Attestor Guild | ATTESTOR-73-002 | Implement `stella attest verify` with policy selection, explainability output, and JSON/table formatting. | Verification command returns structured report; exit codes match pass/fail; integration tests pass. |
| CLI-ATTEST-74-001 | TODO | CLI Attestor Guild | ATTESTOR-73-003 | Implement `stella attest list` with filters (subject, type, issuer, scope) and pagination. | Command outputs table/JSON; tests cover filters. |
| CLI-ATTEST-74-002 | TODO | CLI Attestor Guild | ATTESTOR-73-003 | Implement `stella attest fetch` to download envelopes and payloads to disk. | Fetch command saves files; checks digests; tests cover air-gap use. |
| CLI-ATTEST-75-001 | TODO | CLI Attestor Guild, KMS Guild | KMS-72-001 | Implement `stella attest key create|import|rotate|revoke` commands. | Key commands work with file/KMS drivers; tests cover rotation/revocation. |
| CLI-ATTEST-75-002 | TODO | CLI Attestor Guild, Export Guild | ATTESTOR-75-001 | Add support for building/verifying attestation bundles in CLI. | Bundle commands functional; verification catches tampering; docs updated. |

127
src/StellaOps.Concelier.Core/TASKS.md
View File

@@ -1,21 +1,106 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|JobCoordinator implementation (create/get/mark status)|BE-Core|Storage.Mongo|DONE – `JobCoordinator` drives Mongo-backed runs.|
|Cron scheduling loop with TimeProvider|BE-Core|Core|DONE – `JobSchedulerHostedService` evaluates cron expressions.|
|Single-flight/lease semantics|BE-Core|Storage.Mongo|DONE – lease acquisition backed by `MongoLeaseStore`.|
|Trigger API contract (Result mapping)|BE-Core|WebService|DONE – `JobTriggerResult` outcomes map to HTTP statuses.|
|Run telemetry enrichment|BE-Core|Observability|DONE – `JobDiagnostics` ties activities & counters into coordinator/scheduler paths.|
|Deterministic params hashing|BE-Core|Core|DONE – `JobParametersHasher` creates SHA256 hash.|
|Golden tests for timeout/cancel|QA|Core|DONE – JobCoordinatorTests cover cancellation timeout path.|
|JobSchedulerBuilder options registry coverage|BE-Core|Core|DONE – added scheduler tests confirming cron/timeout/lease metadata persists via JobSchedulerOptions.|
|Plugin discovery + DI glue with PluginHost|BE-Core|Plugin libs|DONE – JobPluginRegistrationExtensions now loads PluginHost routines and wires connector/exporter registrations.|
|Harden lease release error handling in JobCoordinator|BE-Core|Storage.Mongo|DONE – lease release failures now logged, wrapped, and drive run failure status; fire-and-forget execution guarded. Verified with `dotnet test --no-build --filter JobCoordinator`.|
|Validate job trigger parameters for serialization|BE-Core|WebService|DONE – trigger parameters normalized/serialized with defensive checks returning InvalidParameters on failure. Full-suite `dotnet test --no-build` currently red from live connector fixture drift (Oracle/JVN/RedHat).|
|FEEDCORE-ENGINE-03-001 Canonical merger implementation|BE-Core|Merge|DONE – `CanonicalMerger` applies GHSA/NVD/OSV conflict rules with deterministic provenance and comprehensive unit coverage. **Coordination:** Connector leads must align mapper outputs with the canonical field expectations before 2025-10-18 so Merge can activate the path globally.|
|FEEDCORE-ENGINE-03-002 Field precedence and tie-breaker map|BE-Core|Merge|DONE – field precedence and freshness overrides enforced via `FieldPrecedence` map with tie-breakers and analytics capture. **Reminder:** Storage/Merge owners review precedence overrides when onboarding new feeds to ensure `decisionReason` tagging stays consistent.|
|Canonical merger parity for description/CWE/canonical metric|BE-Core|Models|DONE (2025-10-15) – merger now populates description/CWEs/canonical metric id with provenance and regression tests cover the new decisions.|
|Reference normalization & freshness instrumentation cleanup|BE-Core, QA|Models|DONE (2025-10-15) – reference keys normalized, freshness overrides applied to union fields, and new tests assert decision logging.|
|FEEDCORE-ENGINE-07-001 – Advisory event log & asOf queries|Team Core Engine & Storage Analytics|FEEDSTORAGE-DATA-07-001|**DONE (2025-10-19)** – Implemented `AdvisoryEventLog` service plus repository contracts, canonical hashing, and lower-cased key normalization with replay support; documented determinism guarantees. Tests: `dotnet test src/StellaOps.Concelier.Core.Tests/StellaOps.Concelier.Core.Tests.csproj`.|
|FEEDCORE-ENGINE-07-002 – Noise prior computation service|Team Core Engine & Data Science|FEEDCORE-ENGINE-07-001|**DONE (2025-10-21)** – Build rule-based learner capturing false-positive priors per package/env, persist summaries, and expose APIs for Excititor/scan suppressors with reproducible statistics.|
|FEEDCORE-ENGINE-07-003 – Unknown state ledger & confidence seeding|Team Core Engine & Storage Analytics|FEEDCORE-ENGINE-07-001|DONE (2025-10-21) – Persisted `unknown_vuln_range/unknown_origin/ambiguous_fix` markers with seeded confidence bands, exposed query surface for Policy, and added canonical serialization fixtures + regression tests.|
# TASKS — Epic 1: Aggregation-Only Contract
> **AOC Reminder:** ingestion aggregates and links only—no precedence, normalization, or severity computation. Derived data lives in Policy/overlay services.
| ID | Status | Owner(s) | Depends on | Notes |
|---|---|---|---|---|
| CONCELIER-CORE-AOC-19-001 `AOC write guard` | TODO | Concelier Core Guild | WEB-AOC-19-001 | Implement repository interceptor that inspects write payloads for forbidden AOC keys, validates provenance/signature presence, and maps violations to `ERR_AOC_00x`. |
| CONCELIER-CORE-AOC-19-002 `Deterministic linkset extraction` | TODO | Concelier Core Guild | CONCELIER-CORE-AOC-19-001 | Build canonical linkset mappers for CVE/GHSA/PURL/CPE/reference extraction from upstream raw payloads, ensuring reconciled-from metadata is tracked and deterministic. |
| CONCELIER-CORE-AOC-19-003 `Idempotent append-only upsert` | TODO | Concelier Core Guild | CONCELIER-STORE-AOC-19-002 | Implement idempotent upsert path using `(vendor, upstreamId, contentHash, tenant)` key, emitting supersedes pointers for new revisions and preventing duplicate inserts. |
| CONCELIER-CORE-AOC-19-004 `Remove ingestion normalization` | TODO | Concelier Core Guild | CONCELIER-CORE-AOC-19-002, POLICY-AOC-19-003 | Strip normalization/dedup/severity logic from ingestion pipelines, delegate derived computations to Policy Engine, and update exporters/tests to consume raw documents only. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-POLICY-20-002 `Linkset enrichment for policy` | TODO | Concelier Core Guild, Policy Guild | CONCELIER-CORE-AOC-19-002, POLICY-ENGINE-20-001 | Strengthen linkset builders with vendor-specific equivalence tables, NEVRA/PURL normalization, and version range parsing to maximize policy join recall; update fixtures + docs. |
## Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-GRAPH-21-001 `SBOM projection enrichment` | TODO | Concelier Core Guild, Cartographer Guild | CONCELIER-POLICY-20-002, CARTO-GRAPH-21-002 | Extend SBOM normalization to emit full relationship graph (depends_on/contains/provides), scope tags, entrypoint annotations, and component metadata required by Cartographer. |
| CONCELIER-GRAPH-21-002 `Change events` | TODO | Concelier Core Guild, Scheduler Guild | CONCELIER-GRAPH-21-001 | Publish change events (new SBOM version, relationship delta) for Cartographer build queue; ensure events include tenant/context metadata. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-LNM-21-001 `Advisory observation schema` | TODO | Concelier Core Guild | CONCELIER-CORE-AOC-19-001 | Introduce immutable `advisory_observations` model with AOC metadata, raw payload pointers, normalized fields, and tenancy guardrails; publish schema definition. |
| CONCELIER-LNM-21-002 `Linkset builder` | TODO | Concelier Core Guild, Data Science Guild | CONCELIER-LNM-21-001 | Implement correlation pipeline (alias graph, PURL overlap, CVSS vector equality, fuzzy title match) that produces `advisory_linksets` with confidence + conflict annotations. |
| CONCELIER-LNM-21-003 `Conflict annotator` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Detect field disagreements (severity, CVSS, ranges, references) and record structured conflicts on linksets; surface to API/UI. |
| CONCELIER-LNM-21-004 `Merge code removal` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Excise existing merge/dedup logic, enforce immutability on observations, and add guards/tests to prevent future merges. |
| CONCELIER-LNM-21-005 `Event emission` | TODO | Concelier Core Guild, Platform Events Guild | CONCELIER-LNM-21-002 | Emit `advisory.linkset.updated` events with delta payloads for downstream Policy Engine/Cartographer consumers; ensure idempotent delivery. |
## Policy Engine + Editor v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-POLICY-23-001 `Evidence indexes` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Add secondary indexes/materialized views to accelerate policy lookups (alias, severity per observation, correlation confidence). Document query contracts for runtime. |
| CONCELIER-POLICY-23-002 `Event guarantees` | TODO | Concelier Core Guild, Platform Events Guild | CONCELIER-LNM-21-005 | Ensure `advisory.linkset.updated` emits at-least-once with idempotent keys and include policy-relevant metadata (confidence, conflict summary). |
## Graph & Vuln Explorer v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-GRAPH-24-001 `Advisory overlay inputs` | TODO | Concelier Core Guild | CONCELIER-POLICY-23-001 | Expose raw advisory observations/linksets with tenant filters for overlay services; no derived counts/severity in ingestion. |
## Reachability v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-SIG-26-001 `Vulnerable symbol exposure` | TODO | Concelier Core Guild, Signals Guild | SIGNALS-24-002 | Expose advisory metadata (affected symbols/functions) via API to enrich reachability scoring; update fixtures. |
## Orchestrator Dashboard
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-ORCH-32-001 `Source registry integration` | TODO | Concelier Core Guild | ORCH-SVC-32-001, AUTH-ORCH-32-001 | Register Concelier data sources with orchestrator (metadata, schedules, rate policies) and wire provenance IDs/security scopes. |
| CONCELIER-ORCH-32-002 `Worker SDK adoption` | TODO | Concelier Core Guild | CONCELIER-ORCH-32-001, WORKER-GO-32-001, WORKER-PY-32-001 | Embed orchestrator worker SDK in ingestion loops, emit heartbeats/progress/artifact hashes, and enforce idempotency keys. |
| CONCELIER-ORCH-33-001 `Control hook compliance` | TODO | Concelier Core Guild | CONCELIER-ORCH-32-002, ORCH-SVC-33-001, ORCH-SVC-33-002 | Honor orchestrator throttle/pause/retry actions, surface structured error classes, and persist safe checkpoints for resume. |
| CONCELIER-ORCH-34-001 `Backfill + ledger linkage` | TODO | Concelier Core Guild | CONCELIER-ORCH-33-001, ORCH-SVC-33-003, ORCH-SVC-34-001 | Execute orchestrator-driven backfills, reuse artifact hashes to avoid duplicates, and link provenance to run ledger exports. |
## Authority-Backed Scopes & Tenancy (Epic 14)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-TEN-48-001 `Tenant-aware linking` | TODO | Concelier Core Guild | AUTH-TEN-47-001 | Ensure advisory normalization/linking runs per tenant with RLS enforcing isolation; emit capability endpoint reporting `merge=false`; update events with tenant context. |
## Observability & Forensics (Epic 15)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-OBS-50-001 `Telemetry adoption` | TODO | Concelier Core Guild, Observability Guild | TELEMETRY-OBS-50-001, TELEMETRY-OBS-50-002 | Replace ad-hoc logging with telemetry core across ingestion/linking pipelines; ensure spans/logs include tenant, source vendor, upstream id, content hash, and trace IDs. |
| CONCELIER-OBS-51-001 `Metrics & SLOs` | TODO | Concelier Core Guild, DevOps Guild | CONCELIER-OBS-50-001, TELEMETRY-OBS-51-001 | Emit metrics for ingest latency (cold/warm), queue depth, aoc violation rate, and publish SLO burn-rate alerts (ingest P95 <30s cold / <5s warm). Ship dashboards + alert configs. |
| CONCELIER-OBS-52-001 `Timeline events` | TODO | Concelier Core Guild | CONCELIER-OBS-50-001, TIMELINE-OBS-52-002 | Emit `timeline_event` records for advisory ingest/normalization/linkset creation with provenance, trace IDs, conflict summaries, and evidence placeholders. |
| CONCELIER-OBS-53-001 `Evidence snapshots` | TODO | Concelier Core Guild, Evidence Locker Guild | CONCELIER-OBS-52-001, EVID-OBS-53-002 | Produce advisory evaluation bundle payloads (raw doc, linkset, normalization diff) for evidence locker; ensure Merkle manifests seeded with content hashes. |
| CONCELIER-OBS-54-001 `Attestation & verification` | TODO | Concelier Core Guild, Provenance Guild | CONCELIER-OBS-53-001, PROV-OBS-54-001 | Attach DSSE attestations for advisory processing batches, expose verification API to confirm bundle integrity, and link attestation IDs back to timeline + ledger. |
| CONCELIER-OBS-55-001 `Incident mode hooks` | TODO | Concelier Core Guild, DevOps Guild | CONCELIER-OBS-51-001, DEVOPS-OBS-55-001 | Increase sampling, capture raw payload snapshots, and extend retention under incident mode; emit activation events + guardrails against PII leak. |
## Air-Gapped Mode (Epic 16)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-AIRGAP-56-001 `Mirror ingestion adapters` | TODO | Concelier Core Guild | AIRGAP-IMP-57-002, MIRROR-CRT-56-001 | Add mirror source adapters reading advisories from imported bundles, preserving source metadata and bundle IDs. Ensure ingestion remains append-only. |
| CONCELIER-AIRGAP-56-002 `Bundle catalog linking` | TODO | Concelier Core Guild, AirGap Importer Guild | CONCELIER-AIRGAP-56-001, AIRGAP-IMP-57-001 | Persist `bundle_id`, `merkle_root`, and time anchor references on observations/linksets for provenance. |
| CONCELIER-AIRGAP-57-001 `Sealed-mode source restrictions` | TODO | Concelier Core Guild, AirGap Policy Guild | CONCELIER-AIRGAP-56-001, AIRGAP-POL-56-001 | Enforce sealed-mode egress rules by disallowing non-mirror connectors and surfacing remediation errors. |
| CONCELIER-AIRGAP-57-002 `Staleness annotations` | TODO | Concelier Core Guild, AirGap Time Guild | CONCELIER-AIRGAP-56-002, AIRGAP-TIME-58-001 | Compute staleness metadata for advisories per bundle and expose via API for Console/CLI badges. |
| CONCELIER-AIRGAP-58-001 `Portable advisory evidence` | TODO | Concelier Core Guild, Evidence Locker Guild | CONCELIER-OBS-53-001, EVID-OBS-54-001 | Package advisory evidence fragments into portable evidence bundles for cross-domain transfer. |
## SDKs & OpenAPI (Epic 17)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-OAS-61-001 `Spec coverage` | TODO | Concelier Core Guild, API Contracts Guild | OAS-61-001 | Update Concelier OAS with advisory observation/linkset endpoints, standard pagination, and source provenance fields. |
| CONCELIER-OAS-61-002 `Examples library` | TODO | Concelier Core Guild | CONCELIER-OAS-61-001 | Provide rich examples for advisories, linksets, conflict annotations used by SDK + docs. |
| CONCELIER-OAS-62-001 `SDK smoke tests` | TODO | Concelier Core Guild, SDK Generator Guild | CONCELIER-OAS-61-001, SDKGEN-63-001 | Add SDK tests covering advisory search, pagination, and conflict handling; ensure source metadata surfaced. |
| CONCELIER-OAS-63-001 `Deprecation headers` | TODO | Concelier Core Guild, API Governance Guild | APIGOV-63-001 | Implement deprecation header support and timeline events for retiring endpoints. |
## Risk Profiles (Epic 18)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-RISK-66-001 `CVSS/KEV providers` | TODO | Concelier Core Guild, Risk Engine Guild | RISK-ENGINE-67-001 | Expose CVSS, KEV, fix availability data via provider APIs with source metadata preserved. |
| CONCELIER-RISK-66-002 `Fix availability signals` | TODO | Concelier Core Guild | CONCELIER-RISK-66-001 | Provide structured fix availability and release metadata consumable by risk engine; document provenance. |
| CONCELIER-RISK-67-001 `Source consensus metrics` | TODO | Concelier Core Guild | CONCELIER-RISK-66-001 | Add consensus counts and confidence scores for linked advisories; ensure explainability includes source digests. |
| CONCELIER-RISK-68-001 `Policy Studio integration` | TODO | Concelier Core Guild, Policy Studio Guild | POLICY-RISK-68-001 | Surface advisory fields in Policy Studio profile editor (signal pickers, reducers). |
| CONCELIER-RISK-69-001 `Notification hooks` | TODO | Concelier Core Guild, Notifications Guild | CONCELIER-RISK-66-002 | Emit events when advisory signals change impacting risk scores (e.g., fix available). |
## Attestor Console (Epic 19)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-ATTEST-73-001 `ScanResults attestation inputs` | TODO | Concelier Core Guild, Attestor Service Guild | ATTEST-TYPES-72-001 | Provide normalized advisory data and linkset digests needed for ScanResults attestations. |
| CONCELIER-ATTEST-73-002 `Transparency metadata` | TODO | Concelier Core Guild | CONCELIER-ATTEST-73-001 | Ensure Conseiller exposes source digests for transparency proofs and explainability. |

7
src/StellaOps.Concelier.Merge/TASKS.md
View File

@@ -24,3 +24,10 @@
|FEEDMERGE-COORD-02-901 Connector deadline check-ins|BE-Merge|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-21)** – Confirm Cccs/Cisco normalized-rule branches land, capture `concelier.merge.normalized_rules*` counter screenshots, and update coordination docs with the results.|
|FEEDMERGE-COORD-02-902 ICS-CISA normalized-rule decision support|BE-Merge, Models|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-23)** – Review ICS-CISA sample advisories, confirm SemVer reuse vs new firmware scheme, pre-stage Models ticket template, and document outcome in coordination docs + tracker files.|
|FEEDMERGE-COORD-02-903 KISA firmware scheme review|BE-Merge, Models|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-24)** – Pair with KISA team on proposed firmware scheme (`kisa.build` or variant), ensure builder alignment, open Models ticket if required, and log decision in coordination docs + tracker files.|
## Link-Not-Merge v1 Transition
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|MERGE-LNM-21-001 Migration plan authoring|BE-Merge, Architecture Guild|CONCELIER-LNM-21-101|Draft `no-merge` migration playbook, documenting backfill strategy, feature flag rollout, and rollback steps for legacy merge pipeline deprecation.|
|MERGE-LNM-21-002 Merge service deprecation|BE-Merge|MERGE-LNM-21-001|Refactor or retire `AdvisoryMergeService` and related pipelines, ensuring callers transition to observation/linkset APIs; add compile-time analyzer preventing merge service usage.|
|MERGE-LNM-21-003 Determinism/test updates|QA Guild, BE-Merge|MERGE-LNM-21-002|Replace merge determinism suites with observation/linkset regression tests verifying no data mutation and conflicts remain visible.|

46
src/StellaOps.Concelier.Storage.Mongo/TASKS.md
View File

@@ -1,24 +1,22 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|MongoBootstrapper to create collections/indexes|BE-Storage|Storage.Mongo|DONE – `MongoBootstrapper` ensures collections & indexes incl. TTL on locks.ttlAt.|
|SourceState repository (get/set/backoff)|BE-Conn-Base|Storage.Mongo|DONE – implemented `MongoSourceStateRepository`.|
|Document/DTO stores with SHA/metadata|BE-Conn-Base|Storage.Mongo|DONE – DocumentStore and DtoStore provide upsert/status lookups.|
|AdvisoryStore (GetAllAsync etc.)|BE-Export|Models|DONE – AdvisoryStore handles upsert + recent/advisory fetches.|
|Job store (runs/active/recent)|BE-Core|Storage.Mongo|DONE – `MongoJobStore` covers create/start/complete queries.|
|Alias and reference secondary indexes|BE-Storage|Models|DONE – bootstrapper builds alias/reference indexes.|
|MergeEvent store|BE-Merge|Models|DONE – MergeEventStore appends/retrieves recent events.|
|ExportState store|BE-Export|Exporters|DONE – ExportStateStore upserts and retrieves exporter metadata.|
|Performance tests for large advisories|QA|Storage.Mongo|DONE – `AdvisoryStorePerformanceTests` exercises large payload upsert/find throughput budgets.|
|Migration playbook for schema/index changes|BE-Storage|Storage.Mongo|DONE – `MongoMigrationRunner` executes `IMongoMigration` steps recorded in `schema_migrations`; see `MIGRATIONS.md`.|
|Raw document retention/TTL strategy|BE-Storage|Storage.Mongo|DONE – retention options flow into `RawDocumentRetentionService` and TTL migrations for `document`/GridFS indexes.|
|Persist last failure reason in SourceState|BE-Storage|Storage.Mongo|DONE – `MongoSourceStateRepository.MarkFailureAsync` stores `lastFailureReason` with length guard + reset on success.|
|AdvisoryStore range primitives deserialization|BE-Storage|Models|DONE – BSON helpers handle `RangePrimitives`; regression test covers SemVer/NEVRA/EVR envelopes persisted through Mongo.|
|FEEDSTORAGE-DATA-03-001 Merge event provenance audit prep|BE-Storage|Merge|DONE – merge events now persist field-level decision reasons via `MergeFieldDecision` documents for analytics. **Coordination:** log any new precedence signals to storage@ so indexes/serializers stay aligned.|
|FEEDSTORAGE-DATA-02-001 Normalized range dual-write + backfill|BE-Storage|Core|**DONE (2025-10-12)** – `AdvisoryStore` honors `EnableSemVerStyle`, dual-writes normalized docs, and SemVer backfill migration registered for staged rollout.|
|FEEDSTORAGE-TESTS-02-004 Restore AdvisoryStore build after normalized versions refactor|QA|Storage.Mongo|DONE – storage tests updated to cover normalized version payloads and new provenance fields. **Heads-up:** QA to watch for fixture bumps touching normalized rule arrays when connectors roll out support.|
|FEEDSTORAGE-DATA-02-002 Provenance decision persistence|BE-Storage|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-12)** – Normalized documents carry decision reasons/source/timestamps with regression coverage verifying SemVer notes + provenance fallbacks.|
|FEEDSTORAGE-DATA-02-003 Normalized versions index creation|BE-Storage|Normalization, Mongo bootstrapper|**DONE (2025-10-12)** – Bootstrapper seeds `normalizedVersions.*` indexes when SemVer style is enabled; docs/tests confirm index presence.|
|FEEDSTORAGE-DATA-04-001 Advisory payload parity (description/CWEs/canonical metric)|BE-Storage|Models, Core|DONE (2025-10-15) – Mongo payloads round-trip new advisory fields; serializer/tests updated, no migration required beyond optional backfill.|
|FEEDSTORAGE-MONGO-08-001 Causal-consistent session plumbing|BE-Storage|Concelier Core DI|**DONE (2025-10-19)** – Scoped session provider now caches causal-consistent handles per scope, repositories accept optional sessions end-to-end, and new Mongo session consistency tests cover read-your-write + post-stepdown monotonic reads.|
|FEEDSTORAGE-DATA-07-001 Advisory statement & conflict collections|Team Normalization & Storage Backbone|FEEDMERGE-ENGINE-07-001|**DONE (2025-10-19)** – Added immutable `advisory_statements`/`advisory_conflicts` collections, bootstrapper + migration ensuring vulnerability/asOf + hash indexes, new stores (`AdvisoryStatementStore`, `AdvisoryConflictStore`), and docs outlining rollback. Tests: `dotnet test src/StellaOps.Concelier.Storage.Mongo.Tests/StellaOps.Concelier.Storage.Mongo.Tests.csproj`.|
# TASKS — Epic 1: Aggregation-Only Contract
> **AOC Reminder:** storage enforces append-only raw documents; no precedence/severity/normalization in ingestion collections.
| ID | Status | Owner(s) | Depends on | Notes |
|---|---|---|---|---|
| CONCELIER-STORE-AOC-19-001 `advisory_raw schema validator` | TODO | Concelier Storage Guild | Mongo cluster ops sign-off | Author MongoDB JSON schema enforcing required fields (`source`, `upstream`, `content`, `linkset`, `tenant`) and forbidding normalized/severity fields. Include migration toggles for staged rollout. |
| CONCELIER-STORE-AOC-19-002 `idempotency unique index` | TODO | Concelier Storage Guild | CONCELIER-STORE-AOC-19-001 | Create compound unique index on `(source.vendor, upstream.upstream_id, upstream.content_hash, tenant)` with backfill script verifying existing data, and document offline validator bootstrap. |
| CONCELIER-STORE-AOC-19-003 `append-only supersedes migration` | TODO | Concelier Storage Guild | CONCELIER-STORE-AOC-19-002 | Introduce migration that freezes legacy `advisories` writes, copies data into `_backup_*`, and backfills supersedes pointers for raw revisions. Provide rollback plan. |
| CONCELIER-STORE-AOC-19-004 `validator deployment playbook` | TODO | Concelier Storage Guild, DevOps Guild | CONCELIER-STORE-AOC-19-001 | Update `MIGRATIONS.md` and Offline Kit docs to cover enabling validators, rolling restarts, and validator smoke tests for air-gapped installs. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-POLICY-20-003 `Selection cursors` | TODO | Concelier Storage Guild | CONCELIER-STORE-AOC-19-002, POLICY-ENGINE-20-003 | Add advisory/vex selection cursors (per policy run) with change stream checkpoints, indexes, and offline migration scripts to support incremental evaluations. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-LNM-21-101 `Observations collections` | TODO | Concelier Storage Guild | CONCELIER-LNM-21-001 | Provision `advisory_observations` and `advisory_linksets` collections with hashed shard keys, TTL for ingest metadata, and required indexes (`aliases`, `purls`, `observation_ids`). |
| CONCELIER-LNM-21-102 `Migration tooling` | TODO | Concelier Storage Guild, DevOps Guild | CONCELIER-LNM-21-101 | Backfill legacy merged advisories into observation/linkset collections, create tombstones for merged docs, and supply rollback scripts. |
| CONCELIER-LNM-21-103 `Blob/store wiring` | TODO | Concelier Storage Guild | CONCELIER-LNM-21-101 | Store large raw payloads in object storage with pointers from observations; update bootstrapper/offline kit to seed sample blobs. |

114
src/StellaOps.Concelier.WebService/TASKS.md
View File

@@ -1,28 +1,86 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDWEB-EVENTS-07-001 Advisory event replay API|Concelier WebService Guild|FEEDCORE-ENGINE-07-001|**DONE (2025-10-19)** – Added `/concelier/advisories/{vulnerabilityKey}/replay` endpoint with optional `asOf`, hex hashes, and conflict payloads; integration covered via `dotnet test src/StellaOps.Concelier.WebService.Tests/StellaOps.Concelier.WebService.Tests.csproj`.|
|Bind & validate ConcelierOptions|BE-Base|WebService|DONE – options bound/validated with failure logging.|
|Mongo service wiring|BE-Base|Storage.Mongo|DONE – wiring delegated to `AddMongoStorage`.|
|Bootstrapper execution on start|BE-Base|Storage.Mongo|DONE – startup calls `MongoBootstrapper.InitializeAsync`.|
|Plugin host options finalization|BE-Base|Plugins|DONE – default plugin directories/search patterns configured.|
|Jobs API contract tests|QA|Core|DONE – WebServiceEndpointsTests now cover success payloads, filtering, and trigger outcome mapping.|
|Health/Ready probes|DevOps|Ops|DONE – `/health` and `/ready` endpoints implemented.|
|Serilog + OTEL integration hooks|BE-Base|Observability|DONE – `TelemetryExtensions` wires Serilog + OTEL with configurable exporters.|
|Register built-in jobs (sources/exporters)|BE-Base|Core|DONE – AddBuiltInConcelierJobs adds fallback scheduler definitions for core connectors and exporters via reflection.|
|HTTP problem details consistency|BE-Base|WebService|DONE – API errors now emit RFC7807 responses with trace identifiers and typed problem categories.|
|Request logging and metrics|BE-Base|Observability|DONE – Serilog request logging enabled with enriched context and web.jobs counters published via OpenTelemetry.|
|Endpoint smoke tests (health/ready/jobs error paths)|QA|WebService|DONE – WebServiceEndpointsTests assert success and problem responses for health, ready, and job trigger error paths.|
|Batch job definition last-run lookup|BE-Base|Core|DONE – definitions endpoint now precomputes kinds array and reuses batched last-run dictionary; manual smoke verified via local GET `/jobs/definitions`.|
|Add no-cache headers to health/readiness/jobs APIs|BE-Base|WebService|DONE – helper applies Cache-Control/Pragma/Expires on all health/ready/jobs endpoints; awaiting automated probe tests once connector fixtures stabilize.|
|Authority configuration parity (FSR1)|DevEx/Concelier|Authority options schema|**DONE (2025-10-10)** – Options post-config loads clientSecretFile fallback, validators normalize scopes/audiences, and sample config documents issuer/credential/bypass settings.|
|Document authority toggle & scope requirements|Docs/Concelier|Authority integration|**DONE (2025-10-21)** – Quickstart now documents staging flag, client credentials, env overrides; operator guide refresh merged. Remaining copy polishing is tracked under `DOCS-CONCELIER-07-201` in `docs/TASKS.md`.|
|Plumb Authority client resilience options|BE-Base|Auth libraries LIB5|**DONE (2025-10-12)** – `Program.cs` wires `authority.resilience.*` + client scopes into `AddStellaOpsAuthClient`; new integration test asserts binding and retries.|
|Author ops guidance for resilience tuning|Docs/Concelier|Plumb Authority client resilience options|**DONE (2025-10-12)** – `docs/21_INSTALL_GUIDE.md` + `docs/ops/concelier-authority-audit-runbook.md` document resilience profiles for connected vs air-gapped installs and reference monitoring cues.|
|Document authority bypass logging patterns|Docs/Concelier|FSR3 logging|**DONE (2025-10-12)** – Updated operator guides clarify `Concelier.Authorization.Audit` fields (route/status/subject/clientId/scopes/bypass/remote) and SIEM triggers.|
|Update Concelier operator guide for enforcement cutoff|Docs/Concelier|FSR1 rollout|**DONE (2025-10-12)** – Installation guide emphasises disabling `allowAnonymousFallback` before 2025-12-31 UTC and connects audit signals to the rollout checklist.|
|Rename plugin drop directory to namespaced path|BE-Base|Plugins|**DONE (2025-10-19)** – Build outputs now target `StellaOps.Concelier.PluginBinaries`/`StellaOps.Authority.PluginBinaries`, plugin host defaults updated, config/docs refreshed, and `dotnet test src/StellaOps.Concelier.WebService.Tests/StellaOps.Concelier.WebService.Tests.csproj --no-restore` covers the change.|
|Authority resilience adoption|Concelier WebService, Docs|Plumb Authority client resilience options|**BLOCKED (2025-10-10)** – Roll out retry/offline knobs to deployment docs and confirm CLI parity once LIB5 lands; unblock after resilience options wired and tested.|
|CONCELIER-WEB-08-201 – Mirror distribution endpoints|Concelier WebService Guild|CONCELIER-EXPORT-08-201, DEVOPS-MIRROR-08-001|**DONE (2025-10-20)** – Mirror endpoints now enforce per-domain rate limits, emit cache headers, honour Authority/WWW-Authenticate, and docs cover auth + smoke workflows.|
> Remark (2025-10-20): Updated ops runbook with token/rate-limit checks and added API tests for Retry-After + unauthorized flows.|
|Wave 0B readiness checkpoint|Team WebService & Authority|Wave 0A completion|BLOCKED (2025-10-19) – FEEDSTORAGE-MONGO-08-001 closed, but remaining Wave 0A items (AUTH-DPOP-11-001, AUTH-MTLS-11-002, PLUGIN-DI-08-001) still open; maintain current DOING workstreams only.|
# TASKS — Epic 1: Aggregation-Only Contract
> **AOC Reminder:** service links and exposes raw data only—no precedence, severity, or hint computation inside Concelier APIs.
| ID | Status | Owner(s) | Depends on | Notes |
|---|---|---|---|---|
| CONCELIER-WEB-AOC-19-001 `Raw ingestion endpoints` | TODO | Concelier WebService Guild | CONCELIER-CORE-AOC-19-001, CONCELIER-STORE-AOC-19-001 | Implement `POST /ingest/advisory`, `GET /advisories/raw*`, and `POST /aoc/verify` minimal API endpoints. Enforce new Authority scopes, inject tenant claims, and surface `AOCWriteGuard` to repository calls. |
| CONCELIER-WEB-AOC-19-002 `AOC observability` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-WEB-AOC-19-001 | Emit `ingestion_write_total`, `aoc_violation_total`, latency histograms, and tracing spans (`ingest.fetch/transform/write`, `aoc.guard`). Wire structured logging to include tenant, source vendor, upstream id, and content hash. |
| CONCELIER-WEB-AOC-19-003 `Schema/guard unit tests` | TODO | QA Guild | CONCELIER-WEB-AOC-19-001 | Add unit tests covering schema validation failures, forbidden field rejections (`ERR_AOC_001/002/006/007`), idempotent upserts, and supersedes chains using deterministic fixtures. |
| CONCELIER-WEB-AOC-19-004 `End-to-end ingest verification` | TODO | Concelier WebService Guild, QA Guild | CONCELIER-WEB-AOC-19-003, CONCELIER-CORE-AOC-19-002 | Create integration tests ingesting large advisory batches (cold/warm) validating linkset enrichment, metrics emission, and reproducible outputs. Capture load-test scripts + doc notes for Offline Kit dry runs. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-POLICY-20-001 `Policy selection endpoints` | TODO | Concelier WebService Guild | WEB-POLICY-20-001, CONCELIER-CORE-AOC-19-004 | Add batch advisory lookup APIs (`/policy/select/advisories`, `/policy/select/vex`) optimized for PURL/ID lists with pagination, tenant scoping, and explain metadata. |
## StellaOps Console (Sprint 23)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-CONSOLE-23-001 `Advisory aggregation views` | TODO | Concelier WebService Guild, BE-Base Platform Guild | CONCELIER-LNM-21-201, CONCELIER-LNM-21-202 | Expose `/console/advisories` endpoints returning aggregation groups (per linkset) with source chips, severity summaries, and provenance metadata for Console list + dashboard cards. Support filters by source, ecosystem, published/modified window, tenant enforcement. |
| CONCELIER-CONSOLE-23-002 `Dashboard deltas API` | TODO | Concelier WebService Guild | CONCELIER-CONSOLE-23-001, CONCELIER-LNM-21-203 | Provide aggregated advisory delta counts (new, modified, conflicting) for Console dashboard + live status ticker; emit structured events for queue lag metrics. Ensure deterministic counts across repeated queries. |
| CONCELIER-CONSOLE-23-003 `Search fan-out helpers` | TODO | Concelier WebService Guild | CONCELIER-CONSOLE-23-001 | Deliver fast lookup endpoints for CVE/GHSA/purl search (linksets, observations) returning evidence fragments for Console global search; implement caching + scope guards. |
## Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-GRAPH-21-003 `SBOM projection API` | TODO | Concelier WebService Guild, Cartographer Guild | CONCELIER-GRAPH-21-001 | Expose normalized SBOM projection endpoint (`/sboms/{id}/projections/graph`) with pagination, tenant guard, and versioning metadata for Cartographer builds. |
| CONCELIER-GRAPH-21-004 `Entry point registry` | TODO | Concelier WebService Guild | CONCELIER-GRAPH-21-003, SBOM-SERVICE-21-001 | Provide entrypoint/service node lookup API for Cartographer path relevance (configurable overrides, tagging) and document contract. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-LNM-21-201 `Observation APIs` | TODO | Concelier WebService Guild, BE-Base Platform Guild | CONCELIER-LNM-21-001 | Add REST endpoints for advisory observations (`GET /advisories/observations`) with filters (alias, purl, source), pagination, and tenancy enforcement. |
| CONCELIER-LNM-21-202 `Linkset APIs` | TODO | Concelier WebService Guild | CONCELIER-LNM-21-002, CONCELIER-LNM-21-003 | Implement linkset read/export endpoints (`/advisories/linksets/{id}`, `/advisories/by-purl/{purl}`, `/advisories/linksets/{id}/export`, `/evidence`) with correlation/conflict payloads and `ERR_AGG_*` mapping. |
| CONCELIER-LNM-21-203 `Ingest events` | TODO | Concelier WebService Guild, Platform Events Guild | CONCELIER-LNM-21-005 | Publish NATS/Redis events for new observations/linksets and ensure idempotent consumer contracts; document event schemas. |
## Graph & Vuln Explorer v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-GRAPH-24-101 `Advisory summary API` | TODO | Concelier WebService Guild | CONCELIER-GRAPH-24-001 | Expose `/advisories/summary` returning raw linkset/observation metadata for overlay services; no derived severity or fix hints. |
| CONCELIER-GRAPH-24-102 `Evidence batch API` | TODO | Concelier WebService Guild | CONCELIER-LNM-21-201 | Add batch fetch for advisory observations/linksets keyed by component sets to feed Graph overlay tooltips efficiently. |
## Vulnerability Explorer (Sprint 29)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-VULN-29-001 `Advisory key normalization` | TODO | Concelier WebService Guild, Data Integrity Guild | CONCELIER-LNM-21-001 | Normalize advisory identifiers (CVE/GHSA/vendor) into canonical `advisory_key`, persist `links[]`, expose raw payload snapshots for Explorer evidence tabs. Include migration/backfill scripts. |
| CONCELIER-VULN-29-002 `Evidence retrieval API` | TODO | Concelier WebService Guild | CONCELIER-VULN-29-001, VULN-API-29-003 | Provide `/vuln/evidence/advisories/{advisory_key}` returning raw advisory docs with provenance, filtering by tenant and source. |
| CONCELIER-VULN-29-004 `Observability enhancements` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-VULN-29-001 | Instrument metrics/logs for advisory normalization (key collisions, withdrawn flags), emit events consumed by Vuln Explorer resolver. |
## Advisory AI (Sprint 31)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-AIAI-31-001 `Paragraph anchors` | TODO | Concelier WebService Guild | CONCELIER-VULN-29-001 | Expose advisory chunk API returning paragraph anchors, section metadata, and token-safe text for Advisory AI retrieval. |
| CONCELIER-AIAI-31-002 `Structured fields` | TODO | Concelier WebService Guild | CONCELIER-AIAI-31-001 | Ensure normalized advisories expose workaround/fix/CVSS fields via API; add caching for summary queries. |
| CONCELIER-AIAI-31-003 `Advisory AI telemetry` | TODO | Concelier WebService Guild, Observability Guild | CONCELIER-AIAI-31-001 | Emit metrics/logs for chunk requests, cache hits, and guardrail blocks triggered by advisory payloads. |
## Observability & Forensics (Epic 15)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-WEB-OBS-50-001 `Telemetry adoption` | TODO | Concelier WebService Guild | TELEMETRY-OBS-50-001, CONCELIER-OBS-50-001 | Adopt telemetry core in web service host, ensure ingest + read endpoints emit trace/log fields (`tenant_id`, `route`, `decision_effect`), and add correlation IDs to responses. |
| CONCELIER-WEB-OBS-51-001 `Observability APIs` | TODO | Concelier WebService Guild | CONCELIER-WEB-OBS-50-001, WEB-OBS-51-001 | Surface ingest health metrics, queue depth, and SLO status via `/obs/concelier/health` endpoint for Console widgets, with caching and tenant partitioning. |
| CONCELIER-WEB-OBS-52-001 `Timeline streaming` | TODO | Concelier WebService Guild | CONCELIER-WEB-OBS-50-001, TIMELINE-OBS-52-003 | Provide SSE stream `/obs/concelier/timeline` bridging to Timeline Indexer with paging tokens, guardrails, and audit logging. |
| CONCELIER-WEB-OBS-53-001 `Evidence locker integration` | TODO | Concelier WebService Guild, Evidence Locker Guild | CONCELIER-OBS-53-001, EVID-OBS-53-003 | Add `/evidence/advisories/*` routes invoking evidence locker snapshots, verifying tenant scopes (`evidence:read`), and returning signed manifest metadata. |
| CONCELIER-WEB-OBS-54-001 `Attestation exposure` | TODO | Concelier WebService Guild | CONCELIER-OBS-54-001, PROV-OBS-54-001 | Provide `/attestations/advisories/*` read APIs surfacing DSSE status, verification summary, and provenance chain for Console/CLI. |
| CONCELIER-WEB-OBS-55-001 `Incident mode toggles` | TODO | Concelier WebService Guild, DevOps Guild | CONCELIER-OBS-55-001, WEB-OBS-55-001 | Implement incident mode toggle endpoints, propagate to orchestrator/locker, and document cooldown/backoff semantics. |
## Air-Gapped Mode (Epic 16)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-WEB-AIRGAP-56-001 `Mirror import APIs` | TODO | Concelier WebService Guild | AIRGAP-IMP-58-001, CONCELIER-AIRGAP-56-001 | Extend ingestion endpoints to register mirror bundle sources, expose bundle catalog queries, and block external feed URLs in sealed mode. |
| CONCELIER-WEB-AIRGAP-56-002 `Airgap status surfaces` | TODO | Concelier WebService Guild | CONCELIER-AIRGAP-57-002, AIRGAP-CTL-56-002 | Add staleness metadata and bundle provenance to advisory APIs (`/advisories/observations`, `/advisories/linksets`). |
| CONCELIER-WEB-AIRGAP-57-001 `Error remediation` | TODO | Concelier WebService Guild, AirGap Policy Guild | AIRGAP-POL-56-001 | Map sealed-mode violations to `AIRGAP_EGRESS_BLOCKED` responses with user guidance. |
| CONCELIER-WEB-AIRGAP-58-001 `Import timeline emission` | TODO | Concelier WebService Guild, AirGap Importer Guild | CONCELIER-WEB-AIRGAP-56-001, TIMELINE-OBS-53-001 | Emit timeline events for bundle ingestion operations with bundle ID, scope, and actor metadata. |
## SDKs & OpenAPI (Epic 17)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-WEB-OAS-61-001 `/.well-known/openapi` | TODO | Concelier WebService Guild | OAS-61-001 | Implement discovery endpoint emitting Concelier spec with version metadata and ETag. |
| CONCELIER-WEB-OAS-61-002 `Error envelope migration` | TODO | Concelier WebService Guild | APIGOV-61-001 | Ensure all API responses use standardized error envelope; update controllers/tests. |
| CONCELIER-WEB-OAS-62-001 `Examples expansion` | TODO | Concelier WebService Guild | CONCELIER-OAS-61-002 | Add curated examples for advisory observations/linksets/conflicts; integrate into dev portal. |
| CONCELIER-WEB-OAS-63-001 `Deprecation headers` | TODO | Concelier WebService Guild, API Governance Guild | APIGOV-63-001 | Add Sunset/Deprecation headers for retiring endpoints and update documentation/notifications. |

14
src/StellaOps.Cryptography.Kms/AGENTS.md Normal file
View File

@@ -0,0 +1,14 @@
# KMS & Key Management Guild Charter
## Mission
Provide key management abstractions and drivers (file, cloud KMS, HSM, FIDO2) for signing and verification workflows.
## Scope
- Key store interfaces, secure configuration loading, and audit logging.
- Drivers for file-based development keys, cloud KMS providers, PKCS#11 HSMs, and FIDO2 devices.
- Key rotation, revocation, and attestation for keys used in signing.
## Definition of Done
- KMS API supports signing, verification, key metadata, rotation, and revocation.
- Drivers pass integration tests and security review.
- CLI/Console can manage keys using these abstractions.

13
src/StellaOps.Cryptography.Kms/TASKS.md Normal file
View File

@@ -0,0 +1,13 @@
# KMS Task Board — Epic 19: Attestor Console
## Sprint 72 – Abstractions & File Driver
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| KMS-72-001 | TODO | KMS Guild | — | Implement KMS interface (sign, verify, metadata, rotate, revoke) and file-based key driver with encrypted at-rest storage. | Interface + file driver operational; unit tests cover sign/verify/rotation; lint passes. |
| KMS-72-002 | TODO | KMS Guild | KMS-72-001 | Add CLI support for importing/exporting file-based keys with password protection. | CLI commands functional; docs updated; integration tests pass. |
## Sprint 73 – Cloud & HSM Integration
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| KMS-73-001 | TODO | KMS Guild | KMS-72-001 | Add cloud KMS driver (e.g., AWS KMS, GCP KMS) with signing and key metadata retrieval. | Cloud driver tested with mock; configuration documented; security review sign-off. |
| KMS-73-002 | TODO | KMS Guild | KMS-72-001 | Implement PKCS#11/HSM driver plus FIDO2 signing support for high assurance workflows. | HSM/FIDO2 drivers tested with hardware stubs; error handling documented. |

Some files were not shown because too many files have changed in this diff Show More