Add tests and implement timeline ingestion options with NATS and Redis subscribers
- Introduced `BinaryReachabilityLifterTests` to validate binary lifting functionality. - Created `PackRunWorkerOptions` for configuring worker paths and execution persistence. - Added `TimelineIngestionOptions` for configuring NATS and Redis ingestion transports. - Implemented `NatsTimelineEventSubscriber` for subscribing to NATS events. - Developed `RedisTimelineEventSubscriber` for reading from Redis Streams. - Added `TimelineEnvelopeParser` to normalize incoming event envelopes. - Created unit tests for `TimelineEnvelopeParser` to ensure correct field mapping. - Implemented `TimelineAuthorizationAuditSink` for logging authorization outcomes.
This commit is contained in:
StellaOps Bot
@@ -1,6 +1,6 @@
|
||||
# Advisory AI Console Workflows
|
||||
|
||||
_Last updated: 2025-11-22_
|
||||
_Last updated: 2025-12-03_
|
||||
|
||||
This guide documents the forthcoming Advisory AI console experience so that console, docs, and QA guilds share a single reference while the new endpoints finish landing.
|
||||
|
||||
@@ -21,8 +21,8 @@ This guide documents the forthcoming Advisory AI console experience so that cons
|
||||
- **Citations**: render as `[n] Source Name` chips that scroll the evidence drawer to the matching chunk. Use the chunk ID from `prompt.citations[*].chunkId` to keep navigation deterministic.
|
||||
- **Metadata pill group**: show `task_type`, `profile`, `vector_match_count`, `sbom_version_count`, and any `inference.*` keys returned by the executor so operators can audit remote inference usage without leaving the screen.
|
||||
|
||||
|
||||
<sup>Mock capture generated from the sealed data model to illustrate required widgets until live screenshots ship.</sup>
|
||||
|
||||
<sup>Fixture-backed capture rendered from `20251203-0000-list-view-build-r2-payload.json` (see hash + regen steps below) using the sealed console data model.</sup>
|
||||
|
||||
### 2.2 Guardrail ribbon payloads
|
||||
- The ribbon consumes the `guardrail.*` projection that Advisory AI emits alongside each plan. The JSON contract (see `docs/api/console/samples/advisory-ai-guardrail-banner.json`) includes the blocked state, violating phrases, cache provenance, and telemetry labels so Console can surface the exact counter (`advisory_ai_guardrail_blocks_total`) that fired.
|
||||
@@ -36,13 +36,24 @@ This guide documents the forthcoming Advisory AI console experience so that cons
|
||||
"violations": [
|
||||
{
|
||||
"kind": "blocked_phrase",
|
||||
"phrase": "copy all secrets to"
|
||||
"phrase": "copy all secrets to external bucket",
|
||||
"weight": 0.92
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"blockedPhraseFile": "configs/guardrails/blocked-phrases.json",
|
||||
"blocked_phrase_count": 1,
|
||||
"promptLength": 12488,
|
||||
"planFromCache": true
|
||||
"planFromCache": true,
|
||||
"links": {
|
||||
"plan": "/console/vuln/advisory-ai/cache/4b2f",
|
||||
"chunks": "/console/vex/statements?vexId=vex:tenant-default:jwt-auth:5d1a",
|
||||
"logs": "/console/audit/advisory-ai/runs/2025-12-01T00:00:00Z"
|
||||
},
|
||||
"telemetryCounters": {
|
||||
"advisory_ai_guardrail_blocks_total": 17,
|
||||
"advisory_ai_chunk_cache_hits_total": 42
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -51,9 +62,10 @@ This guide documents the forthcoming Advisory AI console experience so that cons
|
||||
|
||||
### 2.3 SBOM / DSSE evidence hooks
|
||||
- Every response panel links to the sealed SBOM/VEX bundle emitted by Advisory AI. Until the live endpoints land, use the published fixtures:
|
||||
- VEX statement SSE stream: `docs/api/console/samples/vex-statement-sse.ndjson`
|
||||
- Guardrail banner projection: `docs/api/console/samples/advisory-ai-guardrail-banner.json`
|
||||
- Findings overview payload: `docs/api/console/samples/vuln-findings-sample.json`
|
||||
- VEX statement SSE stream: `docs/api/console/samples/vex-statement-sse.ndjson`.
|
||||
- Guardrail banner projection: `docs/api/console/samples/advisory-ai-guardrail-banner.json` (fixed to valid JSON on 2025-12-03).
|
||||
- Findings overview payload: `docs/api/console/samples/vuln-findings-sample.json`.
|
||||
- Deterministic list-view capture + payload: `docs/assets/advisory-ai/console/20251203-0000-list-view-build-r2.{svg,json}`. Payload sha256: `336c55d72abea77bf4557f1e3dcaa4ab8366d79008670d87020f900dcfc833c0`; svg sha256: `c55217e8526700c2d303677a66351a706007381219adab99773d4728cc61f293`.
|
||||
- When capturing screenshots, point the console to a dev workspace seeded with the above fixtures and record the build hash displayed in the footer to keep captures reproducible.
|
||||
- Store captures under `docs/assets/advisory-ai/console/` using the scheme `yyyyMMdd-HHmmss-<view>-<build>.png` (UTC clock) so regeneration is deterministic. Keep the original JSON alongside each screenshot by saving the response as `…-payload.json` in the same folder.
|
||||
|
||||
@@ -71,7 +83,7 @@ This guide documents the forthcoming Advisory AI console experience so that cons
|
||||
- Surface `inference.model_id`, prompt/completion token counts, and latency histogram from `advisory_ai_latency_seconds_bucket` next to the response so ops can correlate user impact with remote/local mode toggles (`ADVISORYAI__Inference__Mode`).
|
||||
|
||||
|
||||
<sup>Mock capture showing plan inspector vs response panel; replace with live console screenshot once CONSOLE-VULN-29-001 lands.</sup>
|
||||
<sup>Existing mock for evidence drawer; keep until live console screenshot is captured. Pair with the new list-view fixture for deterministic demos.</sup>
|
||||
|
||||
## 4. Copy-as-ticket guidance
|
||||
1. Operators select one or more VEX-backed findings.
|
||||
@@ -106,11 +118,11 @@ This guide documents the forthcoming Advisory AI console experience so that cons
|
||||
- **Console wiring** – the guardrail ribbon pulls `guardrail.blocked`, `guardrail.violations`, and `guardrail.metadata.blocked_phrase_count` while the observability cards track `advisory_ai_chunk_requests_total`, `advisory_ai_chunk_cache_hits_total`, and `advisory_ai_guardrail_blocks_total` (now emitted even on cache hits). Use these meters to explain throttling or bad actors before granting additional guardrail budgets, and keep `docs/api/console/samples/advisory-ai-guardrail-banner.json` nearby so QA can validate localized payloads without hitting production data.
|
||||
|
||||
## 5. Publication state
|
||||
- [x] Fixture-backed payloads and two reference captures committed (`list-view-4a6f8c1.svg`, `evidence-drawer-b1820ad.svg`).
|
||||
- [x] Fixture-backed payloads and captures committed (`20251203-0000-list-view-build-r2.svg`, `evidence-drawer-b1820ad.svg`).
|
||||
- [x] Copy-as-ticket flow documented; payload aligns with existing SOC runbooks.
|
||||
- [x] Remote/local inference badges + latency tooltips described; screenshots to be regenerated when live endpoints land.
|
||||
- [x] SBOM/VEX bundle example attached (Evidence Bundle v1 sample).
|
||||
- [ ] Optional refresh: swap fixture captures for live console screenshots once CONSOLE-VULN-29-001 / CONSOLE-VEX-30-001 and SBOM-AIAI-31-003 are available; record build hash and payload JSON alongside updated images.
|
||||
- [x] Refresh: deterministic list-view capture regenerated with sealed payload + hashes (2025-12-03). Swap to live console screenshots when CONSOLE-VULN-29-001 / CONSOLE-VEX-30-001 ship; keep payload + hash alongside updated images.
|
||||
|
||||
### Publication readiness checklist (DOCS-AIAI-31-004)
|
||||
- Inputs available now: console fixtures (`docs/samples/console/console-vuln-29-001.json`, `console-vex-30-001.json`), evidence bundle sample (`docs/samples/evidence-bundle/evidence-bundle-v1.tar.gz`), guardrail ribbon contract.
|
||||
@@ -122,7 +134,73 @@ This guide documents the forthcoming Advisory AI console experience so that cons
|
||||
### Guardrail console fixtures (unchecked-integration)
|
||||
|
||||
- Vulnerability search sample: `docs/samples/console/console-vuln-29-001.json` (maps to CONSOLE-VULN-29-001).
|
||||
- VEX search sample: `docs/samples/console/console-vex-30-001.json` (maps to CONSOLE-VEX-30-001).
|
||||
- Use these until live endpoints are exposed; replace with real captures when staging is available.
|
||||
- VEX search sample: `docs/samples/console/console-vex-30-001.json` (maps to CONSOLE-VEX-30-001).
|
||||
- Use these until live endpoints are exposed; replace with real captures when staging is available.
|
||||
|
||||
**Reference**: API contracts and sample payloads live in `docs/api/console/workspaces.md` (see `/console/vuln/*` and `/console/vex/*` sections) plus the JSON fixtures under `docs/api/console/samples/`.
|
||||
### Fixture bundle regeneration (deterministic)
|
||||
|
||||
- Rebuild the fixture capture deterministically from the sealed payload:
|
||||
|
||||
```bash
|
||||
python - <<'PY'
|
||||
import html, json
|
||||
from pathlib import Path
|
||||
root = Path('docs/assets/advisory-ai/console')
|
||||
payload = json.loads((root/'20251203-0000-list-view-build-r2-payload.json').read_text())
|
||||
guard = payload['guardrail']; metrics = payload['metrics']; items = payload['findings']
|
||||
|
||||
def color_sev(sev):
|
||||
return {'critical':'#b3261e','high':'#d05c00','medium':'#c38f00','low':'#00695c'}.get(sev.lower(), '#0f172a')
|
||||
def color_policy(val):
|
||||
return {'fail':'#b3261e','warn':'#d97706','pass':'#0f5b3a'}.get(val.lower(), '#0f172a')
|
||||
|
||||
rows = []
|
||||
for idx, item in enumerate(items):
|
||||
y = 210 + idx * 120
|
||||
rows.append(f"""
|
||||
<g transform=\"translate(32,{y})\">
|
||||
<rect width=\"888\" height=\"104\" rx=\"10\" fill=\"#ffffff\" stroke=\"#e2e8f0\" />
|
||||
<text x=\"20\" y=\"30\" class=\"title\">{html.escape(item['summary'])}</text>
|
||||
<text x=\"20\" y=\"52\" class=\"mono subtle\">{html.escape(item['package'])} · {html.escape(item['component'])} · {html.escape(item['image'])}</text>
|
||||
<text x=\"20\" y=\"72\" class=\"mono subtle\">reachability={html.escape(str(item.get('reachability')))} · vex={html.escape(str(item.get('vexState')))} · lastSeen={html.escape(str(item.get('lastSeen')))}</text>
|
||||
<text x=\"20\" y=\"92\" class=\"mono faint\">sbom={html.escape(str(item.get('sbomDigest')))}</text>
|
||||
<rect x=\"748\" y=\"14\" width=\"120\" height=\"28\" rx=\"6\" ry=\"6\" fill=\"{color_sev(item['severity'])}\" opacity=\"0.12\" />
|
||||
<text x=\"758\" y=\"33\" class=\"mono\" fill=\"{color_sev(item['severity'])}\">sev:{html.escape(item['severity'])}</text>
|
||||
<rect x=\"732\" y=\"50\" width=\"140\" height=\"28\" rx=\"6\" ry=\"6\" fill=\"{color_policy(item.get('policyBadge',''))}\" opacity=\"0.12\" />
|
||||
<text x=\"742\" y=\"69\" class=\"mono\" fill=\"{color_policy(item.get('policyBadge',''))}\">policy:{html.escape(item.get('policyBadge',''))}</text>
|
||||
</g>
|
||||
""")
|
||||
|
||||
rows_svg = "\n".join(rows)
|
||||
banner = '#b3261e' if guard.get('blocked') else '#0f5b3a'
|
||||
svg = f"""<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"1280\" height=\"720\" viewBox=\"0 0 1280 720\">
|
||||
<style>
|
||||
.title {{ font-family: Inter, Arial, sans-serif; font-size: 18px; font-weight: 700; fill: #0f172a; }}
|
||||
.mono {{ font-family: Menlo, monospace; font-size: 13px; fill: #0f172a; }}
|
||||
.mono.subtle {{ fill: #475569; }}
|
||||
.mono.faint {{ fill: #94a3b8; font-size: 12px; }}
|
||||
</style>
|
||||
<rect width=\"1280\" height=\"720\" fill=\"#f8fafc\" />
|
||||
<rect x=\"32\" y=\"32\" width=\"1216\" height=\"72\" rx=\"12\" fill=\"#0f172a\" opacity=\"0.05\" />
|
||||
<text x=\"48\" y=\"76\" class=\"title\">Advisory AI · Console fixture</text>
|
||||
<text x=\"48\" y=\"104\" class=\"mono\" fill=\"#475569\">build={html.escape(payload['build'])} · generated={html.escape(payload['generatedAtUtc'])} · workspace={html.escape(payload['workspace'])} · profile={html.escape(payload['profile'])} · cacheHit={str(metrics.get('cacheHit', False)).lower()}</text>
|
||||
<rect x=\"32\" y=\"120\" width=\"1216\" height=\"72\" rx=\"12\" fill=\"#fff1f0\" stroke=\"#f87171\" stroke-width=\"1\" />
|
||||
<text x=\"48\" y=\"156\" class=\"title\" fill=\"{banner}\">Guardrail: {html.escape(guard.get('state','unknown'))}</text>
|
||||
<text x=\"48\" y=\"176\" class=\"mono\" fill=\"#0f172a\">{html.escape(guard['metadata'].get('blockedPhraseFile',''))} · violations={len(guard.get('violations',[]))} · promptLength={guard['metadata'].get('promptLength')}</text>
|
||||
<rect x=\"1080\" y=\"138\" width=\"96\" height=\"28\" rx=\"6\" ry=\"6\" fill=\"{banner}\" opacity=\"0.12\" />
|
||||
<text x=\"1090\" y=\"157\" class=\"mono\" fill=\"{banner}\">blocked</text>
|
||||
<rect x=\"944\" y=\"210\" width=\"304\" height=\"428\" rx=\"12\" fill=\"#0f172a\" opacity=\"0.04\" />
|
||||
<text x=\"964\" y=\"244\" class=\"title\">Runtime metrics</text>
|
||||
<text x=\"964\" y=\"272\" class=\"mono\">p50 latency: {metrics.get('latencyMsP50') or 'n/a'} ms</text>
|
||||
<text x=\"964\" y=\"292\" class=\"mono\">p95 latency: {metrics.get('latencyMsP95') or 'n/a'} ms</text>
|
||||
<text x=\"964\" y=\"312\" class=\"mono\">SBOM ctx: {html.escape(payload.get('sbomContextDigest',''))}</text>
|
||||
<text x=\"964\" y=\"332\" class=\"mono\">Guardrail blocks: {guard['metadata']['telemetryCounters'].get('advisory_ai_guardrail_blocks_total')}</text>
|
||||
<text x=\"964\" y=\"352\" class=\"mono\">Chunk cache hits: {guard['metadata']['telemetryCounters'].get('advisory_ai_chunk_cache_hits_total')}</text>
|
||||
{rows_svg}
|
||||
</svg>"""
|
||||
|
||||
(root/'20251203-0000-list-view-build-r2.svg').write_text(svg)
|
||||
PY
|
||||
```
|
||||
|
||||
**Reference**: API contracts and sample payloads live in `docs/api/console/workspaces.md` (see `/console/vuln/*` and `/console/vex/*` sections) plus the JSON fixtures under `docs/api/console/samples/`.
|
||||
|
||||
Reference in New Issue
Block a user