docs consolidation

This commit is contained in:
StellaOps Bot
2025-12-24 12:38:14 +02:00
parent 7503c19b8f
commit 9a08d10b89
215 changed files with 2188 additions and 9623 deletions

View File

@@ -7,7 +7,7 @@
- Deterministic pagination (`continuationToken`), stable sorting, and explicit audit trails.
## Headers
- `X-StellaOps-Tenant` (required)
- `X-Stella-Tenant` (required; legacy alias: `X-StellaOps-Tenant`)
- `X-Stella-Project` (optional)
- `X-Stella-Trace-Id` (required)
- `X-Stella-Request-Id` (required; defaults to trace ID)

View File

@@ -1,58 +1,57 @@
# Console Search & Downloads · Draft v0.2
# Console Search and Downloads
Scope: unblock WEB-CONSOLE-23-004/005 by defining deterministic ranking, caching rules, and the download manifest structure (including signed metadata option) for console search and offline bundle downloads. Final guild sign-off still required.
This document defines deterministic ranking, caching rules, and the download manifest shape used by Console search and export-style workflows.
## 1) Deterministic search ranking
- Primary sort: `severity (desc)` `exploitScore (desc)` `reachability (reachable > unknown > unreachable)` `policyBadge (fail > warn > pass > waived)` `vexState (under_investigation > fixed > not_affected > unknown)` `findingId (asc)`.
- Secondary tie-breakers (when above fields absent): `advisoryId (asc)` then `product (asc)`.
- All pages are pre-sorted server-side; clients MUST NOT re-order.
- Primary sort: `severity (desc)` then `exploitScore (desc)` then `reachability (reachable > unknown > unreachable)` then `policyBadge (fail > warn > pass > waived)` then `vexState (under_investigation > fixed > not_affected > unknown)` then `findingId (asc)`.
- Secondary tie-breakers (when primary fields are absent): `advisoryId (asc)` then `product (asc)`.
- Pages are pre-sorted server-side; clients MUST NOT re-order results.
## 2) Caching + freshness
- Response headers: `Cache-Control: public, max-age=300, stale-while-revalidate=60, stale-if-error=300`.
- `ETag` is a stable SHA-256 over the sorted payload; clients send `If-None-Match` for revalidation.
- `Last-Modified` reflects the newest `updatedAt` in the result set.
- Retry/backoff guidance: honor `Retry-After` when present; default client backoff `1s,2s,4s,8s` capped at 30s.
- Deterministic page cursors: opaque base64url, signed; include `sortKeys` and `tenant` to avoid cross-tenant reuse.
## 2) Caching and freshness
- Response headers (recommended defaults): `Cache-Control: private, max-age=300, stale-while-revalidate=60, stale-if-error=300`.
- `ETag` SHOULD be a stable SHA-256 over a canonicalized, sorted payload so identical inputs produce identical validators; clients send `If-None-Match` for revalidation.
- `Last-Modified` SHOULD reflect the newest `updatedAt` present in the result set.
- Retry/backoff: honor `Retry-After` when present; otherwise use deterministic client backoff (for example `1s, 2s, 4s, 8s` capped at 30s).
- Page cursors: opaque base64url, signed; bind to `tenant` and effective sort keys to avoid cross-tenant or cross-sort reuse.
## 3) Download manifest (for `/console/downloads` and export outputs)
Top-level:
```jsonc
{
"version": "2025-12-07",
"exportId": "console-export::tenant-default::2025-12-07::0009",
"tenantId": "tenant-default",
"generatedAt": "2025-12-07T10:15:00Z",
"items": [
{
"type": "vuln", // advisory|vex|policy|scan|chart|bundle
"id": "CVE-2024-12345",
"format": "json",
"url": "https://downloads.local/exports/0009/vuln/CVE-2024-12345.json?sig=...",
"sha256": "f1c5…",
"size": 18432
}
],
"checksums": {
"manifest": "sha256:8bbf…",
"bundle": "sha256:12ae…" // optional when a tar/zip bundle is produced
},
"expiresAt": "2025-12-14T10:15:00Z"
}
```
## 3) Download manifest
When a Console workflow produces downloadable artifacts (individual documents or a bundle), services can return a manifest describing:
- What items are available,
- Where to download them (signed URLs or gateway-relative links),
- How to verify them offline (checksums and optional signature metadata).
### 3.1 Signed metadata
- Optional DSSE envelope for `checksums.manifest`, using `sha256` digest and `application/json` payload type `stellaops.console.manifest`.
- Envelope is attached as `manifest.dsse` or provided via `Link: <...>; rel="alternate"; type="application/dsse+json"`.
- Signers: Authority-issued short-lived key scoped to `console:export`.
Top-level fields:
- `version` (string): manifest schema/version label used by the producer.
- `exportId` (string): stable export identifier (content-addressable or run-id style).
- `tenantId` (string): tenant scope for this export.
- `generatedAt` (RFC 3339 / ISO-8601 UTC): generation timestamp.
- `items[]` (array): downloadable artifacts.
- `checksums` (object): checksum(s) for the manifest itself and optional bundle artifact.
- `expiresAt` (RFC 3339 / ISO-8601 UTC): optional expiry for signed URLs.
### 3.2 Error handling
- Known error codes: `ERR_CONSOLE_DOWNLOAD_INVALID_CURSOR`, `ERR_CONSOLE_DOWNLOAD_EXPIRED`, `ERR_CONSOLE_DOWNLOAD_RATE_LIMIT`, `ERR_CONSOLE_DOWNLOAD_UNAVAILABLE`.
- On error, respond with deterministic JSON body including `requestId` and `retryAfterSeconds` when applicable.
Item fields:
- `type` (string): artifact type (for example `vuln`, `vex`, `policy`, `scan`, `bundle`).
- `id` (string): stable identifier within the type (CVE id, statement id, export id, etc).
- `format` (string): `json`, `ndjson`, `tar.gz`, etc.
- `url` (string): download URL (often signed).
- `sha256` (string): lowercase hex digest of the bytes at `url`.
- `size` (int): byte size.
## 4) Sample manifest
- `docs/api/console/samples/console-download-manifest.json` illustrates the exact shape above.
### Signed metadata (optional)
Producers MAY ship a DSSE envelope for the manifest checksum:
- Payload type: `stellaops.console.manifest`
- Media type: `application/json`
- Distribution: alongside the manifest (for example `manifest.dsse`) or via a link relation (for example `rel=\"alternate\"` with `type=\"application/dsse+json\"`).
## 4) Error handling
Known error codes for download/manifest flows:
- `ERR_CONSOLE_DOWNLOAD_INVALID_CURSOR`
- `ERR_CONSOLE_DOWNLOAD_EXPIRED`
- `ERR_CONSOLE_DOWNLOAD_RATE_LIMIT`
- `ERR_CONSOLE_DOWNLOAD_UNAVAILABLE`
On error, return the standard error envelope with deterministic fields (stable code/message, plus request/trace identifiers).
## 5) Samples and fixtures
- Download manifest example: `docs/api/console/samples/console-download-manifest.json`
## 5) Open items for guild sign-off
- Final TTL values for `max-age` and `stale-*`.
- Whether DSSE envelope is mandatory for sealed tenants.
- Maximum bundle size / item count caps (proposal: 1000 items, 500 MiB compressed per export).

View File

@@ -1,31 +1,29 @@
# Console Workspaces API
_Tracking: CONSOLE-VULN-29-001, CONSOLE-VEX-30-001, DOCS-AIAI-31-004_
## 1. Goals and scope
## 1. Goals & Scope
Console workspaces provide tenant-scoped, read-only aggregates for operators and automation:
The console workspaces provide read-only aggregates for Advisory AI operators:
- `/console/vuln/*` surfaces tenant-scoped findings annotated with policy verdicts, VEX justifications, Scheduler reachability signals, and Advisory AI rationale.
- `/console/vex/*` streams the underlying VEX statements, conflicts, and justification summaries (with SSE support for live updates).
- `/console/vuln/*` surfaces findings annotated with policy verdicts, VEX context, reachability signals, and evidence pointers.
- `/console/vex/*` streams underlying VEX statements and summaries (optionally via SSE).
All endpoints MUST:
1. Remain deterministic (stable sort keys, ISO-8601 UTC timestamps, stable identifiers).
2. Enforce tenant isolation for every request.
3. Be offline-friendly by supporting export flows and fixture-based operation in air-gapped environments.
1. Remain deterministic offline (stable sort keys, ISO-8601 UTC timestamps, hashed assets).
2. Operate with Authority-issued DPoP or mTLS client credentials that include `console:read` and either `vuln:read` or `vex:read`.
3. Respect tenant isolation every request carries `X-StellaOps-Tenant`.
## 2. Shared Request/Response Conventions
## 2. Shared request/response conventions
| Requirement | Description |
| --- | --- |
| Headers | `Authorization: DPoP <token>`, `DPoP: <proof>`, `X-StellaOps-Tenant: <tenantId>`, `Accept: application/json` (or `text/event-stream` for SSE). |
| Pagination | Cursor-based via `pageToken`; defaults to 50 items, max 200. Cursors are opaque, base64url, signed. |
| Auth headers | `Authorization: Bearer <token>` and optional proof headers (e.g., `DPoP: <proof>`) depending on deployment profile. |
| Tenant header | Required. See `docs/api/gateway/tenant-auth.md`; prefer `X-Stella-Tenant` (legacy alias: `X-StellaOps-Tenant`). |
| Pagination | Cursor-based via `pageToken`; defaults to 50 items, max 200. Cursors are opaque, base64url, and signed. |
| Sorting | Findings sorted by `(severity desc, exploitScore desc, findingId asc)`. Statements sorted by `(lastUpdated desc, statementId asc)`. |
| Dates | RFC 3339 / ISO-8601 UTC (e.g., `2025-11-08T12:02:11Z`). |
| Determinism | All arrays must be pre-sorted; no server-generated uuids in responses. |
| Dates | RFC 3339 / ISO-8601 UTC (e.g., `2025-01-02T03:04:05Z`). |
| Determinism | Arrays are pre-sorted; no server-generated random UUIDs in responses. |
## 3. Vulnerability Workspace (`/console/vuln/*`)
## 3. Vulnerability workspace (`/console/vuln/*`)
### 3.1 `GET /console/vuln/findings`
@@ -35,14 +33,14 @@ Query parameters:
| --- | --- | --- |
| `pageToken` | string | Optional cursor from previous response. |
| `pageSize` | int | 1-200, default 50. |
| `severity` | string[] | Accepts `critical`, `high`, `medium`, `low`, `info`. |
| `severity` | string[] | `critical`, `high`, `medium`, `low`, `info`. |
| `product` | string[] | SBOM `purl` or image digest anchors. |
| `policyBadge` | string[] | `pass`, `warn`, `fail`, `waived`. |
| `vexState` | string[] | `not_affected`, `fixed`, `under_investigation`, etc. |
| `reachability` | string[] | `reachable`, `unreachable`, `unknown`. |
| `search` | string | Substring match on CVE/GHSA/KEV ID (case-insensitive). |
| `search` | string | Substring match on CVE/GHSA/KEV id (case-insensitive). |
Response body:
Response body (example):
```jsonc
{
@@ -63,21 +61,21 @@ Response body:
"vex": {
"statementId": "vex:tenant-default:jwt-auth:5d1a",
"state": "under_investigation",
"justification": "Advisory AI flagged reachable path via Scheduler run 42."
"justification": "Operator triage pending."
},
"reachability": {
"status": "reachable",
"lastObserved": "2025-11-07T23:11:04Z",
"signalsVersion": "signals-2025.310.1"
"lastObserved": "2025-01-02T03:04:05Z",
"signalsVersion": "signals-<version>"
},
"evidence": {
"sbomDigest": "sha256:6c81",
"policyRunId": "policy-run::2025-11-07::ca9f",
"attestationId": "dsse://authority/attest/84a2"
"sbomDigest": "sha256:6c81deadbeef...",
"policyRunId": "policy-run::<id>",
"attestationId": "dsse://authority/attest/<id>"
},
"timestamps": {
"firstSeen": "2025-10-31T04:22:18Z",
"lastSeen": "2025-11-07T23:16:51Z"
"firstSeen": "2025-01-01T00:00:00Z",
"lastSeen": "2025-01-02T03:05:06Z"
}
}
],
@@ -102,77 +100,17 @@ Response body:
```
### 3.2 `GET /console/vuln/facets`
Returns the full facet catalog (counts by severity, product, policy badge, VEX state, reachability, KEV flag). Designed for sidebar filters without paging; identical parameter surface as `/findings`.
### 3.3 `GET /console/vuln/{findingId}`
Returns the full finding document, including evidence timeline, policy overlays, and export-ready metadata:
```jsonc
{
"findingId": "tenant-default:advisory-ai:sha256:5d1a",
"details": {
"description": "jsonwebtoken <10.0.0 allows algorithm downgrade.",
"references": [
"https://nvd.nist.gov/vuln/detail/CVE-2024-12345",
"https://github.com/auth0/node-jsonwebtoken/security/advisories/GHSA-45mw-4jw3-g2wg"
],
"exploitAvailability": "known_exploit"
},
"policyBadges": [
{
"policyId": "policy://tenant-default/runtime-hardening",
"verdict": "fail",
"explainUrl": "https://console.local/policy/runs/policy-run::2025-11-07::ca9f"
}
],
"vex": {
"statementId": "vex:tenant-default:jwt-auth:5d1a",
"state": "under_investigation",
"justification": "Runtime telemetry confirmed exploitation path.",
"impactStatement": "Token exchange service remains exposed until patch 2025.11.2.",
"remediations": [
{
"type": "patch",
"description": "Upgrade jwt-auth-service to 2025.11.2.",
"deadline": "2025-11-12T00:00:00Z"
}
]
},
"reachability": {
"status": "reachable",
"callPathSamples": [
"api-gateway -> jwt-auth-service -> jsonwebtoken.verify"
],
"lastUpdated": "2025-11-07T23:11:04Z"
},
"evidence": {
"sbom": {
"digest": "sha256:6c81…",
"componentPath": [
"/src/jwt-auth/package.json",
"/src/jwt-auth/node_modules/jsonwebtoken"
]
},
"attestations": [
{
"type": "scan-report",
"attestationId": "dsse://authority/attest/84a2",
"signer": "attestor@stella-ops.org",
"bundleDigest": "sha256:e2bb…"
}
]
},
"timestamps": {
"firstSeen": "2025-10-31T04:22:18Z",
"lastSeen": "2025-11-07T23:16:51Z",
"vexLastUpdated": "2025-11-07T23:10:09Z"
}
}
```
Returns a full finding document, including evidence timeline, policy overlays, and export-ready metadata.
### 3.4 `POST /console/vuln/tickets`
Create ticket/export payloads in a deterministic, auditable way.
```jsonc
POST /console/vuln/tickets
{
@@ -189,37 +127,7 @@ POST /console/vuln/tickets
}
```
Response:
```jsonc
{
"ticketId": "console-ticket::tenant-default::2025-11-08::00018",
"payload": {
"version": "2025-11-01",
"tenant": "tenant-default",
"findings": [
{ "findingId": "tenant-default:advisory-ai:sha256:5d1a", "severity": "high" },
{ "findingId": "tenant-default:advisory-ai:sha256:9bf4", "severity": "critical" }
],
"policyBadge": "fail",
"vexSummary": "2 reachable findings pending patch.",
"attachments": [
{
"type": "json",
"name": "console-ticket-20251108.json",
"digest": "sha256:1fdd…",
"contentType": "application/json",
"expiresAt": "2025-11-15T00:00:00Z"
}
]
},
"auditEventId": "console.ticket.export::2025-11-08::00018"
}
```
Requests emit `console.ticket.export` audit events (tenant, user, selection counts, target system).
## 4. VEX Workspace (`/console/vex/*`)
## 4. VEX workspace (`/console/vex/*`)
### 4.1 `GET /console/vex/statements`
@@ -243,17 +151,14 @@ Response (paged JSON):
"product": "registry.local/ops/auth:2025.10.0",
"status": "under_investigation",
"justification": "exploit_observed",
"lastUpdated": "2025-11-07T23:10:09Z",
"lastUpdated": "2025-01-02T03:04:05Z",
"source": {
"type": "advisory_ai",
"modelBuild": "aiai-console-2025-10-28",
"modelBuild": "aiai-console-<build>",
"confidence": 0.74
},
"links": [
{
"rel": "finding",
"href": "/console/vuln/findings/tenant-default:advisory-ai:sha256:5d1a"
}
{ "rel": "finding", "href": "/console/vuln/findings/tenant-default:advisory-ai:sha256:5d1a" }
]
}
],
@@ -261,150 +166,16 @@ Response (paged JSON):
}
```
When `Accept: text/event-stream`, the endpoint emits events (see §4.3) instead of paged JSON.
### 4.2 SSE streaming
### 4.2 `GET /console/vex/statements/{statementId}`
Some deployments expose live updates as SSE (`text/event-stream`). When enabled, stream payloads SHOULD be valid NDJSON and remain deterministic for a given event id.
Returns the canonical statement plus provenance extracts. SSE clients can call this endpoint when they need full bodies after receiving a summary event.
## 5. Samples and fixtures
### 4.3 `GET /console/vex/events` (SSE)
Deterministic samples live under `docs/api/console/samples/` and are used by documentation and offline validation:
- `docs/api/console/samples/vuln-findings-sample.json`
- `docs/api/console/samples/vex-statement-sse.ndjson`
- `docs/api/console/samples/console-export-manifest.json`
- `docs/api/console/samples/exception-schema-sample.json`
Streams live updates for VEX statements affecting the tenant:
- Event types: `statement.created`, `statement.updated`, `statement.deleted`, `statement.conflict`.
- Fields: `id`, `advisoryId`, `product`, `vexState`, `severityHint`, `policyBadge`, `conflictSummary`, `sequence`.
- Replay: Clients include `Last-Event-ID`; server resumes from sequence.
- Heartbeats every 15 seconds (`event: keepalive`, `data: {}`).
Example event payload:
```jsonc
event: statement.updated
data: {
"statementId": "vex:tenant-default:jwt-auth:5d1a",
"advisoryId": "CVE-2024-12345",
"product": "registry.local/ops/auth:2025.10.0",
"state": "fixed",
"justification": "solution_available",
"sequence": 4182,
"updatedAt": "2025-11-08T11:44:32Z"
}
```
## 5. Signals & Scheduler Integration
- Reachability data is materialized by Scheduler delta jobs (`SCHED-CONSOLE-23-001`). `/console/vuln/findings` should cache the most recent job ID and expose `signalsVersion`.
- VEX justification fields reference Excititor statement IDs; ensure the gateway checks Excititor availability and degrades gracefully (returns `state: unavailable` plus telemetry).
- Scheduler must publish `console.vuln.refresh` events whenever advisory/VEX deltas warrant workspace refresh; console SSE endpoint may piggyback on the same Redis/NATS channel.
## 6. Determinism & Offline Notes
1. All responses are compressible JSON; no CDN fonts/assets referenced.
2. SSE endpoints must tolerate sealed mode by operating on loopback addresses only.
3. `authority-sealed-ci.json` (see DEVOPS-AIRGAP-57-002) is the evidence Authority consumes before enabling these APIs for sealed tenants; configure `airGap.sealedMode` and set `requiresAirgapSealConfirmation: true` on the Console client so `/token` refuses requests until the sealed harness uploads a fresh, passing artefact. Console responses echo `sealed: true/false` flags for UI badges once Authority has confirmed the evidence.
## 7. Sample Payloads for Docs
- `docs/api/console/samples/vuln-findings-sample.json` exported via `scripts/generate-console-samples.ts` (placeholder script to be added when backend lands).
- `docs/api/console/samples/vex-statement-sse.ndjson` contains 5 chronological SSE events for screenshot reproduction.
> Until backend implementations ship, use the examples above to unblock DOCS-AIAI-31-004; replace them with live captures once the gateway endpoints are available in staging.
## Exports (draft contract v0.4 for sign-off)
### Routes
- `POST /console/exports` — start an evidence bundle export job.
- `GET /console/exports/{exportId}` — fetch job status, manifest link, and download locations.
- `GET /console/exports/{exportId}/events` — SSE stream of job progress (optional).
### Security / headers
- `Authorization: DPoP <token>`
- `DPoP: <proof>`
- `X-StellaOps-Tenant: <tenantId>`
- `Idempotency-Key: <uuid>` (recommended for POST)
- `Accept: application/json` (status) or `text/event-stream` (events)
- Required scopes: `console:read` AND `console:export` (proposal).
### Request body (POST)
```jsonc
{
"scope": { "tenantId": "t1", "projectId": "p1" },
"sources": [
{ "type": "advisory", "ids": ["CVE-2024-12345"] },
{ "type": "vex", "ids": ["vex:tenant-default:jwt-auth:5d1a"] }
],
"formats": ["json", "ndjson", "csv"],
"attestations": { "include": true, "sigstoreBundle": true, "dsse": true },
"notify": { "webhooks": ["https://hooks.local/export"], "email": ["secops@example.com"] },
"priority": "normal"
}
```
### Response: 202 Accepted
- `exportId`, `status: queued|running|succeeded|failed|expired`
- `estimateSeconds`, `retryAfter` (seconds)
- `links`: `{ status: url, events?: url }`
### Response: GET status
```jsonc
{
"exportId": "console-export::tenant-default::2025-12-06::0007",
"status": "running",
"estimateSeconds": 420,
"outputs": [
{
"type": "manifest",
"format": "json",
"url": "https://exports.local/tenant-default/0007/manifest.json?sig=...",
"sha256": "sha256:c0ffee...",
"dsseUrl": "https://exports.local/tenant-default/0007/manifest.dsse?sig=...",
"expiresAt": "2025-12-06T13:10:00Z"
}
],
"progress": { "percent": 42, "itemsCompleted": 210, "itemsTotal": 500, "assetsReady": 12 },
"errors": []
}
```
### Response: SSE events
- `started`: `{ exportId, status }`
- `progress`: `{ exportId, percent, itemsCompleted, itemsTotal }`
- `asset_ready`: `{ exportId, type, id, url, sha256, format }`
- `completed`: `{ exportId, status: "succeeded", manifestUrl, manifestDsseUrl? }`
- `failed`: `{ exportId, status: "failed", code, message, retryAfterSeconds? }`
### Manifest shape (downloaded via outputs)
- Ordering: sort items by `(type asc, id asc, format asc, url asc)`.
- `version`: string (date), `exportId`, `tenantId`, `generatedAt`, `expiresAt`
- `items[]`: `{ type: advisory|vex|policy|scan|chart|bundle, id, format, url, sha256, size }`
- `checksums`: `{ manifest: "sha256:<digest>", bundle?: "sha256:<digest>" }`
- Optional DSSE envelope for manifest: `manifest.dsse` (payload type `stellaops.console.manifest`).
### Limits (proposed)
- Max request body 256 KiB; max sources 50; max outputs 1000 assets/export.
- Max bundle size 500 MiB compressed.
- Default job timeout 30 minutes; idle SSE timeout 60s; backoff via `Retry-After`.
### Determinism, caching, retry
- Responses set `Cache-Control: public, max-age=300, stale-while-revalidate=60, stale-if-error=300`.
- `ETag` is SHA-256 over sorted payload; clients send `If-None-Match`.
- Respect `Retry-After`; client backoff `1s,2s,4s,8s` capped at 30s.
- Cursors (if introduced later) MUST be opaque, base64url, signed with tenant + sortKeys.
### Error codes (proposal)
- `ERR_CONSOLE_EXPORT_INVALID_SOURCE`
- `ERR_CONSOLE_EXPORT_TOO_LARGE`
- `ERR_CONSOLE_EXPORT_RATE_LIMIT`
- `ERR_CONSOLE_EXPORT_UNAVAILABLE`
- `ERR_CONSOLE_EXPORT_EXPIRED`
### Samples
- Request: `docs/api/console/samples/console-export-request.json`
- Status: `docs/api/console/samples/console-export-status.json`
- Manifest: `docs/api/console/samples/console-export-manifest.json`
- Events: `docs/api/console/samples/console-export-events.ndjson`
### Open items (needs guild sign-off)
- Final scopes list (`console:export` vs broader `console:*`).
- Final limits and error codes; checksum manifest format; attestation options.
- Caching/tie-break rules for downstream `/console/search` and `/console/downloads`.
When contracts change, update the fixtures in lockstep and keep diffs deterministic (stable key ordering, stable timestamps, stable ids).