113 lines
5.8 KiB
Markdown
113 lines
5.8 KiB
Markdown
# Excititor Advisory-AI Evidence Contract (v1)
|
|
|
|
Updated: 2025-11-18 · Scope: EXCITITOR-AIAI-31-004 (Phase 119)
|
|
|
|
This note defines the deterministic, aggregation-only contract that Excititor exposes to Advisory AI and Lens consumers. It covers the `/v1/vex/evidence/chunks` NDJSON stream plus the projection rules for observation IDs, signatures, and provenance metadata.
|
|
|
|
## Goals
|
|
- **Deterministic & replayable**: stable ordering, no implicit clocks, fixed schemas.
|
|
- **Aggregation-only**: no consensus/inference; raw supplier statements plus signatures and AOC (Aggregation-Only Contract) guardrails.
|
|
- **Offline-friendly**: chunked NDJSON; no cross-tenant lookups; portable enough for mirror/air-gap bundles.
|
|
|
|
## Endpoint
|
|
- `GET /v1/vex/evidence/chunks`
|
|
- **Query**:
|
|
- `tenant` (required)
|
|
- `vulnerabilityId` (optional, repeatable) — CVE, GHSA, etc.
|
|
- `productKey` (optional, repeatable) — PURLish key used by Advisory AI.
|
|
- `cursor` (optional) — stable pagination token.
|
|
- `limit` (optional) — max records per stream chunk (default 500, max 2000).
|
|
- **Response**: `Content-Type: application/x-ndjson`
|
|
- Each line is a single evidence record (see schema below).
|
|
- Ordered by `(tenant, vulnerabilityId, productKey, observationId, statementId)` to stay deterministic.
|
|
|
|
## Evidence record schema (NDJSON)
|
|
```json
|
|
{
|
|
"tenant": "acme",
|
|
"vulnerabilityId": "CVE-2024-1234",
|
|
"productKey": "pkg:pypi/django@3.2.24",
|
|
"observationId": "obs-3cf9d6e4-…",
|
|
"statementId": "stmt-9c1d…",
|
|
"source": {
|
|
"supplier": "upstream:osv",
|
|
"documentId": "osv:GHSA-xxxx-yyyy",
|
|
"retrievedAt": "2025-11-10T12:34:56Z",
|
|
"signatureStatus": "missing|unverified|verified"
|
|
},
|
|
"aoc": {
|
|
"violations": [
|
|
{ "code": "EVIDENCE_SIGNATURE_MISSING", "surface": "ingest" }
|
|
]
|
|
},
|
|
"evidence": {
|
|
"type": "vex.statement",
|
|
"payload": { "...supplier-normalized-fields..." }
|
|
},
|
|
"provenance": {
|
|
"hash": "sha256:...",
|
|
"canonicalUri": "https://mirror.example/bundles/…",
|
|
"bundleId": "mirror-bundle-001"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Field notes
|
|
- `observationId` is stable and maps 1:1 to internal storage; Advisory AI must cite it when emitting narratives.
|
|
- `statementId` remains unique within an observation.
|
|
- `signatureStatus` is pass-through from ingest; no interpretation beyond `missing|unverified|verified`.
|
|
- `aoc.violations` enumerates guardrail violations without blocking delivery.
|
|
- `evidence.payload` is supplier-shaped; we **do not** merge or rank.
|
|
- `provenance.hash` is the SHA-256 of the supplier document bytes; `canonicalUri` points to the mirror bundle when available.
|
|
|
|
## Determinism rules
|
|
- Ordering: fixed sort above; pagination cursor is derived from the last emitted `(tenant, vulnerabilityId, productKey, observationId, statementId)`.
|
|
- Clocks: All timestamps are UTC ISO-8601 with `Z`.
|
|
- No server-generated randomness; record content is idempotent for identical upstream inputs.
|
|
|
|
## AOC guardrails
|
|
- Enforced surfaces: ingest, `/v1/vex/aoc/verify`, and chunk emission.
|
|
- Violations are reported via `aoc.violations` and metric `excititor.vex.aoc.guard_violations`.
|
|
- No statements are dropped due to AOC; consumers decide how to act.
|
|
|
|
## Telemetry (counters/logs-only until span sink arrives)
|
|
- `excititor.vex.chunks.requests` — by `tenant`, `outcome`, `truncated`.
|
|
- `excititor.vex.chunks.bytes` — histogram of NDJSON stream sizes.
|
|
- `excititor.vex.chunks.records` — histogram of records per stream.
|
|
- Existing observation metrics (`excititor.vex.observation.*`) remain unchanged.
|
|
|
|
## Error handling
|
|
- 400 for invalid tenant or mutually exclusive filters.
|
|
- 429 with `Retry-After` when throttle budgets exceeded.
|
|
- 503 on upstream store/transient failures; responses remain NDJSON-free on error.
|
|
|
|
## Offline / mirror readiness
|
|
- When mirror bundles are configured, `provenance.canonicalUri` points to the local bundle path; otherwise it is omitted.
|
|
- All payloads are side-effect free; no remote fetches occur while streaming.
|
|
|
|
## Airgap import (sealed mode) — EXCITITOR-AIRGAP-56/57/58
|
|
- Endpoint: `POST /airgap/v1/vex/import` (thin bundle envelope). Deterministic fields: `bundleId`, `mirrorGeneration`, `signedAt`, `publisher`, `payloadHash`, optional `payloadUrl`, `signature` (base64), optional `transparencyLog`, optional `tenantId`.
|
|
- Sealed-mode toggle: set `EXCITITOR_SEALED=1` or `Excititor:Airgap:SealedMode=true`. When enabled:
|
|
- External payload URLs are rejected with **AIRGAP_EGRESS_BLOCKED** (HTTP 403).
|
|
- Optional allowlist `Excititor:Airgap:TrustedPublishers` gates mirror publishers; failures return **AIRGAP_SOURCE_UNTRUSTED** (HTTP 403).
|
|
- Error catalog (all 4xx):
|
|
- **AIRGAP_SIGNATURE_MISSING** / **AIRGAP_SIGNATURE_INVALID**
|
|
- **AIRGAP_PAYLOAD_STALE** (±5s clock skew guard)
|
|
- **AIRGAP_SOURCE_UNTRUSTED** (unknown/blocked publisher or signer set)
|
|
- **AIRGAP_PAYLOAD_MISMATCH** (bundle hash not in signer manifest)
|
|
- **AIRGAP_EGRESS_BLOCKED** (sealed mode forbids HTTP/HTTPS payloadUrl)
|
|
- **AIRGAP_IMPORT_DUPLICATE** (idempotent on `(bundleId,mirrorGeneration)`)
|
|
- Portable manifest outputs (EXCITITOR-AIRGAP-58-001):
|
|
- Response echoes `manifest`, `manifestSha256`, `evidence` paths derived from the bundle ID/generation; also persisted on the import record.
|
|
- Evidence Locker linkage: `evidence/{bundleId}/{generation}/bundle.ndjson` path recorded for downstream replay/export.
|
|
- Timeline events (deterministic order, ISO timestamps):
|
|
- `airgap.import.started`, `airgap.import.completed`, `airgap.import.failed`
|
|
- Attributes: `{tenantId,bundleId,generation,stalenessSeconds?,errorCode?}`
|
|
- Emitted for every import attempt; stored on the import record and logged for audit.
|
|
|
|
## Samples
|
|
- NDJSON sample: `docs/samples/excititor/chunks-sample.ndjson` (hashes in `.sha256`) aligned to the schema above.
|
|
|
|
## Versioning
|
|
- Contract version: `v1` (this document). Changes must be additive; breaking changes require `v2` path and updated doc.
|