stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

consolidation of some of the modules, localization fixes, product advisories work, qa work

Browse Source
This commit is contained in:
master
2026-03-05 03:54:22 +02:00
parent 7bafcc3eef
commit 8e1cb9448d
3878 changed files with 72600 additions and 46861 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
docs-archived/modules/excititor/AGENTS.md Normal file
View File

@@ -0,0 +1,34 @@
# Excititor agent guide
## Mission
Excititor converts heterogeneous VEX feeds into raw observations and linksets that honour the Aggregation-Only Contract.
## Key docs
- [Module README](./README.md)
- [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md)
## How to get started
1. Open sprint file `/docs/implplan/SPRINT_*.md` and locate the stories referencing this module.
2. Review ./TASKS.md for local follow-ups and confirm status transitions (TODO → DOING → DONE/BLOCKED).
3. Read the architecture and README for domain context before editing code or docs.
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
## Guardrails
- Honour the Aggregation-Only Contract where applicable (see ../../aoc/aggregation-only-contract.md).
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
- Update runbooks/observability assets when operational characteristics change.
## Required Reading
- `docs/modules/excititor/README.md`
- `docs/modules/excititor/architecture.md`
- `docs/modules/excititor/implementation_plan.md`
- `docs/modules/platform/architecture-overview.md`
## Working Agreement
- 1. Update task status to `DOING`/`DONE` in both correspoding sprint file `/docs/implplan/SPRINT_*.md` and the local `TASKS.md` when you start or finish work.
- 2. Review this charter and the Required Reading documents before coding; confirm prerequisites are met.
- 3. Keep changes deterministic (stable ordering, timestamps, hashes) and align with offline/air-gap expectations.
- 4. Coordinate doc updates, tests, and cross-guild communication whenever contracts or workflows change.
- 5. Revert to `TODO` if you pause the task without shipping changes; leave notes in commit/PR descriptions for context.

76
docs-archived/modules/excititor/README.md Normal file
View File

@@ -0,0 +1,76 @@
# StellaOps Excititor
Excititor converts heterogeneous VEX feeds into raw observations and linksets that honour the Aggregation-Only Contract.
## Latest updates (2025-12-05)
- Chunk API documentation remains blocked until CI is green and a pinned OpenAPI spec + deterministic samples are available.
- Sprint tracker `docs/implplan/SPRINT_0333_0001_0001_docs_modules_excititor.md` and module `TASKS.md` mirror status.
- Observability/runbook assets remain in `operations/observability.md` and `observability/` (timeline, locker manifests); dashboards stay offline-import friendly.
- Prior updates (2025-11-05): Link-Not-Merge readiness and consensus beta note (`../../implplan/archived/updates/2025-11-05-excitor-consensus-beta.md`), observability guide additions, DSSE packaging guidance, and Policy/CLI follow-ups tracked in SPRINT_200.
- Link-Not-Merge readiness: release note [Excitor consensus beta](../../implplan/archived/updates/2025-11-05-excitor-consensus-beta.md) captures how Excititor feeds power the Excititor consensus beta (sample payload in [consensus JSON](../../vex/consensus-json.md)).
- Added [observability guide](operations/observability.md) describing the evidence metrics emitted by `EXCITITOR-AIAI-31-003` (request counters, statement histogram, signature status, guard violations) so Ops/Lens can alert on misuse.
- README now points policy/UI teams to the upcoming consensus integration work.
- DSSE packaging for consensus bundles and Export Center hooks are documented in the [beta release note](../../implplan/archived/updates/2025-11-05-excitor-consensus-beta.md); operators mirroring Excititor exports must verify detached JWS artefacts (`bundle.json.jws`) alongside each bundle.
- Follow-ups called out in the release note (Policy weighting knobs `POLICY-ENGINE-30-101`, CLI verb `CLI-VEX-30-002`) remain in-flight and are tracked in `/docs/implplan/SPRINT_200_documentation_process.md`.
## Release references
- Consensus beta payload reference: [docs/vex/consensus-json.md](../../vex/consensus-json.md)
- Export Center offline packaging: [docs/modules/export-center/devportal-offline.md](../export-center/devportal-offline.md)
- Historical release log: [docs/implplan/archived/updates/](../../implplan/archived/updates/)
## Responsibilities
- Fetch OpenVEX/CSAF/CycloneDX statements via restart-only connectors.
- Store immutable VEX observations with full provenance.
- Publish linksets and events that drive policy suppression decisions.
- Provide deterministic exports for Offline Kit and downstream tooling.
## Key components
- `StellaOps.Excititor.WebService` scheduler/API host.
- Connector libraries under `StellaOps.Excititor.Connector.*`.
- Normalization helpers and exporters in `StellaOps.Excititor.*`.
## Integrations & dependencies
- Policy Engine for evidence queries.
- UI/CLI for conflict visibility and explanation.
- Notify for VEX-driven alerts.
## Operational notes
- PostgreSQL (schema `vex`) for observation storage and job metadata.
- Offline kit packaging aligned with Concelier merges.
- Connector-specific runbooks (see `docs/modules/concelier/operations/connectors`).
- Ubuntu CSAF provenance knobs: [`operations/ubuntu-csaf.md`](operations/ubuntu-csaf.md) captures TrustWeight/Tier, cosign, and fingerprint configuration for the sprint 120 enrichment.
## Backlog references
- DOCS-LNM-22-006 / DOCS-LNM-22-007 (shared with Concelier).
- CLI-EXC-25-001..002 follow-up for CLI parity.
## Epic alignment
- **Epic 1 – AOC enforcement:** maintain immutable VEX observations, provenance, and AOC verifier coverage.
- **Epic 7 – VEX Consensus Lens:** supply trustworthy raw inputs, trust metadata, and consensus hooks for the lens computations.
- **Epic 8 – Advisory AI:** expose citation-ready VEX payloads for the advisory assistant pipeline.
## Implementation Status
### Objectives
- Maintain deterministic behaviour and offline parity across releases
- Keep documentation, telemetry, and runbooks aligned with the latest sprint outcomes
### Key Milestones
- **Epic 1 – AOC enforcement:** enforce immutable VEX observation schema, provenance capture, and guardrails
- **Epic 7 – VEX Consensus Lens:** provide lens-ready metadata (issuer trust, temporal scoping) and consensus APIs
- **Epic 8 – Advisory AI:** guarantee citation-ready payloads and normalized context for AI summaries/explainers
### Recent Delivery Status
- Chunk API documentation remains blocked until CI is green and a pinned OpenAPI spec with deterministic samples are available
- Link-Not-Merge readiness and consensus beta completed with DSSE packaging guidance
- Observability guide additions and policy/CLI follow-ups tracked in sprint files
### Workstreams
- Backlog grooming: reconcile open stories with module roadmap
- Implementation: collaborate with service owners to land feature work
- Validation: extend tests/fixtures to preserve determinism and provenance requirements
### Coordination
- Review ./AGENTS.md before picking up new work
- Sync with cross-cutting teams noted in sprint files
- Update plan whenever scope, dependencies, or guardrails change

1082
docs-archived/modules/excititor/architecture.md Normal file
View File

File diff suppressed because it is too large Load Diff

43
docs-archived/modules/excititor/attestation-plan.md Normal file
View File

@@ -0,0 +1,43 @@
# Excititor Attestation Plan (Sprint 110)
## Goals
- Align Excititor chunk API and attestation envelopes with Evidence Locker contract.
- Provide offline-ready chunk submission/attestation flow for VEX evidence.
## Chunk API shape (`/vex/evidence/chunks`)
- POST body (NDJSON, deterministic order by `chunk_id`):
```json
{
"chunk_id": "uuid",
"tenant": "acme",
"source": "ghsa",
"schema": "stellaops.vex.chunk.v1",
"items": [ {"advisory_id":"GHSA-123","status":"affected","purl":"pkg:npm/foo@1.0.0"} ],
"provenance": {"fetched_at":"2025-11-20T00:00:00Z","artifact_sha":"abc"}
}
```
- At submission, Excititor returns `chunk_digest` (sha256 of canonical JSON) and queue id.
## Attestation envelope
- Subject: `chunk_digest` from above.
- Predicates attached:
- `stellaops.vex.chunk.meta.v1` (tenant, source, schema version, item count).
- `stellaops.vex.chunk.integrity.v1` (sha256 per item block, canonical order).
- Optional `stellaops.transparency.v1` (Rekor UUID/logIndex) when online.
- Envelope format: DSSE using Evidence Locker provider registry; signing profile mirrors Evidence Locker bundle profile for tenant.
## DSSE bundling rules
- Deterministic JSON (sorted keys) before hashing.
- Canonical NDJSON for chunk payload; no gzip inside envelope.
- Attach verification report alongside attestation as `chunk-verify.json` (hashes + signature check results).
## Sample payloads
- `docs/modules/excititor/samples/chunk-sample.ndjson`
- `docs/modules/excititor/samples/chunk-attestation-sample.json`
## Integration points
- Evidence Locker contract v1 (see `docs/modules/evidence-locker/attestation-contract.md`).
- Concelier LNM schemas (observations remain aggregation-only; attestation is evidence, not merge).
## Ownership
- Excititor Guild (primary); Evidence Locker Guild reviewer.

13
docs-archived/modules/excititor/changes.md Normal file
View File

@@ -0,0 +1,13 @@
# Excititor Changes Log
This file records breaking or behavior-changing updates for the Excititor module.
Update this log whenever public contracts, schemas, or workflows change.
## Format
- Date (UTC)
- Change summary
- Impacted contracts or schemas
- Migration notes (if required)
## Entries
- 2026-01-30: Log initialized. No breaking changes recorded.

36
docs-archived/modules/excititor/connectors/connector-signer-metadata.md Normal file
View File

@@ -0,0 +1,36 @@
# Connector signer metadata (v1.0.0)
**Scope.** Defines the canonical, offline-friendly metadata for Excititor connectors that validate signed feeds (MSRC CSAF, Oracle OVAL, Ubuntu OVAL, StellaOps mirror OpenVEX). The file is consumed by WebService/Worker composition roots and by Offline Kits to pin trust material deterministically.
**Location & format.**
- Schema: `docs/modules/excititor/schemas/connector-signer-metadata.schema.json` (JSON Schema 2020‑12).
- Sample: `docs/modules/excititor/samples/connector-signer-metadata-sample.json` (aligns with schema).
- Expected production artifact: NDJSON or JSON stamped per release; store in offline kits alongside connector bundles.
## Required fields (summary)
- `schemaVersion` — must be `1.0.0`.
- `generatedAt` — ISO-8601 UTC timestamp for the metadata file.
- `connectors[]` — one entry per connector:
- `connectorId` — stable slug, e.g., `excititor-msrc-csaf`.
- `provider { name, slug }` — human label and slug.
- `issuerTier` — `tier-0`, `tier-1`, `tier-2`, or `untrusted` (aligns with trust weighting).
- `signers[]` — one per signing path; each has `usage` (`csaf|oval|openvex|bulk-meta|attestation`) and `fingerprints[]` (algorithm + format + value). Optional `keyLocator` and `certificateChain` for offline key retrieval.
- `bundle` — reference to the sealed bundle containing the feed/signing material (`kind`: `oci-referrer|oci-tag|file|tuf`, plus `uri`, optional `digest`, `publishedAt`).
- Optional `validFrom`, `validTo`, `revoked`, `notes` for rollover and incident handling.
## Rollover / migration guidance
1) **Author the metadata** using the schema and place the JSON next to connector bundles in the offline kit (`out/connectors/<provider>/signer-metadata.json`).
2) **Validate** with `dotnet tool run njsonschema validate connector-signer-metadata.schema.json connector-signer-metadata.json` (or `ajv validate`).
3) **Wire connector code** to load the file on startup (Worker + WebService) and pin signers per `connectorId`; reject feeds whose fingerprints are absent or marked `revoked=true` or out of `validFrom/To` range.
- Connectors look for `STELLAOPS_CONNECTOR_SIGNER_METADATA_PATH` (absolute/relative) and enrich provenance metadata automatically when present.
4) **Rollover keys** by appending a new `signers` entry and setting a future `validFrom`; keep the previous signer until all mirrors have caught up. Use `issuerTier` downgrades to quarantine while keeping history.
5) **Mirror references**: store the referenced bundles/keys under OCI tags or TUF targets already shipped in the offline kit so no live network is required.
6) **Record decisions** in sprint Decisions & Risks when changing trust tiers or fingerpints; update this doc if formats change.
## Sample entries (non-production)
See `docs/modules/excititor/samples/connector-signer-metadata-sample.json` for MSRC, Oracle, Ubuntu, and StellaOps example entries. These fingerprints are illustrative only; replace with real values before shipping.
## Consumer expectations
- Deterministic: sort connectors alphabetically before persistence; avoid clock-based defaults.
- Offline-first: all `keyLocator`/`bundle.uri` values must resolve inside the air-gap kit (OCI/TUF/file).
- Observability: emit a structured warning when metadata is missing or stale (>7 days) and fail closed for missing signers.

112
docs-archived/modules/excititor/evidence-contract.md Normal file
View File

@@ -0,0 +1,112 @@
# Excititor Advisory-AI Evidence Contract (v1)
Updated: 2025-11-18 · Scope: EXCITITOR-AIAI-31-004 (Phase 119)
This note defines the deterministic, aggregation-only contract that Excititor exposes to Advisory AI and Lens consumers. It covers the `/v1/vex/evidence/chunks` NDJSON stream plus the projection rules for observation IDs, signatures, and provenance metadata.
## Goals
- **Deterministic & replayable**: stable ordering, no implicit clocks, fixed schemas.
- **Aggregation-only**: no consensus/inference; raw supplier statements plus signatures and AOC (Aggregation-Only Contract) guardrails.
- **Offline-friendly**: chunked NDJSON; no cross-tenant lookups; portable enough for mirror/air-gap bundles.
## Endpoint
- `GET /v1/vex/evidence/chunks`
- **Query**:
- `tenant` (required)
- `vulnerabilityId` (optional, repeatable) — CVE, GHSA, etc.
- `productKey` (optional, repeatable) — PURLish key used by Advisory AI.
- `cursor` (optional) — stable pagination token.
- `limit` (optional) — max records per stream chunk (default 500, max 2000).
- **Response**: `Content-Type: application/x-ndjson`
- Each line is a single evidence record (see schema below).
- Ordered by `(tenant, vulnerabilityId, productKey, observationId, statementId)` to stay deterministic.
## Evidence record schema (NDJSON)
```json
{
"tenant": "acme",
"vulnerabilityId": "CVE-2024-1234",
"productKey": "pkg:pypi/django@3.2.24",
"observationId": "obs-3cf9d6e4-…",
"statementId": "stmt-9c1d…",
"source": {
"supplier": "upstream:osv",
"documentId": "osv:GHSA-xxxx-yyyy",
"retrievedAt": "2025-11-10T12:34:56Z",
"signatureStatus": "missing|unverified|verified"
},
"aoc": {
"violations": [
{ "code": "EVIDENCE_SIGNATURE_MISSING", "surface": "ingest" }
]
},
"evidence": {
"type": "vex.statement",
"payload": { "...supplier-normalized-fields..." }
},
"provenance": {
"hash": "sha256:...",
"canonicalUri": "https://mirror.example/bundles/…",
"bundleId": "mirror-bundle-001"
}
}
```
### Field notes
- `observationId` is stable and maps 1:1 to internal storage; Advisory AI must cite it when emitting narratives.
- `statementId` remains unique within an observation.
- `signatureStatus` is pass-through from ingest; no interpretation beyond `missing|unverified|verified`.
- `aoc.violations` enumerates guardrail violations without blocking delivery.
- `evidence.payload` is supplier-shaped; we **do not** merge or rank.
- `provenance.hash` is the SHA-256 of the supplier document bytes; `canonicalUri` points to the mirror bundle when available.
## Determinism rules
- Ordering: fixed sort above; pagination cursor is derived from the last emitted `(tenant, vulnerabilityId, productKey, observationId, statementId)`.
- Clocks: All timestamps are UTC ISO-8601 with `Z`.
- No server-generated randomness; record content is idempotent for identical upstream inputs.
## AOC guardrails
- Enforced surfaces: ingest, `/v1/vex/aoc/verify`, and chunk emission.
- Violations are reported via `aoc.violations` and metric `excititor.vex.aoc.guard_violations`.
- No statements are dropped due to AOC; consumers decide how to act.
## Telemetry (counters/logs-only until span sink arrives)
- `excititor.vex.chunks.requests` — by `tenant`, `outcome`, `truncated`.
- `excititor.vex.chunks.bytes` — histogram of NDJSON stream sizes.
- `excititor.vex.chunks.records` — histogram of records per stream.
- Existing observation metrics (`excititor.vex.observation.*`) remain unchanged.
## Error handling
- 400 for invalid tenant or mutually exclusive filters.
- 429 with `Retry-After` when throttle budgets exceeded.
- 503 on upstream store/transient failures; responses remain NDJSON-free on error.
## Offline / mirror readiness
- When mirror bundles are configured, `provenance.canonicalUri` points to the local bundle path; otherwise it is omitted.
- All payloads are side-effect free; no remote fetches occur while streaming.
## Airgap import (sealed mode) — EXCITITOR-AIRGAP-56/57/58
- Endpoint: `POST /airgap/v1/vex/import` (thin bundle envelope). Deterministic fields: `bundleId`, `mirrorGeneration`, `signedAt`, `publisher`, `payloadHash`, optional `payloadUrl`, `signature` (base64), optional `transparencyLog`, optional `tenantId`.
- Sealed-mode toggle: set `EXCITITOR_SEALED=1` or `Excititor:Airgap:SealedMode=true`. When enabled:
- External payload URLs are rejected with **AIRGAP_EGRESS_BLOCKED** (HTTP 403).
- Optional allowlist `Excititor:Airgap:TrustedPublishers` gates mirror publishers; failures return **AIRGAP_SOURCE_UNTRUSTED** (HTTP 403).
- Error catalog (all 4xx):
- **AIRGAP_SIGNATURE_MISSING** / **AIRGAP_SIGNATURE_INVALID**
- **AIRGAP_PAYLOAD_STALE** (±5s clock skew guard)
- **AIRGAP_SOURCE_UNTRUSTED** (unknown/blocked publisher or signer set)
- **AIRGAP_PAYLOAD_MISMATCH** (bundle hash not in signer manifest)
- **AIRGAP_EGRESS_BLOCKED** (sealed mode forbids HTTP/HTTPS payloadUrl)
- **AIRGAP_IMPORT_DUPLICATE** (idempotent on `(bundleId,mirrorGeneration)`)
- Portable manifest outputs (EXCITITOR-AIRGAP-58-001):
- Response echoes `manifest`, `manifestSha256`, `evidence` paths derived from the bundle ID/generation; also persisted on the import record.
- Evidence Locker linkage: `evidence/{bundleId}/{generation}/bundle.ndjson` path recorded for downstream replay/export.
- Timeline events (deterministic order, ISO timestamps):
- `airgap.import.started`, `airgap.import.completed`, `airgap.import.failed`
- Attributes: `{tenantId,bundleId,generation,stalenessSeconds?,errorCode?}`
- Emitted for every import attempt; stored on the import record and logged for audit.
## Samples
- NDJSON sample: `docs/modules/excititor/samples/chunks-sample.ndjson` (hashes in `.sha256`) aligned to the schema above.
## Versioning
- Contract version: `v1` (this document). Changes must be additive; breaking changes require `v2` path and updated doc.

87
docs-archived/modules/excititor/graph-overlays.md Normal file
View File

@@ -0,0 +1,87 @@
# Excititor Graph Overlay Contract (v1.0.0)
_Updated: 2025-12-10 | Owners: Excititor Core + UI Guilds | Scope: EXCITITOR-GRAPH-21-001..005, EXCITITOR-POLICY-20-001/002, EXCITITOR-RISK-66-001_
## Purpose
Defines the graph-ready overlay built from Link-Not-Merge observations/linksets so Console, Vuln Explorer, Policy, and Risk surfaces consume a single deterministic shape. This freezes the contract for Postgres materialization and cache APIs, unblocking Sprint 0120 tasks.
## Schema
- JSON Schema: `docs/modules/excititor/schemas/vex_overlay.schema.json` (draft 2020-12, schemaVersion `1.0.0`).
- Required fields: `schemaVersion`, `generatedAt`, `tenant`, `purl`, `advisoryId`, `source`, `status`, `observations[]`, `provenance`.
- Status enum: `affected|not_affected|under_investigation|fixed|unknown`.
- Ordering: observations are sorted by `source, advisoryId, fetchedAt` (Link-Not-Merge invariant) and emitted in that order. Overlays are returned in request PURL order, then by `advisoryId`, then `source`.
- Provenance: carries `linksetId`, `linksetHash`, `observationHashes[]`, optional `policyHash`, `sbomContextHash`, and `planCacheKey` for replay.
## Postgres materialization (IAppendOnlyLinksetStore)
- Table `vex_overlays` (materialized cache):
- Primary key: `(tenant, purl, advisory_id, source)`.
- Columns: `status`, `justifications` (jsonb), `conflicts` (jsonb), `observations` (jsonb), `provenance` (jsonb), `cached_at`, `ttl_seconds`, `schema_version`.
- Indexes: unique `(tenant, purl, advisory_id, source)`, plus `(tenant, cached_at)` for TTL sweeps.
- Overlay rows are regenerated when linkset hash or observation hash set changes; cache evictions use `cached_at + ttl_seconds`.
- Linksets and observation hashes come from the append-only linkset store (`IAppendOnlyLinksetStore`) to preserve Aggregation-Only Contract guarantees.
## API shape (Graph/Vuln Explorer)
- Endpoint: `GET /v1/graph/overlays?purl=<purl>&purl=<purl>&includeJustifications=true|false`.
- Response items follow `vex_overlay.schema.json`; `cache` stanza signals `cached`, `cachedAt`, and `ttlSeconds`.
- Cursoring: stable order (input PURL list) with `nextPageToken` based on `(tenant, purl, advisoryId, source, generatedAt)`.
- Telemetry: `excititor.graph.overlays.cache{tenant,hit}` counter; `excititor.graph.overlays.latency_ms` histogram tagged with `cached`.
## Sample (abridged)
```json
{
"schemaVersion": "1.0.0",
"generatedAt": "2025-12-10T00:00:00Z",
"tenant": "tenant-default",
"purl": "pkg:maven/org.example/foo@1.2.3",
"advisoryId": "GHSA-xxxx-yyyy-zzzz",
"source": "ghsa",
"status": "affected",
"justifications": [
{
"kind": "known_affected",
"reason": "Upstream GHSA reports affected range <1.3.0.",
"evidence": ["concelier:ghsa:obs:6561e41b3e3f4a6e9d3b91c1"],
"weight": 0.8
}
],
"conflicts": [
{
"field": "affected.versions",
"reason": "vendor_range_differs",
"values": ["<1.2.0", "<=1.3.0"],
"sourceIds": ["concelier:redhat:obs:...","concelier:ghsa:obs:..."]
}
],
"observations": [
{
"id": "concelier:ghsa:obs:6561e41b3e3f4a6e9d3b91c1",
"contentHash": "sha256:1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd",
"fetchedAt": "2025-11-19T00:00:00Z"
}
],
"provenance": {
"linksetId": "concelier:ghsa:linkset:6561e41b3e3f4a6e9d3b91d0",
"linksetHash": "sha256:deaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddead",
"observationHashes": ["sha256:1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd"],
"policyHash": "sha256:0f7c...9ad3",
"sbomContextHash": "sha256:421af53f9eeba6903098d292fbd56f98be62ea6130b5161859889bf11d699d18",
"planCacheKey": "tenant-default|pkg:maven/org.example/foo@1.2.3|GHSA-xxxx-yyyy-zzzz"
},
"cache": {
"cached": true,
"cachedAt": "2025-12-10T00:00:00Z",
"ttlSeconds": 300
}
}
```
## Validation & determinism
- Validate overlays against `vex_overlay.schema.json` in CI and during materialization; reject or warn when fields drift.
- Deterministic ordering: input PURL order, then `advisoryId`, then `source`; observation list sorted by `source, advisoryId, fetchedAt`.
- No mutation: overlays are append-only; regeneration inserts a new row/version, leaving prior cache entries for audit until TTL expires.
## Handoff
- Consumers (Console, Vuln Explorer, Policy Engine, Risk) should treat `vex_overlay.schema.json` as the authoritative contract.
- Offline kits must bundle the schema file and sample payloads under `docs/modules/excititor/samples/` with SHA256 manifests.
- Future schema versions must bump `schemaVersion` and add migration notes to this document and `docs/modules/excititor/architecture.md`.
- Policy and Risk surfaces in WebService now read overlays directly (with claim-store fallback for policy tests) to produce lookup and risk feeds; overlay cache/store are selected per tenant (in-memory by default, Postgres `vex.graph_overlays` when configured).

24
docs-archived/modules/excititor/implementation_plan.md Normal file
View File

@@ -0,0 +1,24 @@
# Excititor Implementation Plan
## Purpose
Provide a living plan for Excititor deliverables, dependencies, and evidence.
## Active work
- Track current sprints under `docs/implplan/SPRINT_*.md` for this module.
- Update this file when new scoped work is approved.
## Near-term deliverables
- TBD (add when sprint is staffed).
## Dependencies
- `docs/modules/excititor/architecture.md`
- `docs/modules/excititor/README.md`
- `docs/modules/platform/architecture-overview.md`
## Evidence of completion
- Code changes under `src/Excititor/**`.
- Tests and fixtures under the module's `__Tests` / `__Libraries`.
- Docs and runbooks under `docs/modules/excititor/**`.
## Notes
- Keep deterministic and offline-first expectations aligned with module AGENTS.

195
docs-archived/modules/excititor/mirrors.md Normal file
View File

@@ -0,0 +1,195 @@
# architecture_excititor_mirrors.md — Excititor Mirror Distribution
> **Status:** Draft (Sprint 7). Complements `docs/modules/excititor/architecture.md` by describing the mirror export surface exposed by `Excititor.WebService` and the configuration hooks used by operators and downstream mirrors.
---
## 0) Purpose
Excititor publishes canonical VEX consensus data. Operators (or StellaOps-managed mirrors) need a deterministic way to sync those exports into downstream environments. Mirror distribution provides:
* A declarative map of export bundles (`json`, `jsonl`, `openvex`, `csaf`) reachable via signed HTTP endpoints under `/excititor/mirror`.
* Thin quota/authentication controls on top of the existing export cache so mirrors cannot starve the web service.
* Stable payload shapes that downstream automation can monitor (index → fetch updates → download artifact → verify signature).
Mirror endpoints are intentionally **read-only**. Write paths (export generation, attestation, cache) remain the responsibility of the export pipeline.
---
## 1) Configuration model
The web service reads mirror configuration from `Excititor:Mirror` (YAML/JSON/appsettings). Each domain groups a set of exports that share rate limits and authentication rules.
```yaml
Excititor:
Mirror:
Domains:
- id: primary
displayName: Primary Mirror
requireAuthentication: false
maxIndexRequestsPerHour: 600
maxDownloadRequestsPerHour: 1200
exports:
- key: consensus
format: json
filters:
vulnId: CVE-2025-0001
productKey: pkg:test/demo
sort:
createdAt: false # descending
limit: 1000
- key: consensus-openvex
format: openvex
filters:
vulnId: CVE-2025-0001
```
### Root settings
| Field | Required | Description |
| --- | --- | --- |
| `outputRoot` | – | Filesystem root where mirror artefacts are written. Defaults to the Excititor file-system artifact store root when omitted. |
| `directoryName` | – | Optional subdirectory created under `outputRoot`; defaults to `mirror`. |
| `targetRepository` | – | Hint propagated to manifests/index files indicating the operator-visible location (for example `s3://mirror/excititor`). |
| `signing` | – | Bundle signing configuration. When enabled, the exporter emits a detached JWS (`bundle.json.jws`) alongside each domain bundle. |
`signing` supports the following fields:
| Field | Required | Description |
| --- | --- | --- |
| `enabled` | – | Toggles detached signing for domain bundles. |
| `algorithm` | – | Signing algorithm identifier (default `ES256`). |
| `keyId` | ✅ (when `enabled`) | Signing key identifier resolved via the configured crypto provider registry. |
| `provider` | – | Optional provider hint when multiple registries are available. |
| `keyPath` | – | Optional PEM path used to seed the provider when the key is not already loaded. |
### Domain field reference
| Field | Required | Description |
| --- | --- | --- |
| `id` | ✅ | Stable identifier. Appears in URLs (`/excititor/mirror/domains/{id}`) and download filenames. |
| `displayName` | – | Human-friendly label surfaced in the `/domains` listing. Falls back to `id`. |
| `requireAuthentication` | – | When `true` the service enforces that the caller is authenticated (Authority token). |
| `maxIndexRequestsPerHour` | – | Per-domain quota for index endpoints. `0`/negative disables the guard. |
| `maxDownloadRequestsPerHour` | – | Per-domain quota for artifact downloads. |
| `exports` | ✅ | Collection of export projections. |
Export-level fields:
| Field | Required | Description |
| --- | --- | --- |
| `key` | ✅ | Unique key within the domain. Used in URLs (`/exports/{key}`) and filenames/bundle entries. |
| `format` | ✅ | One of `json`, `jsonl`, `openvex`, `csaf`. Maps to `VexExportFormat`. |
| `filters` | – | Key/value pairs executed via `VexQueryFilter`. Keys must match export data source columns (e.g., `vulnId`, `productKey`). |
| `sort` | – | Key/boolean map (false = descending). |
| `limit`, `offset`, `view` | – | Optional query bounds passed through to the export query. |
⚠️ **Misconfiguration:** invalid formats or missing keys cause exports to be flagged with `status` in the index response; they are not exposed downstream.
---
## 2) HTTP surface
Routes are grouped under `/excititor/mirror`.
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/domains` | Returns configured domains with quota metadata. |
| `GET` | `/domains/{domainId}` | Domain detail (auth/quota + export keys). `404` for unknown domains. |
| `GET` | `/domains/{domainId}/index` | Lists exports with exportId, query signature, format, artifact digest, attestation metadata, and size. Applies index quota. |
| `GET` | `/domains/{domainId}/exports/{exportKey}` | Returns manifest metadata (single export). `404` if unknown/missing. |
| `GET` | `/domains/{domainId}/exports/{exportKey}/download` | Streams export content from the artifact store. Applies download quota. |
Responses are serialized via `VexCanonicalJsonSerializer` ensuring stable ordering. Download responses include a content-disposition header naming the file `<domain>-<export>.<ext>`.
### Error handling
* `401` – authentication required (`requireAuthentication=true`).
* `404` – domain/export not found or manifest not persisted.
* `429` – per-domain quota exceeded (`Retry-After` header set in seconds).
* `503` – export misconfiguration (invalid format/query).
---
## 3) Rate limiting
`MirrorRateLimiter` implements a simple rolling 1-hour window using `IMemoryCache`. Each domain has two quotas:
* `index` scope → `maxIndexRequestsPerHour`
* `download` scope → `maxDownloadRequestsPerHour`
`0` or negative limits disable enforcement. Quotas are best-effort (per-instance). For HA deployments, configure sticky routing at the ingress or replace the limiter with a distributed implementation.
---
## 4) Interaction with export pipeline
Mirror endpoints consume manifests produced by the export engine (`MongoVexExportStore`). They do **not** trigger new exports. Operators must configure connectors/exporters to keep targeted exports fresh (see `EXCITITOR-EXPORT-01-005/006/007`).
Recommended workflow:
1. Define export plans at the export layer (JSON/OpenVEX/CSAF).
2. Configure mirror domains mapping to those plans.
3. Downstream mirror automation:
* `GET /domains/{id}/index`
* Compare `exportId` / `consensusRevision`
* `GET /download` when new
* Verify digest + attestation
When the export engine runs, it materializes the following artefacts under `outputRoot/<directoryName>`:
- `index.json` – canonical index listing each configured domain, manifest/bundle descriptors (with SHA-256 digests), and available export keys.
- `<domain>/manifest.json` – per-domain summary with export metadata (query signature, consensus/score digests, source providers) and a descriptor pointing at the bundle.
- `<domain>/bundle.json` – canonical payload containing serialized consensus, score envelopes, and normalized VEX claims for the matching export definitions.
- `<domain>/bundle.json.jws` – optional detached JWS when signing is enabled.
Downstream automation reads `manifest.json`/`bundle.json` directly, while `/excititor/mirror` endpoints stream the same artefacts through authenticated HTTP.
---
## 5) Operational guidance
* Track quota utilisation via HTTP 429 metrics (configure structured logging or OTEL counters when rate limiting triggers).
* Mirror domains can be deployed per tenant (e.g., `tenant-a`, `tenant-b`) with different auth requirements.
* Ensure the underlying artifact stores (`FileSystem`, `S3`, offline bundle) retain artefacts long enough for mirrors to sync.
* For air-gapped mirrors, combine mirror endpoints with the Offline Kit (see `docs/OFFLINE_KIT.md`).
---
## 6) Future alignment
* Replace manual export definitions with generated mirror bundle manifests once `EXCITITOR-EXPORT-01-007` ships.
* Extend `/index` payload with quiet-provenance when `EXCITITOR-EXPORT-01-006` adds that metadata.
* Integrate domain manifests with DevOps mirror profiles (`DEVOPS-MIRROR-08-001`) so helm/compose overlays can enable or disable domains declaratively.
---
## 7) Runbook & observability checklist (Sprint 22 demo refresh · 2025-11-07)
### Daily / on-call checks
1. **Index freshness** – watch `excitor_mirror_export_latency_seconds` (p95 < 180) grouped by `domainId`. If latency grows past 10 minutes, verify the export worker queue (`stellaops-export-worker` logs) and ensure PostgreSQL `vex.exports` has entries newer than `now()-10m`.
2. **Quota exhaustion** – alert on `excitor_mirror_quota_exhausted_total{scope="download"}` increases. When triggered, inspect structured logs (`MirrorDomainId`, `QuotaScope`, `RemoteIp`) and either raise limits or throttle abusive clients.
3. **Bundle signature health** – metric `excitor_mirror_bundle_signature_verified_total` should match download counts when signing enabled. Deltas indicate missing `.jws` files; rebuild the bundle via export job or copy artefacts from the authority mirror cache.
4. **HTTP errors** – dashboards should track 4xx/5xx rates split by route; repeated `503` statuses imply misconfigured exports. Check `mirror/index` logs for `status=misconfigured`.
### Incident steps
1. Use `GET /excititor/mirror/domains/{id}/index` to capture current manifests. Attach the response to the incident log for reproducibility.
2. For quota incidents, temporarily raise `maxIndexRequestsPerHour`/`maxDownloadRequestsPerHour` via the `Excititor:Mirror:Domains` config override, redeploy, then work with the consuming team on caching.
3. For stale exports, trigger the export job (`Excititor.ExportRunner`) and confirm the artefacts are written to `outputRoot/<domain>`.
4. Validate DSSE artefacts by running `cosign verify-blob --certificate-rekor-url=<rekor> --bundle <domain>/bundle.json --signature <domain>/bundle.json.jws`.
### Logging fields (structured)
| Field | Description |
| --- | --- |
| `MirrorDomainId` | Domain handling the request (matches `id` in config). |
| `QuotaScope` | `index` / `download`, useful when alerting on quota events. |
| `ExportKey` | Included in download logs to pinpoint misconfigured exports. |
| `BundleDigest` | SHA-256 of the artefact; compare with index payload when debugging corruption. |
### OTEL signals
- **Counters:** `excitor.mirror.requests`, `excitor.mirror.quota_blocked`, `excitor.mirror.signature.failures`.
- **Histograms:** `excitor.mirror.download.duration`, `excitor.mirror.export.latency`.
- **Spans:** `mirror.index`, `mirror.download` include attributes `mirror.domain`, `mirror.export.key`, and `mirror.quota.remaining`.
Add these instruments via the `MirrorEndpoints` middleware; see `StellaOps.Excititor.WebService/Telemetry/MirrorMetrics.cs`.

39
docs-archived/modules/excititor/observability/locker-manifest.md Normal file
View File

@@ -0,0 +1,39 @@
# Excititor Locker Manifest (OBS-53-001)
Defines the manifest for evidence snapshots stored in Evidence Locker / sealed-mode bundles.
## Manifest structure
```json
{
"tenant": "default",
"manifestId": "locker:excititor:2025-11-23:0001",
"createdAt": "2025-11-23T23:10:00Z",
"items": [
{
"observationId": "vex:obs:sha256:...",
"providerId": "ubuntu-csaf",
"contentHash": "sha256:...",
"linksetId": "CVE-2024-0001:pkg:maven/org.demo/app@1.2.3",
"dsseEnvelopeHash": "sha256:...",
"provenance": {
"source": "mirror|ingest",
"mirrorGeneration": 12,
"exportCenterManifest": "sha256:..."
}
}
],
"merkleRoot": "sha256:...", // over `items[*].contentHash`
"signature": null, // populated in OBS-54-001 (DSSE)
"metadata": {"sealed": true}
}
```
## Rules
- `items` sorted by `observationId`, then `providerId`.
- `merkleRoot` uses SHA-256 over concatenated item hashes (stable order above).
- `signature` is a DSSE envelope (hash recorded in `dsseEnvelopeHash`) when OBS-54-001 is enabled; otherwise `null`.
- Manifests are immutable; version using `manifestId` suffix.
## Storage and replay
- Store manifests alongside payloads in object storage; key prefix: `locker/excititor/<tenant>/<manifestId>`.
- Replay tools must verify `merkleRoot` before loading payloads; reject if mismatched.

43
docs-archived/modules/excititor/observability/timeline-events.md Normal file
View File

@@ -0,0 +1,43 @@
# Excititor Timeline Events (OBS-52-001)
Defines the event envelope for evidence timelines emitted by Excititor. All fields are aggregation-only; no consensus/merge logic.
## Envelope
```json
{
"type": "excititor.timeline.v1",
"tenant": "default",
"eventId": "urn:uuid:...",
"timestamp": "2025-11-23T23:10:00Z",
"traceId": "beefcafe...",
"spanId": "deadb33f...",
"source": "excititor.web",
"kind": "observation|linkset",
"action": "ingest|update|backfill|replay",
"observationId": "vex:obs:sha256:...",
"linksetId": "CVE-2024-0001:pkg:maven/org.demo/app@1.2.3",
"justifications": ["component_not_present"],
"conflicts": [
{"providerId": "suse-csaf", "status": "fixed", "justification": null}
],
"evidenceHash": "sha256:...", // content-addressed payload hash
"dsseEnvelopeHash": "sha256:...", // if attested (see OBS-54-001)
"metadata": {"connector": "ubuntu-csaf", "mirrorGeneration": 12}
}
```
## Semantics
- `eventId` is stable per write; retries reuse the same ID.
- `timestamp` must be UTC; derive from TimeProvider.
- `traceId`/`spanId` propagate ingestion traces; if tracing is disabled, set both to `null`.
- `kind` + `action` drive downstream storage and alerting.
- `evidenceHash` is the raw document hash; `dsseEnvelopeHash` appears only when OBS-54-001 is enabled.
## Determinism
- Sort `justifications` and `conflicts` ascending by providerId/status before emit.
- Emit at-most-once per storage write; idempotent consumers rely on `(eventId, tenant)`.
## Transport
- Default topic: `excititor.timeline.v1` (NATS/Valkey). Subject includes tenant: `excititor.timeline.v1.<tenant>`.
- Payload size should stay <32 KiB; truncate conflict arrays with `truncated=true` flag if needed (keep hash counts deterministic).

24
docs-archived/modules/excititor/operations/chunk-api-user-guide.md Normal file
View File

@@ -0,0 +1,24 @@
# Using the Chunk API
Endpoint: `POST /vex/evidence/chunks`
- Content-Type: `application/x-ndjson`
- See schema: `docs/modules/excititor/schemas/vex-chunk-api.yaml`
Response: `202 Accepted`
```json
{ "chunk_digest": "sha256:…", "queue_id": "uuid" }
```
Operational notes
- Deterministic hashing: server recomputes `chunk_digest` from canonical JSON; mismatches return 400.
- Limits: default 500 items, max 2000 (aligned with Program.cs guard).
- Telemetry: metrics under `StellaOps.Excititor.Chunks` (see chunk-telemetry.md).
- Headers: correlation/trace headers echoed (`X-Stella-TraceId`, `X-Stella-CorrelationId`).
Example curl
```bash
curl -X POST https://excitor.local/vex/evidence/chunks \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/x-ndjson" \
--data-binary @docs/modules/excititor/samples/chunk-sample.ndjson
```

26
docs-archived/modules/excititor/operations/chunk-telemetry.md Normal file
View File

@@ -0,0 +1,26 @@
# Excititor Chunk Telemetry (Sprint 110)
## Metrics (Meter: `StellaOps.Excititor.Chunks`)
- `vex_chunks_ingested_total` (counter) — tags: `tenant`, `source`, `status` (`accepted|rejected`), `reason` (nullable for accepted). Increments per chunk submitted.
- `vex_chunks_item_count` (histogram, unit=items) — records item count per chunk.
- `vex_chunks_payload_bytes` (histogram, unit=bytes) — measured from NDJSON payload length.
- `vex_chunks_latency_ms` (histogram) — end-to-end ingestion latency per request.
## Logs
- `vex.chunk.ingest.accepted` — includes `chunk_id`, `tenant`, `source`, `item_count`, `chunk_digest`.
- `vex.chunk.ingest.rejected` — includes `chunk_id`, `tenant`, `source`, `reason`, validation errors (summarized).
## Wiring steps
1. Register `ChunkTelemetry` as singleton with shared `Meter` instance.
2. In `/vex/evidence/chunks` handler, compute `chunk_digest` deterministically from canonical JSON and emit counters/histograms via `ChunkTelemetry`.
3. Log using structured templates above; avoid request bodies in logs.
4. Expose metrics via default ASP.NET metrics export (Prometheus/OpenTelemetry) already configured in WebService.
## Determinism & offline posture
- Do not include host-specific paths or timestamps in metric dimensions.
- Histogram buckets: use standard OTEL defaults; no runtime-generated buckets.
- Keep meter name stable; adding new instruments requires version note in sprint Decisions & Risks.
## Ownership
- Implementer: Excititor Observability Guild
- Reviewers: Evidence Locker Guild (for parity with attestation metrics)

31
docs-archived/modules/excititor/operations/consensus-removal-runbook.md Normal file
View File

@@ -0,0 +1,31 @@
# Excititor Consensus Removal Runbook (AOC-19-004)
- **Date:** 2025-11-21
- **Scope:** EXCITITOR-CORE-AOC-19-004
- **Goal:** Eliminate legacy consensus/merged severity fields so Excititor remains aggregation-only.
## Cutover steps
1) **Freeze consensus refresh** — `DisableConsensus=true` (default) forces refresh loop off. Keep this enabled during migration.
2) **Schema cleanup** — migrate collections to remove or null legacy fields:
- `vex_consensus` / `vex_consensus_holds`: drop/ignore fields `consensusDigest`, `policyVersion`, `policyRevisionId`, `policyDigest`, `summary`, `signals`, `status` (merged) once Policy takes over.
- `vex_observations` / materialized exports: ensure no merged severity/status fields are written.
- `vex_mirror` exports: stop emitting consensus JSON; retain raw observations only.
3) **Telemetry:** emit counter `excititor.ingest.consensus.disabled` (tags `tenant`, `source`, `connectorId`) once per batch to prove cutover.
4) **Guards:** AOC guards reject any incoming/derived field in `{mergedSeverity, consensusScore, computedStatus}`.
5) **Backfill:** run one-off job to set `consensusDisabled=true` on legacy records and remove merged fields without touching raw observations.
6) **Verification:** regression checklist (per tenant):
- No writes to `vex_consensus*` collections after cutover.
- Ingest + export fixtures show only raw observations/linksets; snapshots deterministic.
- Telemetry counter present; absence of consensus refresh logs.
## Config
```
Excititor:Worker:
DisableConsensus: true # keep true post-cutover
```
## Test plan (after disk space is restored)
- Unit: AOC guard rejects merged fields.
- Integration (Mongo2Go): ingest batch containing merged fields → rejected; telemetry counter increments.
- Worker: start with DisableConsensus=true → consensus refresh loop does not schedule; log once at startup.

125
docs-archived/modules/excititor/operations/evidence-api.md Normal file
View File

@@ -0,0 +1,125 @@
# Excititor Advisory-AI evidence APIs (projection + chunks)
> Covers the read-only evidence surfaces shipped in Sprints 119–120: `/v1/vex/observations/{vulnerabilityId}/{productKey}` and `/v1/vex/evidence/chunks`.
## Scope and determinism
- **Aggregation-only**: no consensus, severity merging, or reachability. Responses carry raw statements plus provenance/signature metadata.
- **Stable ordering**: both endpoints sort by `lastSeen` DESC; pagination uses a deterministic `limit`.
- **Limits**: observation projection default `limit=200`, max `500`; chunk stream default `limit=500`, max `2000`.
- **Tenancy**: reads respect `X-Stella-Tenant` when provided; otherwise fall back to `DefaultTenant` configuration.
- **Auth**: bearer token with `vex.read` scope required.
## `/v1/vex/observations/{vulnerabilityId}/{productKey}`
- **Response**: JSON object with `vulnerabilityId`, `productKey`, `generatedAt`, `totalCount`, `truncated`, `statements[]`.
- **Statement fields**: `observationId`, `providerId`, `status`, `justification`, `detail`, `firstSeen`, `lastSeen`, `scope{key,name,version,purl,cpe,componentIdentifiers[]}`, `anchors[]`, `document{digest,format,revision,sourceUri}`, `signature{type,keyId,issuer,verifiedAt}`.
- **Filters**:
- `providerId` (multi-valued, comma-separated)
- `status` (values in `VexClaimStatus`)
- `since` (ISO-8601, UTC)
- `limit` (ints within bounds)
- **Mapping back to storage**:
- `observationId` = `{providerId}:{document.digest}`
- `document.digest` locates the raw record in `vex_raw`.
- `anchors` contain JSON pointers/paragraph locators from source metadata.
Headers:
- `Excititor-Results-Truncated: true|false`
- `Excititor-Results-Total: <int>`
## `/v1/vex/evidence/chunks`
- **Query params**: `vulnerabilityId` (required), `productKey` (required), optional `providerId`, `status`, `since`, `limit`.
- **Limits**: default `limit=500`, max `2000`.
- **Response**: **NDJSON** stream; each line is a `VexEvidenceChunkResponse`.
- **Chunk fields**: `observationId`, `linksetId`, `vulnerabilityId`, `productKey`, `providerId`, `status`, `justification`, `detail`, `scopeScore` (from confidence or signals), `firstSeen`, `lastSeen`, `scope{...}`, `document{digest,format,sourceUri,revision}`, `signature{type,subject,issuer,keyId,verifiedAt,transparencyRef}`, `metadata` (flattened additionalMetadata).
- **Headers**: `Excititor-Results-Total`, `Excititor-Results-Truncated` (mirrors projection API naming).
- **Streaming guidance (SDK/clients)**:
- Use HTTP client that supports response streaming; read line-by-line and JSON-deserialize per line.
- Treat stream as an NDJSON list up to `limit`; no outer array.
- Back-off or paginate by adjusting `since` or narrowing providers/statuses.
OpenAPI (excerpt):
```yaml
paths:
/v1/vex/evidence/chunks:
get:
summary: Stream evidence chunks for a vulnerability/product
parameters:
- in: query
name: vulnerabilityId
schema: { type: string }
required: true
- in: query
name: productKey
schema: { type: string }
required: true
- in: query
name: providerId
schema: { type: string }
description: Comma-separated provider ids
- in: query
name: status
schema: { type: string }
description: Comma-separated VEX statuses
- in: query
name: since
schema: { type: string, format: date-time }
- in: query
name: limit
schema: { type: integer, minimum: 1, maximum: 2000, default: 500 }
responses:
"200":
description: NDJSON stream of VexEvidenceChunkResponse
headers:
Excititor-Results-Total: { schema: { type: integer } }
Excititor-Results-Truncated: { schema: { type: boolean } }
content:
application/x-ndjson:
schema:
type: string
description: One JSON object per line (VexEvidenceChunkResponse)
```
Example (curl):
```bash
curl -s -H "Authorization: Bearer <token>" \
-H "X-Stella-Tenant: acme" \
"https://exc.example.test/v1/vex/evidence/chunks?vulnerabilityId=CVE-2025-0001&productKey=pkg:docker/demo&limit=2" |
head -n 2
```
Sample NDJSON line:
```json
{"observationId":"provider-a:4d2f...","linksetId":"CVE-2025-0001:pkg:docker/demo","vulnerabilityId":"CVE-2025-0001","productKey":"pkg:docker/demo","providerId":"provider-a","status":"Affected","justification":"ComponentNotPresent","detail":"demo detail","scopeScore":0.9,"firstSeen":"2025-11-10T12:00:00Z","lastSeen":"2025-11-12T12:00:00Z","scope":{"key":"pkg:docker/demo","name":"demo","version":"1.0.0","purl":"pkg:docker/demo","cpe":null,"componentIdentifiers":["component-a"]},"document":{"digest":"sha256:e7...","format":"sbomcyclonedx","sourceUri":"https://example.test/vex.json","revision":"r1"},"signature":{"type":"cosign","subject":"demo","issuer":"issuer","keyId":"kid","verifiedAt":"2025-11-12T12:00:00Z","transparencyRef":null},"metadata":{}}
```
## `/v1/vex/attestations/{attestationId}`
- **Purpose**: Lookup attestation provenance (supplier ↔ observation/linkset ↔ product/vulnerability) without touching consensus.
- **Response**: `VexAttestationPayload` with fields:
- `attestationId`, `supplierId`, `observationId`, `linksetId`, `vulnerabilityId`, `productKey`, `justificationSummary`, `issuedAt`, `metadata{}`.
- **Semantics**:
- `attestationId` matches the export/attestation ID used when signing (Resolve/Worker flows).
- `observationId`/`linksetId` map back to evidence identifiers; clients can stitch provenance for citations.
- **Auth**: `vex.read` scope; tenant header optional (payloads are tenant-agnostic).
## Error model
- Standard API envelope with `ValidationProblem` for missing required params.
- `scope` failures return `403` with problem details.
- Tenancy parse failures return `400`.
## Backwards compatibility
- No legacy routes are deprecated by these endpoints; they are additive and remain aggregation-only.
## References
- Implementation: `src/Excititor/StellaOps.Excititor.WebService/Program.cs` (`/v1/vex/observations/**`, `/v1/vex/evidence/chunks`).
- Telemetry: `src/Excititor/StellaOps.Excititor.WebService/Telemetry/EvidenceTelemetry.cs` (`excititor.vex.observation.*`, `excititor.vex.chunks.*`).
- Data model: `src/Excititor/StellaOps.Excititor.WebService/Contracts/VexObservationContracts.cs`, `Contracts/VexEvidenceChunkContracts.cs`.

52
docs-archived/modules/excititor/operations/graph-linkouts-implementation.md Normal file
View File

@@ -0,0 +1,52 @@
# Excititor · Graph Linkouts & Overlays — Implementation Notes (Graph 21-001/002/005/24-101/24-102)
- **Date:** 2025-11-21
- **Scope:** EXCITITOR-GRAPH-21-001, EXCITITOR-GRAPH-21-002, EXCITITOR-GRAPH-21-005
- **Status:** Implementation guidance (storage wiring pending).
## Endpoints
1) **Linkouts (21-001)**
- `POST /internal/graph/linkouts`
- Body: `tenant`, `purls[]` (max 500), `includeJustifications?`, `includeProvenance?`
- Response: ordered by input `purls`; each item includes `advisories[]` (`advisoryId`, `source`, `status`, `justification?`, `modifiedAt`, `evidenceHash`, `connectorId`, `dsseEnvelopeHash?`) plus `conflicts[]`; `notFound[]`.
2) **Overlays (21-002)**
- `GET /v1/graph/overlays?purl=<purl>&purl=<purl>&includeJustifications=true|false`
- Response per PURL: `summary` counts (`open`, `not_affected`, `under_investigation`, `no_statement`), `latestModifiedAt`, `justifications[]` (unique, sorted), `provenance` (`sources[]`, `lastEvidenceHash`), `cached`, `cacheAgeMs`.
3) **Status summaries (24-101)**
- `GET /v1/graph/status?purl=<purl>&purl=<purl>`
- Response mirrors overlay summaries but omits justification payloads; includes `sources[]`, `lastEvidenceHash`, `cached`, `cacheAgeMs`. Intended for Vuln Explorer status colouring.
4) **Batch observations for tooltips (24-102)**
- `GET /v1/graph/observations?purl=<purl>[&purl=...]&includeJustifications=true|false[&limitPerPurl=50][&cursor=<base64>]`
- Response per PURL: ordered `observations[]` (`observationId`, `advisoryId`, `status`, `justification?`, `providerId`, `modifiedAt`, `evidenceHash`, `dsseEnvelopeHash?`) plus `truncated`; top-level `nextCursor`, `hasMore` enable paging. Limits enforced per PURL and globally.
## Storage & Indexes (21-005)
- `vex_observations` indexes:
- `{ tenant: 1, component.purl: 1, advisoryId: 1, source: 1, modifiedAt: -1 }`
- Sparse `{ tenant: 1, component.purl: 1, status: 1 }`
- Optional materialized `vex_overlays` cache: unique `{ tenant: 1, purl: 1 }`, TTL on `cachedAt` driven by `excititor:graph:overlayTtlSeconds` (default 300s); payload must validate against `docs/modules/excititor/schemas/vex_overlay.schema.json` (schemaVersion 1.0.0). Bundle sample payload `docs/modules/excititor/samples/vex-overlay-sample.json` in Offline Kits.
## Determinism
- Ordering: input PURL order → `advisoryId` → `source` for linkouts; overlays follow input order.
- Truncation: max 200 advisories per PURL; when truncated, include `truncated: true` and `nextCursor` (`advisoryId`, `source`).
## Config knobs
- `excititor:graph:overlayTtlSeconds` (default 300)
- `excititor:graph:maxPurls` (default 500)
- `excititor:graph:maxAdvisoriesPerPurl` (default 200)
- `excititor:graph:maxTooltipItemsPerPurl` (default 50)
- `excititor:graph:maxTooltipTotal` (default 1000)
## Telemetry
- Counter `excititor.graph.linkouts.requests` tags: `tenant`, `includeJustifications`, `includeProvenance`.
- Counter `excititor.graph.overlays.cache` tags: `tenant`, `hit` (`true|false`).
- Histogram `excititor.graph.linkouts.latency.ms` tags: `tenant`.
## Steps to implement
- Bind `GraphOptions` to `Excititor:Graph`.
- Add endpoints to WebService with tenant guard; enforce limits.
- Implement overlay cache with deterministic sort; respect TTL; surface `cached` + `cacheAgeMs`.
- Backfill Mongo indexes above.
- Integration tests (WebApplicationFactory + Mongo2Go) for ordering, truncation, cache metadata, tenant isolation.

62
docs-archived/modules/excititor/operations/observability.md Normal file
View File

@@ -0,0 +1,62 @@
# Excititor Observability Guide
> Added 2025-11-14 alongside Sprint 119 (`EXCITITOR-AIAI-31-003`). Complements the AirGap/mirror runbooks under the same folder.
Excititor’s evidence APIs now emit first-class OpenTelemetry metrics so Lens, Advisory AI, and Ops can detect misuse or missing provenance without paging through logs. This document lists the counters/histograms shipped by the WebService (`src/Excititor/StellaOps.Excititor.WebService`) and how to hook them into your exporters/dashboards.
## Telemetry prerequisites
- Enable `Excititor:Telemetry` in the service configuration (`appsettings.*`), ensuring **metrics** export is on. The WebService automatically adds the evidence meter (`StellaOps.Excititor.WebService.Evidence`) alongside the ingestion meter.
- Deploy at least one OTLP or console exporter (see `TelemetryExtensions.ConfigureExcititorTelemetry`). If your region lacks OTLP transport, fall back to scraping the console exporter for smoke tests.
- Coordinate with the Ops/Signals guild to provision the span/metric sinks referenced in `docs/modules/platform/architecture-overview.md#observability`.
## Metrics reference
| Metric | Type | Description | Key dimensions |
| --- | --- | --- | --- |
| `excititor.vex.observation.requests` | Counter | Number of `/v1/vex/observations/{vulnerabilityId}/{productKey}` requests handled. | `tenant`, `outcome` (`success`, `error`, `cancelled`), `truncated` (`true/false`) |
| `excititor.vex.observation.statement_count` | Histogram | Distribution of statements returned per observation projection request. | `tenant`, `outcome` |
| `excititor.vex.signature.status` | Counter | Signature status per statement (missing vs. unverified). | `tenant`, `status` (`missing`, `unverified`) |
| `excititor.vex.aoc.guard_violations` | Counter | Aggregated count of Aggregation-Only Contract violations detected by the WebService (ingest + `/v1/vex/aoc/verify`). | `tenant`, `surface` (`ingest`, `aoc_verify`, etc.), `code` (AOC error code) |
| `excititor.vex.chunks.requests` | Counter | Requests to `/v1/vex/evidence/chunks` (NDJSON stream). | `tenant`, `outcome` (`success`,`error`,`cancelled`), `truncated` (`true/false`) |
| `excititor.vex.chunks.bytes` | Histogram | Size of NDJSON chunk streams served (bytes). | `tenant`, `outcome` |
| `excititor.vex.chunks.records` | Histogram | Count of evidence records emitted per chunk stream. | `tenant`, `outcome` |
> All metrics originate from the `EvidenceTelemetry` helper (`src/Excititor/StellaOps.Excititor.WebService/Telemetry/EvidenceTelemetry.cs`). When disabled (telemetry off), the helper is inert.
### Dashboard hints
- **Advisory-AI readiness** – alert when `excititor.vex.signature.status{status="missing"}` spikes for a tenant, indicating connectors aren’t supplying signatures.
- **Guardrail monitoring** – graph `excititor.vex.aoc.guard_violations` per `code` to catch upstream feed regressions before they pollute Evidence Locker or Lens caches.
- **Capacity planning** – histogram percentiles of `excititor.vex.observation.statement_count` feed API sizing (higher counts mean Advisory AI is requesting broad scopes).
## Operational steps
1. **Enable telemetry**: set `Excititor:Telemetry:EnableMetrics=true`, configure OTLP endpoints/headers as described in `TelemetryExtensions`.
2. **Add dashboards**: import panels referencing the metrics above (see Grafana JSON snippets in Ops repo once merged).
3. **Alerting**: add rules for high guard violation rates, missing signatures, and abnormal chunk bytes/record counts. Tie alerts back to connectors via tenant metadata.
4. **Post-deploy checks**: after each release, verify metrics emit by curling `/v1/vex/observations/...` and `/v1/vex/evidence/chunks`, watching the console exporter (dev) or OTLP (prod).
## SLOs (Sprint 119 – OBS-51-001)
The following SLOs apply to Excititor evidence read paths when telemetry is enabled. Record them in the shared SLO registry and alert via the platform alertmanager.
| Surface | SLI | Target | Window | Burn alert | Notes |
| --- | --- | --- | --- | --- | --- |
| `/v1/vex/observations` | p95 latency | ≤ 450 ms | 7d | 2 % over 1h | Measured on successful responses only; tenant scoped. |
| `/v1/vex/observations` | freshness | ≥ 99 % within 5 min of upstream ingest | 7d | 5 % over 4h | Derived from arrival minus `createdAt`; requires ingest clocks in UTC. |
| `/v1/vex/observations` | signature presence | ≥ 98 % statements with signature present | 7d | 3 % over 24h | Use `excititor.vex.signature.status{status="missing"}`. |
| `/v1/vex/evidence/chunks` | p95 stream duration | ≤ 600 ms | 7d | 2 % over 1h | From request start to last NDJSON write; excludes client disconnects. |
| `/v1/vex/evidence/chunks` | truncation rate | ≤ 1 % truncated streams | 7d | 1 % over 1h | `excititor.vex.chunks.records` with `truncated=true`. |
| AOC guardrail | zero hard violations | 0 | continuous | immediate | Any `excititor.vex.aoc.guard_violations` with severity `error` pages ops. |
Implementation notes:
- Emit latency/freshness SLOs via OTEL views that pre-aggregate by tenant and route to the platform SLO backend; keep bucket boundaries aligned with 50/100/250/450/650/1000 ms.
- Freshness SLI derived from ingest timestamps; ensure clocks are synchronized (NTP) and stored in UTC.
- For air-gapped deployments without OTEL sinks, scrape console exporter and push to offline Prometheus; same thresholds apply.
## Related documents
- `docs/modules/excititor/architecture.md` – API contract, AOC guardrails, connector responsibilities.
- `docs/modules/excititor/mirrors.md` – AirGap/mirror ingestion checklist (feeds into `EXCITITOR-AIRGAP-56/57`).
- `docs/modules/platform/architecture-overview.md#observability` – platform-wide telemetry guidance.

39
docs-archived/modules/excititor/operations/tenant-authority-client.md Normal file
View File

@@ -0,0 +1,39 @@
# Excititor Tenant Authority Client (AOC-19-013)
- **Date:** 2025-11-21
- **Scope:** EXCITITOR-CORE-AOC-19-013
- **Files:** `src/Excititor/StellaOps.Excititor.Worker/Auth/TenantAuthorityClientFactory.cs`
## Contract
- Every outbound Authority call must carry `X-Tenant` header and use tenant-specific base URL.
- Base URLs and optional client credentials are configured under `Excititor:Authority:` with per-tenant keys.
- Factory throws when tenant is missing or not configured to prevent cross-tenant leakage.
## Configuration shape
```json
{
"Excititor": {
"Authority": {
"BaseUrls": {
"alpha": "https://authority.alpha.local/",
"bravo": "https://authority.bravo.local/"
},
"ClientIds": {
"alpha": "alpha-client-id"
},
"ClientSecrets": {
"alpha": "alpha-secret"
}
}
}
}
```
## Implementation notes
- `TenantAuthorityClientFactory` (worker) enforces tenant presence and configured base URL; adds `Accept: application/json` and `X-Tenant` headers.
- Registered in DI via `Program.cs` with options binding to `Excititor:Authority`.
- Intended to be reused by WebService/Worker components once disk space block is resolved.
## Next steps
- Wire factory into services that call Authority (WebService + Worker jobs), replacing any tenant-agnostic HttpClient usages.
- Add integration tests to ensure cross-tenant calls reject when config missing or header mismatched.

66
docs-archived/modules/excititor/operations/ubuntu-csaf.md Normal file
View File

@@ -0,0 +1,66 @@
# Ubuntu CSAF connector runbook
> Updated 2025-11-09 alongside sprint 110/120 trust-provenance work.
## Purpose
- Ingest Ubuntu USN/CSAF statements via the restart-only connector (`StellaOps.Excititor.Connectors.Ubuntu.CSAF`).
- Preserve Aggregation-Only Contract guarantees while surfacing issuance provenance (`vex.provenance.*`) for VEX Lens and Policy Engine.
- Allow operators to tune trust weighting (tiers, fingerprints, cosign issuers) without recompiling the connector.
## Configuration keys
| Key | Default | Notes |
| --- | --- | --- |
| `Excititor:Connectors:Ubuntu:IndexUri` | `https://ubuntu.com/security/csaf/index.json` | Ubuntu CSAF index. Override only when mirroring the feed. |
| `...:Channels` | `["stable"]` | List of channel names to poll. Order preserved for deterministic cursoring. |
| `...:MetadataCacheDuration` | `4h` | How long to cache catalog metadata before re-fetching. |
| `...:PreferOfflineSnapshot` / `OfflineSnapshotPath` / `PersistOfflineSnapshot` | `false` / `null` / `true` | Enable when running from Offline Kit bundles. Snapshot path must be reachable/read-only under sealed deployments. |
| `...:TrustWeight` | `0.75` | Baseline trust weight (0–1). Lens multiplies this by freshness/justification modifiers. |
| `...:TrustTier` | `"distro"` | Friendly tier label surfaced via `vex.provenance.trust.tier` (e.g., `distro-trusted`, `community`). |
| `...:CosignIssuer` / `CosignIdentityPattern` | `null` | Supply when Ubuntu publishes cosign attestations (issuer URL and identity regex). Required together. |
| `...:PgpFingerprints` | `[]` | Ordered list of trusted PGP fingerprints. Emitted verbatim as `vex.provenance.pgp.fingerprints`. |
## Example `appsettings.json`
```jsonc
{
"Excititor": {
"Connectors": {
"Ubuntu": {
"IndexUri": "https://mirror.example.com/security/csaf/index.json",
"Channels": ["stable", "esm-apps"],
"TrustWeight": 0.82,
"TrustTier": "distro-trusted",
"CosignIssuer": "https://issuer.ubuntu.com",
"CosignIdentityPattern": "spiffe://ubuntu/vex/*",
"PgpFingerprints": [
"0123456789ABCDEF0123456789ABCDEF01234567",
"89ABCDEF0123456789ABCDEF0123456789ABCDEF"
],
"PreferOfflineSnapshot": true,
"OfflineSnapshotPath": "/opt/stella/offline/ubuntu/index.json"
}
}
}
}
```
## Environment variable cheatsheet
```
Excititor__Connectors__Ubuntu__TrustWeight=0.9
Excititor__Connectors__Ubuntu__TrustTier=distro-critical
Excititor__Connectors__Ubuntu__PgpFingerprints__0=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Excititor__Connectors__Ubuntu__PgpFingerprints__1=BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
Excititor__Connectors__Ubuntu__CosignIssuer=https://issuer.ubuntu.com
Excititor__Connectors__Ubuntu__CosignIdentityPattern=spiffe://ubuntu/vex/*
```
## Operational checklist
1. **Before enabling** – import the Ubuntu PGP bundle (Offline Kit provides `certificates/ubuntu-vex.gpg`) and set the fingerprints so provenance metadata stays deterministic.
2. **Validate provenance output** – run `dotnet test src/Excititor/__Tests/StellaOps.Excititor.Connectors.Ubuntu.CSAF.Tests --filter FetchAsync_IngestsNewDocument` to ensure the connector emits the `vex.provenance.*` fields expected by VEX Lens.
3. **Monitor Lens weights** – Grafana panels `VEX Lens / Trust Inputs` show the weight/tier captured per provider. Ubuntu rows should reflect the configured `TrustWeight` and fingerprints.
4. **Rotate fingerprints** – update `PgpFingerprints` when Canonical rotates signing keys. Apply the change, restart Excititor workers, verify the provenance metadata, then trigger a targeted Lens recompute for Ubuntu issuers.
5. **Offline mode** – populate `OfflineSnapshotPath` via Offline Kit bundles before toggling `PreferOfflineSnapshot`. Keep snapshots in the sealed `/opt/stella/offline` hierarchy for auditability.
## Troubleshooting
- **Connector refuses to start** – check logs for `InvalidOperationException` referencing `CosignIssuer`/`CosignIdentityPattern` or missing snapshot path; the validator enforces complete pairs and on-disk paths.
- **Lens still sees default weights** – confirm the Excititor deployment picked up the new settings (view `/excititor/health` JSON → `connectors.providers[].options`). Lens only overrides when the provenance payload includes `vex.provenance.trust.*` fields.
- **PGP mismatch alerts** – if Lens reports fingerprint mismatches, ensure the list ordering matches Canonical’s published order; duplicates are trimmed, so provide each fingerprint once.

18
docs-archived/modules/excititor/samples/chunk-attestation-sample.json Normal file
View File

@@ -0,0 +1,18 @@
{
"subject_digest": "sha256:112233",
"predicates": {
"stellaops.vex.chunk.meta.v1": {
"tenant": "acme",
"source": "ghsa",
"schema": "stellaops.vex.chunk.v1",
"item_count": 1
},
"stellaops.vex.chunk.integrity.v1": {
"items": [
{"ordinal": 0, "sha256": "abc"}
]
}
},
"signing_profile": "sovereign-default",
"transparency": null
}

1
docs-archived/modules/excititor/samples/chunk-sample.ndjson Normal file
View File

@@ -0,0 +1 @@
{"chunk_id":"11111111-2222-3333-4444-555555555555","tenant":"acme","source":"ghsa","schema":"stellaops.vex.chunk.v1","items":[{"advisory_id":"GHSA-123","status":"affected","purl":"pkg:npm/foo@1.0.0"}],"provenance":{"fetched_at":"2025-11-20T00:00:00Z","artifact_sha":"abc"}}

2
docs-archived/modules/excititor/samples/chunks-sample.ndjson Normal file
View File

@@ -0,0 +1,2 @@
{"tenant":"demo","vulnerabilityId":"CVE-2024-1234","productKey":"pkg:pypi/django@3.2.24","observationId":"obs-001","statementId":"stmt-001","source":{"supplier":"upstream:osv","documentId":"osv:CVE-2024-1234","retrievedAt":"2025-11-18T12:00:00Z","signatureStatus":"missing"},"aoc":{"violations":[]},"evidence":{"type":"vex.statement","payload":{"status":"not_affected","justification":"component_not_present"}},"provenance":{"hash":"sha256:dummyhash","canonicalUri":null,"bundleId":null}}
{"tenant":"demo","vulnerabilityId":"CVE-2024-2345","productKey":"pkg:pypi/requests@2.31.0","observationId":"obs-002","statementId":"stmt-001","source":{"supplier":"upstream:osv","documentId":"osv:CVE-2024-2345","retrievedAt":"2025-11-18T12:05:00Z","signatureStatus":"unverified"},"aoc":{"violations":[{"code":"EVIDENCE_SIGNATURE_MISSING","surface":"ingest"}]},"evidence":{"type":"vex.statement","payload":{"status":"affected","impact":"info","details":"placeholder"}},"provenance":{"hash":"sha256:dummyhash2","canonicalUri":null,"bundleId":null}}

1
docs-archived/modules/excititor/samples/chunks-sample.ndjson.sha256 Normal file
View File

@@ -0,0 +1 @@
4d638b24d6f8f703bcbcac23a0185265f3db5defb9f3d7f33b7be7fccc0de738 docs/modules/excititor/samples/chunks-sample.ndjson

93
docs-archived/modules/excititor/samples/connector-signer-metadata-sample.json Normal file
View File

@@ -0,0 +1,93 @@
{
"schemaVersion": "1.0.0",
"generatedAt": "2025-11-20T00:00:00Z",
"connectors": [
{
"connectorId": "excititor:msrc",
"provider": { "name": "Microsoft Security Response Center", "slug": "msrc" },
"issuerTier": "tier-1",
"signers": [
{
"usage": "csaf",
"fingerprints": [
{"alg": "sha256", "format": "pgp", "value": "F1C3D9E4A7B28C5FD6E1A203B947C2A0C5D8BEEF"},
{"alg": "sha256", "format": "x509-spki", "value": "5A1F4C0E9B27D0C64EAC1F22C3F501AA9FCB77AC8B1D4F9F3EA7E6B4CE90F311"}
],
"keyLocator": "oci://mirror.stella.local/keys/msrc-csaf@sha256:793dd8a6..."
}
],
"bundle": {
"kind": "oci-referrer",
"uri": "oci://mirror.stella.local/msrc/csaf:2025-11-19",
"digest": "sha256:4b8c9fd6e479e1b6dcd2e7ed93a85c1c7d6052f7b4a6b83471e44f5c9c2a1f30",
"publishedAt": "2025-11-19T12:00:00Z"
},
"validFrom": "2025-11-01"
},
{
"connectorId": "excititor:oracle",
"provider": { "name": "Oracle", "slug": "oracle" },
"issuerTier": "tier-1",
"signers": [
{
"usage": "oval",
"fingerprints": [
{"alg": "sha256", "format": "x509-spki", "value": "6E3AC4A95BD5402F4C7E9B2371190E0F3B3C11C7B42B88652E7EE0F659A0D202"}
],
"keyLocator": "file://offline-kits/oracle/oval/signing-chain.pem",
"certificateChain": ["-----BEGIN CERTIFICATE-----\nMIID...oracle-root...\n-----END CERTIFICATE-----"]
}
],
"bundle": {
"kind": "file",
"uri": "file://offline-kits/oracle/oval/oval-feed-2025-11-18.tar.gz",
"digest": "sha256:b13b1b84af1da7ee3433e0c6c0cc28a8b5c7d3e52d93b9f86d4a4b0f1dcd8f05",
"publishedAt": "2025-11-18T09:30:00Z"
},
"validFrom": "2025-10-15"
},
{
"connectorId": "excititor:oci.openvex.attest",
"provider": { "name": "StellaOps Mirror", "slug": "stella-mirror" },
"issuerTier": "tier-0",
"signers": [
{
"usage": "openvex",
"fingerprints": [
{"alg": "sha256", "format": "cosign", "value": "a0c1d4e5f6b7982134d56789e0fab12345cdef6789abcdeffedcba9876543210"}
],
"keyLocator": "oci://mirror.stella.local/keys/stella-mirror-openvex:1",
"certificateChain": []
}
],
"bundle": {
"kind": "oci-tag",
"uri": "oci://mirror.stella.local/stellaops/openvex:2025-11-19",
"digest": "sha256:77f6c0b8f2c9845c7d0a4f3b783b0caf00cce6fb899319ff69cb941fe2c58010",
"publishedAt": "2025-11-19T15:00:00Z"
},
"validFrom": "2025-11-15"
},
{
"connectorId": "excititor:ubuntu",
"provider": { "name": "Ubuntu Security", "slug": "ubuntu" },
"issuerTier": "tier-2",
"signers": [
{
"usage": "oval",
"fingerprints": [
{"alg": "sha256", "format": "pgp", "value": "7D19E3B4A5F67C103CB0B4DE0FA28F90D6E4C1D2"}
],
"keyLocator": "tuf://mirror.stella.local/tuf/ubuntu/targets/oval-signing.pub"
}
],
"bundle": {
"kind": "tuf",
"uri": "tuf://mirror.stella.local/tuf/ubuntu/oval/targets/oval-2025-11-18.tar.gz",
"digest": "sha256:e41c4fc15132f8848e9924a1a0f1a247d3c56da87b7735b6c6d8cbe64f0f07e5",
"publishedAt": "2025-11-18T07:00:00Z"
},
"validFrom": "2025-11-01"
}
]
}

1
docs-archived/modules/excititor/samples/connector-signer-metadata-sample.json.sha256 Normal file
View File

@@ -0,0 +1 @@
a2f0986d938d877adf01a76b7a9e79cc148f330e57348569619485feb994df1d connector-signer-metadata-sample.json

50
docs-archived/modules/excititor/samples/vex-overlay-sample.json Normal file
View File

@@ -0,0 +1,50 @@
{
"schemaVersion": "1.0.0",
"generatedAt": "2025-12-10T00:00:00Z",
"tenant": "tenant-default",
"purl": "pkg:maven/org.example/foo@1.2.3",
"advisoryId": "GHSA-xxxx-yyyy-zzzz",
"source": "ghsa",
"status": "affected",
"justifications": [
{
"kind": "known_affected",
"reason": "Upstream GHSA reports affected range <1.3.0.",
"evidence": ["concelier:ghsa:obs:6561e41b3e3f4a6e9d3b91c1"],
"weight": 0.8
}
],
"conflicts": [
{
"field": "affected.versions",
"reason": "vendor_range_differs",
"values": ["<1.2.0", "<=1.3.0"],
"sourceIds": [
"concelier:redhat:obs:6561e41b3e3f4a6e9d3b91a1",
"concelier:ghsa:obs:6561e41b3e3f4a6e9d3b91c1"
]
}
],
"observations": [
{
"id": "concelier:ghsa:obs:6561e41b3e3f4a6e9d3b91c1",
"contentHash": "sha256:1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd",
"fetchedAt": "2025-11-19T00:00:00Z"
}
],
"provenance": {
"linksetId": "concelier:ghsa:linkset:6561e41b3e3f4a6e9d3b91d0",
"linksetHash": "sha256:deaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddeaddead",
"observationHashes": [
"sha256:1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd"
],
"policyHash": "sha256:0f7c0f7c0f7c0f7c0f7c0f7c0f7c0f7c0f7c0f7c0f7c0f7c0f7c0f7c0f7c0f7c",
"sbomContextHash": "sha256:421af53f9eeba6903098d292fbd56f98be62ea6130b5161859889bf11d699d18",
"planCacheKey": "tenant-default|pkg:maven/org.example/foo@1.2.3|GHSA-xxxx-yyyy-zzzz"
},
"cache": {
"cached": true,
"cachedAt": "2025-12-10T00:00:00Z",
"ttlSeconds": 300
}
}

125
docs-archived/modules/excititor/schemas/connector-signer-metadata.schema.json Normal file
View File

@@ -0,0 +1,125 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://stellaops.dev/schemas/excititor/connector-signer-metadata.schema.json",
"title": "Excititor Connector Signer Metadata",
"type": "object",
"additionalProperties": false,
"required": ["schemaVersion", "generatedAt", "connectors"],
"properties": {
"schemaVersion": {
"type": "string",
"pattern": "^1\\.0\\.0$"
},
"generatedAt": {
"type": "string",
"format": "date-time"
},
"connectors": {
"type": "array",
"minItems": 1,
"items": {
"$ref": "#/$defs/connector"
}
}
},
"$defs": {
"connector": {
"type": "object",
"additionalProperties": false,
"required": [
"connectorId",
"provider",
"issuerTier",
"signers"
],
"properties": {
"connectorId": {
"type": "string",
"pattern": "^[a-z0-9:-\\.]+$"
},
"provider": {
"type": "object",
"additionalProperties": false,
"required": ["name", "slug"],
"properties": {
"name": { "type": "string", "minLength": 3 },
"slug": { "type": "string", "pattern": "^[a-z0-9-]+$" }
}
},
"issuerTier": {
"type": "string",
"enum": ["tier-0", "tier-1", "tier-2", "untrusted"]
},
"signers": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/signer" }
},
"bundle": { "$ref": "#/$defs/bundleRef" },
"validFrom": { "type": "string", "format": "date" },
"validTo": { "type": "string", "format": "date" },
"revoked": { "type": "boolean", "default": false },
"notes": { "type": "string", "maxLength": 2000 }
}
},
"signer": {
"type": "object",
"additionalProperties": false,
"required": ["usage", "fingerprints"],
"properties": {
"usage": {
"type": "string",
"enum": ["csaf", "oval", "openvex", "bulk-meta", "attestation"]
},
"fingerprints": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/fingerprint" }
},
"keyLocator": {
"type": "string",
"description": "Path or URL (mirror/OCI/TUF) where the signing key or certificate chain can be retrieved in offline kits."
},
"certificateChain": {
"type": "array",
"items": { "type": "string" },
"description": "Optional PEM-encoded certificates for x509/cosign keys."
}
}
},
"fingerprint": {
"type": "object",
"additionalProperties": false,
"required": ["alg", "value"],
"properties": {
"alg": {
"type": "string",
"enum": ["sha256", "sha512", "sha1"]
},
"format": {
"type": "string",
"enum": ["pgp", "x509-spki", "x509-ski", "cosign", "pem"]
},
"value": {
"type": "string",
"minLength": 16,
"maxLength": 128
}
}
},
"bundleRef": {
"type": "object",
"additionalProperties": false,
"required": ["kind", "uri"],
"properties": {
"kind": {
"type": "string",
"enum": ["oci-referrer", "oci-tag", "file", "tuf"]
},
"uri": { "type": "string", "minLength": 8 },
"digest": { "type": "string", "minLength": 32 },
"publishedAt": { "type": "string", "format": "date-time" }
}
}
}
}

305
docs-archived/modules/excititor/schemas/issuer_directory_contract.md Normal file
View File

@@ -0,0 +1,305 @@
# Issuer Directory Contract v1.0.0
**Status:** APPROVED
**Version:** 1.0.0
**Effective:** 2025-12-19
**Owner:** VEX Lens Guild + Issuer Directory Guild
**Sprint:** SPRINT_0129_0001_0001 (unblocks VEXLENS-30-003)
---
## 1. Purpose
The Issuer Directory provides a registry of known VEX statement issuers with trust metadata, signing key information, and provenance tracking.
## 2. Data Model
### 2.1 Issuer Entity
```csharp
public sealed record Issuer
{
/// <summary>Unique issuer identifier (e.g., "vendor:redhat", "cert:cisa").</summary>
public required string IssuerId { get; init; }
/// <summary>Issuer category.</summary>
public required IssuerCategory Category { get; init; }
/// <summary>Display name.</summary>
public required string DisplayName { get; init; }
/// <summary>Trust tier assignment.</summary>
public required IssuerTrustTier TrustTier { get; init; }
/// <summary>Official website URL.</summary>
public string? WebsiteUrl { get; init; }
/// <summary>Security advisory feed URL.</summary>
public string? AdvisoryFeedUrl { get; init; }
/// <summary>Registered signing keys.</summary>
public ImmutableArray<SigningKeyInfo> SigningKeys { get; init; }
/// <summary>Products/ecosystems this issuer is authoritative for.</summary>
public ImmutableArray<string> AuthoritativeFor { get; init; }
/// <summary>When this issuer record was created.</summary>
public DateTimeOffset CreatedAt { get; init; }
/// <summary>When this issuer record was last updated.</summary>
public DateTimeOffset UpdatedAt { get; init; }
/// <summary>Whether issuer is active.</summary>
public bool IsActive { get; init; } = true;
}
```
### 2.2 Issuer Category
```csharp
public enum IssuerCategory
{
/// <summary>Software vendor/maintainer.</summary>
Vendor = 0,
/// <summary>Linux distribution.</summary>
Distribution = 1,
/// <summary>CERT/security response team.</summary>
Cert = 2,
/// <summary>Security research organization.</summary>
SecurityResearch = 3,
/// <summary>Community project.</summary>
Community = 4,
/// <summary>Commercial security vendor.</summary>
Commercial = 5
}
```
### 2.3 Signing Key Info
```csharp
public sealed record SigningKeyInfo
{
/// <summary>Key fingerprint (SHA-256).</summary>
public required string Fingerprint { get; init; }
/// <summary>Key type (pgp, x509, sigstore).</summary>
public required string KeyType { get; init; }
/// <summary>Key algorithm (rsa, ecdsa, ed25519).</summary>
public string? Algorithm { get; init; }
/// <summary>Key size in bits.</summary>
public int? KeySize { get; init; }
/// <summary>Key creation date.</summary>
public DateTimeOffset? CreatedAt { get; init; }
/// <summary>Key expiration date.</summary>
public DateTimeOffset? ExpiresAt { get; init; }
/// <summary>Whether key is currently valid.</summary>
public bool IsValid { get; init; } = true;
/// <summary>Public key location (URL or inline).</summary>
public string? PublicKeyUri { get; init; }
}
```
## 3. Pre-Registered Issuers
### 3.1 Authoritative Tier (Trust Tier 0)
| Issuer ID | Display Name | Category | Authoritative For |
|-----------|--------------|----------|-------------------|
| `vendor:redhat` | Red Hat Product Security | Vendor | `pkg:rpm/redhat/*`, `pkg:oci/registry.redhat.io/*` |
| `vendor:canonical` | Ubuntu Security Team | Distribution | `pkg:deb/ubuntu/*` |
| `vendor:debian` | Debian Security Team | Distribution | `pkg:deb/debian/*` |
| `vendor:suse` | SUSE Security Team | Distribution | `pkg:rpm/suse/*`, `pkg:rpm/opensuse/*` |
| `vendor:microsoft` | Microsoft Security Response | Vendor | `pkg:nuget/*` (Microsoft packages) |
| `vendor:oracle` | Oracle Security | Vendor | `pkg:maven/com.oracle.*/*` |
| `vendor:apache` | Apache Security Team | Community | `pkg:maven/org.apache.*/*` |
| `vendor:google` | Google Security Team | Vendor | `pkg:golang/google.golang.org/*` |
### 3.2 Trusted Tier (Trust Tier 1)
| Issuer ID | Display Name | Category |
|-----------|--------------|----------|
| `cert:cisa` | CISA | Cert |
| `cert:nist` | NIST NVD | Cert |
| `cert:github` | GitHub Security Advisories | SecurityResearch |
| `cert:snyk` | Snyk Security | Commercial |
| `research:oss-fuzz` | Google OSS-Fuzz | SecurityResearch |
### 3.3 Community Tier (Trust Tier 2)
| Issuer ID | Display Name | Category |
|-----------|--------------|----------|
| `community:osv` | OSV (Open Source Vulnerabilities) | Community |
| `community:vulndb` | VulnDB | Community |
## 4. API Endpoints
### 4.1 List Issuers
```
GET /api/v1/issuers
```
Query Parameters:
- `category`: Filter by category
- `trust_tier`: Filter by trust tier
- `active`: Filter by active status (default: true)
- `limit`: Max results (default: 100)
- `cursor`: Pagination cursor
### 4.2 Get Issuer
```
GET /api/v1/issuers/{issuerId}
```
### 4.3 Register Issuer (Admin)
```
POST /api/v1/issuers
Authorization: Bearer {admin_token}
{
"issuerId": "vendor:acme",
"category": "vendor",
"displayName": "ACME Security",
"trustTier": "trusted",
"websiteUrl": "https://security.acme.example",
"advisoryFeedUrl": "https://security.acme.example/feed.json",
"authoritativeFor": ["pkg:npm/@acme/*"]
}
```
### 4.4 Register Signing Key (Admin)
```
POST /api/v1/issuers/{issuerId}/keys
Authorization: Bearer {admin_token}
{
"fingerprint": "sha256:abc123...",
"keyType": "pgp",
"algorithm": "rsa",
"keySize": 4096,
"publicKeyUri": "https://security.acme.example/keys/signing.asc"
}
```
### 4.5 Lookup by Fingerprint
```
GET /api/v1/issuers/by-fingerprint/{fingerprint}
```
Returns the issuer associated with a signing key fingerprint.
## 5. Trust Tier Resolution
### 5.1 Automatic Assignment
When a VEX statement is received:
1. **Check signature:** If signed, lookup issuer by key fingerprint
2. **Check domain:** Match issuer by advisory feed domain
3. **Check authoritativeFor:** Match issuer by product PURL patterns
4. **Fallback:** Assign `Unknown` tier if no match
### 5.2 Override Rules
Operators can configure trust overrides:
```yaml
# etc/vexlens.yaml
issuer_overrides:
- issuer_id: "community:custom-feed"
trust_tier: "trusted" # Promote community to trusted
- issuer_id: "vendor:untrusted-vendor"
trust_tier: "community" # Demote vendor to community
```
## 6. Issuer Verification
### 6.1 PGP Signature Verification
```csharp
public interface IIssuerVerifier
{
/// <summary>
/// Verifies a VEX document signature against registered issuer keys.
/// </summary>
Task<IssuerVerificationResult> VerifyAsync(
byte[] documentBytes,
byte[] signatureBytes,
CancellationToken cancellationToken = default);
}
public sealed record IssuerVerificationResult
{
public bool IsValid { get; init; }
public string? IssuerId { get; init; }
public string? KeyFingerprint { get; init; }
public IssuerTrustTier? TrustTier { get; init; }
public string? VerificationError { get; init; }
}
```
### 6.2 Sigstore Verification
For Sigstore-signed documents:
1. Verify Rekor inclusion proof
2. Extract OIDC identity from certificate
3. Match identity to registered issuer
4. Return issuer info with trust tier
## 7. Database Schema
```sql
CREATE TABLE vex.issuers (
issuer_id TEXT PRIMARY KEY,
category TEXT NOT NULL,
display_name TEXT NOT NULL,
trust_tier INT NOT NULL DEFAULT 3,
website_url TEXT,
advisory_feed_url TEXT,
authoritative_for TEXT[] DEFAULT '{}',
is_active BOOLEAN DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE vex.issuer_signing_keys (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
issuer_id TEXT NOT NULL REFERENCES vex.issuers(issuer_id),
fingerprint TEXT NOT NULL UNIQUE,
key_type TEXT NOT NULL,
algorithm TEXT,
key_size INT,
public_key_uri TEXT,
is_valid BOOLEAN DEFAULT TRUE,
created_at TIMESTAMPTZ,
expires_at TIMESTAMPTZ,
registered_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX idx_issuer_signing_keys_fingerprint ON vex.issuer_signing_keys(fingerprint);
CREATE INDEX idx_issuers_trust_tier ON vex.issuers(trust_tier);
```
---
## Changelog
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0 | 2025-12-19 | Initial release |

82
docs-archived/modules/excititor/schemas/vex-chunk-api.yaml Normal file
View File

@@ -0,0 +1,82 @@
openapi: 3.1.0
info:
title: StellaOps Excititor Chunk API
version: "0.1.0"
description: |
Frozen for Sprint 110 (EXCITITOR-AIAI-31-002). Aligns with Evidence Locker attestation contract v1.
servers:
- url: https://excitor.local
paths:
/vex/evidence/chunks:
post:
summary: Submit VEX evidence chunk (aggregation-only)
requestBody:
required: true
content:
application/x-ndjson:
schema:
$ref: '#/components/schemas/VexChunk'
responses:
'202':
description: Accepted for processing
content:
application/json:
schema:
type: object
required: [chunk_digest, queue_id]
properties:
chunk_digest:
type: string
description: sha256 of canonical chunk JSON
queue_id:
type: string
description: Background job identifier
'400':
description: Validation error
components:
schemas:
VexChunk:
type: object
required: [chunk_id, tenant, source, schema, items, provenance]
properties:
chunk_id:
type: string
format: uuid
tenant:
type: string
source:
type: string
description: feed id (e.g., ghsa, nvd)
schema:
type: string
enum: [stellaops.vex.chunk.v1]
items:
type: array
items:
type: object
required: [advisory_id, status, purl]
properties:
advisory_id:
type: string
status:
type: string
enum: [affected, unaffected, under_investigation, fixed, unknown]
purl:
type: string
justification:
type: string
last_observed:
type: string
format: date-time
provenance:
type: object
required: [fetched_at, artifact_sha]
properties:
fetched_at:
type: string
format: date-time
artifact_sha:
type: string
signature:
type: object
nullable: true

271
docs-archived/modules/excititor/schemas/vex_normalization_contract.md Normal file
View File

@@ -0,0 +1,271 @@
# VEX Normalization Contract v1.0.0
**Status:** APPROVED
**Version:** 1.0.0
**Effective:** 2025-12-19
**Owner:** VEX Lens Guild
**Sprint:** SPRINT_0129_0001_0001 (unblocks VEXLENS-30-001 through 30-011)
---
## 1. Purpose
This contract defines the normalization rules for VEX (Vulnerability Exploitability eXchange) documents from multiple sources into a canonical StellaOps internal representation.
## 2. Supported Input Formats
| Format | Version | Parser |
|--------|---------|--------|
| OpenVEX | 0.2.0+ | `OpenVexParser` |
| CycloneDX VEX | 1.5+ | `CycloneDxVexParser` |
| CSAF VEX | 2.0 | `CsafVexParser` |
## 3. Canonical Representation
### 3.1 NormalizedVexStatement
```csharp
public sealed record NormalizedVexStatement
{
/// <summary>Unique statement identifier (deterministic hash).</summary>
public required string StatementId { get; init; }
/// <summary>CVE or vulnerability identifier.</summary>
public required string VulnerabilityId { get; init; }
/// <summary>Normalized status (not_affected, affected, fixed, under_investigation).</summary>
public required VexStatus Status { get; init; }
/// <summary>Justification code (when status = not_affected).</summary>
public VexJustification? Justification { get; init; }
/// <summary>Human-readable impact statement.</summary>
public string? ImpactStatement { get; init; }
/// <summary>Action statement for remediation.</summary>
public string? ActionStatement { get; init; }
/// <summary>Products affected by this statement.</summary>
public required ImmutableArray<ProductIdentifier> Products { get; init; }
/// <summary>Source document metadata.</summary>
public required VexSourceMetadata Source { get; init; }
/// <summary>Statement timestamp (UTC, ISO-8601).</summary>
public required DateTimeOffset Timestamp { get; init; }
/// <summary>Issuer information.</summary>
public required IssuerInfo Issuer { get; init; }
}
```
### 3.2 VexStatus Enum
```csharp
public enum VexStatus
{
/// <summary>Product is not affected by the vulnerability.</summary>
NotAffected = 0,
/// <summary>Product is affected and vulnerable.</summary>
Affected = 1,
/// <summary>Product was affected but is now fixed.</summary>
Fixed = 2,
/// <summary>Impact is being investigated.</summary>
UnderInvestigation = 3
}
```
### 3.3 VexJustification Enum
```csharp
public enum VexJustification
{
/// <summary>Component is not present.</summary>
ComponentNotPresent = 0,
/// <summary>Vulnerable code is not present.</summary>
VulnerableCodeNotPresent = 1,
/// <summary>Vulnerable code is not in execute path.</summary>
VulnerableCodeNotInExecutePath = 2,
/// <summary>Vulnerable code cannot be controlled by adversary.</summary>
VulnerableCodeCannotBeControlledByAdversary = 3,
/// <summary>Inline mitigations exist.</summary>
InlineMitigationsAlreadyExist = 4
}
```
## 4. Normalization Rules
### 4.1 Status Mapping
| Source Format | Source Value | Normalized Status |
|---------------|--------------|-------------------|
| OpenVEX | `not_affected` | NotAffected |
| OpenVEX | `affected` | Affected |
| OpenVEX | `fixed` | Fixed |
| OpenVEX | `under_investigation` | UnderInvestigation |
| CycloneDX | `notAffected` | NotAffected |
| CycloneDX | `affected` | Affected |
| CycloneDX | `resolved` | Fixed |
| CycloneDX | `inTriage` | UnderInvestigation |
| CSAF | `not_affected` | NotAffected |
| CSAF | `known_affected` | Affected |
| CSAF | `fixed` | Fixed |
| CSAF | `under_investigation` | UnderInvestigation |
### 4.2 Justification Mapping
| Source Format | Source Value | Normalized Justification |
|---------------|--------------|--------------------------|
| OpenVEX | `component_not_present` | ComponentNotPresent |
| OpenVEX | `vulnerable_code_not_present` | VulnerableCodeNotPresent |
| OpenVEX | `vulnerable_code_not_in_execute_path` | VulnerableCodeNotInExecutePath |
| OpenVEX | `vulnerable_code_cannot_be_controlled_by_adversary` | VulnerableCodeCannotBeControlledByAdversary |
| OpenVEX | `inline_mitigations_already_exist` | InlineMitigationsAlreadyExist |
| CycloneDX | Same as OpenVEX (camelCase) | Same mapping |
| CSAF | `component_not_present` | ComponentNotPresent |
| CSAF | `vulnerable_code_not_present` | VulnerableCodeNotPresent |
| CSAF | `vulnerable_code_not_in_execute_path` | VulnerableCodeNotInExecutePath |
| CSAF | `vulnerable_code_cannot_be_controlled_by_adversary` | VulnerableCodeCannotBeControlledByAdversary |
| CSAF | `inline_mitigations_already_exist` | InlineMitigationsAlreadyExist |
### 4.3 Product Identifier Normalization
Products are normalized to PURL (Package URL) format:
```
pkg:{ecosystem}/{namespace}/{name}@{version}?{qualifiers}#{subpath}
```
| Source | Extraction Method |
|--------|-------------------|
| OpenVEX | Direct from `product.id` if PURL, else construct from `product.identifiers` |
| CycloneDX | From `bom-ref` PURL or construct from `component.purl` |
| CSAF | From `product_id` → `product_identification_helper.purl` |
### 4.4 Statement ID Generation
Statement IDs are deterministic SHA-256 hashes:
```csharp
public static string GenerateStatementId(
string vulnerabilityId,
VexStatus status,
IEnumerable<string> productPurls,
string issuerId,
DateTimeOffset timestamp)
{
var input = $"{vulnerabilityId}|{status}|{string.Join(",", productPurls.OrderBy(p => p))}|{issuerId}|{timestamp:O}";
var hash = SHA256.HashData(Encoding.UTF8.GetBytes(input));
return $"stmt:{Convert.ToHexString(hash).ToLowerInvariant()[..32]}";
}
```
## 5. Issuer Directory Integration
Normalized statements include issuer information from the Issuer Directory:
```csharp
public sealed record IssuerInfo
{
/// <summary>Issuer identifier (e.g., "vendor:redhat", "vendor:canonical").</summary>
public required string IssuerId { get; init; }
/// <summary>Display name.</summary>
public required string DisplayName { get; init; }
/// <summary>Trust tier (authoritative, trusted, community, unknown).</summary>
public required IssuerTrustTier TrustTier { get; init; }
/// <summary>Issuer's signing key fingerprints (if signed).</summary>
public ImmutableArray<string> SigningKeyFingerprints { get; init; }
}
public enum IssuerTrustTier
{
Authoritative = 0, // Vendor/maintainer of the product
Trusted = 1, // Known security research org
Community = 2, // Community contributor
Unknown = 3 // Unverified source
}
```
## 6. API Governance
### 6.1 Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/v1/vex/statements` | GET | Query normalized statements |
| `/api/v1/vex/statements/{id}` | GET | Get specific statement |
| `/api/v1/vex/normalize` | POST | Normalize a VEX document |
| `/api/v1/vex/issuers` | GET | List known issuers |
| `/api/v1/vex/issuers/{id}` | GET | Get issuer details |
### 6.2 Query Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `vulnerability` | string | Filter by CVE/vulnerability ID |
| `product` | string | Filter by PURL (URL-encoded) |
| `status` | enum | Filter by VEX status |
| `issuer` | string | Filter by issuer ID |
| `since` | datetime | Statements after timestamp |
| `limit` | int | Max results (default: 100, max: 1000) |
| `cursor` | string | Pagination cursor |
### 6.3 Response Format
```json
{
"statements": [
{
"statementId": "stmt:a1b2c3d4e5f6...",
"vulnerabilityId": "CVE-2024-1234",
"status": "not_affected",
"justification": "vulnerable_code_not_in_execute_path",
"products": ["pkg:npm/lodash@4.17.21"],
"issuer": {
"issuerId": "vendor:lodash",
"displayName": "Lodash Maintainers",
"trustTier": "authoritative"
},
"timestamp": "2024-12-19T10:30:00Z"
}
],
"cursor": "next_page_token",
"total": 42
}
```
## 7. Precedence Rules
When multiple statements exist for the same vulnerability+product:
1. **Timestamp:** Later statements supersede earlier ones
2. **Trust Tier:** Higher trust tiers take precedence (Authoritative > Trusted > Community > Unknown)
3. **Specificity:** More specific product matches win (exact version > version range > package)
## 8. Validation
All normalized statements must pass:
1. `vulnerabilityId` matches CVE/GHSA/vendor pattern
2. `status` is a valid enum value
3. `products` contains at least one valid PURL
4. `timestamp` is valid ISO-8601 UTC
5. `issuer.issuerId` exists in Issuer Directory or is marked Unknown
---
## Changelog
| Version | Date | Changes |
|---------|------|---------|
| 1.0.0 | 2025-12-19 | Initial release |

149
docs-archived/modules/excititor/schemas/vex_overlay.schema.json Normal file
View File

@@ -0,0 +1,149 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://stellaops.dev/schemas/excititor/vex_overlay.schema.json",
"title": "Excititor VEX Overlay",
"description": "Graph-ready overlay built from Link-Not-Merge observations and linksets. Immutable and append-only; ordered for deterministic pagination and caching.",
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"generatedAt",
"tenant",
"purl",
"advisoryId",
"source",
"status",
"observations",
"provenance"
],
"properties": {
"schemaVersion": {
"type": "string",
"enum": ["1.0.0"]
},
"generatedAt": {
"type": "string",
"format": "date-time"
},
"tenant": {
"type": "string",
"description": "Tenant identifier used for storage partitioning."
},
"purl": {
"type": "string",
"description": "Normalized package URL for the component."
},
"advisoryId": {
"type": "string",
"description": "Upstream advisory identifier (e.g., GHSA, RHSA, CVE)."
},
"source": {
"type": "string",
"description": "Linkset source identifier (matches Concelier linkset source)."
},
"status": {
"type": "string",
"enum": [
"affected",
"not_affected",
"under_investigation",
"fixed",
"unknown"
]
},
"justifications": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["kind", "reason"],
"properties": {
"kind": {
"type": "string",
"description": "Reason code aligned to VEX statement taxonomy."
},
"reason": {
"type": "string",
"description": "Human-readable justification text."
},
"evidence": {
"type": "array",
"items": {
"type": "string",
"description": "Observation or linkset id contributing to this justification."
}
},
"weight": {
"type": "number",
"minimum": 0,
"maximum": 1,
"description": "Optional confidence weight."
}
}
}
},
"conflicts": {
"type": "array",
"description": "Conflicts detected in linkset normalization.",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["field", "reason"],
"properties": {
"field": { "type": "string" },
"reason": { "type": "string" },
"values": {
"type": "array",
"items": { "type": "string" }
},
"sourceIds": {
"type": "array",
"items": { "type": "string" }
}
}
}
},
"observations": {
"type": "array",
"description": "Ordered list of Link-Not-Merge observation references feeding this overlay.",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["id", "contentHash", "fetchedAt"],
"properties": {
"id": { "type": "string" },
"contentHash": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
"fetchedAt": { "type": "string", "format": "date-time" }
}
},
"minItems": 1
},
"provenance": {
"type": "object",
"additionalProperties": false,
"required": ["linksetId", "linksetHash", "observationHashes"],
"properties": {
"linksetId": { "type": "string" },
"linksetHash": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
"observationHashes": {
"type": "array",
"items": { "type": "string", "pattern": "^sha256:[A-Fa-f0-9]{64}$" },
"minItems": 1
},
"policyHash": { "type": "string" },
"sbomContextHash": { "type": "string" },
"planCacheKey": { "type": "string" },
"generatedBy": { "type": "string" }
}
},
"cache": {
"type": "object",
"additionalProperties": false,
"properties": {
"cached": { "type": "boolean" },
"cachedAt": { "type": "string", "format": "date-time" },
"ttlSeconds": { "type": "integer", "minimum": 0 }
}
}
}
}

37
docs-archived/modules/excititor/schemas/vex_raw.schema.json Normal file
View File

@@ -0,0 +1,37 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://stellaops.dev/schemas/excititor/vex_raw.schema.json",
"title": "Excititor VEX Raw Document",
"$comment": "Note (2025-12): The gridFsObjectId field is legacy. Since Sprint 4400, all large content is stored in PostgreSQL with RustFS. This field exists only for backward compatibility with migrated data.",
"type": "object",
"additionalProperties": true,
"required": ["_id", "providerId", "format", "sourceUri", "retrievedAt", "digest"],
"properties": {
"_id": {
"type": "string",
"description": "Content-addressed digest; equals `digest`."
},
"providerId": { "type": "string", "minLength": 1 },
"format": { "type": "string", "enum": ["csaf", "cyclonedx", "openvex"] },
"sourceUri": { "type": "string", "minLength": 1 },
"retrievedAt": { "type": "string", "format": "date-time" },
"digest": { "type": "string", "minLength": 32 },
"content": {
"oneOf": [
{ "type": "string", "contentEncoding": "base64" },
{ "type": "string" }
],
"description": "Inline payload if below size threshold; may be empty when stored in RustFS (legacy: GridFS prior to Sprint 4400)."
},
"gridFsObjectId": {
"anyOf": [
{ "type": "string" },
{ "type": "null" }
]
},
"metadata": {
"type": "object",
"additionalProperties": { "type": "string" }
}
}
}

104
docs-archived/modules/excititor/scoring.md Normal file
View File

@@ -0,0 +1,104 @@
## Status
This document tracks the future-looking risk scoring model for Excititor. The calculation below is not active yet; Sprint 7 work will add the required schema fields, policy controls, and services. Until that ships, Excititor emits consensus statuses without numeric scores.
## Scoring model (target state)
**S = Gate(VEX_status) × W_trust(source) × [Severity_base × (1 + α·KEV + β·EPSS)]**
* **Gate(VEX_status)**: `affected`/`under_investigation` → 1, `not_affected`/`fixed` → 0. A trusted “not affected” or “fixed” still zeroes the score.
* **W_trust(source)**: normalized policy weight (baseline 0‒1). Policies may opt into >1 boosts for signed vendor feeds once Phase 1 closes.
* **Severity_base**: canonical numeric severity from Concelier (CVSS or org-defined scale).
* **KEV flag**: 0/1 boost when CISA Known Exploited Vulnerabilities applies.
* **EPSS**: probability [0,1]; bounded multiplier.
* **α, β**: configurable coefficients (default α=0.25, β=0.5) stored in policy.
Safeguards: freeze boosts when product identity is unknown, clamp outputs ≥0, and log every factor in the audit trail.
## Implementation roadmap
| Phase | Scope | Artifacts |
| --- | --- | --- |
| **Phase 1 – Schema foundations** | Extend Excititor consensus/claims and Concelier canonical advisories with severity, KEV, EPSS, and expose α/β + weight ceilings in policy. | Sprint 7 tasks `EXCITITOR-CORE-02-001`, `EXCITITOR-POLICY-02-001`, `EXCITITOR-STORAGE-02-001`, `FEEDCORE-ENGINE-07-001`. |
| **Phase 2 – Deterministic score engine** | Implement a scoring component that executes alongside consensus and persists score envelopes with hashes. | Planned task `EXCITITOR-CORE-02-002` (backlog). |
| **Phase 3 – Surfacing & enforcement** | Expose scores via WebService/CLI, integrate with Concelier noise priors, and enforce policy-based suppressions. | To be scheduled after Phase 2. |
## Policy controls (Phase 1)
Operators tune scoring inputs through the Excititor policy document:
```yaml
excititor:
policy:
weights:
vendor: 1.10 # per-tier weight
ceiling: 1.40 # max clamp applied to tiers and overrides (1.0‒5.0)
providerOverrides:
trusted.vendor: 1.35
scoring:
alpha: 0.30 # KEV boost coefficient (defaults to 0.25)
beta: 0.60 # EPSS boost coefficient (defaults to 0.50)
```
* All weights (tiers + overrides) are clamped to `[0, weights.ceiling]` with structured warnings when a value is out of range or not a finite number.
* `weights.ceiling` itself is constrained to `[1.0, 5.0]`, preserving prior behaviour when omitted.
* `scoring.alpha` / `scoring.beta` accept non-negative values up to 5.0; values outside the range fall back to defaults and surface diagnostics to operators.
## Data model (after Phase 1)
```json
{
"vulnerabilityId": "CVE-2025-12345",
"product": "pkg:name@version",
"consensus": {
"status": "affected",
"policyRevisionId": "rev-12",
"policyDigest": "0D9AEC…"
},
"signals": {
"severity": {"scheme": "CVSS:3.1", "score": 7.5},
"kev": true,
"epss": 0.40
},
"policy": {
"weight": 1.15,
"alpha": 0.25,
"beta": 0.5
},
"score": {
"value": 10.8,
"generatedAt": "2025-11-05T14:12:30Z",
"audit": [
"gate:affected",
"weight:1.15",
"severity:7.5",
"kev:1",
"epss:0.40"
]
}
}
```
## Operational guidance
* **Inputs**: Concelier delivers severity/KEV/EPSS via the advisory event log; Excititor connectors load VEX statements. Policy owns trust tiers and coefficients.
* **Processing**: the scoring engine (Phase 2) runs next to consensus, storing results with deterministic hashes so exports and attestations can reference them.
* **Consumption**: WebService/CLI will return consensus plus score; scanners may suppress findings only when policy-authorized VEX gating and signed score envelopes agree.
## Pseudocode (Phase 2 preview)
```python
def risk_score(gate, weight, severity, kev, epss, alpha, beta, freeze_boosts=False):
if gate == 0:
return 0
if freeze_boosts:
kev, epss = 0, 0
boost = 1 + alpha * kev + beta * epss
return max(0, weight * severity * boost)
```
## FAQ
* **Can operators opt out?** Set α=β=0 or keep weights ≤1.0 via policy.
* **What about missing signals?** Treat them as zero and log the omission.
* **When will this ship?** Phase 1 is planned for Sprint 7; later phases depend on connector coverage and attestation delivery.

515
docs-archived/modules/excititor/trust-lattice.md Normal file
View File

@@ -0,0 +1,515 @@
# VEX Trust Lattice Specification
> **Status**: Implementation Complete (Sprint 7100)
> **Version**: 1.0.0
> **Last Updated**: 2025-12-22
> **Source Advisory**: `docs/product/advisories/archived/22-Dec-2026 - Building a Trust Lattice for VEX Sources.md`
## 1. Overview
The VEX Trust Lattice provides a mathematically rigorous framework for converting heterogeneous VEX claims from multiple sources into a single, signed, reproducible verdict with a numeric confidence and a complete audit trail.
### Goals
1. **Explainability**: Every verdict includes a full breakdown of how it was computed
2. **Reproducibility**: Same inputs always produce identical verdicts (deterministic)
3. **Auditability**: Signed verdict manifests with pinned inputs for regulatory compliance
4. **Tunability**: Per-tenant, per-source trust configuration without code changes
### Non-Goals
- Real-time vulnerability detection (handled by Scanner)
- VEX document ingestion (handled by Excititor core)
- Policy enforcement (handled by Policy Engine)
---
## 2. Trust Vector Model
Each VEX source is assigned a 3-component trust vector scored in the range [0..1].
### 2.1 Provenance (P)
Measures cryptographic and process integrity of the source.
| Score | Description |
|-------|-------------|
| 1.00 | DSSE-signed, timestamped, Rekor/Git anchored, key in allow-list, rotation policy OK |
| 0.75 | DSSE-signed + public key known, but no transparency log |
| 0.40 | Unsigned but retrieved via authenticated, immutable artifact repo |
| 0.10 | Opaque/CSV/email/manual import |
### 2.2 Coverage (C)
Measures how well the statement's scope maps to the target asset.
| Score | Description |
|-------|-------------|
| 1.00 | Exact package + version/build digest + feature/flag context matched |
| 0.75 | Exact package + version range matched; partial feature context |
| 0.50 | Product-level only; maps via CPE/PURL family |
| 0.25 | Family-level heuristics; no version proof |
### 2.3 Replayability (R)
Measures whether the claim can be deterministically re-derived.
| Score | Description |
|-------|-------------|
| 1.00 | All inputs pinned (feeds, SBOM hash, ruleset hash, lattice version); replays byte-identical |
| 0.60 | Inputs mostly pinned; non-deterministic ordering tolerated but stable outcome |
| 0.20 | Ephemeral APIs; no snapshot |
### 2.4 Weight Configuration
The base trust score is computed as:
```
BaseTrust(S) = wP * P + wC * C + wR * R
```
**Default weights:**
- `wP = 0.45` (Provenance)
- `wC = 0.35` (Coverage)
- `wR = 0.20` (Replayability)
Weights are tunable per policy and sum to 1.0.
---
## 3. Claim Scoring
### 3.1 Base Trust Calculation
```csharp
double BaseTrust(double P, double C, double R, TrustWeights W)
=> W.wP * P + W.wC * C + W.wR * R;
```
### 3.2 Claim Strength Multipliers (M)
Each VEX claim carries a strength multiplier based on evidence quality:
| Strength | Value | Description |
|----------|-------|-------------|
| ExploitabilityWithReachability | 1.00 | Exploitability analysis + reachability proof subgraph provided |
| ConfigWithEvidence | 0.80 | Config/feature-flag reason with evidence |
| VendorBlanket | 0.60 | Vendor blanket statement |
| UnderInvestigation | 0.40 | "Under investigation" |
### 3.3 Freshness Decay (F)
Time-decay curve with configurable half-life:
```csharp
double Freshness(DateTime issuedAt, DateTime cutoff, double halfLifeDays = 90, double floor = 0.35)
{
var ageDays = (cutoff - issuedAt).TotalDays;
var decay = Math.Exp(-Math.Log(2) * ageDays / halfLifeDays);
return Math.Max(decay, floor);
}
```
**Parameters:**
- `halfLifeDays = 90` (default): Score halves every 90 days
- `floor = 0.35` (default): Minimum freshness unless revoked
### 3.4 ClaimScore Formula
```
ClaimScore = BaseTrust(S) * M * F
```
**Example calculation:**
```
Source: Red Hat (Vendor)
P = 0.90, C = 0.75, R = 0.60
BaseTrust = 0.45*0.90 + 0.35*0.75 + 0.20*0.60 = 0.405 + 0.2625 + 0.12 = 0.7875
Claim: ConfigWithEvidence (M = 0.80)
Freshness: 30 days old (F = 0.79)
ClaimScore = 0.7875 * 0.80 * 0.79 = 0.498
```
---
## 4. Lattice Merge Algorithm
### 4.1 Partial Ordering
Claims are ordered by a tuple: `(scope_specificity, ClaimScore)`.
Scope specificity levels:
1. Exact digest match (highest)
2. Exact version match
3. Version range match
4. Product family match
5. Platform match (lowest)
### 4.2 Conflict Detection
Conflicts occur when claims for the same (CVE, Asset) have different statuses:
```csharp
bool HasConflict(IEnumerable<Claim> claims)
=> claims.Select(c => c.Status).Distinct().Count() > 1;
```
### 4.3 Conflict Penalty
When conflicts exist, apply a penalty to weaker/older claims:
```csharp
const double ConflictPenalty = 0.25;
if (contradictory)
{
var strongest = claims.OrderByDescending(c => c.Score).First();
foreach (var claim in claims.Where(c => c.Status != strongest.Status))
{
claim.AdjustedScore = claim.Score * (1 - ConflictPenalty);
}
}
```
### 4.4 Winner Selection
Final verdict is selected by:
```csharp
var winner = scored
.OrderByDescending(x => (x.Claim.ScopeSpecificity, x.AdjustedScore))
.First();
```
### 4.5 Audit Trail Generation
Every merge produces:
```csharp
public sealed record MergeResult
{
public VexStatus Status { get; init; }
public double Confidence { get; init; }
public ImmutableArray<VerdictExplanation> Explanations { get; init; }
public ImmutableArray<string> EvidenceRefs { get; init; }
public string PolicyHash { get; init; }
public string LatticeVersion { get; init; }
}
```
---
## 5. Policy Gates
Gates are evaluated after merge to enforce policy requirements.
### 5.1 MinimumConfidenceGate
Requires minimum confidence by environment for certain statuses.
```yaml
gates:
minimumConfidence:
enabled: true
thresholds:
production: 0.75
staging: 0.60
development: 0.40
applyToStatuses:
- not_affected
- fixed
```
**Behavior**: Fails if confidence < threshold for specified statuses.
### 5.2 UnknownsBudgetGate
Limits exposure to unknown/unscored dependencies.
```yaml
gates:
unknownsBudget:
enabled: true
maxUnknownCount: 5
maxCumulativeUncertainty: 2.0
```
**Behavior**: Fails if:
- `#unknown_deps > maxUnknownCount`, OR
- `sum(1 - ClaimScore) > maxCumulativeUncertainty`
### 5.3 SourceQuotaGate
Prevents single-source dominance without corroboration.
```yaml
gates:
sourceQuota:
enabled: true
maxInfluencePercent: 60
corroborationDelta: 0.10
```
**Behavior**: Fails if single source influence > 60% AND no second source within delta=0.10.
### 5.4 ReachabilityRequirementGate
Requires reachability proof for critical vulnerabilities.
```yaml
gates:
reachabilityRequirement:
enabled: true
severityThreshold: CRITICAL
requiredForStatuses:
- not_affected
bypassReasons:
- component_not_present
```
**Behavior**: Fails if `not_affected` on CRITICAL CVE without reachability proof (unless bypass reason applies).
---
## 6. Deterministic Replay
### 6.1 Input Pinning
To guarantee "same inputs → same verdict", pin:
- SBOM digest(s)
- Vuln feed snapshot IDs
- VEX document digests
- Reachability graph IDs
- Policy file hash
- Lattice version
- Clock cutoff (evaluation timestamp)
### 6.2 Verdict Manifest
```json
{
"manifestId": "verd:tenant:asset:cve:1234567890",
"tenant": "acme-corp",
"assetDigest": "sha256:abc123...",
"vulnerabilityId": "CVE-2025-12345",
"inputs": {
"sbomDigests": ["sha256:..."],
"vulnFeedSnapshotIds": ["nvd:2025-12-22"],
"vexDocumentDigests": ["sha256:..."],
"reachabilityGraphIds": ["graph:..."],
"clockCutoff": "2025-12-22T12:00:00Z"
},
"result": {
"status": "not_affected",
"confidence": 0.82,
"explanations": [...]
},
"policyHash": "sha256:...",
"latticeVersion": "1.2.0",
"evaluatedAt": "2025-12-22T12:00:01Z",
"manifestDigest": "sha256:..."
}
```
### 6.3 Signing
Verdict manifests are signed using DSSE with predicate type:
```
https://stella-ops.org/attestations/vex-verdict/1
```
### 6.4 Replay Verification
```
POST /api/v1/authority/verdicts/{manifestId}/replay
Response:
{
"success": true,
"originalManifest": {...},
"replayedManifest": {...},
"differences": [],
"signatureValid": true
}
```
---
## 7. Configuration Reference
### Full Configuration Example
```yaml
# etc/trust-lattice.yaml
version: "1.0"
trustLattice:
weights:
provenance: 0.45
coverage: 0.35
replayability: 0.20
freshness:
halfLifeDays: 90
floor: 0.35
conflictPenalty: 0.25
defaults:
vendor:
provenance: 0.90
coverage: 0.70
replayability: 0.60
distro:
provenance: 0.80
coverage: 0.85
replayability: 0.60
internal:
provenance: 0.85
coverage: 0.95
replayability: 0.90
gates:
minimumConfidence:
enabled: true
thresholds:
production: 0.75
staging: 0.60
development: 0.40
unknownsBudget:
enabled: true
maxUnknownCount: 5
maxCumulativeUncertainty: 2.0
sourceQuota:
enabled: true
maxInfluencePercent: 60
corroborationDelta: 0.10
reachabilityRequirement:
enabled: true
severityThreshold: CRITICAL
```
---
## 8. API Reference
### Endpoints
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/v1/excititor/verdicts/{manifestId}` | Get verdict manifest |
| GET | `/api/v1/excititor/verdicts` | List verdicts (paginated) |
| POST | `/api/v1/authority/verdicts/{manifestId}/replay` | Verify replay |
| GET | `/api/v1/authority/verdicts/{manifestId}/download` | Download signed manifest |
See `docs/API_CLI_REFERENCE.md` for complete API documentation.
---
## 9. Examples
### Example 1: High-Confidence Verdict
**Input:**
- Red Hat VEX: `not_affected` with `component_not_present`
- Ubuntu VEX: `not_affected` with `component_not_present`
**Calculation:**
```
Red Hat: BaseTrust=0.78, M=0.80, F=0.95 → ClaimScore=0.59
Ubuntu: BaseTrust=0.72, M=0.80, F=0.90 → ClaimScore=0.52
No conflict (both agree)
Winner: Red Hat (higher score)
Confidence: 0.59
Gates: All pass (> 0.40 threshold)
```
### Example 2: Conflict Resolution
**Input:**
- Vendor VEX: `not_affected`
- Internal scan: `affected`
**Calculation:**
```
Vendor: ClaimScore=0.65
Internal: ClaimScore=0.55
Conflict detected → penalty applied
Internal adjusted: 0.55 * 0.75 = 0.41
Winner: Vendor
Confidence: 0.65
Note: Conflict recorded in audit trail
```
---
---
## 10. Implementation Reference
### 10.1 Source Files
| Component | Location |
|-----------|----------|
| TrustVector | `src/Excititor/__Libraries/StellaOps.Excititor.Core/TrustVector/TrustVector.cs` |
| TrustWeights | `src/Excititor/__Libraries/StellaOps.Excititor.Core/TrustVector/TrustWeights.cs` |
| ClaimStrength | `src/Excititor/__Libraries/StellaOps.Excititor.Core/TrustVector/ClaimStrength.cs` |
| FreshnessCalculator | `src/Excititor/__Libraries/StellaOps.Excititor.Core/TrustVector/FreshnessCalculator.cs` |
| DefaultTrustVectors | `src/Excititor/__Libraries/StellaOps.Excititor.Core/TrustVector/DefaultTrustVectors.cs` |
| ProvenanceScorer | `src/Excititor/__Libraries/StellaOps.Excititor.Core/TrustVector/ProvenanceScorer.cs` |
| CoverageScorer | `src/Excititor/__Libraries/StellaOps.Excititor.Core/TrustVector/CoverageScorer.cs` |
| ReplayabilityScorer | `src/Excititor/__Libraries/StellaOps.Excititor.Core/TrustVector/ReplayabilityScorer.cs` |
| SourceClassificationService | `src/Excititor/__Libraries/StellaOps.Excititor.Core/TrustVector/SourceClassificationService.cs` |
| ClaimScoreMerger | `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/ClaimScoreMerger.cs` |
| MinimumConfidenceGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/MinimumConfidenceGate.cs` |
| UnknownsBudgetGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/UnknownsBudgetGate.cs` |
| SourceQuotaGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/SourceQuotaGate.cs` |
| ReachabilityRequirementGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/ReachabilityRequirementGate.cs` |
| TrustVectorCalibrator | `src/Excititor/__Libraries/StellaOps.Excititor.Core/Calibration/TrustVectorCalibrator.cs` |
### 10.2 Configuration Files
| File | Purpose |
|------|---------|
| `etc/trust-lattice.yaml.sample` | Trust vector weights, freshness parameters, default vectors |
| `etc/policy-gates.yaml.sample` | Gate thresholds and enable/disable flags |
| `etc/excititor-calibration.yaml.sample` | Calibration learning parameters |
### 10.3 Database Schema
- **Calibration manifests**: `src/Excititor/__Libraries/StellaOps.Excititor.Storage.Postgres/Migrations/002_calibration_schema.sql`
- **Verdict storage**: See Authority module for verdict manifest persistence
### 10.4 Test Coverage
| Test Suite | Location |
|------------|----------|
| TrustVector tests | `src/Excititor/__Tests/StellaOps.Excititor.Core.Tests/TrustVector/` |
| ClaimScoreMerger tests | `src/Policy/__Tests/StellaOps.Policy.Tests/TrustLattice/` |
| Gate tests | `src/Policy/__Tests/StellaOps.Policy.Tests/Gates/` |
| Calibration tests | `src/Excititor/__Tests/StellaOps.Excititor.Core.Tests/Calibration/` |
---
## Related Documentation
- [Excititor Architecture](./architecture.md)
- [Verdict Manifest Specification](../authority/verdict-manifest.md)
- [Policy Gates Configuration](../policy/architecture.md)
- [API Reference](../../API_CLI_REFERENCE.md)
---
*Document Version: 1.0.0*
*Sprint: 7100.0003.0002*
*Created: 2025-12-22*

322
docs-archived/modules/excititor/vex_linksets_api.md Normal file
View File

@@ -0,0 +1,322 @@
# Excititor VEX Observation & Linkset APIs
> Implementation reference for Sprint 121 (`EXCITITOR-LNM-21-201`, `EXCITITOR-LNM-21-202`). Documents the REST endpoints implemented in `src/Excititor/StellaOps.Excititor.WebService/Endpoints/ObservationEndpoints.cs` and `LinksetEndpoints.cs`.
## Authentication & Headers
All endpoints require:
- **Authorization**: Bearer token with `vex.read` scope
- **X-Stella-Tenant**: Tenant identifier (required)
## /vex/observations
### List observations with filters
```
GET /vex/observations?vulnerabilityId=CVE-2024-0001&productKey=pkg:maven/org.demo/app@1.2.3&limit=50
GET /vex/observations?providerId=ubuntu-csaf&limit=50
```
**Query Parameters:**
- `vulnerabilityId` + `productKey` (required together) - Filter by vulnerability and product
- `providerId` - Filter by provider
- `limit` (optional, default: 50, max: 100) - Number of results
- `cursor` (optional) - Pagination cursor from previous response
**Response 200:**
```json
{
"items": [
{
"observationId": "vex:obs:sha256:abc123...",
"tenant": "default",
"providerId": "ubuntu-csaf",
"vulnerabilityId": "CVE-2024-0001",
"productKey": "pkg:maven/org.demo/app@1.2.3",
"status": "affected",
"createdAt": "2025-11-18T12:34:56Z",
"lastObserved": "2025-11-18T12:34:56Z",
"purls": ["pkg:maven/org.demo/app@1.2.3"]
}
],
"nextCursor": "MjAyNS0xMS0xOFQxMjozNDo1NlonfHZleDpvYnM6c2hhMjU2OmFiYzEyMy4uLg=="
}
```
**Error Responses:**
- `400 ERR_PARAMS` - At least one filter is required
- `400 ERR_TENANT` - X-Stella-Tenant header is required
- `403` - Missing required scope
### Get observation by ID
```
GET /vex/observations/{observationId}
```
**Response 200:**
```json
{
"observationId": "vex:obs:sha256:abc123...",
"tenant": "default",
"providerId": "ubuntu-csaf",
"streamId": "ubuntu-csaf-vex",
"upstream": {
"upstreamId": "USN-9999-1",
"documentVersion": "2024.10.22",
"fetchedAt": "2025-11-18T12:34:00Z",
"receivedAt": "2025-11-18T12:34:05Z",
"contentHash": "sha256:...",
"signature": {
"type": "cosign",
"keyId": "ubuntu-vex-prod",
"issuer": "https://token.actions.githubusercontent.com",
"verifiedAt": "2025-11-18T12:34:10Z"
}
},
"content": {
"format": "csaf",
"specVersion": "2.0"
},
"statements": [
{
"vulnerabilityId": "CVE-2024-0001",
"productKey": "pkg:maven/org.demo/app@1.2.3",
"status": "affected",
"lastObserved": "2025-11-18T12:34:56Z",
"locator": "#/statements/0",
"justification": "component_not_present",
"introducedVersion": null,
"fixedVersion": "1.2.4"
}
],
"linkset": {
"aliases": ["USN-9999-1"],
"purls": ["pkg:maven/org.demo/app@1.2.3"],
"cpes": [],
"references": [{"type": "advisory", "url": "https://ubuntu.com/security/notices/USN-9999-1"}]
},
"createdAt": "2025-11-18T12:34:56Z"
}
```
**Error Responses:**
- `404 ERR_NOT_FOUND` - Observation not found
### Count observations
```
GET /vex/observations/count
```
**Response 200:**
```json
{
"count": 12345
}
```
## /vex/linksets
### List linksets with filters
At least one filter is required: `vulnerabilityId`, `productKey`, `providerId`, or `hasConflicts=true`.
```
GET /vex/linksets?vulnerabilityId=CVE-2024-0001&limit=50
GET /vex/linksets?productKey=pkg:maven/org.demo/app@1.2.3&limit=50
GET /vex/linksets?providerId=ubuntu-csaf&limit=50
GET /vex/linksets?hasConflicts=true&limit=50
```
**Query Parameters:**
- `vulnerabilityId` - Filter by vulnerability ID
- `productKey` - Filter by product key
- `providerId` - Filter by provider
- `hasConflicts` - Filter to linksets with disagreements (true/false)
- `limit` (optional, default: 50, max: 100) - Number of results
- `cursor` (optional) - Pagination cursor
**Response 200:**
```json
{
"items": [
{
"linksetId": "sha256:tenant:CVE-2024-0001:pkg:maven/org.demo/app@1.2.3",
"tenant": "default",
"vulnerabilityId": "CVE-2024-0001",
"productKey": "pkg:maven/org.demo/app@1.2.3",
"providerIds": ["ubuntu-csaf", "suse-csaf"],
"statuses": ["affected", "fixed"],
"aliases": [],
"purls": [],
"cpes": [],
"references": [],
"disagreements": [
{
"providerId": "suse-csaf",
"status": "fixed",
"justification": null,
"confidence": 0.85
}
],
"observations": [
{"observationId": "vex:obs:...", "providerId": "ubuntu-csaf", "status": "affected", "confidence": 0.9},
{"observationId": "vex:obs:...", "providerId": "suse-csaf", "status": "fixed", "confidence": 0.85}
],
"createdAt": "2025-11-18T12:34:56Z"
}
],
"nextCursor": null
}
```
**Error Responses:**
- `400 ERR_AGG_PARAMS` - At least one filter is required
### Get linkset by ID
```
GET /vex/linksets/{linksetId}
```
**Response 200:**
```json
{
"linksetId": "sha256:...",
"tenant": "default",
"vulnerabilityId": "CVE-2024-0001",
"productKey": "pkg:maven/org.demo/app@1.2.3",
"providerIds": ["ubuntu-csaf", "suse-csaf"],
"statuses": ["affected", "fixed"],
"confidence": "low",
"hasConflicts": true,
"disagreements": [
{
"providerId": "suse-csaf",
"status": "fixed",
"justification": null,
"confidence": 0.85
}
],
"observations": [
{"observationId": "vex:obs:...", "providerId": "ubuntu-csaf", "status": "affected", "confidence": 0.9},
{"observationId": "vex:obs:...", "providerId": "suse-csaf", "status": "fixed", "confidence": 0.85}
],
"createdAt": "2025-11-18T12:00:00Z",
"updatedAt": "2025-11-18T12:34:56Z"
}
```
**Error Responses:**
- `400 ERR_AGG_PARAMS` - linksetId is required
- `404 ERR_AGG_NOT_FOUND` - Linkset not found
### Lookup linkset by vulnerability and product
```
GET /vex/linksets/lookup?vulnerabilityId=CVE-2024-0001&productKey=pkg:maven/org.demo/app@1.2.3
```
**Response 200:** Same as Get linkset by ID
**Error Responses:**
- `400 ERR_AGG_PARAMS` - vulnerabilityId and productKey are required
- `404 ERR_AGG_NOT_FOUND` - No linkset found for the specified vulnerability and product
### Count linksets
```
GET /vex/linksets/count
```
**Response 200:**
```json
{
"total": 5000,
"withConflicts": 127
}
```
### List linksets with conflicts (shorthand)
```
GET /vex/linksets/conflicts?limit=50
```
**Response 200:** Same format as List linksets
## Error Codes
| Code | Description |
|------|-------------|
| `ERR_PARAMS` | Missing or invalid query parameters (observations) |
| `ERR_TENANT` | X-Stella-Tenant header is required |
| `ERR_NOT_FOUND` | Observation not found |
| `ERR_AGG_PARAMS` | Missing or invalid query parameters (linksets) |
| `ERR_AGG_NOT_FOUND` | Linkset not found |
## Pagination
- Uses cursor-based pagination with base64-encoded `timestamp|id` cursors
- Default limit: 50, Maximum limit: 100
- Cursors are opaque; treat as strings and pass back unchanged
## Determinism
- Results are sorted by timestamp (descending), then by ID
- Array fields are sorted lexicographically
- Status enums are lowercase strings
## SDK Example (TypeScript)
```typescript
const listObservations = async (
baseUrl: string,
token: string,
tenant: string,
vulnerabilityId: string,
productKey: string
) => {
const params = new URLSearchParams({
vulnerabilityId,
productKey,
limit: "100"
});
const response = await fetch(`${baseUrl}/vex/observations?${params}`, {
headers: {
Authorization: `Bearer ${token}`,
"X-Stella-Tenant": tenant
}
});
if (!response.ok) {
const error = await response.json();
throw new Error(`${error.error.code}: ${error.error.message}`);
}
return response.json();
};
const getLinksetWithConflicts = async (
baseUrl: string,
token: string,
tenant: string
) => {
const response = await fetch(`${baseUrl}/vex/linksets/conflicts?limit=50`, {
headers: {
Authorization: `Bearer ${token}`,
"X-Stella-Tenant": tenant
}
});
return response.json();
};
```
## Related Documentation
- `vex_observations.md` - VEX Observation domain model and storage schema
- `evidence-contract.md` - Evidence bundle format and attestation
- `AGENTS.md` - Component development guidelines

232
docs-archived/modules/excititor/vex_observations.md Normal file
View File

@@ -0,0 +1,232 @@
# VEX Observation Model (`vex_observations`)
> Authored 2025-11-14 for Sprint 120 (`EXCITITOR-LNM-21-001`). This document is the canonical schema description for Excititor's immutable observation records. It unblocks downstream documentation tasks (`DOCS-LNM-22-002`) and aligns the WebService/Worker data structures with PostgreSQL persistence.
Excititor ingests heterogeneous VEX statements, normalizes them under the Aggregation-Only Contract (AOC), and persists each normalized statement as a **VEX observation**. These observations are the source of truth for:
- Advisory AI citation APIs (`/v1/vex/observations/{vulnerabilityId}/{productKey}`)
- Graph/Vuln Explorer overlays (batch observation APIs)
- Evidence Locker + portable bundle manifests
- Policy Engine materialization and audit trails
All observation documents are immutable. New information creates a new observation record linked by `observationId`; supersedence happens through Graph/Lens layers, not by mutating this collection.
## Storage & routing
| Aspect | Value |
| --- | --- |
| Table | `vex_observations` (PostgreSQL) |
| Upstream generator | `VexObservationProjectionService` (WebService) and Worker normalization pipeline |
| Primary key | `{tenant, observationId}` |
| Required indexes | `{tenant, vulnerabilityId}`, `{tenant, productKey}`, `{tenant, document.digest}`, `{tenant, providerId, status}` |
| Source of truth for | `/v1/vex/observations`, Graph batch APIs, Excititor → Evidence Locker replication |
## Canonical document shape
```jsonc
{
"tenant": "default",
"observationId": "vex:obs:sha256:...",
"vulnerabilityId": "CVE-2024-12345",
"productKey": "pkg:maven/org.example/app@1.2.3",
"providerId": "ubuntu-csaf",
"status": "affected", // matches VexClaimStatus enum
"justification": {
"type": "component_not_present",
"reason": "Package not shipped in this profile",
"detail": "Binary not in base image"
},
"detail": "Free-form vendor detail",
"confidence": {
"score": 0.9,
"level": "high",
"method": "vendor"
},
"signals": {
"severity": {
"scheme": "cvss3.1",
"score": 7.8,
"label": "High",
"vector": "CVSS:3.1/..."
},
"kev": true,
"epss": 0.77
},
"scope": {
"key": "pkg:deb/ubuntu/apache2@2.4.58-1",
"purls": [
"pkg:deb/ubuntu/apache2@2.4.58-1",
"pkg:docker/example/app@sha256:..."
],
"cpes": ["cpe:2.3:a:apache:http_server:2.4.58:*:*:*:*:*:*:*"]
},
"anchors": [
"#/statements/0/justification",
"#/statements/0/detail"
],
"document": {
"format": "csaf",
"digest": "sha256:abc123...",
"revision": "2024-10-22T09:00:00Z",
"sourceUri": "https://ubuntu.com/security/notices/USN-0000-1",
"signature": {
"type": "cosign",
"issuer": "https://token.actions.githubusercontent.com",
"keyId": "ubuntu-vex-prod",
"verifiedAt": "2024-10-22T09:01:00Z",
"transparencyLogReference": "rekor://UUID",
"trust": {
"tenantId": "default",
"issuerId": "ubuntu",
"effectiveWeight": 0.9,
"tenantOverrideApplied": false,
"retrievedAtUtc": "2024-10-22T09:00:30Z"
}
}
},
"aoc": {
"guardVersion": "2024.10.0",
"violations": [], // non-empty -> stored + surfaced
"ingestedAt": "2024-10-22T09:00:05Z",
"retrievedAt": "2024-10-22T08:59:59Z"
},
"metadata": {
"provider-hint": "Mainline feed",
"source-channel": "mirror"
}
}
```
### Field notes
- **`tenant`** – logical tenant resolved by WebService based on headers or default configuration.
- **`observationId`** – deterministic hash (sha256) over `{tenant, vulnerabilityId, productKey, providerId, statementDigest}`. Never reused.
- **`status` + `justification`** – follow the OpenVEX semantics enforced by `StellaOps.Excititor.Core.VexClaim`.
- **`scope`** – includes canonical `key` plus normalized PURLs/CPES; deterministic ordering.
- **`anchors`** – optional JSON-pointer hints pointing to the source document sections; stored as trimmed strings.
- **`document.signature`** – mirrors `VexSignatureMetadata`; empty if upstream feed lacks signatures.
- **`aoc.violations`** – stored if the guard detected non-fatal issues; fatal issues never create an observation.
- **`metadata`** – reserved for deterministic provider hints; keys follow `vex.*` prefix guidance.
## Determinism & AOC guarantees
1. **Write-once** – once inserted, observation documents never change. New evidence creates a new `observationId`.
2. **Sorted collections** – arrays (`anchors`, `purls`, `cpes`) are sorted lexicographically before persistence.
3. **Guard metadata** – `aoc.guardVersion` records the guard library version (`docs/aoc/guard-library.md`), enabling audits.
4. **Signatures** – only verification metadata proven by the Worker is stored; WebService never recomputes trust.
5. **Time normalization** – all timestamps stored as UTC ISO-8601 strings (PostgreSQL `timestamptz`).
## API mapping
| API | Source fields | Notes |
| --- | --- | --- |
| `GET /vex/observations` | `tenant`, `vulnerabilityId`, `productKey`, `providerId` | List observations with filters. Implemented in `ObservationEndpoints.cs`. |
| `GET /vex/observations/{observationId}` | `tenant`, `observationId` | Get single observation by ID with full detail. |
| `GET /vex/observations/count` | `tenant` | Count all observations for tenant. |
| `/v1/vex/observations/{vuln}/{product}` | `tenant`, `vulnerabilityId`, `productKey`, `scope`, `statements[]` | Response uses `VexObservationProjectionService` to render `statements`, `document`, and `signature` fields. |
| `/vex/aoc/verify` | `document.digest`, `providerId`, `aoc` | Replays guard validation for recent digests; guard violations here align with `aoc.violations`. |
| Evidence batch API (Graph) | `statements[]`, `scope`, `signals`, `anchors` | Format optimized for overlays; reduces `document` to digest/URI. |
## Related work
- `EXCITITOR-GRAPH-24-*` relies on this schema to build overlays.
- `DOCS-LNM-22-002` (Link-Not-Merge documentation) references this file.
- `EXCITITOR-ATTEST-73-*` uses `document.digest` + `signature` to embed provenance in attestation payloads.
---
## Rekor Transparency Log Linkage
**Sprint Reference**: `SPRINT_20260117_002_EXCITITOR_vex_rekor_linkage`
VEX observations can be attested to the Sigstore Rekor transparency log, providing an immutable, publicly verifiable record of when each observation was recorded. This supports:
- **Auditability**: Independent verification that an observation existed at a specific time
- **Non-repudiation**: Cryptographic proof of observation provenance
- **Supply chain compliance**: Evidence for regulatory and security requirements
- **Offline verification**: Stored inclusion proofs enable air-gapped verification
### Rekor Linkage Fields
The following fields are added to `vex_observations` when an observation is attested:
| Field | Type | Description |
|-------|------|-------------|
| `rekor_uuid` | TEXT | Rekor entry UUID (64-char hex) |
| `rekor_log_index` | BIGINT | Monotonically increasing log position |
| `rekor_integrated_time` | TIMESTAMPTZ | When entry was integrated into log |
| `rekor_log_url` | TEXT | Rekor server URL where submitted |
| `rekor_inclusion_proof` | JSONB | RFC 6962 inclusion proof for offline verification |
| `rekor_linked_at` | TIMESTAMPTZ | When linkage was recorded locally |
### Schema Extension
```sql
-- V20260117__vex_rekor_linkage.sql
ALTER TABLE excititor.vex_observations
ADD COLUMN IF NOT EXISTS rekor_uuid TEXT,
ADD COLUMN IF NOT EXISTS rekor_log_index BIGINT,
ADD COLUMN IF NOT EXISTS rekor_integrated_time TIMESTAMPTZ,
ADD COLUMN IF NOT EXISTS rekor_log_url TEXT,
ADD COLUMN IF NOT EXISTS rekor_inclusion_proof JSONB,
ADD COLUMN IF NOT EXISTS rekor_linked_at TIMESTAMPTZ;
-- Indexes for Rekor queries
CREATE INDEX idx_vex_observations_rekor_uuid
ON excititor.vex_observations(rekor_uuid)
WHERE rekor_uuid IS NOT NULL;
CREATE INDEX idx_vex_observations_pending_rekor
ON excititor.vex_observations(created_at)
WHERE rekor_uuid IS NULL;
```
### API Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/attestations/rekor/observations/{id}` | POST | Attest observation to Rekor |
| `/attestations/rekor/observations/batch` | POST | Batch attestation |
| `/attestations/rekor/observations/{id}/verify` | GET | Verify Rekor linkage |
| `/attestations/rekor/pending` | GET | List observations pending attestation |
### CLI Commands
```bash
# Show observation with Rekor details
stella vex observation show <id> --show-rekor
# Attest an observation to Rekor
stella vex observation attest <id> [--rekor-url URL]
# Verify Rekor linkage
stella vex observation verify-rekor <id> [--offline]
# List pending attestations
stella vex observation list-pending
```
### Inclusion Proof Structure
```jsonc
{
"treeSize": 1234567,
"rootHash": "base64-encoded-root-hash",
"logIndex": 12345,
"hashes": [
"base64-hash-1",
"base64-hash-2",
"base64-hash-3"
]
}
```
### Verification Modes
| Mode | Network | Use Case |
|------|---------|----------|
| Online | Required | Full verification against live Rekor |
| Offline | Not required | Verify using stored inclusion proof |
Offline mode uses the stored `rekor_inclusion_proof` to verify the Merkle path locally. This is essential for air-gapped environments.