git.stella-ops.org/docs/ARCHITECTURE_EXCITITOR_MIRRORS.md

# architecture_excititor_mirrors.md — Excititor Mirror Distribution

> **Status:** Draft (Sprint 7). Complements `docs/ARCHITECTURE_EXCITITOR.md` by describing the mirror export surface exposed by `Excititor.WebService` and the configuration hooks used by operators and downstream mirrors.

---

## 0) Purpose

Excititor publishes canonical VEX consensus data. Operators (or StellaOps-managed mirrors) need a deterministic way to sync those exports into downstream environments. Mirror distribution provides:

* A declarative map of export bundles (`json`, `jsonl`, `openvex`, `csaf`) reachable via signed HTTP endpoints under `/excititor/mirror`.
* Thin quota/authentication controls on top of the existing export cache so mirrors cannot starve the web service.
* Stable payload shapes that downstream automation can monitor (index → fetch updates → download artifact → verify signature).

Mirror endpoints are intentionally **read-only**. Write paths (export generation, attestation, cache) remain the responsibility of the export pipeline.

---

## 1) Configuration model

The web service reads mirror configuration from `Excititor:Mirror` (YAML/JSON/appsettings). Each domain groups a set of exports that share rate limits and authentication rules.

```yaml
Excititor:
  Mirror:
    Domains:
      - id: primary
        displayName: Primary Mirror
        requireAuthentication: false
        maxIndexRequestsPerHour: 600
        maxDownloadRequestsPerHour: 1200
        exports:
          - key: consensus
            format: json
            filters:
              vulnId: CVE-2025-0001
              productKey: pkg:test/demo
            sort:
              createdAt: false     # descending
            limit: 1000
          - key: consensus-openvex
            format: openvex
            filters:
              vulnId: CVE-2025-0001
```

### Root settings

| Field | Required | Description |
| --- | --- | --- |
| `outputRoot` | – | Filesystem root where mirror artefacts are written. Defaults to the Excititor file-system artifact store root when omitted. |
| `directoryName` | – | Optional subdirectory created under `outputRoot`; defaults to `mirror`. |
| `targetRepository` | – | Hint propagated to manifests/index files indicating the operator-visible location (for example `s3://mirror/excititor`). |
| `signing` | – | Bundle signing configuration. When enabled, the exporter emits a detached JWS (`bundle.json.jws`) alongside each domain bundle. |

`signing` supports the following fields:

| Field | Required | Description |
| --- | --- | --- |
| `enabled` | – | Toggles detached signing for domain bundles. |
| `algorithm` | – | Signing algorithm identifier (default `ES256`). |
| `keyId` | ✅ (when `enabled`) | Signing key identifier resolved via the configured crypto provider registry. |
| `provider` | – | Optional provider hint when multiple registries are available. |
| `keyPath` | – | Optional PEM path used to seed the provider when the key is not already loaded. |

### Domain field reference

| Field | Required | Description |
| --- | --- | --- |
| `id` | ✅ | Stable identifier. Appears in URLs (`/excititor/mirror/domains/{id}`) and download filenames. |
| `displayName` | – | Human-friendly label surfaced in the `/domains` listing. Falls back to `id`. |
| `requireAuthentication` | – | When `true` the service enforces that the caller is authenticated (Authority token). |
| `maxIndexRequestsPerHour` | – | Per-domain quota for index endpoints. `0`/negative disables the guard. |
| `maxDownloadRequestsPerHour` | – | Per-domain quota for artifact downloads. |
| `exports` | ✅ | Collection of export projections. |

Export-level fields:

| Field | Required | Description |
| --- | --- | --- |
| `key` | ✅ | Unique key within the domain. Used in URLs (`/exports/{key}`) and filenames/bundle entries. |
| `format` | ✅ | One of `json`, `jsonl`, `openvex`, `csaf`. Maps to `VexExportFormat`. |
| `filters` | – | Key/value pairs executed via `VexQueryFilter`. Keys must match export data source columns (e.g., `vulnId`, `productKey`). |
| `sort` | – | Key/boolean map (false = descending). |
| `limit`, `offset`, `view` | – | Optional query bounds passed through to the export query. |

⚠️ **Misconfiguration:** invalid formats or missing keys cause exports to be flagged with `status` in the index response; they are not exposed downstream.

---

## 2) HTTP surface

Routes are grouped under `/excititor/mirror`.

| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/domains` | Returns configured domains with quota metadata. |
| `GET` | `/domains/{domainId}` | Domain detail (auth/quota + export keys). `404` for unknown domains. |
| `GET` | `/domains/{domainId}/index` | Lists exports with exportId, query signature, format, artifact digest, attestation metadata, and size. Applies index quota. |
| `GET` | `/domains/{domainId}/exports/{exportKey}` | Returns manifest metadata (single export). `404` if unknown/missing. |
| `GET` | `/domains/{domainId}/exports/{exportKey}/download` | Streams export content from the artifact store. Applies download quota. |

Responses are serialized via `VexCanonicalJsonSerializer` ensuring stable ordering. Download responses include a content-disposition header naming the file `<domain>-<export>.<ext>`.

### Error handling

* `401` – authentication required (`requireAuthentication=true`).
* `404` – domain/export not found or manifest not persisted.
* `429` – per-domain quota exceeded (`Retry-After` header set in seconds).
* `503` – export misconfiguration (invalid format/query).

---

## 3) Rate limiting

`MirrorRateLimiter` implements a simple rolling 1-hour window using `IMemoryCache`. Each domain has two quotas:

* `index` scope → `maxIndexRequestsPerHour`
* `download` scope → `maxDownloadRequestsPerHour`

`0` or negative limits disable enforcement. Quotas are best-effort (per-instance). For HA deployments, configure sticky routing at the ingress or replace the limiter with a distributed implementation.

---

## 4) Interaction with export pipeline

Mirror endpoints consume manifests produced by the export engine (`MongoVexExportStore`). They do **not** trigger new exports. Operators must configure connectors/exporters to keep targeted exports fresh (see `EXCITITOR-EXPORT-01-005/006/007`).

Recommended workflow:

1. Define export plans at the export layer (JSON/OpenVEX/CSAF).
2. Configure mirror domains mapping to those plans.
3. Downstream mirror automation:
   * `GET /domains/{id}/index`
   * Compare `exportId` / `consensusRevision`
   * `GET /download` when new
   * Verify digest + attestation

When the export engine runs, it materializes the following artefacts under `outputRoot/<directoryName>`:

- `index.json` – canonical index listing each configured domain, manifest/bundle descriptors (with SHA-256 digests), and available export keys.
- `<domain>/manifest.json` – per-domain summary with export metadata (query signature, consensus/score digests, source providers) and a descriptor pointing at the bundle.
- `<domain>/bundle.json` – canonical payload containing serialized consensus, score envelopes, and normalized VEX claims for the matching export definitions.
- `<domain>/bundle.json.jws` – optional detached JWS when signing is enabled.

Downstream automation reads `manifest.json`/`bundle.json` directly, while `/excititor/mirror` endpoints stream the same artefacts through authenticated HTTP.

---

## 5) Operational guidance

* Track quota utilisation via HTTP 429 metrics (configure structured logging or OTEL counters when rate limiting triggers).
* Mirror domains can be deployed per tenant (e.g., `tenant-a`, `tenant-b`) with different auth requirements.
* Ensure the underlying artifact stores (`FileSystem`, `S3`, offline bundle) retain artefacts long enough for mirrors to sync.
* For air-gapped mirrors, combine mirror endpoints with the Offline Kit (see `docs/24_OFFLINE_KIT.md`).

---

## 6) Future alignment

* Replace manual export definitions with generated mirror bundle manifests once `EXCITITOR-EXPORT-01-007` ships.
* Extend `/index` payload with quiet-provenance when `EXCITITOR-EXPORT-01-006` adds that metadata.
* Integrate domain manifests with DevOps mirror profiles (`DEVOPS-MIRROR-08-001`) so helm/compose overlays can enable or disable domains declaratively.