# architecture_excititor_mirrors.md — Excititor Mirror Distribution

> **Status:** Draft (Sprint 7). Complements `docs/modules/excititor/architecture.md` by describing the mirror export surface exposed by `Excititor.WebService` and the configuration hooks used by operators and downstream mirrors.

---

## 0) Purpose

Excititor publishes canonical VEX consensus data. Operators (or StellaOps-managed mirrors) need a deterministic way to sync those exports into downstream environments. Mirror distribution provides:

* A declarative map of export bundles (`json`, `jsonl`, `openvex`, `csaf`) reachable via signed HTTP endpoints under `/excititor/mirror`.
* Thin quota/authentication controls on top of the existing export cache so mirrors cannot starve the web service.
* Stable payload shapes that downstream automation can monitor (index → fetch updates → download artifact → verify signature).

Mirror endpoints are intentionally **read-only**. Write paths (export generation, attestation, cache) remain the responsibility of the export pipeline.

---

## 1) Configuration model

The web service reads mirror configuration from `Excititor:Mirror` (YAML/JSON/appsettings). Each domain groups a set of exports that share rate limits and authentication rules.

```yaml
Excititor:
  Mirror:
    Domains:
      - id: primary
        displayName: Primary Mirror
        requireAuthentication: false
        maxIndexRequestsPerHour: 600
        maxDownloadRequestsPerHour: 1200
        exports:
          - key: consensus
            format: json
            filters:
              vulnId: CVE-2025-0001
              productKey: pkg:test/demo
            sort:
              createdAt: false     # descending
            limit: 1000
          - key: consensus-openvex
            format: openvex
            filters:
              vulnId: CVE-2025-0001
```

### Root settings

| Field | Required | Description |
| --- | --- | --- |
| `outputRoot` | – | Filesystem root where mirror artefacts are written. Defaults to the Excititor file-system artifact store root when omitted. |
| `directoryName` | – | Optional subdirectory created under `outputRoot`; defaults to `mirror`. |
| `targetRepository` | – | Hint propagated to manifests/index files indicating the operator-visible location (for example `s3://mirror/excititor`). |
| `signing` | – | Bundle signing configuration. When enabled, the exporter emits a detached JWS (`bundle.json.jws`) alongside each domain bundle. |

`signing` supports the following fields:

| Field | Required | Description |
| --- | --- | --- |
| `enabled` | – | Toggles detached signing for domain bundles. |
| `algorithm` | – | Signing algorithm identifier (default `ES256`). |
| `keyId` | ✅ (when `enabled`) | Signing key identifier resolved via the configured crypto provider registry. |
| `provider` | – | Optional provider hint when multiple registries are available. |
| `keyPath` | – | Optional PEM path used to seed the provider when the key is not already loaded. |

### Domain field reference

| Field | Required | Description |
| --- | --- | --- |
| `id` | ✅ | Stable identifier. Appears in URLs (`/excititor/mirror/domains/{id}`) and download filenames. |
| `displayName` | – | Human-friendly label surfaced in the `/domains` listing. Falls back to `id`. |
| `requireAuthentication` | – | When `true` the service enforces that the caller is authenticated (Authority token). |
| `maxIndexRequestsPerHour` | – | Per-domain quota for index endpoints. `0`/negative disables the guard. |
| `maxDownloadRequestsPerHour` | – | Per-domain quota for artifact downloads. |
| `exports` | ✅ | Collection of export projections. |

Export-level fields:

| Field | Required | Description |
| --- | --- | --- |
| `key` | ✅ | Unique key within the domain. Used in URLs (`/exports/{key}`) and filenames/bundle entries. |
| `format` | ✅ | One of `json`, `jsonl`, `openvex`, `csaf`. Maps to `VexExportFormat`. |
| `filters` | – | Key/value pairs executed via `VexQueryFilter`. Keys must match export data source columns (e.g., `vulnId`, `productKey`). |
| `sort` | – | Key/boolean map (false = descending). |
| `limit`, `offset`, `view` | – | Optional query bounds passed through to the export query. |

⚠️ **Misconfiguration:** invalid formats or missing keys cause exports to be flagged with `status` in the index response; they are not exposed downstream.

---

## 2) HTTP surface

Routes are grouped under `/excititor/mirror`.

| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/domains` | Returns configured domains with quota metadata. |
| `GET` | `/domains/{domainId}` | Domain detail (auth/quota + export keys). `404` for unknown domains. |
| `GET` | `/domains/{domainId}/index` | Lists exports with exportId, query signature, format, artifact digest, attestation metadata, and size. Applies index quota. |
| `GET` | `/domains/{domainId}/exports/{exportKey}` | Returns manifest metadata (single export). `404` if unknown/missing. |
| `GET` | `/domains/{domainId}/exports/{exportKey}/download` | Streams export content from the artifact store. Applies download quota. |

Responses are serialized via `VexCanonicalJsonSerializer` ensuring stable ordering. Download responses include a content-disposition header naming the file `<domain>-<export>.<ext>`.

### Error handling

* `401` – authentication required (`requireAuthentication=true`).
* `404` – domain/export not found or manifest not persisted.
* `429` – per-domain quota exceeded (`Retry-After` header set in seconds).
* `503` – export misconfiguration (invalid format/query).

---

## 3) Rate limiting

`MirrorRateLimiter` implements a simple rolling 1-hour window using `IMemoryCache`. Each domain has two quotas:

* `index` scope → `maxIndexRequestsPerHour`
* `download` scope → `maxDownloadRequestsPerHour`

`0` or negative limits disable enforcement. Quotas are best-effort (per-instance). For HA deployments, configure sticky routing at the ingress or replace the limiter with a distributed implementation.

---

## 4) Interaction with export pipeline

Mirror endpoints consume manifests produced by the export engine (`MongoVexExportStore`). They can also be generated on demand via the mirror domain management API (`POST /api/v1/mirror/domains/{domainId}/generate`).

### 4.1 MirrorExportScheduler (background bundle refresh)

`MirrorExportScheduler` is a `BackgroundService` that periodically checks configured mirror domains for stale export bundles and triggers regeneration when source data has been updated since the last bundle generation.

**Behavior:**

1. On startup, loads all configured mirror domains from DB and config.
2. Every N minutes (configurable via `RefreshIntervalMinutes`, default 60), iterates all domains.
3. For each domain, checks whether any source in the export filters has been updated since the last bundle generation by comparing connector state timestamps.
4. If stale, triggers bundle regeneration via `IMirrorBundleRegenerator`.
5. Logs bundle generation metrics (duration, advisory count, stale/regenerated counts).
6. Exposes per-domain staleness via `GetDomainStatuses()` and the `/api/v1/mirror/domains/{domainId}/status` endpoint.

**Configuration:**

| Setting | Default | Description |
| --- | --- | --- |
| `AutoRefreshEnabled` | `true` | Set to `false` to disable the scheduler entirely. Required for air-gap deployments where bundles are imported, not generated locally. |
| `RefreshIntervalMinutes` | `60` | How often the scheduler checks for stale exports. Minimum 1 minute. |
| `Enabled` | `true` | Global mirror distribution enable flag. When `false`, the scheduler skips refresh cycles. |

**Staleness detection** uses `IVexConnectorStateRepository` to find the latest source update timestamp across all connectors. If any source was updated after the last bundle generation timestamp, the export is considered stale.

### 4.2 Export workflow

Recommended workflow:

1. **Define mirror domains** via the UI wizard or API (`POST /api/v1/mirror/domains`).
2. **Configure exports** with filter shorthands (see 4.3 below).
3. **Automatic refresh**: `MirrorExportScheduler` regenerates stale bundles periodically.
4. **On-demand generation**: `POST /api/v1/mirror/domains/{domainId}/generate` for immediate bundle creation.
5. **Downstream mirror automation**:
   * `GET /domains/{id}/index`
   * Compare `exportId` / `consensusRevision`
   * `GET /download` when new
   * Verify digest + attestation

When the export engine runs, it materializes the following artefacts under `outputRoot/<directoryName>`:

- `index.json` -- canonical index listing each configured domain, manifest/bundle descriptors (with SHA-256 digests), and available export keys.
- `<domain>/manifest.json` -- per-domain summary with export metadata (query signature, consensus/score digests, source providers) and a descriptor pointing at the bundle.
- `<domain>/bundle.json` -- canonical payload containing serialized consensus, score envelopes, and normalized VEX claims for the matching export definitions.
- `<domain>/bundle.json.jws` -- optional detached JWS when signing is enabled.

Downstream automation reads `manifest.json`/`bundle.json` directly, while `/excititor/mirror` endpoints stream the same artefacts through authenticated HTTP.

### 4.3 Multi-value filter support and category shorthands

Export filters in `MirrorExportOptions.Filters` support three expansion modes via the `ResolveFilters()` method:

**sourceCategory shorthand** -- resolves a category name (or comma-separated list) to all source IDs in that category:

```json
{
  "Key": "linux-distros",
  "Format": "openvex",
  "Filters": { "sourceCategory": "Distribution" }
}
```

This resolves to all 10 distribution sources (Debian, Ubuntu, Alpine, SUSE, RHEL, CentOS, Fedora, Arch, Gentoo, Astra Linux). Multiple categories are comma-separated: `"Exploit,Container,Ics,PackageManager"`.

**sourceTag shorthand** -- resolves a tag (or comma-separated list) to all sources carrying that tag:

```json
{
  "Key": "linux-all",
  "Format": "json",
  "Filters": { "sourceTag": "linux" }
}
```

**Comma-separated sourceVendor** -- OR semantics across multiple vendors:

```json
{
  "Key": "custom-selection",
  "Format": "json",
  "Filters": { "sourceVendor": "debian,ubuntu,alpine" }
}
```

All resolved values are sorted alphabetically for deterministic query signatures. Existing single-value filters remain backward-compatible.

**Supported categories** (14 total, matching the `SourceCategory` enum):

`Primary`, `Vendor`, `Distribution`, `Ecosystem`, `Cert`, `Csaf`, `Threat`, `Exploit`, `Container`, `Hardware`, `Ics`, `PackageManager`, `Mirror`, `Other`.

---

## 5) Mirror configuration UI

The mirror configuration UI provides three Angular components for managing mirror domains from the StellaOps Console, accessible under the Advisory & VEX Sources integration hub.

### 5.1 Mirror Domain Builder wizard

Route: `advisory-vex-sources/mirror/new`

A 3-step wizard for creating mirror domains from the source catalog:

**Step 1 -- Select Sources**: Displays the full source catalog grouped by the 14 categories. Operators can select sources individually or by category (checking a category header selects all sources in that category). Shorthand buttons provide quick selections: "All Primary", "All Distributions", "All Ecosystem", "All CERTs", "Everything". A live summary panel shows selected source count by category.

**Step 2 -- Configure Domain**: Auto-generates a domain ID and display name from the selection. Operators configure the export format (JSON, JSONL, OpenVEX, CSAF, CycloneDX), rate limits (index requests/hour, download requests/hour), authentication requirement, signing options, and optionally create multiple exports per domain.

**Step 3 -- Review & Create**: Summary card showing domain name, source count across categories, export format, and rate limits. Displays the resolved filter JSON that will be stored. "Create Domain" calls `POST /api/v1/mirror/domains`. An optional "Generate Now" checkbox triggers immediate bundle generation after creation.

### 5.2 Mirror Dashboard

Route: `advisory-vex-sources/mirror`

Displays all configured mirror domains as status cards:

* **Top bar**: Mirror mode indicator (Direct/Mirror/Hybrid), "Create Domain" button, global mirror health summary.
* **Domain cards**: Each card shows domain name, export count and format, source count with category pills, last generated timestamp, bundle size, and staleness indicator (fresh/stale/never generated). Actions: Regenerate, Edit, Delete, View Endpoints.
* **Consumer config panel**: For Hybrid or Mirror mode, shows the consumer mirror URL, connection status, and last sync time.
* **Empty state**: If no domains are configured, displays a "Create your first mirror domain" call-to-action linking to the wizard.

### 5.3 Catalog mirror integration

The source catalog header (`advisory-source-catalog.component.ts`) includes mirror context:

* Mirror mode badge (Direct/Mirror/Hybrid) in the catalog header.
* "Configure Mirror" link navigating to the mirror dashboard.
* "Create Mirror Domain" button navigating to the wizard with pre-selected sources based on currently enabled sources.
* Mirror domain count and total bundle advisory count in the stats bar (when domains exist).

---

## 6) Operational guidance

* Track quota utilisation via HTTP 429 metrics (configure structured logging or OTEL counters when rate limiting triggers).
* Mirror domains can be deployed per tenant (e.g., `tenant-a`, `tenant-b`) with different auth requirements.
* Ensure the underlying artifact stores (`FileSystem`, `S3`, offline bundle) retain artefacts long enough for mirrors to sync.
* For air-gapped mirrors, combine mirror endpoints with the Offline Kit (see `docs/OFFLINE_KIT.md`).

---

## 7) Future alignment

* Replace manual export definitions with generated mirror bundle manifests once `EXCITITOR-EXPORT-01-007` ships.
* Extend `/index` payload with quiet-provenance when `EXCITITOR-EXPORT-01-006` adds that metadata.
* Integrate domain manifests with DevOps mirror profiles (`DEVOPS-MIRROR-08-001`) so helm/compose overlays can enable or disable domains declaratively.

---

## 8) Runbook & observability checklist (Sprint 22 demo refresh · 2025-11-07)

### Daily / on-call checks
1. **Index freshness** – watch `excitor_mirror_export_latency_seconds` (p95 < 180) grouped by `domainId`. If latency grows past 10 minutes, verify the export worker queue (`stellaops-export-worker` logs) and ensure PostgreSQL `vex.exports` has entries newer than `now()-10m`.
2. **Quota exhaustion** – alert on `excitor_mirror_quota_exhausted_total{scope="download"}` increases. When triggered, inspect structured logs (`MirrorDomainId`, `QuotaScope`, `RemoteIp`) and either raise limits or throttle abusive clients.
3. **Bundle signature health** – metric `excitor_mirror_bundle_signature_verified_total` should match download counts when signing enabled. Deltas indicate missing `.jws` files; rebuild the bundle via export job or copy artefacts from the authority mirror cache.
4. **HTTP errors** – dashboards should track 4xx/5xx rates split by route; repeated `503` statuses imply misconfigured exports. Check `mirror/index` logs for `status=misconfigured`.

### Incident steps
1. Use `GET /excititor/mirror/domains/{id}/index` to capture current manifests. Attach the response to the incident log for reproducibility.
2. For quota incidents, temporarily raise `maxIndexRequestsPerHour`/`maxDownloadRequestsPerHour` via the `Excititor:Mirror:Domains` config override, redeploy, then work with the consuming team on caching.
3. For stale exports, trigger the export job (`Excititor.ExportRunner`) and confirm the artefacts are written to `outputRoot/<domain>`.
4. Validate DSSE artefacts by running `cosign verify-blob --certificate-rekor-url=<rekor> --bundle <domain>/bundle.json --signature <domain>/bundle.json.jws`.

### Logging fields (structured)
| Field | Description |
| --- | --- |
| `MirrorDomainId` | Domain handling the request (matches `id` in config). |
| `QuotaScope` | `index` / `download`, useful when alerting on quota events. |
| `ExportKey` | Included in download logs to pinpoint misconfigured exports. |
| `BundleDigest` | SHA-256 of the artefact; compare with index payload when debugging corruption. |

### OTEL signals
- **Counters:** `excitor.mirror.requests`, `excitor.mirror.quota_blocked`, `excitor.mirror.signature.failures`.
- **Histograms:** `excitor.mirror.download.duration`, `excitor.mirror.export.latency`.
- **Spans:** `mirror.index`, `mirror.download` include attributes `mirror.domain`, `mirror.export.key`, and `mirror.quota.remaining`.

Add these instruments via the `MirrorEndpoints` middleware; see `StellaOps.Excititor.WebService/Telemetry/MirrorMetrics.cs`.