Add unit and integration tests for VexCandidateEmitter and SmartDiff repositories
- Implemented comprehensive unit tests for VexCandidateEmitter to validate candidate emission logic based on various scenarios including absent and present APIs, confidence thresholds, and rate limiting. - Added integration tests for SmartDiff PostgreSQL repositories, covering snapshot storage and retrieval, candidate storage, and material risk change handling. - Ensured tests validate correct behavior for storing, retrieving, and querying snapshots and candidates, including edge cases and expected outcomes.
This commit is contained in:
master
@@ -6,21 +6,21 @@
|
||||
|
||||
| Method | Path | Description | Scopes |
|
||||
| ------ | ---- | ----------- | ------ |
|
||||
| `GET` | `/api/v1/scheduler/runs` | List runs for the current tenant (filter by schedule, state, createdAfter, cursor). | `scheduler.runs.read` |
|
||||
| `GET` | `/api/v1/scheduler/runs/{runId}` | Retrieve run details. | `scheduler.runs.read` |
|
||||
| `GET` | `/api/v1/scheduler/runs/{runId}/deltas` | Fetch deterministic delta metadata for the specified run. | `scheduler.runs.read` |
|
||||
| `GET` | `/api/v1/scheduler/runs/queue/lag` | Snapshot queue depth per transport/queue for console dashboards. | `scheduler.runs.read` |
|
||||
| `GET` | `/api/v1/scheduler/runs/{runId}/stream` | Server-sent events (SSE) stream for live progress, queue lag, and heartbeats. | `scheduler.runs.read` |
|
||||
| `POST` | `/api/v1/scheduler/runs` | Create an ad-hoc run bound to an existing schedule. | `scheduler.runs.write` |
|
||||
| `POST` | `/api/v1/scheduler/runs/{runId}/cancel` | Transition a run to `cancelled` when still in a non-terminal state. | `scheduler.runs.manage` |
|
||||
| `POST` | `/api/v1/scheduler/runs/{runId}/retry` | Clone a terminal run into a new manual retry, preserving provenance. | `scheduler.runs.manage` |
|
||||
| `POST` | `/api/v1/scheduler/runs/preview` | Resolve impacted images using the ImpactIndex without enqueuing work. | `scheduler.runs.preview` |
|
||||
| `GET` | `/api/v1/scheduler/policies/simulations` | List policy simulations for the current tenant (filters: policyId, status, since, limit). | `policy:simulate` |
|
||||
| `GET` | `/api/v1/scheduler/policies/simulations/{simulationId}` | Retrieve simulation status snapshot. | `policy:simulate` |
|
||||
| `GET` | `/api/v1/scheduler/policies/simulations/{simulationId}/stream` | SSE stream emitting simulation status, queue lag, and heartbeats. | `policy:simulate` |
|
||||
| `POST` | `/api/v1/scheduler/policies/simulations` | Enqueue a policy simulation (mode=`simulate`) with optional SBOM inputs and metadata. | `policy:simulate` |
|
||||
| `POST` | `/api/v1/scheduler/policies/simulations/{simulationId}/cancel` | Request cancellation for an in-flight simulation. | `policy:simulate` |
|
||||
| `POST` | `/api/v1/scheduler/policies/simulations/{simulationId}/retry` | Clone a terminal simulation into a new run preserving inputs/metadata. | `policy:simulate` |
|
||||
| `GET` | `/api/v1/scheduler/runs` | List runs for the current tenant (filter by schedule, state, createdAfter, cursor). | `scheduler.runs.read` |
|
||||
| `GET` | `/api/v1/scheduler/runs/{runId}` | Retrieve run details. | `scheduler.runs.read` |
|
||||
| `GET` | `/api/v1/scheduler/runs/{runId}/deltas` | Fetch deterministic delta metadata for the specified run. | `scheduler.runs.read` |
|
||||
| `GET` | `/api/v1/scheduler/runs/queue/lag` | Snapshot queue depth per transport/queue for console dashboards. | `scheduler.runs.read` |
|
||||
| `GET` | `/api/v1/scheduler/runs/{runId}/stream` | Server-sent events (SSE) stream for live progress, queue lag, and heartbeats. | `scheduler.runs.read` |
|
||||
| `POST` | `/api/v1/scheduler/runs` | Create an ad-hoc run bound to an existing schedule. | `scheduler.runs.write` |
|
||||
| `POST` | `/api/v1/scheduler/runs/{runId}/cancel` | Transition a run to `cancelled` when still in a non-terminal state. | `scheduler.runs.manage` |
|
||||
| `POST` | `/api/v1/scheduler/runs/{runId}/retry` | Clone a terminal run into a new manual retry, preserving provenance. | `scheduler.runs.manage` |
|
||||
| `POST` | `/api/v1/scheduler/runs/preview` | Resolve impacted images using the ImpactIndex without enqueuing work. | `scheduler.runs.preview` |
|
||||
| `GET` | `/api/v1/scheduler/policies/simulations` | List policy simulations for the current tenant (filters: policyId, status, since, limit). | `policy:simulate` |
|
||||
| `GET` | `/api/v1/scheduler/policies/simulations/{simulationId}` | Retrieve simulation status snapshot. | `policy:simulate` |
|
||||
| `GET` | `/api/v1/scheduler/policies/simulations/{simulationId}/stream` | SSE stream emitting simulation status, queue lag, and heartbeats. | `policy:simulate` |
|
||||
| `POST` | `/api/v1/scheduler/policies/simulations` | Enqueue a policy simulation (mode=`simulate`) with optional SBOM inputs and metadata. | `policy:simulate` |
|
||||
| `POST` | `/api/v1/scheduler/policies/simulations/{simulationId}/cancel` | Request cancellation for an in-flight simulation. | `policy:simulate` |
|
||||
| `POST` | `/api/v1/scheduler/policies/simulations/{simulationId}/retry` | Clone a terminal simulation into a new run preserving inputs/metadata. | `policy:simulate` |
|
||||
|
||||
All endpoints require a tenant context (`X-Tenant-Id`) and the appropriate scheduler scopes. Development mode allows header-based auth; production deployments must rely on Authority-issued tokens (OpTok + DPoP).
|
||||
|
||||
@@ -80,12 +80,12 @@ GET /api/v1/scheduler/runs?scheduleId=sch_4f2c7d9e0a2b4c64a0e7b5f9d65c1234&state
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"runs": [
|
||||
{
|
||||
"schemaVersion": "scheduler.run@1",
|
||||
"id": "run_c7b4e9d2f6a04f8784a40476d8a2f771",
|
||||
"tenantId": "tenant-alpha",
|
||||
{
|
||||
"runs": [
|
||||
{
|
||||
"schemaVersion": "scheduler.run@1",
|
||||
"id": "run_c7b4e9d2f6a04f8784a40476d8a2f771",
|
||||
"tenantId": "tenant-alpha",
|
||||
"scheduleId": "sch_4f2c7d9e0a2b4c64a0e7b5f9d65c1234",
|
||||
"trigger": "manual",
|
||||
"state": "planning",
|
||||
@@ -103,13 +103,13 @@ GET /api/v1/scheduler/runs?scheduleId=sch_4f2c7d9e0a2b4c64a0e7b5f9d65c1234&state
|
||||
"reason": {
|
||||
"manualReason": "Nightly backfill"
|
||||
},
|
||||
"createdAt": "2025-10-26T03:12:45Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
When additional pages are available the response includes `"nextCursor": "<base64>"`. Clients pass this cursor via `?cursor=` to fetch the next deterministic slice (ordering = `createdAt desc, id desc`).
|
||||
"createdAt": "2025-10-26T03:12:45Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
When additional pages are available the response includes `"nextCursor": "<base64>"`. Clients pass this cursor via `?cursor=` to fetch the next deterministic slice (ordering = `createdAt desc, id desc`).
|
||||
|
||||
## Cancel Run
|
||||
|
||||
@@ -148,33 +148,33 @@ POST /api/v1/scheduler/runs/run_c7b4e9d2f6a04f8784a40476d8a2f771/cancel
|
||||
|
||||
## Impact Preview
|
||||
|
||||
`/api/v1/scheduler/runs/preview` resolves impacted images via the ImpactIndex without mutating state. When `scheduleId` is provided the schedule selector is reused; callers may alternatively supply an explicit selector.
|
||||
|
||||
## Retry Run
|
||||
|
||||
`POST /api/v1/scheduler/runs/{runId}/retry` clones a terminal run into a new manual run with `retryOf` pointing to the original identifier. Retry is scope-gated with `scheduler.runs.manage`; the new run’s `reason.manualReason` gains a `retry-of:<runId>` suffix for provenance.
|
||||
|
||||
## Run deltas
|
||||
|
||||
`GET /api/v1/scheduler/runs/{runId}/deltas` returns an immutable, deterministically sorted array of delta summaries (`[imageDigest, severity slices, KEV hits, attestations]`).
|
||||
|
||||
## Queue lag snapshot
|
||||
|
||||
`GET /api/v1/scheduler/runs/queue/lag` exposes queue depth summaries for planner/runner transports. The payload includes `capturedAt`, `totalDepth`, `maxDepth`, and ordered queue entries (transport + queue + depth). Console uses this for backlog dashboards and alert thresholds.
|
||||
|
||||
## Live stream (SSE)
|
||||
|
||||
`GET /api/v1/scheduler/runs/{runId}/stream` emits server-sent events for:
|
||||
|
||||
- `initial` — full run snapshot
|
||||
- `stateChanged` — state/started/finished transitions
|
||||
- `segmentProgress` — stats updates
|
||||
- `deltaSummary` — deltas available
|
||||
- `queueLag` — periodic queue snapshots
|
||||
- `heartbeat` — uptime keep-alive (default 5s)
|
||||
- `completed` — terminal summary
|
||||
|
||||
The stream is tolerant to clients reconnecting (idempotent payloads, deterministic ordering) and honours tenant scope plus cancellation tokens.
|
||||
`/api/v1/scheduler/runs/preview` resolves impacted images via the ImpactIndex without mutating state. When `scheduleId` is provided the schedule selector is reused; callers may alternatively supply an explicit selector.
|
||||
|
||||
## Retry Run
|
||||
|
||||
`POST /api/v1/scheduler/runs/{runId}/retry` clones a terminal run into a new manual run with `retryOf` pointing to the original identifier. Retry is scope-gated with `scheduler.runs.manage`; the new run’s `reason.manualReason` gains a `retry-of:<runId>` suffix for provenance.
|
||||
|
||||
## Run deltas
|
||||
|
||||
`GET /api/v1/scheduler/runs/{runId}/deltas` returns an immutable, deterministically sorted array of delta summaries (`[imageDigest, severity slices, KEV hits, attestations]`).
|
||||
|
||||
## Queue lag snapshot
|
||||
|
||||
`GET /api/v1/scheduler/runs/queue/lag` exposes queue depth summaries for planner/runner transports. The payload includes `capturedAt`, `totalDepth`, `maxDepth`, and ordered queue entries (transport + queue + depth). Console uses this for backlog dashboards and alert thresholds.
|
||||
|
||||
## Live stream (SSE)
|
||||
|
||||
`GET /api/v1/scheduler/runs/{runId}/stream` emits server-sent events for:
|
||||
|
||||
- `initial` — full run snapshot
|
||||
- `stateChanged` — state/started/finished transitions
|
||||
- `segmentProgress` — stats updates
|
||||
- `deltaSummary` — deltas available
|
||||
- `queueLag` — periodic queue snapshots
|
||||
- `heartbeat` — uptime keep-alive (default 5s)
|
||||
- `completed` — terminal summary
|
||||
|
||||
The stream is tolerant to clients reconnecting (idempotent payloads, deterministic ordering) and honours tenant scope plus cancellation tokens.
|
||||
|
||||
```http
|
||||
POST /api/v1/scheduler/runs/preview
|
||||
@@ -216,106 +216,106 @@ POST /api/v1/scheduler/runs/preview
|
||||
|
||||
### Integration notes
|
||||
|
||||
* Run creation and cancellation produce audit entries under category `scheduler.run` with correlation metadata when provided.
|
||||
* The preview endpoint relies on the ImpactIndex stub in development. Production deployments must register the concrete index implementation before use.
|
||||
* Planner/worker orchestration tasks will wire run creation to queueing in SCHED-WORKER-16-201/202.
|
||||
|
||||
## Policy simulations
|
||||
|
||||
The policy simulation APIs mirror the run endpoints but operate on policy-mode jobs (`mode=simulate`) scoped by tenant and RBAC (`policy:simulate`).
|
||||
|
||||
### Create simulation
|
||||
|
||||
```http
|
||||
POST /api/v1/scheduler/policies/simulations
|
||||
X-Tenant-Id: tenant-alpha
|
||||
Authorization: Bearer <OpTok>
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"policyId": "P-7",
|
||||
"policyVersion": 4,
|
||||
"priority": "normal",
|
||||
"metadata": {
|
||||
"source": "console.review"
|
||||
},
|
||||
"inputs": {
|
||||
"sbomSet": ["sbom:S-318", "sbom:S-42"],
|
||||
"captureExplain": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
HTTP/1.1 201 Created
|
||||
Location: /api/v1/scheduler/policies/simulations/run:P-7:20251103T153000Z:e4d1a9b2
|
||||
{
|
||||
"simulation": {
|
||||
"schemaVersion": "scheduler.policy-run-status@1",
|
||||
"runId": "run:P-7:20251103T153000Z:e4d1a9b2",
|
||||
"tenantId": "tenant-alpha",
|
||||
"policyId": "P-7",
|
||||
"policyVersion": 4,
|
||||
"mode": "simulate",
|
||||
"status": "queued",
|
||||
"priority": "normal",
|
||||
"queuedAt": "2025-11-03T15:30:00Z",
|
||||
"stats": {
|
||||
"components": 0,
|
||||
"rulesFired": 0,
|
||||
"findingsWritten": 0,
|
||||
"vexOverrides": 0
|
||||
},
|
||||
"inputs": {
|
||||
"sbomSet": ["sbom:S-318", "sbom:S-42"],
|
||||
"captureExplain": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Canonical payload lives in `samples/api/scheduler/policy-simulation-status.json`.
|
||||
|
||||
### List and fetch simulations
|
||||
|
||||
- `GET /api/v1/scheduler/policies/simulations?policyId=P-7&status=queued&limit=25`
|
||||
- `GET /api/v1/scheduler/policies/simulations/{simulationId}`
|
||||
|
||||
The response envelope mirrors `policy-run-status` but uses `simulations` / `simulation` wrappers. All metadata keys are lower-case; retries append `retry-of=<priorRunId>` for provenance.
|
||||
|
||||
### Cancel and retry
|
||||
|
||||
- `POST /api/v1/scheduler/policies/simulations/{simulationId}/cancel`
|
||||
- Marks the job as `cancellationRequested` and surfaces the reason. Worker execution honours this flag before leasing.
|
||||
- `POST /api/v1/scheduler/policies/simulations/{simulationId}/retry`
|
||||
- Clones a terminal simulation, preserving inputs/metadata and adding `metadata.retry-of` pointing to the original run ID. Returns `409 Conflict` when the simulation is not terminal.
|
||||
|
||||
### Live stream (SSE)
|
||||
|
||||
`GET /api/v1/scheduler/policies/simulations/{simulationId}/stream` emits:
|
||||
|
||||
- `retry` — reconnection hint (milliseconds) emitted before events.
|
||||
- `initial` — current simulation snapshot.
|
||||
- `status` — status/attempt/stat updates.
|
||||
- `queueLag` — periodic queue depth summary (shares payload with run streams).
|
||||
- `heartbeat` — keep-alive ping (default 5s; configurable under `Scheduler:RunStream`).
|
||||
- `completed` — terminal summary (`succeeded`, `failed`, or `cancelled`).
|
||||
- `notFound` — emitted if the run record disappears while streaming.
|
||||
|
||||
Heartbeats, queue lag summaries, and the reconnection directive are sent immediately after connection so Console clients receive deterministic telemetry when loading a simulation workspace.
|
||||
|
||||
### Metrics
|
||||
|
||||
```
|
||||
GET /api/v1/scheduler/policies/simulations/metrics
|
||||
X-Tenant-Id: tenant-alpha
|
||||
Authorization: Bearer <OpTok>
|
||||
```
|
||||
|
||||
Returns queue depth and latency summaries tailored for simulation dashboards and alerting. Response properties align with the metric names exposed via OTEL (`policy_simulation_queue_depth`, `policy_simulation_latency_seconds`). Canonical payload lives at `samples/api/scheduler/policy-simulation-metrics.json`.
|
||||
|
||||
- `policy_simulation_queue_depth.total` — pending simulation jobs (aggregate of `pending`, `dispatching`, `submitted`).
|
||||
- `policy_simulation_latency.*` — latency percentiles (seconds) computed from the most recent terminal simulations.
|
||||
|
||||
> **Note:** When Mongo storage is not configured the metrics provider is disabled and the endpoint responds with `501 Not Implemented`.
|
||||
* Run creation and cancellation produce audit entries under category `scheduler.run` with correlation metadata when provided.
|
||||
* The preview endpoint relies on the ImpactIndex stub in development. Production deployments must register the concrete index implementation before use.
|
||||
* Planner/worker orchestration tasks will wire run creation to queueing in SCHED-WORKER-16-201/202.
|
||||
|
||||
## Policy simulations
|
||||
|
||||
The policy simulation APIs mirror the run endpoints but operate on policy-mode jobs (`mode=simulate`) scoped by tenant and RBAC (`policy:simulate`).
|
||||
|
||||
### Create simulation
|
||||
|
||||
```http
|
||||
POST /api/v1/scheduler/policies/simulations
|
||||
X-Tenant-Id: tenant-alpha
|
||||
Authorization: Bearer <OpTok>
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"policyId": "P-7",
|
||||
"policyVersion": 4,
|
||||
"priority": "normal",
|
||||
"metadata": {
|
||||
"source": "console.review"
|
||||
},
|
||||
"inputs": {
|
||||
"sbomSet": ["sbom:S-318", "sbom:S-42"],
|
||||
"captureExplain": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
HTTP/1.1 201 Created
|
||||
Location: /api/v1/scheduler/policies/simulations/run:P-7:20251103T153000Z:e4d1a9b2
|
||||
{
|
||||
"simulation": {
|
||||
"schemaVersion": "scheduler.policy-run-status@1",
|
||||
"runId": "run:P-7:20251103T153000Z:e4d1a9b2",
|
||||
"tenantId": "tenant-alpha",
|
||||
"policyId": "P-7",
|
||||
"policyVersion": 4,
|
||||
"mode": "simulate",
|
||||
"status": "queued",
|
||||
"priority": "normal",
|
||||
"queuedAt": "2025-11-03T15:30:00Z",
|
||||
"stats": {
|
||||
"components": 0,
|
||||
"rulesFired": 0,
|
||||
"findingsWritten": 0,
|
||||
"vexOverrides": 0
|
||||
},
|
||||
"inputs": {
|
||||
"sbomSet": ["sbom:S-318", "sbom:S-42"],
|
||||
"captureExplain": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Canonical payload lives in `samples/api/scheduler/policy-simulation-status.json`.
|
||||
|
||||
### List and fetch simulations
|
||||
|
||||
- `GET /api/v1/scheduler/policies/simulations?policyId=P-7&status=queued&limit=25`
|
||||
- `GET /api/v1/scheduler/policies/simulations/{simulationId}`
|
||||
|
||||
The response envelope mirrors `policy-run-status` but uses `simulations` / `simulation` wrappers. All metadata keys are lower-case; retries append `retry-of=<priorRunId>` for provenance.
|
||||
|
||||
### Cancel and retry
|
||||
|
||||
- `POST /api/v1/scheduler/policies/simulations/{simulationId}/cancel`
|
||||
- Marks the job as `cancellationRequested` and surfaces the reason. Worker execution honours this flag before leasing.
|
||||
- `POST /api/v1/scheduler/policies/simulations/{simulationId}/retry`
|
||||
- Clones a terminal simulation, preserving inputs/metadata and adding `metadata.retry-of` pointing to the original run ID. Returns `409 Conflict` when the simulation is not terminal.
|
||||
|
||||
### Live stream (SSE)
|
||||
|
||||
`GET /api/v1/scheduler/policies/simulations/{simulationId}/stream` emits:
|
||||
|
||||
- `retry` — reconnection hint (milliseconds) emitted before events.
|
||||
- `initial` — current simulation snapshot.
|
||||
- `status` — status/attempt/stat updates.
|
||||
- `queueLag` — periodic queue depth summary (shares payload with run streams).
|
||||
- `heartbeat` — keep-alive ping (default 5s; configurable under `Scheduler:RunStream`).
|
||||
- `completed` — terminal summary (`succeeded`, `failed`, or `cancelled`).
|
||||
- `notFound` — emitted if the run record disappears while streaming.
|
||||
|
||||
Heartbeats, queue lag summaries, and the reconnection directive are sent immediately after connection so Console clients receive deterministic telemetry when loading a simulation workspace.
|
||||
|
||||
### Metrics
|
||||
|
||||
```
|
||||
GET /api/v1/scheduler/policies/simulations/metrics
|
||||
X-Tenant-Id: tenant-alpha
|
||||
Authorization: Bearer <OpTok>
|
||||
```
|
||||
|
||||
Returns queue depth and latency summaries tailored for simulation dashboards and alerting. Response properties align with the metric names exposed via OTEL (`policy_simulation_queue_depth`, `policy_simulation_latency_seconds`). Canonical payload lives at `samples/api/scheduler/policy-simulation-metrics.json`.
|
||||
|
||||
- `policy_simulation_queue_depth.total` — pending simulation jobs (aggregate of `pending`, `dispatching`, `submitted`).
|
||||
- `policy_simulation_latency.*` — latency percentiles (seconds) computed from the most recent terminal simulations.
|
||||
|
||||
> **Note:** When PostgreSQL storage is not configured the metrics provider is disabled and the endpoint responds with `501 Not Implemented`.
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
- `GET /api/v1/scheduler/policies/simulations/metrics` (scope: `policy:simulate`)
|
||||
- Returns queue depth grouped by status plus latency percentiles derived from the most recent sample window (default 200 terminal runs).
|
||||
- Surface area is unchanged from the implementation in Sprint 27 week 1; consumers should continue to rely on the contract in `samples/api/scheduler/policy-simulation-metrics.json`.
|
||||
- When backing storage is not Mongo the endpoint responds `501 Not Implemented`.
|
||||
- When backing storage is not PostgreSQL the endpoint responds `501 Not Implemented`.
|
||||
|
||||
## 2. Completion webhooks
|
||||
|
||||
|
||||
Reference in New Issue
Block a user