feat(api): Implement Console Export Client and Models

- Added ConsoleExportClient for managing export requests and responses. - Introduced ConsoleExportRequest and ConsoleExportResponse models. - Implemented methods for creating and retrieving exports with appropriate headers. feat(crypto): Add Software SM2/SM3 Cryptography Provider - Implemented SmSoftCryptoProvider for software-only SM2/SM3 cryptography. - Added support for signing and verification using SM2 algorithm. - Included hashing functionality with SM3 algorithm. - Configured options for loading keys from files and environment gate checks. test(crypto): Add unit tests for SmSoftCryptoProvider - Created comprehensive tests for signing, verifying, and hashing functionalities. - Ensured correct behavior for key management and error handling. feat(api): Enhance Console Export Models - Expanded ConsoleExport models to include detailed status and event types. - Added support for various export formats and notification options. test(time): Implement TimeAnchorPolicyService tests - Developed tests for TimeAnchorPolicyService to validate time anchors. - Covered scenarios for anchor validation, drift calculation, and policy enforcement.
2025-12-07 00:27:33 +02:00
parent 9bd6a73926
commit 0de92144d2
229 changed files with 32351 additions and 1481 deletions
--- a/docs/api/console/workspaces.md
+++ b/docs/api/console/workspaces.md
@@ -310,42 +310,84 @@ data: {

 > Until backend implementations ship, use the examples above to unblock DOCS-AIAI-31-004; replace them with live captures once the gateway endpoints are available in staging.

-## Exports (draft contract)
+## Exports (draft contract v0.3)

-Routes
+### Routes
 - `POST /console/exports` — start an evidence bundle export job.
 - `GET /console/exports/{exportId}` — fetch job status and download locations.
 - `GET /console/exports/{exportId}/events` — SSE stream of job progress (optional).

-Headers
- `Authorization: Bearer <token>`
+### Security / headers
+- `Authorization: DPoP <token>`
+- `DPoP: <proof>`
 - `X-StellaOps-Tenant: <tenantId>`
 - `Idempotency-Key: <uuid>` (recommended for POST)
 - `Accept: application/json` (status) or `text/event-stream` (events)
+- Required scopes: `console:read` AND `console:export` (proposal).

-Request body (POST /console/exports)
- `scope`: `{ tenantId, projectId? }`
- `sources`: array of `{ type: "advisory"|"vex"|"policy"|"scan", ids: string[] }`
- `formats`: array of `"json"|"csv"|"ndjson"|"pdf"`
- `attestations`: `{ include: boolean, sigstoreBundle?: boolean }`
- `notify`: `{ webhooks?: string[], email?: string[] }`
- `priority`: `"low"|"normal"|"high"`
+### Request body (POST)
+```jsonc
+{
+  "scope": { "tenantId": "t1", "projectId": "p1" },
+  "sources": [ { "type": "advisory", "ids": ["CVE-2024-12345"] } ],
+  "formats": ["json", "ndjson", "csv"],
+  "attestations": { "include": true, "sigstoreBundle": true },
+  "notify": { "webhooks": ["https://hooks.local/export"], "email": ["secops@example.com"] },
+  "priority": "normal"
+}
+```

-Responses
- `202 Accepted` with `exportId`, `status: queued|running|succeeded|failed|expired`, `estimateSeconds`, `retryAfter`.
- Status payload includes presigned download URLs, checksum manifest, and error list when failed.
- SSE events emit `started`, `progress` (percent, item counts), `asset_ready` (uri, sha256), `completed`, `failed` (code, message).
+### Response: 202 Accepted
+- `exportId`: string
+- `status`: `queued|running|succeeded|failed|expired`
+- `estimateSeconds`: int
+- `retryAfter`: int seconds (for polling)
+- `links`: `{ status: url, events?: url }`

-Proposed limits
+### Response: GET status
+```jsonc
+{
+  "exportId": "console-export::tenant-default::2025-12-06::0007",
+  "status": "running",
+  "estimateSeconds": 420,
+  "outputs": [
+    { "type": "manifest", "format": "json", "url": "https://.../manifest.json?sig=...", "sha256": "...", "expiresAt": "2025-12-06T13:10:00Z" }
+  ],
+  "progress": { "percent": 42, "itemsCompleted": 210, "itemsTotal": 500, "assetsReady": 12 },
+  "errors": []
+}
+```
+
+### Response: SSE events
+- `started`: `{ exportId, status }`
+- `progress`: `{ exportId, percent, itemsCompleted, itemsTotal }`
+- `asset_ready`: `{ exportId, type, id, url, sha256 }`
+- `completed`: `{ exportId, status: "succeeded", manifestUrl }`
+- `failed`: `{ exportId, status: "failed", code, message }`
+
+### Manifest shape (downloaded via outputs)
+- `version`: string (date)
+- `exportId`, `tenantId`, `generatedAt`
+- `items[]`: `{ type: advisory|vex|policy|scan, id, url, sha256 }`
+- `checksums`: `{ manifest, bundle }`
+
+### Limits (proposed)
 - Max request body 256 KiB; max sources 50; max outputs 1000 assets/export.
- Default job timeout 30 minutes; idle SSE timeout 60s; backoff header `Retry-After`.
+- Default job timeout 30 minutes; idle SSE timeout 60s; backoff via `Retry-After`.

-Samples (draft)
+### Error codes (proposal)
+- `ERR_CONSOLE_EXPORT_INVALID_SOURCE`
+- `ERR_CONSOLE_EXPORT_TOO_LARGE`
+- `ERR_CONSOLE_EXPORT_RATE_LIMIT`
+- `ERR_CONSOLE_EXPORT_UNAVAILABLE`
+
+### Samples
 - Request: `docs/api/console/samples/console-export-request.json`
 - Status: `docs/api/console/samples/console-export-status.json`
 - Manifest: `docs/api/console/samples/console-export-manifest.json`
 - Events: `docs/api/console/samples/console-export-events.ndjson`

-Open items (needs owner sign-off)
- Final schema (fields, limits, error codes), checksum manifest format, attestation options.
+### Open items (needs guild sign-off)
+- Final scopes list (`console:export` vs broader `console:*`).
+- Final limits and error codes; checksum manifest format; attestation options.
 - Caching/tie-break rules for downstream `/console/search` and `/console/downloads`.
--- a/docs/api/gateway/export-center.md
+++ b/docs/api/gateway/export-center.md
@@ -1,17 +1,79 @@
 # Export Center Gateway Contract (draft placeholder)

-**Status:** TODO · awaiting Export Center Guild inputs
+**Status:** Draft v0.2 · owner-proposed

 ## Scope
 - Profile, run, download, and distribution routes proxied via Web gateway.
 - Tenant scoping, RBAC/ABAC, streaming limits, retention/encryption parameters, signed URL policy.

-## Needed from owners
- OpenAPI/JSON schema for: profiles, runs, downloads, distributions (OCI/object storage).
- Range/streaming limits; retry/backoff guidance; checksum/manifest format.
- Required headers (tenant/project, idempotency, auth) and rate limits.
- Example payloads/NDJSON streams for happy-path and error cases.
+## Endpoints
+- `GET /export-center/profiles` — list export profiles (tenant-scoped).
+- `POST /export-center/runs` — start an export run.
+- `GET /export-center/runs/{runId}` — run status and artifacts.
+- `GET /export-center/runs/{runId}/events` — SSE for run progress.
+- `GET /export-center/distributions/{id}` — fetch signed URLs for OCI/object storage distribution.

-## TODO
- Replace this file with the ratified contract and sample payloads.
- Record schema hash and date when published; link from Web II sprint Execution Log.
+## Security / headers
+- `Authorization: DPoP <token>`; `DPoP: <proof>`
+- `X-StellaOps-Tenant: <tenantId>` (required)
+- `X-StellaOps-Project: <projectId>` (optional)
+- `Idempotency-Key` (recommended for POST)
+- Required scopes (proposal): `export:read`, `export:write`.
+
+## Request: POST /export-center/runs
+```jsonc
+{
+  "profileId": "export-profile::tenant-default::daily-vex",
+  "targets": ["vex", "advisory", "policy"],
+  "formats": ["json", "ndjson"],
+  "distribution": {
+    "type": "oci",
+    "ref": "registry.local/exports/daily",
+    "signing": { "enabled": true, "keyRef": "k8s://secrets/eks/oci-signer" }
+  },
+  "retentionDays": 30,
+  "encryption": { "enabled": true, "kmsKey": "kms://tenant-default/key1" },
+  "priority": "normal"
+}
+```
+
+## Response: 202 Accepted
+- `runId`, `status: queued|running|succeeded|failed|expired`, `estimateSeconds`, `retryAfter`.
+
+## Response: GET run
+```jsonc
+{
+  "runId": "export-run::tenant-default::2025-12-06::0003",
+  "status": "running",
+  "profileId": "export-profile::tenant-default::daily-vex",
+  "startedAt": "2025-12-06T10:00:00Z",
+  "outputs": [
+    { "type": "manifest", "format": "json", "url": "https://exports.local/.../manifest.json?sig=...", "sha256": "...", "expiresAt": "2025-12-06T16:00:00Z" }
+  ],
+  "progress": { "percent": 35, "itemsCompleted": 70, "itemsTotal": 200 },
+  "errors": []
+}
+```
+
+## SSE events
+- `started`, `progress`, `artifact_ready` (url, sha256, type), `completed`, `failed` (code, message).
+
+## Limits (proposal)
+- Max request body 256 KiB; max targets 50; default timeout 60 minutes.
+- Idle SSE timeout 60s; backoff with `Retry-After`.
+
+## Error codes (proposal)
+- `ERR_EXPORT_PROFILE_NOT_FOUND`
+- `ERR_EXPORT_REQUEST_INVALID`
+- `ERR_EXPORT_TOO_LARGE`
+- `ERR_EXPORT_RATE_LIMIT`
+- `ERR_EXPORT_DISTRIBUTION_FAILED`
+
+## Samples
+- Profile list sample: _todo_
+- Run request/response: see above snippets.
+- Events NDJSON: _todo_
+
+## Outstanding (for finalization)
+- Confirm scopes, limits, distribution signing rules, and manifest checksum requirements.
+- Provide full OpenAPI/JSON schema and sample artifacts for OCI/object storage distributions.
--- a/docs/api/graph/overlay-schema.md
+++ b/docs/api/graph/overlay-schema.md
@@ -1,16 +1,42 @@
 # Graph Overlay & Cache Schema (draft placeholder)

-**Status:** TODO · awaiting Graph Platform Guild ratification
+**Status:** Draft v0.2 · owner-proposed

 ## Scope
 - Overlay/cache schema for graph tiles used by Web gateway and UI overlays.
 - Validation rules for bbox/zoom/path; pagination tokens; deterministic ordering.
 - Error codes and sampling/telemetry fields.

-## Needed from owners
- JSON schema (or OpenAPI fragment) for overlay response and cache metadata.
- Allowed zoom levels/coordinate system; max nodes/edges per tile; hashing/etag rules.
- Sample overlay bundle (happy path + rate-limit + validation error).
+## Schema (draft)
+```jsonc
+{
+  "version": "2025-12-06",
+  "tenantId": "tenant-default",
+  "tile": {
+    "id": "graph-tile::asset::<hash>::z8/x12/y5",
+    "bbox": { "minX": -122.41, "minY": 37.77, "maxX": -122.38, "maxY": 37.79 },
+    "zoom": 8,
+    "etag": "c0ffee-etag"
+  },
+  "nodes": [ { "id": "asset:...", "kind": "asset|component|vuln", "label": "", "severity": "high|medium|low|info", "reachability": "reachable|unreachable|unknown", "attributes": {} } ],
+  "edges": [ { "id": "edge-1", "source": "nodeId", "target": "nodeId", "type": "depends_on|contains|evidence", "weight": 0.0 } ],
+  "overlays": {
+    "policy": [ { "nodeId": "nodeId", "badge": "pass|warn|fail|waived", "policyId": "", "verdictAt": "2025-12-05T09:00:00Z" } ],
+    "vex": [ { "nodeId": "nodeId", "state": "not_affected|fixed|under_investigation|affected", "statementId": "", "lastUpdated": "2025-12-05T09:10:00Z" } ],
+    "aoc": [ { "nodeId": "nodeId", "status": "pass|fail|warn", "lastVerified": "2025-12-05T10:11:12Z" } ]
+  },
+  "telemetry": { "generationMs": 0, "cache": "hit|miss", "samples": 0 }
+}
+```

-## TODO
- Insert ratified schema + samples; note schema hash/date; link from Web II sprint log.
+## Constraints (proposal)
+- Max nodes per tile: 2,000; max edges: 4,000.
+- Zoom range: 0–12; tiles must include bbox and etag.
+- Arrays must be pre-sorted: nodes by `id`, edges by `id`, overlays by `nodeId` then `policyId|statementId`.
+
+## Samples
+- `docs/api/graph/samples/overlay-sample.json`
+
+## Outstanding
+- Confirm max sizes, allowed edge types, and etag hashing rule.
+- Provide validation error example and rate-limit headers for gateway responses.
--- a/docs/api/graph/samples/overlay-sample.json
+++ b/docs/api/graph/samples/overlay-sample.json
@@ -0,0 +1,75 @@
+{
+  "version": "2025-12-06",
+  "tenantId": "tenant-default",
+  "tile": {
+    "id": "graph-tile::asset::sha256:abc123::z8/x12/y5",
+    "bbox": {
+      "minX": -122.41,
+      "minY": 37.77,
+      "maxX": -122.38,
+      "maxY": 37.79
+    },
+    "zoom": 8,
+    "etag": "c0ffee-overlay-etag"
+  },
+  "nodes": [
+    {
+      "id": "asset:registry.local/library/app@sha256:abc123",
+      "kind": "asset",
+      "label": "app:1.2.3",
+      "severity": "high",
+      "reachability": "reachable",
+      "aoc": { "summary": "pass", "lastVerified": "2025-12-05T10:11:12Z" },
+      "attributes": {
+        "purl": "pkg:docker/app@sha256:abc123",
+        "componentCount": 42
+      }
+    },
+    {
+      "id": "component:pkg:npm/jsonwebtoken@9.0.2",
+      "kind": "component",
+      "label": "jsonwebtoken@9.0.2",
+      "severity": "high",
+      "reachability": "reachable"
+    }
+  ],
+  "edges": [
+    {
+      "id": "edge-1",
+      "source": "asset:registry.local/library/app@sha256:abc123",
+      "target": "component:pkg:npm/jsonwebtoken@9.0.2",
+      "type": "depends_on",
+      "weight": 0.87
+    }
+  ],
+  "overlays": {
+    "policy": [
+      {
+        "nodeId": "component:pkg:npm/jsonwebtoken@9.0.2",
+        "badge": "fail",
+        "policyId": "policy://tenant-default/runtime-hardening",
+        "verdictAt": "2025-12-05T09:00:00Z"
+      }
+    ],
+    "vex": [
+      {
+        "nodeId": "component:pkg:npm/jsonwebtoken@9.0.2",
+        "state": "under_investigation",
+        "statementId": "vex:tenant-default:jwt:2025-12-05",
+        "lastUpdated": "2025-12-05T09:10:00Z"
+      }
+    ],
+    "aoc": [
+      {
+        "nodeId": "asset:registry.local/library/app@sha256:abc123",
+        "status": "pass",
+        "lastVerified": "2025-12-05T10:11:12Z"
+      }
+    ]
+  },
+  "telemetry": {
+    "generationMs": 120,
+    "cache": "hit",
+    "samples": 3
+  }
+}
--- a/docs/api/signals/reachability-contract.md
+++ b/docs/api/signals/reachability-contract.md
@@ -1,15 +1,66 @@
 # Signals Reachability API Contract (draft placeholder)

-**Status:** TODO · awaiting Signals Guild
+**Status:** Draft v0.2 · owner-proposed

 ## Scope
 - `/signals/callgraphs`, `/signals/facts`, reachability scoring overlays feeding UI/Web.
 - Deterministic fixtures for SIG-26 chain (columns/badges, call paths, timelines, overlays, coverage).

-## Needed from owners
- OpenAPI/JSON schema for callgraphs and facts (request/response, pagination, ETags).
- Reachability score model, states, and filtering parameters.
- Fixture bundle (JSON/NDJSON) with checksums and performance budgets (target FPS/node caps).
+## Endpoints
+- `GET /signals/callgraphs` — returns call paths contributing to reachability.
+- `GET /signals/facts` — returns reachability/coverage facts.

-## TODO
- Replace with ratified contract and fixtures; record schema hash/date; link from Web V and UI III logs.
+Common headers: `Authorization: DPoP <token>`, `DPoP: <proof>`, `X-StellaOps-Tenant`, optional `If-None-Match`.
+Pagination: cursor via `pageToken`; default 50, max 200.
+ETag: required on responses; clients must send `If-None-Match` for cache validation.
+
+### Callgraphs response (draft)
+```jsonc
+{
+  "tenantId": "tenant-default",
+  "assetId": "registry.local/library/app@sha256:abc123",
+  "paths": [
+    {
+      "id": "path-1",
+      "source": "api-gateway",
+      "target": "jwt-auth-service",
+      "hops": [
+        { "service": "api-gateway", "endpoint": "/login", "timestamp": "2025-12-05T10:00:00Z" },
+        { "service": "jwt-auth-service", "endpoint": "/verify", "timestamp": "2025-12-05T10:00:01Z" }
+      ],
+      "evidence": { "traceId": "trace-abc", "spanCount": 2, "score": 0.92 }
+    }
+  ],
+  "pagination": { "nextPageToken": null },
+  "etag": "sig-callgraphs-etag"
+}
+```
+
+### Facts response (draft)
+```jsonc
+{
+  "tenantId": "tenant-default",
+  "facts": [
+    {
+      "id": "fact-1",
+      "type": "reachability",
+      "assetId": "registry.local/library/app@sha256:abc123",
+      "component": "pkg:npm/jsonwebtoken@9.0.2",
+      "status": "reachable",
+      "confidence": 0.88,
+      "observedAt": "2025-12-05T10:10:00Z",
+      "signalsVersion": "signals-2025.310.1"
+    }
+  ],
+  "pagination": { "nextPageToken": "..." },
+  "etag": "sig-facts-etag"
+}
+```
+
+### Samples
+- Callgraphs: `docs/api/signals/samples/callgraph-sample.json`
+- Facts: `docs/api/signals/samples/facts-sample.json`
+
+### Outstanding
+- Finalize score model, accepted `type` values, and max page size.
+- Provide OpenAPI/JSON schema and error codes.
--- a/docs/api/signals/samples/callgraph-sample.json
+++ b/docs/api/signals/samples/callgraph-sample.json
@@ -0,0 +1,23 @@
+{
+  "tenantId": "tenant-default",
+  "assetId": "registry.local/library/app@sha256:abc123",
+  "paths": [
+    {
+      "id": "path-1",
+      "source": "api-gateway",
+      "target": "jwt-auth-service",
+      "hops": [
+        { "service": "api-gateway", "endpoint": "/login", "timestamp": "2025-12-05T10:00:00Z" },
+        { "service": "jwt-auth-service", "endpoint": "/verify", "timestamp": "2025-12-05T10:00:01Z" }
+      ],
+      "evidence": {
+        "traceId": "trace-abc",
+        "spanCount": 2,
+        "score": 0.92
+      }
+    }
+  ],
+  "pagination": {
+    "nextPageToken": null
+  }
+}
--- a/docs/api/signals/samples/facts-sample.json
+++ b/docs/api/signals/samples/facts-sample.json
@@ -0,0 +1,26 @@
+{
+  "tenantId": "tenant-default",
+  "facts": [
+    {
+      "id": "fact-1",
+      "type": "reachability",
+      "assetId": "registry.local/library/app@sha256:abc123",
+      "component": "pkg:npm/jsonwebtoken@9.0.2",
+      "status": "reachable",
+      "confidence": 0.88,
+      "observedAt": "2025-12-05T10:10:00Z",
+      "signalsVersion": "signals-2025.310.1"
+    },
+    {
+      "id": "fact-2",
+      "type": "coverage",
+      "assetId": "registry.local/library/app@sha256:abc123",
+      "metric": "sensors_present",
+      "value": 0.94,
+      "observedAt": "2025-12-05T10:11:00Z"
+    }
+  ],
+  "pagination": {
+    "nextPageToken": "eyJmYWN0SWQiOiJmYWN0LTIifQ"
+  }
+}
--- a/docs/api/vex-consensus-sample.ndjson
+++ b/docs/api/vex-consensus-sample.ndjson
@@ -0,0 +1,11 @@
+event: started
+data: {"tenantId":"tenant-default","streamId":"vex-consensus::2025-12-06","status":"running"}
+
+event: consensus_update
+data: {"statementId":"vex:tenant-default:jwt-auth:5d1a","state":"under_investigation","justification":"reachable path confirmed","validFrom":"2025-12-06T10:00:00Z","validUntil":"2025-12-20T00:00:00Z","sources":["signals","policy"],"etag":"vex-etag-123"}
+
+event: consensus_update
+data: {"statementId":"vex:tenant-default:openssl:7b2c","state":"not_affected","justification":"no call-path and patched","validFrom":"2025-12-05T00:00:00Z","validUntil":"2026-01-01T00:00:00Z","sources":["sbom","scanner"],"etag":"vex-etag-456"}
+
+event: completed
+data: {"streamId":"vex-consensus::2025-12-06","status":"succeeded"}
--- a/docs/api/vex-consensus.md
+++ b/docs/api/vex-consensus.md
@@ -1,14 +1,25 @@
 # VEX Consensus Stream Contract (draft placeholder)

-**Status:** TODO · awaiting VEX Lens Guild
+**Status:** Draft v0.2 · owner-proposed

 ## Scope
 - `/vex/consensus` streaming APIs via Web gateway with tenant RBAC/ABAC, caching, and telemetry.

-## Needed from owners
- SSE/stream envelope (fields, heartbeats, retry/backoff headers), sample NDJSON stream.
- RBAC/ABAC requirements and caching rules; idempotency/correlation headers.
- Error codes and rate limits.
+## Endpoint
+- `GET /vex/consensus/stream` — SSE stream of consensus VEX statements per tenant.

-## TODO
- Insert finalized contract + samples; note schema hash/date; reference in Web V sprint log.
+Headers: `Authorization: DPoP <token>`, `DPoP: <proof>`, `X-StellaOps-Tenant`, optional `If-None-Match`.
+Scopes (proposal): `vex:read` and `vex:consensus`.
+
+Events (draft)
+- `started`: `{ tenantId, streamId, status }`
+- `consensus_update`: `{ statementId, state, justification, validFrom, validUntil, sources[], etag }`
+- `heartbeat`: `{ streamId, ts }`
+- `completed`: `{ streamId, status }`
+- `failed`: `{ streamId, code, message }`
+
+Rate limits: heartbeats every 30s; idle timeout 90s; backoff via `Retry-After` header on reconnect.
+
+Samples: `docs/api/vex-consensus-sample.ndjson`
+
+Outstanding: finalize scopes, error codes, cache/etag semantics, and add pagination/replay guidance.