wip: doctor/cli/docs/api to vector db consolidation; api hardening for descriptions, tenant, and scopes; migrations and conversions of all DALs to EF v10
This commit is contained in:
@@ -2,7 +2,10 @@
|
||||
|
||||
Scanner analyses container images layer-by-layer, producing deterministic SBOM fragments, diffs, and signed reports.
|
||||
|
||||
## Latest updates (2025-12-12)
|
||||
## Latest updates (2026-02-22)
|
||||
- Unknowns API surface is now registered in Scanner (`/api/v1/unknowns`) with tenant-scoped query predicates and tenant conflict handling via shared request-context resolution.
|
||||
- Tenant isolation hardening for triage/finding evidence APIs (Sprint `20260222.057`): triage query/status/rationale/replay services now require explicit tenant context, triage persistence includes `tenant_id`, and cross-tenant finding lookups resolve as deterministic misses. See `./endpoint-registration-matrix.md`.
|
||||
- Tenant-argument parity hardening for API-backed tenant tables (Sprint `20260222.057`, `SCAN-TEN-13`): source-run and secret-exception APIs now enforce tenant-scoped repository lookups for `scanner.sbom_source_runs` and `secret_exception_pattern`.
|
||||
- Deterministic SBOM composition fixture published at `docs/modules/scanner/fixtures/deterministic-compose/` with DSSE, `_composition.json`, BOM, and hashes; doc `deterministic-sbom-compose.md` promoted to Ready v1.0 with offline verification steps.
|
||||
- Node analyzer now ingests npm/yarn/pnpm lockfiles, emitting `DeclaredOnly` components with lock provenance. The CLI companion command `stella node lock-validate` runs the collector offline, surfaces declared-only or missing-lock packages, and emits telemetry via `stellaops.cli.node.lock_validate.count`. See `docs/modules/scanner/analyzers-node.md` and bench scenario `node_detection_gaps_fixture`.
|
||||
- Python analyzer picks up `requirements*.txt`, `Pipfile.lock`, and `poetry.lock`, tagging installed distributions with lock provenance and generating declared-only components for policy. Use `stella python lock-validate` to run the same checks locally before images are built.
|
||||
@@ -37,6 +40,7 @@ Scanner analyses container images layer-by-layer, producing deterministic SBOM f
|
||||
- ./operations/analyzers-grafana-dashboard.json
|
||||
- ./operations/rustfs-migration.md
|
||||
- ./operations/entrypoint.md
|
||||
- ./endpoint-registration-matrix.md
|
||||
- ./analyzers-node.md
|
||||
- ./analyzers-go.md
|
||||
- ./operations/secret-leak-detection.md
|
||||
|
||||
@@ -416,17 +416,42 @@ Scanner now exposes a deterministic VEX+reachability matrix filter for triage pr
|
||||
- API surface: `POST /api/v1/scans/vex-reachability/filter` accepts finding batches and returns annotated decisions plus action summary counts.
|
||||
- Determinism: batch order is preserved, rule IDs are explicit, and no network lookups are required for matrix evaluation.
|
||||
|
||||
### 5.5.7 Vulnerability-first triage clustering APIs (Sprint 20260208_063)
|
||||
### 5.5.7 Vulnerability-first triage clustering APIs (Sprint 20260208_063)
|
||||
|
||||
Scanner triage now includes deterministic exploit-path clustering primitives for vulnerability-first triage workflows:
|
||||
|
||||
- Core clustering service: `StellaOps.Scanner.Triage/Services/ExploitPathGroupingService` groups findings using common call-chain prefix similarity with configurable thresholds.
|
||||
- Inbox enhancements: `GET /api/v1/triage/inbox` supports `similarityThreshold`, `sortBy`, and `descending` for deterministic cluster filtering/sorting.
|
||||
- Cluster statistics: `GET /api/v1/triage/inbox/clusters/stats` returns per-cluster severity counts, reachability distribution, and priority scores.
|
||||
- Batch triage actions: `POST /api/v1/triage/inbox/clusters/{pathId}/actions` applies one action to all findings in the cluster and emits deterministic action records.
|
||||
- Offline/determinism posture: no network calls, stable ordering by IDs/path IDs, deterministic path-ID hashing, and replayable batch payload digests.
|
||||
|
||||
### 5.6 DSSE attestation (via Signer/Attestor)
|
||||
- Batch triage actions: `POST /api/v1/triage/inbox/clusters/{pathId}/actions` applies one action to all findings in the cluster and emits deterministic action records.
|
||||
- Offline/determinism posture: no network calls, stable ordering by IDs/path IDs, deterministic path-ID hashing, and replayable batch payload digests.
|
||||
|
||||
### 5.5.8 Triage tenant isolation contract (Sprint 20260222_057)
|
||||
|
||||
Scanner triage and finding evidence APIs enforce tenant-aware access at endpoint, service, and persistence layers:
|
||||
|
||||
- Tenant context is resolved by `ScannerRequestContextResolver` (canonical claim `tenant`, compatibility claim aliases, compatibility header aliases, and conflict detection).
|
||||
- Triage/finding service contracts require explicit `tenantId` and all retrieval/update paths filter by tenant before resolving finding/scan identity.
|
||||
- Triage schema includes tenant discriminators (`triage_scan.tenant_id`, `triage_finding.tenant_id`), and active-case uniqueness includes `tenant_id` to prevent cross-tenant collisions.
|
||||
- Cross-tenant finding lookups resolve as deterministic not-found responses rather than revealing record existence.
|
||||
|
||||
### 5.5.9 Unknowns API tenant activation (Sprint 20260222_057 follow-up)
|
||||
|
||||
Scanner now registers the `/api/v1/unknowns` endpoint group in `Program.cs` with explicit `scanner.scans.read` authorization and tenant-aware query semantics:
|
||||
|
||||
- Request tenant resolution uses `ScannerRequestContextResolver` with canonical/compatibility claim-header handling and deterministic conflict failures (`tenant_conflict`).
|
||||
- Unknown list/detail/evidence/history/stats/bands handlers call a tenant-scoped query service that filters by `tenant_id`.
|
||||
- Cross-tenant detail lookups resolve as deterministic not-found responses (`404`).
|
||||
|
||||
### 5.5.10 API-backed tenant table parity (Sprint 20260222_057 SCAN-TEN-13)
|
||||
|
||||
Scanner API flows that operate on tenant-partitioned tables now require tenant arguments at repository boundaries:
|
||||
|
||||
- Source run APIs (`/api/v1/sources/{sourceId}/runs`, `/api/v1/sources/{sourceId}/runs/{runId}`) pass tenant into `ISbomSourceRunRepository` for `GetByIdAsync`, `ListForSourceAsync`, and `GetStatsAsync`; SQL predicates include `tenant_id = @tenantId`.
|
||||
- Secret exception APIs (`/api/v1/secrets/config/exceptions/{tenantId}/{exceptionId}`) use tenant-scoped repository methods for get/update/delete on `secret_exception_pattern`, removing ID-only tenant-agnostic operations.
|
||||
- Generic webhook ingress by `sourceId` remains compatibility-tolerant when tenant context is absent, but enforces tenant ownership when context is present.
|
||||
|
||||
### 5.6 DSSE attestation (via Signer/Attestor)
|
||||
|
||||
* WebService constructs **predicate** with `image_digest`, `stellaops_version`, `license_id`, `policy_digest?` (when emitting **final reports**), timestamps.
|
||||
* Calls **Signer** (requires **OpTok + PoE**); Signer verifies **entitlement + scanner image integrity** and returns **DSSE bundle**.
|
||||
|
||||
88
docs/modules/scanner/endpoint-registration-matrix.md
Normal file
88
docs/modules/scanner/endpoint-registration-matrix.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# Scanner Endpoint Registration Matrix
|
||||
|
||||
Last updated: 2026-02-23 (Sprint `20260222.057`, tasks `SCAN-TEN-08`, `SCAN-TEN-11`, `SCAN-TEN-12`, and `SCAN-TEN-13`).
|
||||
|
||||
This file is the Scanner WebService source-of-truth for endpoint registration intent and authorization posture.
|
||||
|
||||
## Active endpoint maps (registered in `Program.cs`)
|
||||
|
||||
| Map method | Base path | Authorization posture | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `MapHealthEndpoints` | `/healthz`, `/readyz` | `scanner.scans.read` | Health and readiness are authenticated. |
|
||||
| `MapObservabilityEndpoints` | `/metrics` | `AllowAnonymous` | Prometheus scrape path is explicitly anonymous. |
|
||||
| `MapOfflineKitEndpoints` | `/offline-kit` | Mixed explicit policies | Offline import/status/manifest/validate policies are explicit per route. |
|
||||
| `MapScanEndpoints` | `/api/v1/scans` | Explicit policies per route | Includes nested scan maps (`callgraphs`, `sbom`, `reachability`, `export`, `evidence`, `approval`, `manifest`, `github-code-scanning`). |
|
||||
| `MapSourcesEndpoints` | `/api/v1/sources` | `scanner.sources.read/write/admin` | Tenant-aware source CRUD and source-run operations (`sbom_source_runs` reads/writes are tenant-filtered). |
|
||||
| `MapWebhookEndpoints` | `/api/v1/webhooks` | `AllowAnonymous` | External provider ingress; tenant-scoped source resolution for name routes. |
|
||||
| `MapSbomUploadEndpoints` | `/api/v1/sbom` | Explicit policies per route | Includes SBOM upload and hot lookup maps. |
|
||||
| `MapReachabilityDriftRootEndpoints` | `/api/v1/reachability` | `scanner.scans.read/write` | Root drift query/compute routes. |
|
||||
| `MapDeltaCompareEndpoints` | `/api/v1/delta` | Explicit policies per route | Delta compare operations. |
|
||||
| `MapSmartDiffEndpoints` | `/api/v1/smart-diff` | Explicit policies per route | Smart diff + VEX candidate workflows. |
|
||||
| `MapBaselineEndpoints` | `/api/v1/baselines` | Explicit policies per route | Baseline operations. |
|
||||
| `MapActionablesEndpoints` | `/api/v1/actionables` | Explicit policies per route | Actionable findings operations. |
|
||||
| `MapCounterfactualEndpoints` | `/api/v1/counterfactuals` | Explicit policies per route | Counterfactual analysis operations. |
|
||||
| `MapProofSpineEndpoints` | `/api/v1/spines` | Explicit policies per route | Proof spine APIs. |
|
||||
| `MapReplayEndpoints` | `/api/v1/replay` | Explicit policies per route | Replay command and verification APIs. |
|
||||
| `MapScoreReplayEndpoints` (feature-gated) | `/api/v1/replay/score` | Explicit policies per route | Enabled only when `scanner:scoreReplay:enabled=true`. |
|
||||
| `MapWitnessEndpoints` | `/api/v1/witnesses` | Explicit policies per route | Witness management APIs. |
|
||||
| `MapEpssEndpoints` | `/api/v1/epss` | Explicit policies per route | EPSS ingestion/query APIs. |
|
||||
| `MapTriageStatusEndpoints` | `/api/v1/triage` | `scanner.triage.read/write` | Triage status workflows. |
|
||||
| `MapTriageInboxEndpoints` | `/api/v1/triage` | `scanner.triage.read/write` | Triage inbox workflows. |
|
||||
| `MapBatchTriageEndpoints` | `/api/v1/triage` | `scanner.triage.read/write` | Bulk triage operations. |
|
||||
| `MapProofBundleEndpoints` | `/api/v1/triage` | `scanner.triage.read` | Triage proof bundle retrieval. |
|
||||
| `MapUnknownsEndpoints` | `/api/v1/unknowns` | `scanner.scans.read` | Tenant-scoped unknown listing/detail/evidence/history/stats/bands routes. |
|
||||
| `MapSecretDetectionSettingsEndpoints` | `/api/v1/secrets/config` | `scanner.secret-settings.*` / `scanner.secret-exceptions.*` | Secret detection config APIs with tenant-scoped exception get/update/delete lookups. |
|
||||
| `MapSecurityAdapterEndpoints` | `/api/v1/security` | Explicit policies per route | Security adapter read-model routes. |
|
||||
| `MapPolicyEndpoints` (feature-gated) | `/api/v1/policy` | Explicit policies per route | Enabled only when policy preview feature flag is true. |
|
||||
| `MapReportEndpoints` | `/api/v1/reports` | `scanner.reports` | Report generation and retrieval routes. |
|
||||
| `MapRuntimeEndpoints` | `/api/v1/runtime` | `scanner.runtime.ingest` and scan policies | Runtime ingest/reconcile APIs. |
|
||||
| `MapSliceEndpoints` | `/api/slices` | Explicit policies per route | Reachability slice query/replay APIs. |
|
||||
|
||||
## Webhook tenant contract (name routes)
|
||||
|
||||
- Name-based webhook routes are:
|
||||
- `POST /api/v1/webhooks/docker/{sourceName}`
|
||||
- `POST /api/v1/webhooks/github/{sourceName}`
|
||||
- `POST /api/v1/webhooks/gitlab/{sourceName}`
|
||||
- `POST /api/v1/webhooks/harbor/{sourceName}`
|
||||
- Tenant context is required for name-based routes (claim or `X-StellaOps-Tenant`/compatibility alias). Missing tenant returns `400` with `tenant_missing`; claim/header mismatch returns `400` with `tenant_conflict`.
|
||||
- Source lookup is tenant-scoped via `GetByNameAsync(tenantId, sourceName)`, so same-name sources in different tenants cannot cross-dispatch.
|
||||
- `POST /api/v1/webhooks/{sourceId}` accepts optional tenant context; if present and it does not match source ownership, the endpoint returns `404`.
|
||||
|
||||
## Triage tenant contract
|
||||
|
||||
- Triage and finding-evidence routes resolve tenant context via `ScannerRequestContextResolver`.
|
||||
- Resolution precedence: claim (`tenant`, compatibility aliases `tid`/`tenant_id`), then headers (`X-StellaOps-Tenant`, `X-Stella-Tenant`, `X-Tenant-Id`), then fallback tenant `default` for triage flows configured with `allowDefaultTenant`.
|
||||
- If multiple claim/header values conflict, Scanner returns `400` with `tenant_conflict`.
|
||||
- Triage-facing contracts now require tenant input (`ITriageStatusService`, `ITriageQueryService`, `IFindingQueryService`, `IGatingReasonService`, `IUnifiedEvidenceService`, `IReplayCommandService`, `IFindingRationaleService`) and enforce tenant-filtered data access.
|
||||
- Triage persistence is tenant partitioned (`triage_scan.tenant_id`, `triage_finding.tenant_id`), and the active-case uniqueness key includes `tenant_id` to avoid cross-tenant collisions.
|
||||
- Cross-tenant finding lookups are treated as deterministic not-found responses (`404`) on triage/finding endpoints.
|
||||
|
||||
## Unknowns tenant contract
|
||||
|
||||
- Unknowns routes resolve tenant context via `ScannerRequestContextResolver` using canonical and compatibility claim/header paths with conflict detection.
|
||||
- Unknown queries are tenant-scoped by data access predicates (`tenant_id::text = @tenantId`) in the Scanner WebService query service.
|
||||
- Missing records across tenant boundaries resolve as deterministic `404` for detail/evidence/history endpoints.
|
||||
|
||||
## Reachability + SmartDiff tenant contract (SCAN-TEN-11)
|
||||
|
||||
- Reachability drift routes (`/api/v1/scans/{scanId}/drift`, `/api/v1/drift/{driftId}/sinks`) now resolve tenant context via `ScannerRequestContextResolver` before repository access.
|
||||
- Drift persistence/retrieval paths are tenant-parameterized end-to-end (`call_graph_snapshots`, `code_changes`, `reachability_drift_results`, `drifted_sinks`) instead of using a fixed tenant scope.
|
||||
- SmartDiff routes (`/api/v1/smart-diff/**`) now resolve tenant context and pass it explicitly to material-change and VEX-candidate stores.
|
||||
- SmartDiff persistence/retrieval paths are tenant-parameterized (`material_risk_changes`, `vex_candidates`), so candidate ID collisions across tenants resolve as deterministic not-found.
|
||||
- Remaining reachability/SmartDiff storage adapters used by these flows are tenant-parameterized as well (`risk_state_snapshots`, `reachability_results`), removing fixed-tenant constants from Postgres adapter implementations.
|
||||
|
||||
## Source-runs + secret-exception tenant contract (SCAN-TEN-13)
|
||||
|
||||
- Source run routes (`/api/v1/sources/{sourceId}/runs`, `/api/v1/sources/{sourceId}/runs/{runId}`) now pass resolved tenant context into `ISbomSourceRunRepository` and SQL predicates (`tenant_id = @tenantId`) for `scanner.sbom_source_runs`.
|
||||
- Secret exception routes (`/api/v1/secrets/config/exceptions/{tenantId}/{exceptionId}`) now use tenant-scoped repository methods for get/update/delete against `secret_exception_pattern` rather than ID-only lookups.
|
||||
- The generic webhook route (`POST /api/v1/webhooks/{sourceId}`) remains compatibility-tolerant for missing tenant context, but enforces ownership match when tenant context is present.
|
||||
|
||||
## Deferred endpoint maps (not registered by design)
|
||||
|
||||
| Map method | Current state | Reason for deferral | Activation prerequisite |
|
||||
| --- | --- | --- | --- |
|
||||
| `MapValidationEndpoints` | Deferred | Validation endpoint stack is not wired in `Program.cs` (validator service graph and option bindings are not part of default webservice startup contract). | Add explicit validation DI and contract docs, then register in `Program.cs`. |
|
||||
| `MapReachabilityEvidenceEndpoints` | Deferred | Legacy standalone `/api/reachability` surface is not part of current `/api/v1/scans`/triage scanner API contract. | Contract decision to promote or remove this surface. |
|
||||
| `MapReachabilityStackEndpoints` | Deferred | Stack endpoints depend on optional repository path and are not part of the current public API registration set. | Contract decision and stable backing repository implementation. |
|
||||
| `MapFidelityEndpoints` | Deferred | Fidelity analyzer endpoints are experimental and require explicit analyzer service contract decisions before exposure. | Product sign-off and DI contract hardening. |
|
||||
Reference in New Issue
Block a user