feat(scanner): Implement Deno analyzer and associated tests
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Added Deno analyzer with comprehensive metadata and evidence structure.
- Created a detailed implementation plan for Sprint 130 focusing on Deno analyzer.
- Introduced AdvisoryAiGuardrailOptions for managing guardrail configurations.
- Developed GuardrailPhraseLoader for loading blocked phrases from JSON files.
- Implemented tests for AdvisoryGuardrailOptions binding and phrase loading.
- Enhanced telemetry for Advisory AI with metrics tracking.
- Added VexObservationProjectionService for querying VEX observations.
- Created extensive tests for VexObservationProjectionService functionality.
- Introduced Ruby language analyzer with tests for simple and complex workspaces.
- Added Ruby application fixtures for testing purposes.
This commit is contained in:
master
2025-11-12 10:01:54 +02:00
parent 0e8655cbb1
commit babb81af52
75 changed files with 3346 additions and 187 deletions

View File

@@ -15,12 +15,13 @@ The service operates strictly downstream of the **Aggregation-Only Contract (AOC
## 1·Responsibilities & Constraints
- Compile and evaluate `stella-dsl@1` policy packs into deterministic verdicts.
- Join SBOM inventory, Concelier advisories, and Excititor VEX evidence via canonical linksets and equivalence tables.
- Materialise effective findings (`effective_finding_{policyId}`) with append-only history and produce explain traces.
- Operate incrementally: react to change streams (advisory/vex/SBOM deltas) with ≤5min SLA.
- Provide simulations with diff summaries for UI/CLI workflows without modifying state.
- Enforce strict determinism guard (no wall-clock, RNG, network beyond allow-listed services) and RBAC + tenancy via Authority scopes.
- Compile and evaluate `stella-dsl@1` policy packs into deterministic verdicts.
- Join SBOM inventory, Concelier advisories, and Excititor VEX evidence via canonical linksets and equivalence tables.
- Materialise effective findings (`effective_finding_{policyId}`) with append-only history and produce explain traces.
- Emit per-finding OpenVEX decisions anchored to reachability evidence, forward them to Signer/Attestor for DSSE/Rekor, and publish the resulting artifacts for bench/verification consumers.
- Operate incrementally: react to change streams (advisory/vex/SBOM deltas) with ≤5min SLA.
- Provide simulations with diff summaries for UI/CLI workflows without modifying state.
- Enforce strict determinism guard (no wall-clock, RNG, network beyond allow-listed services) and RBAC + tenancy via Authority scopes.
- Support sealed/air-gapped deployments with offline bundles and sealed-mode hints.
Non-goals: policy authoring UI (handled by Console), ingestion or advisory normalisation (Concelier), VEX consensus (Excititor), runtime enforcement (Zastava).
@@ -110,8 +111,9 @@ Key notes:
| **Materialiser** (`Materialization/`) | Upsert effective findings, append history, manage explain bundle exports. | Mongo transactions per SBOM chunk. |
| **Orchestrator** (`Runs/`) | Change-stream ingestion, fairness, retry/backoff, queue writer. | Works with Scheduler Models DTOs. |
| **API** (`Api/`) | Minimal API endpoints, DTO validation, problem responses, idempotency. | Generated clients for CLI/UI. |
| **Observability** (`Telemetry/`) | Metrics (`policy_run_seconds`, `rules_fired_total`), traces, structured logs. | Sampled rule-hit logs with redaction. |
| **Offline Adapter** (`Offline/`) | Bundle export/import (policies, simulations, runs), sealed-mode enforcement. | Uses DSSE signing via Signer service. |
| **Observability** (`Telemetry/`) | Metrics (`policy_run_seconds`, `rules_fired_total`), traces, structured logs. | Sampled rule-hit logs with redaction. |
| **Offline Adapter** (`Offline/`) | Bundle export/import (policies, simulations, runs), sealed-mode enforcement. | Uses DSSE signing via Signer service. |
| **VEX Decision Emitter** (`Vex/Emitter/`) | Build OpenVEX statements, attach reachability evidence hashes, request DSSE signing, and persist artifacts for Export Center / bench repo. | New (Sprint401); integrates with Signer predicate `stella.ops/vexDecision@v1` and Attestor Rekor logging. |
---
@@ -172,22 +174,36 @@ Determinism guard instrumentation wraps the evaluator, rejecting access to forbi
---
## 6·Run Orchestration & Incremental Flow
- **Change streams:** Concelier and Excititor publish document changes to the scheduler queue (`policy.trigger.delta`). Payload includes `tenant`, `source`, `linkset digests`, `cursor`.
- **Orchestrator:** Maintains per-tenant backlog; merges deltas until time/size thresholds met, then enqueues `PolicyRunRequest`.
- **Queue:** Mongo queue with lease; each job assigned `leaseDuration`, `maxAttempts`.
## 6·Run Orchestration & Incremental Flow
- **Change streams:** Concelier and Excititor publish document changes to the scheduler queue (`policy.trigger.delta`). Payload includes `tenant`, `source`, `linkset digests`, `cursor`.
- **Orchestrator:** Maintains per-tenant backlog; merges deltas until time/size thresholds met, then enqueues `PolicyRunRequest`.
- **Queue:** Mongo queue with lease; each job assigned `leaseDuration`, `maxAttempts`.
- **Workers:** Lease jobs, execute evaluation pipeline, report status (success/failure/canceled). Failures with recoverable errors requeue with backoff; determinism or schema violations mark job `failed` and raise incident event.
- **Fairness:** Round-robin per `{tenant, policyId}`; emergency jobs (`priority=emergency`) jump queue but limited via circuit breaker.
- **Replay:** On demand, orchestrator rehydrates run via stored cursors and exports sealed bundle for audit/CI determinism checks.
- **Batch evaluation service (`/api/policy/eval/batch`):** Stateless evaluator powering Findings Ledger and replay/offline workflows. Requests contain canonical ledger events plus optional current projection; responses return status/severity/labels/rationale without mutating state. Policy Engine enforces per-tenant cost budgets, caches results by `(tenantId, policyVersion, eventHash, projectionHash)`, and falls back to inline evaluation when the remote service is disabled.
---
## 7·Security & Tenancy
- **Auth:** All API calls pass through Authority gateway; DPoP tokens enforced for service-to-service (Policy Engine service principal). CLI/UI tokens include scope claims.
- **Scopes:** Mutations require `policy:*` scopes corresponding to action; `effective:write` restricted to service identity.
### 6.1·VEX decision attestation pipeline
1. **Verdict capture.** Each evaluation result contains `{findingId, cve, productKey, reachabilityState, evidenceRefs}` plus SBOM and runtime CAS hashes.
2. **OpenVEX serialization.** `VexDecisionEmitter` builds an OpenVEX document with one statement per `(cve, productKey)` and fills:
- `status`, `justification`, `status_notes`, `impact_statement`, `action_statement`.
- `products` (purl) and `evidence` array referencing `reachability.json`, `sbom.cdx.json`, `runtimeFacts`.
3. **DSSE signing.** The emitter calls Signer `POST /api/v1/signer/sign/dsse` with predicate `stella.ops/vexDecision@v1`. Signer verifies PoE + scanner integrity and returns a DSSE envelope (`decision.dsse.json`).
4. **Transparency (optional).** When Rekor integration is enabled, Attestor logs the DSSE payload and returns `{uuid, logIndex, checkpoint}` which we persist next to the decision.
5. **Export.** API/CLI endpoints expose `decision.openvex.json`, `decision.dsse.json`, `rekor.txt`, and evidence metadata so Export Center + bench automation can mirror them into `bench/findings/**` as defined in the [VEX Evidence Playbook](../../benchmarks/vex-evidence-playbook.md).
All payloads are immutable and include analyzer fingerprints (`scanner.native@sha256:...`, `policyEngine@sha256:...`) so replay tooling can recompute identical digests. Determinism tests cover both the OpenVEX JSON and the DSSE payload bytes.
---
## 7·Security & Tenancy
- **Auth:** All API calls pass through Authority gateway; DPoP tokens enforced for service-to-service (Policy Engine service principal). CLI/UI tokens include scope claims.
- **Scopes:** Mutations require `policy:*` scopes corresponding to action; `effective:write` restricted to service identity.
- **Tenancy:** All queries filter by `tenant`. Service identity uses `tenant-global` for shared policies; cross-tenant reads prohibited unless `policy:tenant-admin` scope present.
- **Secrets:** Configuration loaded via environment variables or sealed secrets; runtime avoids writing secrets to logs.
- **Determinism guard:** Static analyzer prevents referencing forbidden namespaces; runtime guard intercepts `DateTime.Now`, `Random`, `Guid`, HTTP clients beyond allow-list.