feat: Implement vulnerability token signing and verification utilities
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Added VulnTokenSigner for signing JWT tokens with specified algorithms and keys.
- Introduced VulnTokenUtilities for resolving tenant and subject claims, and sanitizing context dictionaries.
- Created VulnTokenVerificationUtilities for parsing tokens, verifying signatures, and deserializing payloads.
- Developed VulnWorkflowAntiForgeryTokenIssuer for issuing anti-forgery tokens with configurable options.
- Implemented VulnWorkflowAntiForgeryTokenVerifier for verifying anti-forgery tokens and validating payloads.
- Added AuthorityVulnerabilityExplorerOptions to manage configuration for vulnerability explorer features.
- Included tests for FilesystemPackRunDispatcher to ensure proper job handling under egress policy restrictions.
This commit is contained in:
master
2025-11-03 10:02:29 +02:00
parent bf2bf4b395
commit b1e78fe412
215 changed files with 19441 additions and 12185 deletions

View File

@@ -232,12 +232,108 @@
- **Policy considerations**: Policies would need repository allowlists, signing requirements, and OS-specific mitigations; defer concrete templates until design spike completes.
- **Next actions**: Execute tasks DOCS-SCANNER-BENCH-62-002/003/004/005/006 as demand signals accrue; only open engineering backlog after demand review approves scope expansion.
### Implementation details
- **StellaOps**:
- Linux analyzers live under `StellaOps.Scanner.Analyzers.OS.(Apk|Dpkg|Rpm)` and inherit from `OsPackageAnalyzerBase`, which normalises package metadata, file evidence, and vendor fields before persisting content-addressed fragments via Surface.FS.
- Analyzer results are converted to `LayerComponentFragment`s by `OsComponentMapper` and cached inside the worker `ScanAnalysisStore`; downstream SBOM assembly keeps layer digests and provenance so usage/inventory views can be replayed deterministically.
- Export Center binds OS fragments to DSSE attestations through Signer/Attestor so operators can prove provenance for every package list in offline bundles.
- **Trivy**:
- Package analyzers under `pkg/fanal/analyzer/pkg/(apk|dpkg|rpm)` walk layer file systems and emit `types.Package` results; the upstream coverage matrix in `docs/docs/coverage/os/*.md` drives which distro manifests and feeds ship with Trivy releases.
- Results are flattened per artifact, leaving layer attribution to Syft-style catalog walks; provenance and diffing are delegated to downstream tooling.
- **Grype**:
- Delegates cataloguing to Syft, then applies matchers in `grype/matcher/{apk,dpkg,rpm}` that resolve distro namespaces and fix statuses using Anchore feeds.
- Evidence is matched at scan time without long-lived fragments, so reproducibility depends on feed snapshots rather than packaged artifacts.
- **Snyk**:
- `snyk container test` uploads image metadata to Snyk SaaS; coverage is dictated by the hosted service and cannot run offline.
- No layer-aware evidence is returned; operators rely on SaaS reports for package presence and risk.
### Detection techniques
| Technique | Artifacts | Analyzer / Module | Merge strategy |
|-----------|-----------|-------------------|----------------|
| Layer package DB parsing | apk, dpkg, rpm status/databases per layer | StellaOps.Scanner.Analyzers.OS.* with RustFS CAS | Produces per-layer fragments keyed by layer digest; composed into inventory/usage SBOM with provenance pointers. |
| Manifest + attestation binding | Distro manifest attestations, vendor signatures | Export Center + Signer/Attestor hand-off | Binds package fragments to DSSE attestations; policy consumes provenance metadata for trust weighting. |
| External SBOM import (interim) | Third-party SBOMs for Windows/macOS | Scanner SBOM import API (planned) | Imports produce declared-only entries flagged for policy review until native analyzers exist. |
| Layer package DB parsing | apk, dpkg, rpm status/databases per layer | `StellaOps.Scanner.Analyzers.OS.(Apk|Dpkg|Rpm)` + Surface.FS | Emits `OSPackageRecord` fragments tagged with layer digest; `OsComponentMapper` writes `LayerComponentFragment`s that SBOM assembly uses to build inventory/usage views with provenance. |
| Manifest + attestation binding | Distro manifest attestations, vendor signatures | Export Center + Signer/Attestor hand-off | Couples OS fragments with DSSE/Rekor proofs; Policy Engine verifies signatures before promotion gates. |
| Linux distro enrichment | Vendor advisory feeds, package-to-CVE mapping | Concelier + Policy Engine | Concelier ingests advisory metadata; Policy joins advisories with package fragments to produce lattice-scored findings. |
| External SBOM import (interim) | Third-party SBOMs for Windows/macOS | Scanner SBOM import API (planned) | Declared-only entries are merged with runtime evidence; policy downgrades unmatched declarations until native analyzers exist. |
## macOS package coverage (Trivy, Grype, Snyk)
### Scorecard
| Dimension | Score (1-5) | Notes |
|-----------|-------------|-------|
| Customer demand | 2 | macOS coverage requests surface intermittently; demand capture ongoing via `windows-macos-demand.md`. |
| Competitive risk | 2 | Competitors lack macOS analyzers today, but roadmap announcements could erode differentiation quickly. |
| Engineering effort | 4 | Requires new collectors (Homebrew, pkgutil receipts, `.app` bundles) plus Offline Kit tap mirroring. |
| Policy/config impact | 4 | Policies must reason over entitlements, notarization, and signed bundle provenance. |
| Offline/air-gap impact | 4 | Tap metadata, notarization caches, and entitlement schemas must ship offline. |
- **Competitor capability**: None of Trivy, Grype, or Snyk provide macOS host/package coverage; their documentation lists Linux-only ecosystems.
- **StellaOps gap**: No macOS analyzers exist; design work is captured in `docs/benchmarks/scanner/deep-dives/macos.md` but implementation/backlog stories are not yet opened.
- **Proposed plan**:
1. Finalise demand assessment (DOCS-SCANNER-BENCH-62-002) using customer interviews, sales telemetry, and support tags.
2. Scope macOS analyzer design spike covering Homebrew cellar parsing, pkgutil receipt ingestion, and `.app` bundle inspection; include entitlements/notarization strategy.
3. Define Policy Engine predicates for entitlements, notarization status, bundle signing chain, and tenant allow lists.
4. Coordinate with Offline Kit guild on mirroring tap metadata, notarization caches, and CRL/OCSP content for air-gapped operation.
- **Milestones**: Northwind Health Services demo (2025-11-10) doubles as POLICY-READINESS-0001 workshop to finalise masking/telemetry defaults before spike approval.
- **Policy considerations**: Need predicates for `macos.entitlement`, `macos.notarized`, `macos.bundle.teamId`, and severity rules for unsigned or unnotarized software; waivers should bind to bundle hash + signer.
- **Next actions**: Keep `windows-macos-demand.md` updated, prepare design brief once threshold met, and log engineering backlog items (SCANNER-ENG-00xx) for macOS collectors and policy integration.
### Implementation details
- **Design references**: `docs/benchmarks/scanner/deep-dives/macos.md` captures proposed collectors (Homebrew, receipts, bundles) and open questions for Security/Policy guilds.
- **Collector outline**:
- Homebrew collector enumerates Cellar manifests under `/usr/local/Cellar` and `/opt/homebrew/Cellar`, mapping taps to PURLs and retaining bottle SHA256 for provenance.
- pkgutil collector parses `/var/db/receipts/*.plist` and `.bom` files to record installer package metadata with deterministic hashes.
- Bundle inspector walks `.app` directories, extracting Info.plist, entitlements, embedded frameworks, and code signing chains.
- **Merge strategy**: Collectors will emit component fragments keyed by bundle/tap identifiers; planned aggregator merges receipts and cellular data where identifiers match, tagging capabilities for Policy Engine.
- **Offline requirements**: Bundle signed rule packs (tap metadata snapshots, entitlements schema) with Offline Kit; ensure notarization cache instructions are documented.
### Detection techniques
| Technique | Artifacts | Analyzer / Module | Merge strategy |
|-----------|-----------|-------------------|----------------|
| Homebrew cellar parsing (planned) | Cellar manifests, `INSTALL_RECEIPT.json`, tap metadata | Planned `StellaOps.Scanner.Analyzers.OS.Mac.Homebrew` | Emits component records keyed by tap + version; merges duplicates and attaches bottle hashes for provenance. |
| pkgutil receipt parsing (planned) | `/var/db/receipts/*.plist` and `.bom` | Planned `StellaOps.Scanner.Analyzers.OS.Mac.Receipts` | Records installer packages with bundle identifiers; merges with bundle evidence when identifiers align. |
| `.app` bundle inspection (planned) | Info.plist, CodeResources, entitlements, signing certs | Planned `StellaOps.Scanner.Analyzers.OS.Mac.Bundles` | Produces capability evidence (entitlements, hardened runtime) and links to receipts/Homebrew entries via bundle id. |
| Launch agent/daemon analysis (planned) | `/Library/Launch*` manifests, `launchctl` exports | Planned runtime usage mapper | Augments EntryTrace usage hints to distinguish active services. |
| Competitor baseline | — | Trivy/Grype/Snyk | No macOS host analyzers; coverage limited to Linux/container contexts. |
## Windows package coverage (Trivy, Grype, Snyk)
### Scorecard
| Dimension | Score (1-5) | Notes |
|-----------|-------------|-------|
| Customer demand | 3 | Windows Server/container adopters continue to request evidence parity; demand captured via `windows-macos-demand.md`. |
| Competitive risk | 2 | Competitors lack Windows analyzers today but could narrow the gap with roadmap announcements. |
| Engineering effort | 5 | Requires MSI/WinSxS parsers, registry collectors, Chocolatey handling, and extensive Offline Kit work. |
| Policy/config impact | 4 | Policies must interpret Authenticode trust, driver/service posture, and legacy installer artefacts. |
| Offline/air-gap impact | 5 | Need to package MSI schemas, feed snapshots, and certificate bundles; storage overhead is significant. |
- **Competitor capability**: Trivy, Grype, and Snyk do not ship Windows host analyzers; tooling focuses on Linux ecosystems and SaaS flows.
- **StellaOps gap**: No Windows analyzers exist yet; design outline under `docs/benchmarks/scanner/deep-dives/windows.md`.
- **Proposed plan**:
1. Complete demand validation and prioritisation (DOCS-SCANNER-BENCH-62-002) alongside macOS signals.
2. Execute engineering spike covering MSI/WinSxS parsing, Chocolatey inventory, and registry-based fallbacks.
3. Define Policy Engine predicates for Authenticode, driver risk, service start mode, and Chocolatey provenance.
4. Coordinate Offline Kit packaging strategy for MSI schemas, feed snapshots, and certificate revocation caches.
- **Policy considerations**: Introduce predicates such as `windows.package.signed(teamId?)`, `windows.driver.kernelMode`, `windows.service.startType`, and waivers tied to product code + signature thumbprint.
- **Next actions**: Maintain demand tracker, execute DOCS-SCANNER-BENCH-62-016, refine `docs/modules/scanner/design/windows-analyzer.md`, and open SCANNER-ENG-0024..0027 backlog tickets for collector and policy work once demand threshold is confirmed.
- **Milestones**: POLICY-READINESS-0002 Authenticode/feed decision due 2025-11-07 (FinSecure PCI blocker) gates Windows analyzer spikes.
### Implementation details
- **Design references**: `docs/benchmarks/scanner/deep-dives/windows.md`, `docs/modules/scanner/design/windows-analyzer.md`, `docs/api/scanner/windows-coverage.md`.
- **Policy readiness**: see `docs/modules/policy/windows-package-readiness.md` for predicate requirements, waiver model, and offline guidance.
- **Collector outline**:
- MSI/WinSxS collector to parse installer databases/manifests and correlate via file hashes.
- Chocolatey collector to read nuspec metadata and install scripts, retaining feed provenance.
- Registry collector to harvest uninstall/service keys, linking to filesystem artefacts and signatures.
- **Merge strategy**: Planned `WindowsComponentMapper` merges MSI, WinSxS, Chocolatey, and registry evidence into unified fragments with provenance metadata; capability overlays capture services/drivers.
- **Offline requirements**: Bundle MSI schema definitions, Chocolatey feed snapshots, Windows Update catalog hashes, and certificate chains; document cache priming steps for air-gapped environments.
### Detection techniques
| Technique | Artifacts | Analyzer / Module | Merge strategy |
|-----------|-----------|-------------------|----------------|
| MSI database parsing (planned) | `Windows/Installer/*.msi` Product/Component/File tables | Planned `StellaOps.Scanner.Analyzers.OS.Windows.Msi` | Emits component records keyed by product/component codes; merges with WinSxS manifests via file hashes. |
| WinSxS manifest parsing (planned) | `Windows/WinSxS/Manifests/*.manifest`, catalog files | Planned `StellaOps.Scanner.Analyzers.OS.Windows.WinSxS` | Maps assemblies to catalogs and MSP patches; merges with MSI output for provenance. |
| Chocolatey package parsing (planned) | `ProgramData/Chocolatey/lib/*`, nuspec metadata | Planned `StellaOps.Scanner.Analyzers.OS.Windows.Choco` | Records package evidence with feed provenance; merges with registry uninstall data. |
| Registry fallback (planned) | Exported uninstall/service hives | Planned `StellaOps.Scanner.Analyzers.OS.Windows.Registry` | Fills gaps for legacy installers and services; merges evidence by install path/signature. |
| Service/driver capability mapping (planned) | SYSTEM hive, DriverStore manifest | Planned capability overlay | Emits runtime capability records (drivers/services) for Policy Engine gating. |
| Competitor baseline | — | Trivy/Grype/Snyk | No Windows host analyzers; operators depend on external tooling. |
## Secrets leak detection (Trivy, Snyk)
### Scorecard
@@ -258,14 +354,29 @@
4. Offer CLI verb (`stella secrets scan`) and integration into existing scan workflows behind an opt-in flag.
5. Expose explain traces detailing rule IDs, masked snippets, and remediation guidance while upholding privacy constraints.
- **Policy considerations**: Deliver policy templates for severity gating, rule packs per tenant, and privacy controls; lattice logic should discount low-confidence matches.
- **Next actions**: open analyzer/CLI backlog work, coordinate with Docs Guild on policy templates, and bundle signed rule packs for Offline Kit distribution.
- **Next actions**: Track execution via SCANNER-ENG-0007 (design/implementation) and DOCS-SCANNER-BENCH-62-007 (policy/docs); bundle signed rule packs for Offline Kit distribution once analyzer stories land.
### Implementation details
- **StellaOps**:
- Operational secret retrieval flows through `Surface.Secrets` providers (Kubernetes, file bundle, inline) with validation policies enforced by `Surface.Validation`; handles remain opaque to analyzers.
- Planned `StellaOps.Scanner.Analyzers.Secrets` plug-in will execute deterministic rule bundles signed by the Export Center signing stack; findings land in `ScanAnalysisStore` alongside component fragments.
- Policy Engine will ingest `secret.leak` evidence with lattice hints (`confidence`, `rule.id`, `masking.applied`) so tenants can tune severities and approvals.
- **Trivy**:
- Secret detection is implemented in `pkg/fanal/secret` with detectors combining regex and entropy heuristics; rules merge into `Result` objects per file with severity weighting.
- Configuration is provided via `trivy-secret.yaml`, enabling per-rule enable/disable and allow lists.
- **Snyk**:
- CLI delegates to Snyk Code (`src/lib/plugins/sast`) which uploads source or image contents to SaaS for analysis; results stream back as issue JSON with remediation tips.
- Offline execution is unsupported; rule updates ship server-side.
- **Grype**:
- No leak detection analyzer; secrets are only used for registry authentication options.
### Detection techniques
| Technique | Artifacts | Analyzer / Module | Merge strategy |
|-----------|-----------|-------------------|----------------|
| Operational secret retrieval | secret:// references resolved via Surface.Secrets providers | Surface.Secrets, Surface.Validation | Injects secrets at runtime; no SBOM entry created; policy ensures provenance of retrieved credentials. |
| Deterministic leak detection (planned) | File content, archives, bytecode | StellaOps.Scanner.Analyzers.Secrets (planned) | Emits secret.leak evidence with masked snippets; Policy Engine merges with package evidence using VEX gating. |
| Competitor leak scanning | Regex/entropy rulesets (Trivy pkg/fanal/secret), Snyk Code SaaS service | Trivy secret analyzer, Snyk Code API | Findings remain separate from SBOM data; StellaOps will map to policy evidence types once analyzer ships. |
| Operational secret retrieval | `secret://` references resolved via `Surface.Secrets` providers | Surface.Secrets, Surface.Validation | Injects handles at runtime only; provenance recorded in scan metadata, nothing added to SBOM inventory. |
| Deterministic leak detection (planned) | File contents, archives, bytecode, container layers | `StellaOps.Scanner.Analyzers.Secrets` plug-in | Produces `secret.leak` records stored in `ScanAnalysisStore`; Policy Engine correlates with component metadata for context-aware enforcement. |
| Policy gating and reporting | Secret evidence + policy templates | Policy Engine, CLI/Export Center | Lattice scores combine confidence + severity; CLI/report output masks payloads while referencing rule IDs for explainability. |
| Competitor leak scanning | Regex/entropy rule sets, SaaS classifiers | Trivy `pkg/fanal/secret`; Snyk Code service | Trivy merges detectors per file; Snyk relies on SaaS analysis; neither binds results to SBOM evidence or deterministic attestations. |
## EntryTrace runtime command resolution (Trivy, Grype, Snyk)
### Scorecard
@@ -314,12 +425,23 @@
- **Policy considerations**: Provide policy predicates for attestation presence, Rekor inclusion, and proof expiry to enforce promotion gates.
- **Next actions**: Track via DOCS-SCANNER-BENCH-62-015 and SCANNER-ENG-0015 for playbook plus tooling updates.
### Implementation details
- **StellaOps**:
- `StellaOps.Signer` generates DSSE envelopes for SBOMs/reports using PoE-scoped keys and forwards them to `StellaOps.Attestor`, which handles Rekor v2 submissions with retries and proof caching.
- Export Center profiles bundle DSSE payloads, Rekor inclusion proofs, and any external attestations into offline-ready archives; CLI and Policy Engine verify proofs before release.
- Notify + Scanner.WebService emit attestation health telemetry so operators can quickly spot and remediate failed Rekor submissions.
- **Trivy / Grype / Snyk**:
- Trivy supports optional Cosign signing but leaves Rekor submission manual; proofs are not bundled with scanner outputs.
- Grype leans on Syft for SBOM export and does not sign outputs.
- Snyk Container/Snyk CLI rely on SaaS-managed signing and do not expose DSSE workflows or offline proof packaging.
### Detection techniques
| Technique | Artifacts | Analyzer / Module | Merge strategy |
|-----------|-----------|-------------------|----------------|
| SBOM emission | CycloneDX/SPDX payloads per scan | Scanner emit pipelines | Generates inventory/usage BOMs stored with CAS hashes for attestation. |
| DSSE signing | DSSE bundles, signing keys | StellaOps.Signer + StellaOps.Attestor | Signs SBOM/report metadata, forwards to Rekor v2, records proof identifiers. |
| Rekor proof packaging | Rekor inclusion proofs, bundle metadata | Export Center attestation packager | Bundles proofs into Offline Kit/export artifacts; Policy verifies before release. |
| Policy enforcement | Attestation & proof evidence | Policy Engine, Scheduler gates | Policies require successful DSSE/Rekor entries before promotion; Scheduler blocks exports lacking proofs. |
| Competitor approach | CLI or SaaS-managed signing | Trivy Cosign integration, Snyk SaaS, Grype none | Operators must integrate manually; no default policy enforcement. |
## Ruby analyzer parity (Trivy, Grype, Snyk)