feat: Implement vulnerability token signing and verification utilities
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Added VulnTokenSigner for signing JWT tokens with specified algorithms and keys.
- Introduced VulnTokenUtilities for resolving tenant and subject claims, and sanitizing context dictionaries.
- Created VulnTokenVerificationUtilities for parsing tokens, verifying signatures, and deserializing payloads.
- Developed VulnWorkflowAntiForgeryTokenIssuer for issuing anti-forgery tokens with configurable options.
- Implemented VulnWorkflowAntiForgeryTokenVerifier for verifying anti-forgery tokens and validating payloads.
- Added AuthorityVulnerabilityExplorerOptions to manage configuration for vulnerability explorer features.
- Included tests for FilesystemPackRunDispatcher to ensure proper job handling under egress policy restrictions.
This commit is contained in:
master
2025-11-03 10:02:29 +02:00
parent bf2bf4b395
commit b1e78fe412
215 changed files with 19441 additions and 12185 deletions

View File

@@ -8,6 +8,8 @@ Policy Engine compiles and evaluates Stella DSL policies deterministically, prod
- [Architecture](./architecture.md)
- [Implementation plan](./implementation_plan.md)
- [Task board](./TASKS.md)
- [Secret leak detection readiness](../policy/secret-leak-detection-readiness.md)
- [Windows package readiness](../policy/windows-package-readiness.md)
## How to get started
1. Open ../../implplan/SPRINTS.md and locate the stories referencing this module.

View File

@@ -1,31 +1,33 @@
# StellaOps Policy Engine
Policy Engine compiles and evaluates Stella DSL policies deterministically, producing explainable findings with full provenance.
## Responsibilities
- Compile `stella-dsl@1` packs into executable graphs.
- Join advisories, VEX evidence, and SBOM inventories to derive effective findings.
- Expose simulation and diff APIs for UI/CLI workflows.
- Emit change-stream driven events for Notify/Scheduler integrations.
## Key components
- `StellaOps.Policy.Engine` service host.
- Shared libraries under `StellaOps.Policy.*` for evaluation, storage, DSL tooling.
## Integrations & dependencies
- MongoDB findings collections, RustFS explain bundles.
- Scheduler for incremental re-evaluation triggers.
- CLI/UI for policy authoring and runs.
## Operational notes
- DSL grammar and lifecycle docs in ../../policy/.
- Observability guidance in ../../observability/policy.md.
- Governance and scope mapping in ../../security/policy-governance.md.
## Backlog references
- DOCS-POLICY-20-001 … DOCS-POLICY-20-012 (completed baseline).
- DOCS-POLICY-23-007 (upcoming command updates).
## Epic alignment
- **Epic 2 Policy Engine & Editor:** deliver deterministic evaluation, DSL infrastructure, explain traces, and incremental runs.
- **Epic 4 Policy Studio:** integrate registry workflows, simulation at scale, approvals, and promotion semantics.
# StellaOps Policy Engine
Policy Engine compiles and evaluates Stella DSL policies deterministically, producing explainable findings with full provenance.
## Responsibilities
- Compile `stella-dsl@1` packs into executable graphs.
- Join advisories, VEX evidence, and SBOM inventories to derive effective findings.
- Expose simulation and diff APIs for UI/CLI workflows.
- Emit change-stream driven events for Notify/Scheduler integrations.
## Key components
- `StellaOps.Policy.Engine` service host.
- Shared libraries under `StellaOps.Policy.*` for evaluation, storage, DSL tooling.
## Integrations & dependencies
- MongoDB findings collections, RustFS explain bundles.
- Scheduler for incremental re-evaluation triggers.
- CLI/UI for policy authoring and runs.
## Operational notes
- DSL grammar and lifecycle docs in ../../policy/.
- Observability guidance in ../../observability/policy.md.
- Governance and scope mapping in ../../security/policy-governance.md.
- Readiness briefs: ../policy/secret-leak-detection-readiness.md, ../policy/windows-package-readiness.md.
- Readiness briefs: ../scanner/design/macos-analyzer.md, ../scanner/design/windows-analyzer.md, ../policy/secret-leak-detection-readiness.md, ../policy/windows-package-readiness.md.
## Backlog references
- DOCS-POLICY-20-001 … DOCS-POLICY-20-012 (completed baseline).
- DOCS-POLICY-23-007 (upcoming command updates).
## Epic alignment
- **Epic 2 Policy Engine & Editor:** deliver deterministic evaluation, DSL infrastructure, explain traces, and incremental runs.
- **Epic 4 Policy Studio:** integrate registry workflows, simulation at scale, approvals, and promotion semantics.

View File

@@ -1,9 +1,11 @@
# Task board — Policy Engine
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------|
| POLICY ENGINE-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| POLICY ENGINE-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| POLICY ENGINE-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |
# Task board — Policy Engine
> Local tasks should link back to ./AGENTS.md and mirror status updates into ../../TASKS.md when applicable.
| ID | Status | Owner(s) | Description | Notes |
|----|--------|----------|-------------|-------|
| POLICY ENGINE-DOCS-0001 | TODO | Docs Guild | Validate that ./README.md aligns with the latest release notes. | See ./AGENTS.md |
| POLICY ENGINE-OPS-0001 | TODO | Ops Guild | Review runbooks/observability assets after next sprint demo. | Sync outcomes back to ../../TASKS.md |
| POLICY ENGINE-ENG-0001 | TODO | Module Team | Cross-check implementation plan milestones against ../../implplan/SPRINTS.md. | Update status via ./AGENTS.md workflow |
| POLICY-READINESS-0001 | DOING (2025-11-03) | Policy Guild, Security Guild | Resolve open questions in `../policy/secret-leak-detection-readiness.md` ahead of SCANNER-ENG-0007. | Decision workshop 2025-11-10 (Northwind demo); cover masking depth, telemetry retention, bundle defaults, tenant overrides. |
| POLICY-READINESS-0002 | DOING (2025-11-03) | Policy Guild, Security Guild, Offline Kit Guild | Review `../policy/windows-package-readiness.md`, set signature verification locus, feed mirroring scopes, and legacy installer posture. | FinSecure PCI blocker; deliver Authenticode/feed decision by 2025-11-07 before analyzer spike kickoff. |

View File

@@ -0,0 +1,80 @@
# Secret Leak Detection Readiness — Policy & Security Brief
> Audience: Policy Guild, Security Guild
> Related backlog: SCANNER-ENG-0007 (deterministic leak detection pipeline), DOCS-SCANNER-BENCH-62-007 (rule bundle documentation), SCANNER-SECRETS-01..03 (Surface.Secrets alignment)
## 1. Goal & scope
- Provide a shared understanding of how the planned `StellaOps.Scanner.Analyzers.Secrets` plug-in will operate so Policy/Security can prepare governance controls in parallel with engineering delivery.
- Document evidence flow, policy predicates, and offline distribution requirements to minimise lead time once implementation lands.
- Capture open questions requiring Policy/Security sign-off (masking rules, tenancy constraints, waiver workflows).
## 2. Proposed evidence pipeline
1. **Source resolution**
- Surface.Secrets providers (Kubernetes, file bundle, inline) continue to resolve operational credentials. Handles remain opaque and never enter analyzer outputs.
2. **Deterministic scanning**
- New plug-in executes signed rule bundles (regex + entropy signatures) stored under `scanner/rules/secrets/`.
- Execution context restricted to read-only layer mount; analyzers emit `secret.leak` evidence with: `{rule.id, rule.version, confidence, severity, mask, file, line}`.
3. **Analysis store persistence**
- Findings are written into `ScanAnalysisStore` (`ScanAnalysisKeys.secretFindings`) so Policy Engine can ingest them alongside component fragments.
4. **Policy overlay**
- Policy predicates (see §3) evaluate evidence, lattice scores, and tenant-scoped allow/deny lists.
- CLI/export surfaces show masked snippets and remediation hints.
5. **Offline parity**
- Rule bundles, signature manifests, and validator hash lists ship with Offline Kit; rule updates must be signed and versioned to preserve determinism.
## 3. Policy Engine considerations
- **New predicates**
- `secret.hasFinding(ruleId?, severity?, confidence?)`
- `secret.bundle.version(requiredVersion)`
- `secret.mask.applied` (bool) — verify masking for high severity hits.
- `secret.path.allowlist` — tenant-configured allow list keyed by digest/path.
- **Lattice weight suggestions**
- High severity & high confidence → escalate to `block` unless waived.
- Low confidence → default to `warn` with optional escalation when multiple matches occur (`secret.match.count >= N`).
- **Waiver workflow**
- Reuse VEX-first lattice approach: require attach of remediation note, ticket reference, and expiration date.
- Ensure waivers attach rule version so upgraded rules re-evaluate automatically.
- **Masking / privacy**
- Minimum masking: first and last 2 characters retained; remainder replaced with `*`.
- Persist masked payload only; full value never leaves scanner context.
## 4. Security guardrails
- Rule bundle signing: Signer issues DSSE envelope for each ruleset; Policy must verify signature before enabling new bundle.
- Tenant isolation: bundle enablement scoped per tenant; defaults deny unknown bundles.
- Telemetry: emit `scanner.secret.finding_total{tenant, ruleId, severity}` with masking applied after count aggregation.
- Access control: restrict retrieval of raw finding payloads to roles with `scanner.secret.read` scope; audits log query + tenant + rule id.
## 5. Offline & update flow
1. Engineering publishes new bundle → Signer signs → Offline Kit includes bundle + manifest.
2. Operators import bundle via CLI (`stella secrets bundle install --path <bundle>`).
- CLI verifies signature, version monotonicity, and rule hash list.
3. Policy update: push config snippet enabling bundle, severity mapping, and waiver templates.
4. Run `stella secrets scan --dry-run` to verify deterministic output against golden fixtures before enabling in production.
## 6. Open questions / owners
| Topic | Question | Owner | Target decision |
| --- | --- | --- | --- |
| Masking depth | Do we mask file paths or only payloads? | Security Guild | Sprint 132 design review |
| Telemetry retention | Should secret finding metrics be sampled or full fidelity? | Policy + Observability Guild | Sprint 132 |
| Default bundles | Which rule families ship enabled-by-default (cloud creds, SSH keys, JWT)? | Security Guild | Sprint 133 |
| Tenant overrides | Format for per-tenant allow lists (path glob vs digest)? | Policy Guild | Sprint 133 |
### Decision tracker
| Decision | Owner(s) | Target date | Status |
| --- | --- | --- | --- |
| Masking depth (paths vs payloads) | Security Guild | 2025-11-10 | Pending — workshop aligned with Northwind demo |
| Telemetry retention granularity | Policy + Observability Guild | 2025-11-10 | Pending |
| Default rule bundles (cloud creds/SSH/JWT) | Security Guild | 2025-11-10 | Draft proposals under review |
| Tenant override format | Policy Guild | 2025-11-10 | Pending |
## 7. Next steps
1. Policy Guild drafts predicate specs + policy templates (map to DOCS-SCANNER-BENCH-62-007 exit criteria).
2. Security Guild reviews signing + masking requirements; align with Surface.Secrets roadmap.
3. Docs Guild (this task) continues maintaining `docs/benchmarks/scanner/deep-dives/secrets.md` with finalized rule taxonomy and references.
4. Engineering provides prototype fixture outputs for review once SCANNER-ENG-0007 spikes begin.
## Coordination
- Capture macOS customer requirements via `docs/benchmarks/scanner/windows-macos-demand.md` (Northwind Health Services).
- Update dashboard (`docs/api/scanner/windows-macos-summary.md`) after readiness decisions.
- Share outcomes from 2025-11-10 macOS demo before finalising POLICY-READINESS-0001.

View File

@@ -0,0 +1,92 @@
# Windows Package Coverage — Policy & Security Readiness Brief
> Audience: Policy Guild, Security Guild, Offline Kit Guild
> Related engineering backlog (proposed): SCANNER-ENG-0024..0027
> Docs linkage: DOCS-SCANNER-BENCH-62-016
## 1. Goal
- Prepare policy and security guidance ahead of Windows analyzer implementation (MSI, WinSxS, Chocolatey, registry).
- Define evidence handling, predicates, waiver expectations, and offline prerequisites so engineering can align during spike execution.
## 2. Evidence pipeline snapshot (from `design/windows-analyzer.md`)
1. **Collection**
- MSI database parsing → component records keyed by ProductCode/ComponentCode.
- WinSxS manifests → assembly identities, catalog signatures.
- Chocolatey packages → nuspec metadata, feed provenance, script hashes.
- Registry exports → uninstall/service entries, legacy installers.
- Driver/service mapper → capability overlays (kernel-mode, auto-start).
2. **Storage**
- Results persisted as `LayerComponentFragment`s plus capability overlays (`ScanAnalysisKeys.capability.windows`).
- Provenance metadata includes signature thumbprint, catalog hash, feed URL, install context.
3. **Downstream**
- Policy Engine consumes component + capability evidence; Export Center bundles MSI manifests, nuspec metadata, catalog hashes.
## 3. Policy predicate requirements
| Predicate | Description | Initial default |
| --- | --- | --- |
| `windows.package.signed(thumbprint?)` | True when Authenticode signature/cert matches allowlist. | Warn on missing signature, fail on mismatched thumbprint for kernel drivers. |
| `windows.package.sourceAllowed(sourceId)` | Validates Chocolatey/nuget feed against tenant allowlist. | Fail if feed not in tenant policy. |
| `windows.driver.kernelMode()` | Flags kernel-mode drivers for extra scrutiny. | Fail when unsigned; warn otherwise. |
| `windows.driver.signedBy(publisher)` | Checks driver publisher matches allowlist. | Warn on unknown publisher. |
| `windows.service.autoStart(name)` | Identifies auto-start services. | Warn if unsigned binary or service not in allowlist. |
| `windows.package.legacyInstaller()` | Legacy EXE-only installers detected via registry. | Warn by default; escalate if binary unsigned. |
Additional considerations:
- Map KB references (from WinSxS/MSP metadata) to vulnerability posture once Policy Engine supports patch layering.
- Provide predicates to waive specific ProductCodes or AssemblyIdentities with expiration.
## 4. Waiver & governance model
- Waiver key: `{productCode, version, signatureThumbprint}` or for drivers `{driverName, serviceName, signatureThumbprint}`.
- Required metadata: remediation ticket, justification, expiry date.
- Automated re-evaluation when version or signature changes.
- Tenants maintain allow lists for Chocolatey feeds and driver publishers via policy configuration.
## 5. Masking & privacy
- Findings should not include raw script contents; provide SHA256 hash and limited excerpt (first/last 8 chars).
- Registry values (install paths, command lines) must be truncated if they contain secrets; rely on Surface.Secrets to manage environment variables referenced during install scripts.
## 6. Offline kit guidance
- Bundle:
- MSI parser binary + schema definitions.
- Chocolatey feed snapshot(s) (nupkg files) with hash manifest.
- Microsoft root/intermediate certificate bundles; optional CRL/OCSP cache instructions.
- Operators must export registry hives (`SOFTWARE`, `SYSTEM`) during image extraction; document PowerShell script and required access.
- Provide checksum manifest to verify feed snapshot integrity.
## 7. Telemetry expectations
- Metrics:
- `scanner.windows.package_total{tenant,signed}` — count packages per signature state.
- `scanner.windows.driver_unsigned_total{tenant}`.
- `scanner.windows.choco_feed_total{tenant,feed}`.
- Logs:
- Include product code, version, signature thumbprint, feed ID (no file paths unless sanitized).
- Traces:
- Annotate collector spans (`collector.windows.msi`, `collector.windows.winsxs`, etc.) with component counts and parsing duration.
## 8. Open questions
| Topic | Question | Owner | Target decision |
| --- | --- | --- | --- |
| Signature verification locus | Scanner vs Policy: where to verify Authenticode signatures + revocation? | Security Guild | Sprint 133 |
| Feed mirroring scope | Default set of Chocolatey feeds to mirror (official/community). | Product + Security Guild | Sprint 133 |
| Legacy installers | Should we block unsigned EXE installers by default or allow warn-only posture? | Policy Guild | Sprint 134 |
| Driver taxonomy | Define high-risk driver categories (kernel-mode, filter drivers) for policy severity. | Policy Guild | Sprint 134 |
### Decision tracker
| Decision | Owner(s) | Target date | Status |
| --- | --- | --- | --- |
| Authenticode verification locus (Scanner vs Policy) | Security Guild | 2025-11-07 | Pending — blocker for FinSecure |
| Chocolatey feed mirroring scope | Product + Security Guild | 2025-11-07 | Draft proposal circulating |
| Legacy installer posture (warn vs fail) | Policy Guild | 2025-11-14 | Not started |
| Driver risk taxonomy | Policy Guild | 2025-11-14 | Not started |
## 9. Next steps
1. Policy Guild drafts predicate specs + policy templates; align with DOCS-SCANNER-BENCH-62-016.
2. Security Guild evaluates signature verification approach and revocation handling (online vs offline CRL cache).
3. Offline Kit Guild scopes snapshot size and update cadence for Chocolatey feeds and certificate bundles.
4. Docs Guild prepares policy/user guidance updates once predicates are finalised.
5. Security Guild to report decision for FinSecure Corp (POLICY-READINESS-0002) by 2025-11-07; feed outcome into dashboards.
## Coordination
- Sync demand signals via `docs/benchmarks/scanner/windows-macos-demand.md`.
- Log policy readiness status in `docs/api/scanner/windows-coverage.md`.
- Update Windows/macOS metrics dashboard when decisions change (`docs/api/scanner/windows-macos-summary.md`).