feat: Implement vulnerability token signing and verification utilities
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Added VulnTokenSigner for signing JWT tokens with specified algorithms and keys. - Introduced VulnTokenUtilities for resolving tenant and subject claims, and sanitizing context dictionaries. - Created VulnTokenVerificationUtilities for parsing tokens, verifying signatures, and deserializing payloads. - Developed VulnWorkflowAntiForgeryTokenIssuer for issuing anti-forgery tokens with configurable options. - Implemented VulnWorkflowAntiForgeryTokenVerifier for verifying anti-forgery tokens and validating payloads. - Added AuthorityVulnerabilityExplorerOptions to manage configuration for vulnerability explorer features. - Included tests for FilesystemPackRunDispatcher to ensure proper job handling under egress policy restrictions.
This commit is contained in:
master
35
docs/modules/scanner/design/README.md
Normal file
35
docs/modules/scanner/design/README.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# Scanner Design Dossiers
|
||||
|
||||
This directory contains deep technical designs for current and upcoming analyzers and surface components.
|
||||
|
||||
## Language analyzers
|
||||
- `ruby-analyzer.md` — lockfile, runtime graph, capability signals for Ruby.
|
||||
|
||||
## Surface & platform contracts
|
||||
- `surface-fs.md`
|
||||
- `surface-env.md`
|
||||
- `surface-validation.md`
|
||||
- `surface-secrets.md`
|
||||
|
||||
## OS ecosystem designs
|
||||
- `macos-analyzer.md` — Homebrew, pkgutil, `.app` bundle plan.
|
||||
- `windows-analyzer.md` — MSI, WinSxS, Chocolatey, registry collectors.
|
||||
|
||||
## Demand & dashboards
|
||||
- `../../benchmarks/scanner/windows-macos-demand.md` — demand tracker.
|
||||
- `../../benchmarks/scanner/windows-macos-interview-template.md` — interview template.
|
||||
- `../../api/scanner/windows-coverage.md` — coverage summary dashboard.
|
||||
- `../../api/scanner/windows-macos-summary.md` — metric snapshot.
|
||||
|
||||
## Utility & reference
|
||||
- `../operations/field-engagement.md` — SE workflow guidance.
|
||||
- `../operations/analyzers.md` — operational runbook.
|
||||
- `../operations/rustfs-migration.md` — storage migration notes.
|
||||
|
||||
## Maintenance tips
|
||||
- Keep demand tracker (`../../benchmarks/scanner/windows-macos-demand.md`) and API dashboards in sync when updating macOS/Windows designs.
|
||||
- Cross-reference policy readiness briefs for associated predicates and waiver models.
|
||||
|
||||
## Policy readiness
|
||||
- `../policy/secret-leak-detection-readiness.md` — secret leak pipeline decisions.
|
||||
- `../policy/windows-package-readiness.md` — Windows analyzer policy decisions.
|
||||
123
docs/modules/scanner/design/macos-analyzer.md
Normal file
123
docs/modules/scanner/design/macos-analyzer.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# macOS Analyzer Design Brief (Draft)
|
||||
|
||||
> Owners: Scanner Guild, Policy Guild, Offline Kit Guild
|
||||
> Related backlog (proposed): SCANNER-ENG-0020..0023, DOCS-SCANNER-BENCH-62-002
|
||||
> Status: Draft pending demand validation (see `docs/benchmarks/scanner/windows-macos-demand.md`)
|
||||
|
||||
## 1. Scope & objectives
|
||||
- Deliver deterministic inventory coverage for macOS container and VM images, focusing on Homebrew, Apple/pkgutil receipts, and `.app` bundles.
|
||||
- Preserve StellaOps principles: per-layer provenance, offline parity, policy explainability, and signed evidence pipelines.
|
||||
- Provide capability signals (entitlements, hardened runtime, launch agents) to enable Policy Engine gating.
|
||||
|
||||
Out of scope (Phase 1):
|
||||
- Dynamic runtime tracing of macOS services (deferred to Zastava/EntryTrace).
|
||||
- iOS/tvOS/visionOS package formats (will reuse bundle inspection where feasible).
|
||||
- Direct notarization ticket validation (delegated to Policy Engine unless otherwise decided).
|
||||
|
||||
## 2. High-level architecture
|
||||
```
|
||||
Scanner.Worker (macOS profile)
|
||||
├─ Surface.Validation (enforce allowlists, bundle size limits)
|
||||
├─ Surface.FS (layer/materialized filesystem)
|
||||
├─ HomebrewCollector (new) -> LayerComponentFragment (brew)
|
||||
├─ PkgutilCollector (new) -> LayerComponentFragment (pkgutil)
|
||||
├─ BundleInspector (new) -> Capability records + component fragments
|
||||
├─ LaunchAgentMapper (optional) -> Usage hints
|
||||
└─ MacOsAggregator (new) -> merges fragments, emits ComponentGraph & capability overlays
|
||||
```
|
||||
|
||||
- Each collector runs deterministically against the mounted filesystem; results persist in `ScanAnalysisStore` under dedicated keys before being normalized by `MacOsComponentMapper`.
|
||||
- Layer digests follow existing `LayerComponentFragment` semantics to maintain diff/replay parity.
|
||||
|
||||
## 3. Collectors & data sources
|
||||
### 3.1 Homebrew collector
|
||||
- Scan `/usr/local/Cellar/**` and `/opt/homebrew/Cellar/**` for `INSTALL_RECEIPT.json` and `*.rb` formula metadata.
|
||||
- Output fields: `tap`, `formula`, `version`, `revision`, `poured_from_bottle`, `installed_with`: [].
|
||||
- Store `bottle.sha256`, `source.url`, and `tapped_from` metadata for provenance.
|
||||
- Map to PURL: `pkg:brew/<tap>/<formula>@<version>?revision=<revision>`.
|
||||
- Guardrails: limit traversal depth, ignore caches (`/Caskroom` optional), enforce 200MB per formula cap (configurable).
|
||||
|
||||
### 3.2 pkgutil receipt collector
|
||||
- Parse `/var/db/receipts/*.plist` for pkg identifiers, version, install time, volume, installer domain.
|
||||
- Use `.bom` files to enumerate installed files; capture path hashes via `osx.BomParser`.
|
||||
- Emit `ComponentRecord`s keyed by `pkgutil:<identifier>` with metadata: `bundleIdentifier`, `installTimeUtc`, `volume`.
|
||||
- Provide dependency mapping between pkg receipts and Homebrew formulae when overlapping file hashes exist (optional Phase 2).
|
||||
|
||||
### 3.3 Bundle inspector
|
||||
- Traverse `/Applications`, `/System/Applications`, `/Users/*/Applications`, `/Library/Application Support`.
|
||||
- For each `.app`:
|
||||
- Read `Info.plist` (bundle id, version, short version, minimum system).
|
||||
- Extract code signing metadata via `codesign --display --xml` style parsing (Team ID, certificate chain, hardened runtime flag).
|
||||
- Parse entitlements (`archived-entitlements.xcent`) and map to capability taxonomy (network, camera, automation, etc.).
|
||||
- Hash `CodeResources` manifest to support provenance comparisons.
|
||||
- Link `.app` bundles to receipts and Homebrew formulae using bundle id or install path heuristics.
|
||||
- Emit capability overlays (e.g., `macos.entitlement("com.apple.security.network.server") == true`).
|
||||
|
||||
### 3.4 Launch agent mapper (optional Phase 1 extension)
|
||||
- Inspect `/Library/Launch{Agents,Daemons}` and user equivalents.
|
||||
- Parse `plist` for program arguments, run conditions, sockets, environment.
|
||||
- Feed derived runtime hints into EntryTrace (`UsageHints`) to mark active binaries vs dormant installations.
|
||||
|
||||
## 4. Aggregation & output
|
||||
- `MacOsComponentMapper` converts collector results into:
|
||||
- `LayerComponentFragment` arrays keyed by synthetic digests (`sha256:stellaops-os-macbrew`, etc.).
|
||||
- `ComponentMetadata` entries capturing tap origin, Team ID, entitlements, notarization flag (when available).
|
||||
- Capability overlays stored under `ScanAnalysisKeys.capability.macOS`.
|
||||
- Export Center enhancements:
|
||||
- Include Homebrew metadata manifests and CodeResources hashes in attested bundles.
|
||||
- Provide optional notarization proof attachments if Policy Engine later requires them.
|
||||
|
||||
## 5. Policy & governance integration
|
||||
- Predicates to introduce:
|
||||
- `macos.bundle.signed(teamId?, hardenedRuntime?)`
|
||||
- `macos.entitlement(name)`
|
||||
- `macos.pkg.receipt(identifier, version?)`
|
||||
- `macos.notarized` (pending Security decision).
|
||||
- Lattice guidance:
|
||||
- Unsigned/unnotarized third-party apps default to `warn`.
|
||||
- High-risk entitlements (camera, screen capture) escalate severity unless whitelisted.
|
||||
- Waiver model similar to existing EntryTrace/Secrets: require bundle hash + Team ID to avoid broad exceptions.
|
||||
|
||||
## 6. Offline kit considerations
|
||||
- Mirror required Homebrew taps (`homebrew/core`, `homebrew/cask`) and freeze metadata per release.
|
||||
- Bundle notarization cache instructions (CRL/OCSP) so operators can prefetch without Apple connectivity.
|
||||
- Package Apple certificate chain updates (WWDR) and provide verification script to ensure validity.
|
||||
- Document disk space impacts (Homebrew cellar snapshots ~500MB per tap snapshot).
|
||||
|
||||
## 7. Testing strategy
|
||||
- Unit tests with fixture directories:
|
||||
- Sample Cellar tree with multiple formulas (bottled vs source builds).
|
||||
- Synthetic pkg receipts + BOM files.
|
||||
- `.app` bundles with varying signing states (signed/notarized/unsigned).
|
||||
- Integration tests:
|
||||
- Build macOS rootfs tarballs via CI job (runner requiring macOS) that mimic layered container conversion.
|
||||
- Verify deterministic output by re-running collectors and diffing results (CI gating).
|
||||
- Offline compliance tests:
|
||||
- Ensure rule bundles and caches load without network by running integration suite in sealed environment.
|
||||
|
||||
## 8. Dependencies & open items
|
||||
| Item | Description | Owner | Status |
|
||||
| --- | --- | --- | --- |
|
||||
| Demand threshold | Need ≥3 qualified customer asks to justify engineering investment | Product Guild | Active (DOCS-SCANNER-BENCH-62-002) |
|
||||
| macOS rootfs fixtures | Require automation to produce macOS layer tarballs for tests | Scanner Guild | Pending design |
|
||||
| Notarization validation | Decide whether scanner performs `spctl --assess` or defers | Security Guild | TBD |
|
||||
| Entitlement taxonomy | Finalize capability grouping for Policy Engine | Policy Guild | TBD |
|
||||
|
||||
## 9. Proposed backlog entries
|
||||
| ID (proposed) | Title | Summary |
|
||||
| --- | --- | --- |
|
||||
| SCANNER-ENG-0020 | Implement Homebrew collector and fragment mapper | Parse cellar manifests, emit brew component records, integrate with Surface.FS/Validation. |
|
||||
| SCANNER-ENG-0021 | Implement pkgutil receipt collector | Parse receipts/BOM, output installer component records with provenance. |
|
||||
| SCANNER-ENG-0022 | Implement macOS bundle inspector & capability overlays | Extract plist/signing/entitlements, produce capability evidence and merge with receipts. |
|
||||
| SCANNER-ENG-0023 | Policy & Offline integration for macOS | Define predicates, CLI toggles, Offline Kit packaging, and documentation. |
|
||||
|
||||
## 10. References
|
||||
- `docs/benchmarks/scanner/deep-dives/macos.md` — competitor snapshot and roadmap summary.
|
||||
- `docs/benchmarks/scanner/windows-macos-demand.md` — demand capture log.
|
||||
- `docs/modules/scanner/design/surface-secrets.md`, `surface-fs.md`, `surface-validation.md` — surface contracts leveraged by collectors.
|
||||
|
||||
Further reading: `../../api/scanner/windows-coverage.md` (summary) and `../../api/scanner/windows-macos-summary.md` (metrics dashboard).
|
||||
|
||||
Policy readiness alignment: see `../policy/secret-leak-detection-readiness.md` (POLICY-READINESS-0001).
|
||||
|
||||
Upcoming milestone: Northwind Health Services demo on 2025-11-10 to validate notarization/entitlement outputs before final policy sign-off.
|
||||
135
docs/modules/scanner/design/windows-analyzer.md
Normal file
135
docs/modules/scanner/design/windows-analyzer.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# Windows Analyzer Design Brief (Draft)
|
||||
|
||||
> Owners: Scanner Guild, Policy Guild, Offline Kit Guild, Security Guild
|
||||
> Related backlog (proposed): SCANNER-ENG-0024..0027, DOCS-SCANNER-BENCH-62-002
|
||||
> Status: Draft — contingent on Windows demand threshold (see `docs/benchmarks/scanner/windows-macos-demand.md`)
|
||||
|
||||
## 1. Objectives & boundaries
|
||||
- Provide deterministic inventory for Windows Server/container images covering MSI/WinSxS assemblies, Chocolatey packages, and registry-derived installers.
|
||||
- Preserve replayability (layer fragments, provenance metadata) and align outputs with existing SBOM/policy pipelines.
|
||||
- Respect sovereignty constraints: offline-friendly, signed rule bundles, no reliance on Windows APIs unavailable in containerized scans.
|
||||
|
||||
Out of scope (Phase 1):
|
||||
- Live registry queries on running Windows hosts (requires runtime agent; defer to Zastava/Runtime roadmap).
|
||||
- Windows Update patch baseline comparison (tracked separately under Runtime/Posture).
|
||||
- UWP/MSIX packages (flagged for follow-up once MSI parity is complete).
|
||||
|
||||
## 2. Architecture overview
|
||||
```
|
||||
Scanner.Worker (Windows profile)
|
||||
├─ Surface.Validation (enforce layer size, path allowlists)
|
||||
├─ Surface.FS (materialized NTFS image via 7z/guestmount)
|
||||
├─ MsiCollector -> LayerComponentFragment (windows-msi)
|
||||
├─ WinSxSCollector -> LayerComponentFragment (windows-winsxs)
|
||||
├─ ChocolateyCollector -> LayerComponentFragment (windows-choco)
|
||||
├─ RegistryCollector -> Evidence overlays (uninstall/services)
|
||||
├─ DriverCapabilityMapper -> Capability overlays (kernel/user drivers)
|
||||
└─ WindowsComponentMapper -> ComponentGraph + capability metadata
|
||||
```
|
||||
|
||||
- Collectors operate on extracted filesystem snapshots; registry access performed on exported hive files produced during image extraction (document in ops runbooks).
|
||||
- `WindowsComponentMapper` normalizes component identities (ProductCode, AssemblyIdentity, Chocolatey package ID) and merges overlapping evidence into deterministic fragments.
|
||||
|
||||
## 3. Collectors
|
||||
### 3.1 MSI collector
|
||||
- Input: `Windows/Installer/*.msi` database files (Jet OLE DB), registry hive exports for product mapping.
|
||||
- Implementation approach:
|
||||
- Use open-source MSI parser (custom or MIT-compatible) to avoid COM dependencies.
|
||||
- Extract Product, Component, File, Feature, Media tables.
|
||||
- Compute SHA256 for installed files via Component table, linking to WinSxS manifests.
|
||||
- Output metadata: `productCode`, `upgradeCode`, `productVersion`, `manufacturer`, `language`, `installContext`, `packageCode`, `sourceList`.
|
||||
- Evidence: file paths with digests, component IDs, CAB/patch references.
|
||||
|
||||
### 3.2 WinSxS collector
|
||||
- Input: `Windows/WinSxS/Manifests/*.manifest`, `Windows/WinSxS/` payload directories, catalog (.cat) files.
|
||||
- Parse XML assembly identities (name, version, processor architecture, public key token, language).
|
||||
- Map to MSI components when file hashes match.
|
||||
- Capture catalog signature thumbprint and optional patch KB references for policy gating.
|
||||
|
||||
### 3.3 Chocolatey collector
|
||||
- Input: `ProgramData/Chocolatey/lib/**`, `ProgramData/Chocolatey/package.backup`, `chocolateyinstall.ps1`, `.nuspec`.
|
||||
- Extract package ID, version, checksum, source feed, installed files and scripts.
|
||||
- Note whether install used cache or remote feed; record script hash for determinism.
|
||||
|
||||
### 3.4 Registry collector
|
||||
- Input: Exported `SOFTWARE` hive covering:
|
||||
- `Microsoft\Windows\CurrentVersion\Uninstall`
|
||||
- `Microsoft\Windows\CurrentVersion\Installer\UserData`
|
||||
- `Microsoft\Windows\CurrentVersion\Run` (startup apps)
|
||||
- Service/driver configuration from `SYSTEM` hive under `Services`.
|
||||
- Emit fallback evidence for installers not captured by MSI/Chocolatey (legacy EXE installers).
|
||||
- Record uninstall strings, install dates, publisher, estimated size, install location.
|
||||
|
||||
### 3.5 Driver & service mapper
|
||||
- Parse `SYSTEM` hive `Services` entries to detect drivers (type=1 or 2) and critical services (start mode auto/boot).
|
||||
- Output capability overlays (e.g., `windows.driver.kernelMode(true)`, `windows.service.autoStart("Spooler")`) for Policy Engine.
|
||||
|
||||
## 4. Component mapping & output
|
||||
- `WindowsComponentMapper`:
|
||||
- Generate `LayerComponentFragment`s with synthetic layer digests (e.g., `sha256:stellaops-windows-msi`).
|
||||
- Build `ComponentIdentity` with PURL-like scheme: `pkg:msi/<productCode>` or `pkg:winsxs/<assemblyIdentity>`.
|
||||
- Include metadata: signature thumbprint, catalog hash, KB references, install context, manufacturer.
|
||||
- Capability overlays stored under `ScanAnalysisKeys.capability.windows` for policy consumption.
|
||||
- Export Center bundling:
|
||||
- Include MSI manifest extracts, WinSxS assembly manifests, Chocolatey nuspec snapshots, and service/driver capability CSV.
|
||||
|
||||
## 5. Policy integration
|
||||
- Predicates to introduce:
|
||||
- `windows.package.signed(expectedThumbprint?)`
|
||||
- `windows.package.unsupportedInstallerType`
|
||||
- `windows.driver.kernelMode`, `windows.driver.unsigned`
|
||||
- `windows.service.autoStart(name)`
|
||||
- `windows.choco.sourceAllowed(feed)`
|
||||
- Lattice approach:
|
||||
- Unsigned kernel drivers → default `fail`.
|
||||
- Unknown installer sources → `warn` with escalation on critical services.
|
||||
- Chocolatey packages from non-whitelisted feeds → configurable severity.
|
||||
- Waiver semantics bind to product code + signature thumbprint; waivers expire when package version changes.
|
||||
|
||||
## 6. Offline kit & distribution
|
||||
- Package:
|
||||
- MSI schema definitions and parser binaries (signed).
|
||||
- Chocolatey feed snapshot (nupkg archives + index) for allow-listed feeds.
|
||||
- Windows catalog certificate chains + optional CRL/OCSP caches.
|
||||
- Documentation:
|
||||
- Provide instructions for exporting registry hives during image extraction (PowerShell script included).
|
||||
- Note disk space expectations (Chocolatey snapshot size, WinSxS manifest volume).
|
||||
|
||||
## 7. Testing strategy
|
||||
- Fixtures:
|
||||
- Sample MSI packages (with/without transforms), WinSxS manifests, Chocolatey packages.
|
||||
- Registry hive exports representing mixed installer types.
|
||||
- Tests:
|
||||
- Unit tests for each collector parsing edge cases (language-specific manifests, transforms, script hashing).
|
||||
- Integration tests using synthetic Windows container image layers (generated via CI on Windows worker).
|
||||
- Determinism checks ensuring repeated runs produce identical fragments.
|
||||
- Security review:
|
||||
- Validate script execution paths (collectors must never execute Chocolatey scripts; inspect only).
|
||||
|
||||
## 8. Dependencies & open questions
|
||||
| Item | Description | Owner | Status |
|
||||
| --- | --- | --- | --- |
|
||||
| MSI parser choice | Select MIT/Apache-compatible parser or build internal reader | Scanner Guild | TBD |
|
||||
| Registry export tooling | Determine standard script/utility for hive exports in container context | Ops Guild | TBD |
|
||||
| Authenticodes verification locus | Decide scanner vs policy responsibility for signature verification | Security Guild | TBD |
|
||||
| Feed mirroring policy | Which Chocolatey feeds to mirror by default | Product + Security Guilds | TBD |
|
||||
|
||||
## 9. Proposed backlog entries
|
||||
| ID (proposed) | Title | Summary |
|
||||
| --- | --- | --- |
|
||||
| SCANNER-ENG-0024 | Implement Windows MSI collector | Parse MSI databases, emit component fragments with provenance metadata. |
|
||||
| SCANNER-ENG-0025 | Implement WinSxS manifest collector | Correlate assemblies with MSI components and catalog signatures. |
|
||||
| SCANNER-ENG-0026 | Implement Chocolatey & registry collectors | Harvest nuspec metadata and uninstall/service registry data. |
|
||||
| SCANNER-ENG-0027 | Policy & Offline integration for Windows | Define predicates, CLI toggles, Offline Kit packaging, documentation. |
|
||||
|
||||
## 10. References
|
||||
- `docs/benchmarks/scanner/deep-dives/windows.md`
|
||||
- `docs/benchmarks/scanner/windows-macos-demand.md`
|
||||
- `docs/modules/scanner/design/macos-analyzer.md` (structure/composition parallels)
|
||||
- Surface design docs (`surface-fs.md`, `surface-validation.md`, `surface-secrets.md`) for interfacing expectations.
|
||||
|
||||
Further reading: `../../api/scanner/windows-coverage.md` (summary) and `../../api/scanner/windows-macos-summary.md` (metrics dashboard).
|
||||
|
||||
Policy readiness alignment: see `../policy/windows-package-readiness.md` (POLICY-READINESS-0002).
|
||||
|
||||
Upcoming milestone: FinSecure Corp PCI review requires Authenticode/feed decision by 2025-11-07 before Windows analyzer spike kickoff.
|
||||
Reference in New Issue
Block a user