up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
Notify Smoke Test / Notify Unit Tests (push) Has been cancelled
Notify Smoke Test / Notifier Service Tests (push) Has been cancelled
Notify Smoke Test / Notification Smoke Test (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Scanner Analyzers / Discover Analyzers (push) Has been cancelled
Scanner Analyzers / Build Analyzers (push) Has been cancelled
Scanner Analyzers / Test Language Analyzers (push) Has been cancelled
Scanner Analyzers / Validate Test Fixtures (push) Has been cancelled
Scanner Analyzers / Verify Deterministic Output (push) Has been cancelled
Signals CI & Image / signals-ci (push) Has been cancelled
Signals Reachability Scoring & Events / reachability-smoke (push) Has been cancelled
Signals Reachability Scoring & Events / sign-and-upload (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
Notify Smoke Test / Notify Unit Tests (push) Has been cancelled
Notify Smoke Test / Notifier Service Tests (push) Has been cancelled
Notify Smoke Test / Notification Smoke Test (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Scanner Analyzers / Discover Analyzers (push) Has been cancelled
Scanner Analyzers / Build Analyzers (push) Has been cancelled
Scanner Analyzers / Test Language Analyzers (push) Has been cancelled
Scanner Analyzers / Validate Test Fixtures (push) Has been cancelled
Scanner Analyzers / Verify Deterministic Output (push) Has been cancelled
Signals CI & Image / signals-ci (push) Has been cancelled
Signals Reachability Scoring & Events / reachability-smoke (push) Has been cancelled
Signals Reachability Scoring & Events / sign-and-upload (push) Has been cancelled
This commit is contained in:
@@ -75,13 +75,25 @@ All endpoints require Authority-issued JWT + DPoP tokens with scopes `export:run
|
||||
| `export_profiles` | Profile definitions (kind, variant, config). | `_id`, `tenant`, `name`, `kind`, `variant`, `config_json`, `created_by`, `created_at`. | Config includes adapter parameters (included record types, compression, encryption). |
|
||||
| `export_runs` | Run state machine and audit info. | `_id`, `profile_id`, `tenant`, `status`, `requested_by`, `selectors`, `policy_snapshot_id`, `started_at`, `completed_at`, `duration_ms`, `error_code`. | Immutable selectors; status transitions recorded in `export_events`. |
|
||||
| `export_inputs` | Resolved input ranges. | `run_id`, `source`, `cursor`, `count`, `hash`. | Enables resumable retries and audit. |
|
||||
| `export_distributions` | Distribution artefacts. | `run_id`, `type` (`http`, `oci`, `object`), `location`, `sha256`, `size_bytes`, `expires_at`. | `expires_at` used for retention policies and automatic pruning. |
|
||||
| `export_events` | Timeline of state transitions and metrics. | `run_id`, `event_type`, `message`, `at`, `metrics`. | Feeds SSE stream and audit trails. |
|
||||
|
||||
## Adapter responsibilities
|
||||
- **JSON (`json:raw`, `json:policy`).**
|
||||
- Ensures canonical casing, timezone normalization, and linkset preservation.
|
||||
- Policy variant embeds policy snapshot metadata (`policy_version`, `inputs_hash`, `decision_trace` fingerprint) and emits evaluated findings as separate files.
|
||||
| `export_distributions` | Distribution artefacts. | `run_id`, `type` (`http`, `oci`, `object`), `location`, `sha256`, `size_bytes`, `expires_at`. | `expires_at` used for retention policies and automatic pruning. |
|
||||
| `export_events` | Timeline of state transitions and metrics. | `run_id`, `event_type`, `message`, `at`, `metrics`. | Feeds SSE stream and audit trails. |
|
||||
|
||||
## Audit bundles (immutable triage exports)
|
||||
|
||||
Audit bundles are a specialized Export Center output: a deterministic, immutable evidence pack for a single subject (and optional time window) suitable for audits and incident response.
|
||||
|
||||
- **Schema**: `docs/schemas/audit-bundle-index.schema.json` (bundle index/manifest with integrity hashes and referenced artefacts).
|
||||
- **Core APIs**:
|
||||
- `POST /v1/audit-bundles` - Create a new bundle (async generation).
|
||||
- `GET /v1/audit-bundles` - List previously created bundles.
|
||||
- `GET /v1/audit-bundles/{bundleId}` - Returns job metadata (`Accept: application/json`) or streams bundle bytes (`Accept: application/octet-stream`).
|
||||
- **Typical contents**: vuln reports, SBOM(s), VEX decisions, policy evaluations, and DSSE attestations, plus an integrity root hash and optional OCI reference.
|
||||
- **Reference**: `docs/product-advisories/archived/27-Nov-2025-superseded/28-Nov-2025 - Vulnerability Triage UX & VEX-First Decisioning.md`.
|
||||
|
||||
## Adapter responsibilities
|
||||
- **JSON (`json:raw`, `json:policy`).**
|
||||
- Ensures canonical casing, timezone normalization, and linkset preservation.
|
||||
- Policy variant embeds policy snapshot metadata (`policy_version`, `inputs_hash`, `decision_trace` fingerprint) and emits evaluated findings as separate files.
|
||||
- Enforces AOC guardrails: no derived modifications to raw evidence fields.
|
||||
- **Trivy (`trivy:db`, `trivy:java-db`).**
|
||||
- Maps StellaOps advisory schema to Trivy DB format, handling namespace collisions and ecosystem-specific ranges.
|
||||
|
||||
@@ -2,13 +2,14 @@
|
||||
|
||||
Scanner analyses container images layer-by-layer, producing deterministic SBOM fragments, diffs, and signed reports.
|
||||
|
||||
## Latest updates (2025-12-03)
|
||||
## Latest updates (2025-12-12)
|
||||
- Deterministic SBOM composition fixture published at `docs/modules/scanner/fixtures/deterministic-compose/` with DSSE, `_composition.json`, BOM, and hashes; doc `deterministic-sbom-compose.md` promoted to Ready v1.0 with offline verification steps.
|
||||
- Node analyzer now ingests npm/yarn/pnpm lockfiles, emitting `DeclaredOnly` components with lock provenance. The CLI companion command `stella node lock-validate` runs the collector offline, surfaces declared-only or missing-lock packages, and emits telemetry via `stellaops.cli.node.lock_validate.count`.
|
||||
- Python analyzer picks up `requirements*.txt`, `Pipfile.lock`, and `poetry.lock`, tagging installed distributions with lock provenance and generating declared-only components for policy. Use `stella python lock-validate` to run the same checks locally before images are built.
|
||||
- Java analyzer now parses `gradle.lockfile`, `gradle/dependency-locks/**/*.lockfile`, and `pom.xml` dependencies via the new `JavaLockFileCollector`, merging lock metadata onto jar evidence and emitting declared-only components when jars are absent. The new CLI verb `stella java lock-validate` reuses that collector offline (table/JSON output) and records `stellaops.cli.java.lock_validate.count{outcome}` for observability.
|
||||
- Worker/WebService now resolve cache roots and feature flags via `StellaOps.Scanner.Surface.Env`; misconfiguration warnings are documented in `docs/modules/scanner/design/surface-env.md` and surfaced through startup validation.
|
||||
- Platform events rollout (2025-10-19) continues to publish scanner.report.ready@1 and scanner.scan.completed@1 envelopes with embedded DSSE payloads (see docs/updates/2025-10-19-scanner-policy.md and docs/updates/2025-10-19-platform-events.md). Service and consumer tests should round-trip the canonical samples under docs/events/samples/.
|
||||
- OS/non-language analyzers: evidence is rootfs-relative, warnings are structured/capped, hashing is bounded, and Linux OS analyzers support surface-cache reuse. See `os-analyzers-evidence.md`.
|
||||
|
||||
## Responsibilities
|
||||
- Expose APIs (WebService) for scan orchestration, diffing, and artifact retrieval.
|
||||
@@ -38,6 +39,7 @@ Scanner analyses container images layer-by-layer, producing deterministic SBOM f
|
||||
- ./operations/entrypoint.md
|
||||
- ./operations/secret-leak-detection.md
|
||||
- ./operations/dsse-rekor-operator-guide.md
|
||||
- ./os-analyzers-evidence.md
|
||||
- ./design/macos-analyzer.md
|
||||
- ./design/windows-analyzer.md
|
||||
- ../benchmarks/scanner/deep-dives/macos.md
|
||||
|
||||
74
docs/modules/scanner/os-analyzers-evidence.md
Normal file
74
docs/modules/scanner/os-analyzers-evidence.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# OS Analyzer Evidence Semantics (Non-Language Scanners)
|
||||
|
||||
This document defines the **evidence contract** produced by OS/non-language analyzers (apk/dpkg/rpm + Windows/macOS OS analyzers) so downstream SBOM/attestation logic can rely on stable, deterministic semantics.
|
||||
|
||||
## Evidence Paths
|
||||
|
||||
- `OSPackageFileEvidence.Path` is **rootfs-relative** and **normalized**:
|
||||
- No leading slash (`/`).
|
||||
- Forward slashes only (`/`), even on Windows inputs.
|
||||
- Never a host path.
|
||||
- Any analyzer-specific absolute path must be converted to rootfs-relative before emission.
|
||||
- Helper: `StellaOps.Scanner.Analyzers.OS.Helpers.OsPath.TryGetRootfsRelative(...)`.
|
||||
|
||||
Examples:
|
||||
|
||||
- Good: `usr/bin/bash`
|
||||
- Bad: `/usr/bin/bash`
|
||||
- Bad: `C:\scans\rootfs\usr\bin\bash`
|
||||
|
||||
## Layer Attribution
|
||||
|
||||
- `OSPackageFileEvidence.LayerDigest` is **best-effort** attribution derived from scan metadata:
|
||||
- `ScanMetadataKeys.LayerDirectories` (optional mapping of layer digest → extracted directory)
|
||||
- `ScanMetadataKeys.CurrentLayerDigest` (fallback/default)
|
||||
- Helper: `StellaOps.Scanner.Analyzers.OS.Helpers.OsFileEvidenceFactory`.
|
||||
|
||||
## Digest & Hashing Strategy
|
||||
|
||||
Default posture is **avoid unbounded hashing**:
|
||||
|
||||
- Prefer package-manager-provided digests when present (`OSPackageFileEvidence.Digests` / `OSPackageFileEvidence.Sha256`).
|
||||
- Compute `sha256` only when:
|
||||
- No digests are present, and
|
||||
- File exists, and
|
||||
- File size is ≤ 16 MiB (`OsFileEvidenceFactory` safeguard).
|
||||
- Primary digest selection for file evidence metadata prefers strongest available:
|
||||
- `sha512` → `sha384` → `sha256` → `sha1` → `md5`
|
||||
|
||||
## Analyzer Warnings
|
||||
|
||||
OS analyzers may emit `AnalyzerWarning` entries (`Code`, `Message`) for partial/edge conditions (missing db, parse errors, unexpected layout).
|
||||
|
||||
Normalization rules (in `OsPackageAnalyzerBase`):
|
||||
|
||||
- Deduplicate by `(Code, Message)`.
|
||||
- Stable sort by `Code` then `Message` (ordinal).
|
||||
- Cap at 50 warnings.
|
||||
|
||||
## OS Analyzer Caching (Surface Cache)
|
||||
|
||||
Linux OS analyzers (apk/dpkg/rpm) support **safe, deterministic reuse** via `ISurfaceCache`:
|
||||
|
||||
- Cache key: `(tenant, analyzerId, rootfsFingerprint)` under namespace `scanner/os/analyzers`.
|
||||
- Fingerprint inputs are intentionally narrow: a single **analyzer-specific** “DB fingerprint file”:
|
||||
- `apk`: `lib/apk/db/installed`
|
||||
- `dpkg`: `var/lib/dpkg/status`
|
||||
- `rpm`: `var/lib/rpm/rpmdb.sqlite` (preferred) or legacy `Packages` fallback
|
||||
- Fingerprint payload includes:
|
||||
- Root path + analyzerId
|
||||
- Relative fingerprint file path
|
||||
- File length + `LastWriteTimeUtc` (ms)
|
||||
- Optional file-content sha256 when the file is ≤ 8 MiB
|
||||
|
||||
Worker wiring:
|
||||
|
||||
- `StellaOps.Scanner.Worker.Processing.CompositeScanAnalyzerDispatcher` records cache hit/miss counters per analyzer.
|
||||
|
||||
## RPM sqlite Reader Notes
|
||||
|
||||
When `rpmdb.sqlite` is present, the reader avoids `SELECT *` and column scanning:
|
||||
|
||||
- Uses `PRAGMA table_info(Packages)` to select a likely RPM header blob column (prefers `hdr`/`header`, excludes `pkgId` when possible).
|
||||
- Queries only `pkgKey` + header blob column for parsing.
|
||||
|
||||
@@ -44,8 +44,9 @@
|
||||
├─ scans/ # scan list, detail, SBOM viewer, diff-by-layer, EntryTrace
|
||||
├─ runtime/ # Zastava posture, drift events, admission decisions
|
||||
├─ policy/ # rules editor (YAML/Rego), exemptions, previews
|
||||
├─ vex/ # VEX explorer (claims, consensus, conflicts)
|
||||
├─ concelier/ # source health, export cursors, rebuild/export triggers
|
||||
├─ vex/ # VEX explorer (claims, consensus, conflicts)
|
||||
├─ triage/ # vulnerability triage (artifact-first), VEX decisions, audit bundles
|
||||
├─ concelier/ # source health, export cursors, rebuild/export triggers
|
||||
├─ attest/ # attestation proofs, verification bundles, Rekor links
|
||||
├─ admin/ # tenants, roles, clients, quotas, licensing posture
|
||||
└─ plugins/ # route plug-ins (lazy remote modules, governed)
|
||||
@@ -106,14 +107,23 @@ Each feature folder builds as a **standalone route** (lazy loaded). All HTTP sha
|
||||
* **Proofs list**: last 7 days Rekor entries; filter by kind (sbom/report/vex).
|
||||
* **Verification**: paste UUID or upload bundle → verify; result with explanations (chain, Merkle path).
|
||||
|
||||
### 3.8 Admin
|
||||
|
||||
* **Tenants/Installations**: view/edit, isolation hints.
|
||||
* **Clients & roles**: Authority clients, role→scope mapping, rotation hints.
|
||||
* **Quotas**: per license plan, counters, throttle events.
|
||||
* **Licensing posture**: last PoE introspection snapshot (redacted), release window.
|
||||
|
||||
---
|
||||
### 3.8 Admin
|
||||
|
||||
* **Tenants/Installations**: view/edit, isolation hints.
|
||||
* **Clients & roles**: Authority clients, role→scope mapping, rotation hints.
|
||||
* **Quotas**: per license plan, counters, throttle events.
|
||||
* **Licensing posture**: last PoE introspection snapshot (redacted), release window.
|
||||
|
||||
### 3.9 Vulnerability triage (VEX-first)
|
||||
|
||||
* **Routes**: `/triage/artifacts`, `/triage/artifacts/:artifactId`, `/triage/audit-bundles`, `/triage/audit-bundles/new`.
|
||||
* **Workspace**: artifact-first split layout (finding cards on the left; explainability tabs on the right: Overview, Reachability, Policy, Attestations).
|
||||
* **VEX decisions**: evidence-first VEX modal with scope + validity + evidence links; bulk apply supported; uses `/v1/vex-decisions`.
|
||||
* **Audit bundles**: "Create immutable audit bundle" UX to build and download an evidence pack; uses `/v1/audit-bundles`.
|
||||
* **Schemas**: `docs/schemas/vex-decision.schema.json`, `docs/schemas/attestation-vuln-scan.schema.json`, `docs/schemas/audit-bundle-index.schema.json`.
|
||||
* **Reference**: `docs/product-advisories/archived/27-Nov-2025-superseded/28-Nov-2025 - Vulnerability Triage UX & VEX-First Decisioning.md`.
|
||||
|
||||
---
|
||||
|
||||
## 4) Auth, sessions & RBAC
|
||||
|
||||
|
||||
@@ -79,7 +79,7 @@ CLI mirrors these endpoints (`stella findings list|view|update|export`). Console
|
||||
|
||||
## 8) VEX-First Triage UX
|
||||
|
||||
> Reference: Product advisory `28-Nov-2025 - Vulnerability Triage UX & VEX-First Decisioning.md`
|
||||
> Reference: Product advisory `docs/product-advisories/archived/27-Nov-2025-superseded/28-Nov-2025 - Vulnerability Triage UX & VEX-First Decisioning.md`
|
||||
|
||||
### 8.1 Evidence-First Finding Cards
|
||||
|
||||
@@ -175,6 +175,8 @@ Immutable audit bundles follow the `AuditBundleIndex` schema (`docs/schemas/audi
|
||||
- `GET /v1/audit-bundles/{bundleId}` - Download bundle (ZIP or OCI)
|
||||
- `GET /v1/audit-bundles` - List previously created bundles
|
||||
|
||||
`GET /v1/audit-bundles/{bundleId}` may use content negotiation: `Accept: application/json` returns job metadata; `Accept: application/octet-stream` streams bundle bytes.
|
||||
|
||||
### 8.6 Industry Pattern Alignment
|
||||
|
||||
The triage UX aligns with industry patterns from:
|
||||
|
||||
Reference in New Issue
Block a user