This commit is contained in:
StellaOps Bot
2025-12-09 00:20:52 +02:00
parent 3d01bf9edc
commit bc0762e97d
261 changed files with 14033 additions and 4427 deletions

View File

@@ -0,0 +1,41 @@
# Dart Analyzer Scope · SCANNER-ENG-0012 (2025-12-08)
## Goals
- Define Dart analyzer for pubspec/pub cache parity with other language analyzers.
- Keep offline-first (no `pub get`), deterministic inventories/graphs, and policy-ready signals.
## Inputs
- `pubspec.yaml` + `pubspec.lock` (dependencies, sources, sdk constraints).
- `.dart_tool/package_config.json` (resolved packages, language version, root URIs).
- AOT artifacts: `*.aot`, `*.snapshot`, `build/` outputs (record presence only).
- Optional Flutter plugins: `ios/`/`android/` platform manifests (metadata only).
## Pipeline (deterministic, offline)
1) **Normalize pubspec/pubspec.lock**:
- Parse lock entries; map sources: `hosted`, `sdk:flutter`, `git`, `path`.
- Emit PURLs (`pkg:pub/<name>@<version>`) with `source` metadata (`hosted.url`, `git.sha`, `path`).
- Enforce sorted components by name.
2) **Package config**:
- Read `.dart_tool/package_config.json`; map package `rootUri`/`packageUri` to build module graph roots.
- Capture `languageVersion` and `generated` timestamp (drop or normalize to `0001-01-01Z` for determinism).
3) **Graph builder**:
- Build dependency edges from `pubspec.lock` -> `package_config` packages; include `sdk:flutter` nodes when present.
- Record `sourceType` (hosted/git/path/sdk) for provenance.
4) **Signals**:
- `dart.sdk` requirement from `environment.sdk`; `flutter` channel/version when present.
- AOT snapshot presence flags (`aot=true`, `snapshot=true`); no binary parsing.
5) **Outputs**:
- Inventory: list of PURLs + source metadata + checksum if provided in lock (hosted `sha256`).
- Graph: edges `(package -> dependency)` sorted.
- Signals: `dart.sdkConstraint`, `flutter.sdk`, `flutter.plugins` (names only), `buildArtifacts` flags.
## Tests & fixtures
- Fixtures under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Dart.Tests/Fixtures/`:
- Hosted-only lockfile, git dependency, path dependency, Flutter project with plugins.
- Determinism tests: stable ordering, normalized timestamps, no network.
- Signal tests: sdk constraint extraction, AOT/snapshot flagging.
## Deliverables
- Design captured here; wire into implementation plan + sprint log.
- Analyzer to live under `StellaOps.Scanner.Analyzers.Lang.Dart` with tests mirroring fixtures.
- Offline posture: never invoke `dart pub`; rely solely on provided lock/config; error clearly when missing lock.

View File

@@ -0,0 +1,44 @@
# Deno Analyzer Scope · SCANNER-ENG-0011 (2025-12-08)
## Goals
- Deliver offline-safe Deno analyzer (lockfile/import graph/runtime signals) that matches Ruby/PHP parity bar.
- Provide deterministic SBOM/inventory outputs and capability signals consumable by Policy/Surface.
## Inputs
- `deno.json` / `deno.jsonc` (tasks, import map refs, npm bridging).
- `deno.lock` v2/v3 (modules, npm section, integrity hashes).
- Optional `import_map.json`; vendor/cache roots (`$DENO_DIR`, `vendor/`).
- CLI flags via Surface.Env: `deno.disable_npm`, `deno.vendor`, `deno.lock_path`, `deno.import_map`.
## Pipeline (deterministic, offline)
1) **Normalize config**: parse `deno.json`/jsonc; resolve `importMap` path; default to repo root import map if present. Sort keys.
2) **Lock resolver**: read `deno.lock`; emit components:
- `npm:` entries → PURL (`pkg:npm/<name>@<version>`) + integrity from `integrity`.
- `specifiers` → source→target map for transitive graph.
- `modules` (remote URLs) → canonical URL + content hash when present; mark `fetchSource: cache`.
3) **Import map & vendor**:
- Apply `imports`/`scopes` to rewrite edges before graph emission.
- If `vendor/` exists, prefer vendored paths; emit `provenance: vendor`.
4) **Graph builder**:
- Build module graph from `specifiers` + import map rewrites; emit edges `(from -> to, kind: import|dynamic|npm)`.
- Recognise `npm:` specifiers; map to npm package node.
- Stable ordering: sort by `from, to`.
5) **Runtime/capability signals**:
- Detect permissions from `tasks` (`--allow-*` flags) and `deno.json` `unstable`/`no-check`.
- Capture `nodeModulesDir` toggle to flag npm bridge.
6) **Outputs**:
- Inventory: npm components + remote module list (`digest`, `source`, `origin`).
- Graph: edges with provenance (`lockfile`, `import_map`, `vendor`).
- Signals: `deno.permissions[]`, `deno.node_compat`, `deno.unstable`.
## Tests & fixtures
- Add fixtures under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Deno.Tests/Fixtures/`:
- lockfile v2 + import map,
- lockfile v3 with npm section,
- vendorized project (`vendor/` present).
- Determinism assertions: sorted edges, stable hash of inventory, no network calls (enforce via stubbed fetcher).
## Deliverables
- Analyzer implementation + tests in `StellaOps.Scanner.Analyzers.Lang.Deno`.
- Doc cross-link to `docs/modules/scanner/implementation_plan.md` and sprint log.
- Offline posture: default `LIVE_FETCH=false` equivalent; rely solely on lock/import map/vendor.

View File

@@ -0,0 +1,42 @@
# Native Reachability Graph Plan (Scanner · Signals Alignment)
## Goals
- Extract native reachability graphs from ELF binaries across layers (stripped and unstripped), emitting:
- Build IDs (`.note.gnu.build-id`) and code IDs per file.
- Symbol digests (purl+symbol) and edges (callgraph) with deterministic ordering.
- Synthetic roots for `_init`, `.init_array`, `.preinit_array`, entry points.
- DSSE graph bundle per layer for Signals ingestion.
- Offline-friendly, deterministic outputs (stable ordering, UTF-8, UTC).
## Inputs
- Layered filesystem with ELF binaries and shared objects.
- Layer metadata: digests from `scanner.rootfs.layers` and `scanner.layer.archives` (when provided).
- Optional runtime proc snapshot for reconciliation (if available via Signals pipeline).
## Approach
- **Discovery**: Walk layer directories; identify ELF binaries (`e_ident`, machine, class). Record per-layer path.
- **Identifiers**: Capture build-id (hash of `.note.gnu.build-id`), fallback to SHA-256 of `.text` when absent; store code-id (PE/ELF-friendly string).
- **Symbols**: Parse `.symtab`/`.dynsym`; compute stable symbol digests (e.g., SHA-256 over symbol bytes + name); include size/address for ordering.
- **Edges**: Build callgraph from relocation/import tables and (when available) `.eh_frame`/`.plt` linkage; emit Unknown edges when target unresolved.
- **Synthetic Roots**: Insert edges from synthetic root nodes (per binary) to `_start`, `_init`, `.init_array` entries.
- **Layer Bundles**: Emit DSSE bundle per layer with edges, symbols, identifiers, and provenance (layer digest, path, sha256).
- **Determinism**: Sort by layer digest, path, symbol name; normalize paths to POSIX separators; timestamps fixed to generation time in UTC ISO-8601.
## Deliverables
- Library: `StellaOps.Scanner.Analyzers.Native` (new) with ELF reader and graph builder.
- Tests: fixtures under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Native.Tests` using stripped/unstripped ELF samples (no network).
- DSSE bundle schema: shared constants/types reused by Signals ingestion.
- Sprint doc links: referenced from `SPRINT_0146_0001_0001_scanner_analyzer_gap_close.md`.
## Task Backlog (initial)
1) Skeleton project `StellaOps.Scanner.Analyzers.Native` + plugin registration for scanner worker.
2) ELF reader: header detection, build-id extraction, code-id calculation, section loader with deterministic sorting.
3) Symbol digests: compute `sha256(name + addr + size + binding)`; emit per-symbol evidence and purl+symbol IDs.
4) Callgraph builder: edges from PLT/relocs/imports; Unknown targets captured; synthetic roots for init arrays.
5) Layer attribution: carry layer digest/source through evidence; emit DSSE bundle per layer with signatures stubbed for now.
6) Tests/fixtures: stripped+unstripped ELF, shared objects, missing build-id, init array edges; golden JSON/NDJSON bundles.
7) Signals alignment: finalize DSSE graph schema and bundle naming; hook into reachability ingestion contract.
## Open Questions
- Final DSSE payload shape (Signals team) — currently assumed `graph.bundle` with edges, symbols, metadata.
- Whether to include debugline info for coverage (could add optional module later).***

View File

@@ -0,0 +1,40 @@
# Runtime Alignment (Scanner ↔ Zastava) · SCANNER-ENG-0014 (2025-12-08)
## Objective
Align Kubernetes/VM target coverage between Scanner and Zastava so runtime signals, job orchestration, and evidence exports stay consistent across clusters and on-prem installs.
## Scope
- Scanner: Worker runtime capture (EntryTrace), Surface.Env/FS detectors, analyzer job manifests, and policy predicates that rely on runtime/container metadata.
- Zastava: runtime observation feeds (system call/ebpf), workload labeling, and admission hooks.
## Alignment Plan
1) **Workload identity contract**
- Standardize labels/annotations for scan jobs and Zastava monitors:
- `stellaops.workload/id`, `tenant`, `project`, `component`, `channel`.
- Container image digest required; tag optional.
- Shared manifest snippet lives in `deploy/helm/stellaops` overlays; reuse in job templates.
2) **Runtime evidence channels**
- Scanner EntryTrace publishes `runtime.events` with fields: `workloadId`, `namespace`, `node`, `edgeType` (syscall/net/fs), `timestamp` (UTC, ISO-8601), `code_id` (when available).
- Zastava observers mirror the same schema on `zastava.runtime.events`; controller stitches by `workloadId` and `imageDigest`.
- Determinism: sort edge batches by `(workloadId, timestamp, edgeType)`.
3) **Kubernetes defaults**
- Namespace allowlist `scanner-runtime`/`zastava-runtime`; service accounts share RBAC for `pods/exec`, `pods/log`, `nodes/proxy` (read-only).
- Feature flags: `scanner.runtime.capture.enabled` (default false), `zastava.attach.enabled` (default false) to keep sealed-mode/offline safe.
4) **VM/bare-metal**
- Use node agent mode: Scanner jobs emit host metadata `hostId`, `osRelease`; Zastava tailers tag events with same ids.
- Shared log shipper config uses file socket paths under `/var/log/stellaops/runtime/*.ndjson`.
5) **Evidence export**
- Export Center receives combined runtime bundle with two streams: `scanner.entrytrace.ndjson`, `zastava.runtime.ndjson`; manifest includes hash of each and workload identity table.
- Offline kit: bundle path `offline/runtime/<runId>/`; deterministic manifests/hashes.
6) **SLOs & alerts**
- Target: runtime event lag < 30s P95; drop rate < 0.5%.
- Alerts wired via Prometheus: `stella_runtime_events_lag_seconds`, `stella_runtime_events_dropped_total`.
## Deliverables
- Update job/observer templates (Helm/Compose) to include shared labels and feature flags.
- Documented schema alignment (this note) referenced from sprint log.
- Tests: determinism checks on merged runtime bundle; label presence asserted in integration harness.
## Next Steps
- Wire labels/flags into `deploy/helm/stellaops` templates and Scanner Worker job manifests.
- Add integration test to ensure EntryTrace and Zastava events with same workload id are coalesced without reordering.

View File

@@ -0,0 +1,42 @@
# SwiftPM Coverage Plan · SCANNER-ENG-0013 (2025-12-08)
## Goals
- Plan Swift Package Manager coverage for Scanner: inventory, dependency graph, xcframework/binary target awareness, runtime hints.
- Keep processing offline and deterministic; no `swift package` execution.
## Inputs
- `Package.swift` (manifest) and `Package.resolved` (v2/v3 lockfile).
- `.build/checkouts/**` (optional for checksum verification only).
- Binary targets: `binaryTarget` entries, xcframeworks under `.xcframework/`.
- Platform hints: `platforms`, `cLanguageStandard`, `cxxLanguageStandard`.
## Pipeline (deterministic, offline)
1) **Resolve lockfile**:
- Parse `Package.resolved`; emit packages with identity, version, repo URL, checksum.
- PURL: `pkg:swift/<identity>@<version>`; include `vcs` metadata (git URL, revision).
- Sort packages by identity.
2) **Manifest signals**:
- Parse `Package.swift` (static parse via tree-sitter Swift or manifest JSON dump if available) to extract:
- products/targets (name, type library/test/executable).
- binary targets (path/url, checksum).
- platform minimum versions.
3) **Graph builder**:
- Edges from targets → dependencies; packages → transitive dependencies from lockfile pins.
- Mark binary targets with `provenance: binary-target` and attach checksum if supplied.
4) **Runtime hints**:
- Collect `unsafeFlags`, linker settings, `swiftSettings`/`cSettings`/`cxxSettings` indicators (e.g., `-enable-library-evolution`).
- Emit `xcframework` presence for Apple platform binaries.
5) **Outputs**:
- Inventory: Swift packages (PURL + checksum/vcs), binary targets (type=binary, checksum/path).
- Graph: package dependency edges; target-to-target edges (optional).
- Signals: platform minimums, binary target flags, unsafe flags presence.
## Tests & fixtures
- Fixtures under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Native.Tests/Fixtures/SwiftPM/`:
- Simple library/executable, binary target with checksum, mixed platform constraints.
- Determinism: stable ordering, normalized checksums, no filesystem time dependency.
## Deliverables
- Implementation to land under `StellaOps.Scanner.Analyzers.Native` (SwiftPM module).
- Documentation cross-link to sprint log and `docs/modules/scanner/implementation_plan.md`.
- Offline posture: never invoke `swift build`; rely solely on `Package.resolved`/manifest; error clearly when lockfile missing.

View File

@@ -0,0 +1,10 @@
{
// Enable IL/dependency edge emission and entrypoint export.
"emitDependencyEdges": true,
"includeEntrypoints": true,
// Optional runtime evidence merge (NDJSON lines with package/target/reason/confidence/source).
// When provided, runtime edges are appended with prefix "edge.runtime".
"runtimeEvidencePath": "runtime-evidence.ndjson",
"runtimeEvidenceConfidence": "medium"
}

View File

@@ -0,0 +1,2 @@
{"package":"stellaops.toolkit","target":"native-lib","reason":"runtime-load","confidence":"medium","source":"trace"}
{"package":"microsoft.extensions.logging","target":"microsoft.extensions.dependencyinjection","reason":"runtime-resolve","confidence":"medium","source":"probe"}