up
This commit is contained in:
41
docs/modules/scanner/design/dart-analyzer-plan.md
Normal file
41
docs/modules/scanner/design/dart-analyzer-plan.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Dart Analyzer Scope · SCANNER-ENG-0012 (2025-12-08)
|
||||
|
||||
## Goals
|
||||
- Define Dart analyzer for pubspec/pub cache parity with other language analyzers.
|
||||
- Keep offline-first (no `pub get`), deterministic inventories/graphs, and policy-ready signals.
|
||||
|
||||
## Inputs
|
||||
- `pubspec.yaml` + `pubspec.lock` (dependencies, sources, sdk constraints).
|
||||
- `.dart_tool/package_config.json` (resolved packages, language version, root URIs).
|
||||
- AOT artifacts: `*.aot`, `*.snapshot`, `build/` outputs (record presence only).
|
||||
- Optional Flutter plugins: `ios/`/`android/` platform manifests (metadata only).
|
||||
|
||||
## Pipeline (deterministic, offline)
|
||||
1) **Normalize pubspec/pubspec.lock**:
|
||||
- Parse lock entries; map sources: `hosted`, `sdk:flutter`, `git`, `path`.
|
||||
- Emit PURLs (`pkg:pub/<name>@<version>`) with `source` metadata (`hosted.url`, `git.sha`, `path`).
|
||||
- Enforce sorted components by name.
|
||||
2) **Package config**:
|
||||
- Read `.dart_tool/package_config.json`; map package `rootUri`/`packageUri` to build module graph roots.
|
||||
- Capture `languageVersion` and `generated` timestamp (drop or normalize to `0001-01-01Z` for determinism).
|
||||
3) **Graph builder**:
|
||||
- Build dependency edges from `pubspec.lock` -> `package_config` packages; include `sdk:flutter` nodes when present.
|
||||
- Record `sourceType` (hosted/git/path/sdk) for provenance.
|
||||
4) **Signals**:
|
||||
- `dart.sdk` requirement from `environment.sdk`; `flutter` channel/version when present.
|
||||
- AOT snapshot presence flags (`aot=true`, `snapshot=true`); no binary parsing.
|
||||
5) **Outputs**:
|
||||
- Inventory: list of PURLs + source metadata + checksum if provided in lock (hosted `sha256`).
|
||||
- Graph: edges `(package -> dependency)` sorted.
|
||||
- Signals: `dart.sdkConstraint`, `flutter.sdk`, `flutter.plugins` (names only), `buildArtifacts` flags.
|
||||
|
||||
## Tests & fixtures
|
||||
- Fixtures under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Dart.Tests/Fixtures/`:
|
||||
- Hosted-only lockfile, git dependency, path dependency, Flutter project with plugins.
|
||||
- Determinism tests: stable ordering, normalized timestamps, no network.
|
||||
- Signal tests: sdk constraint extraction, AOT/snapshot flagging.
|
||||
|
||||
## Deliverables
|
||||
- Design captured here; wire into implementation plan + sprint log.
|
||||
- Analyzer to live under `StellaOps.Scanner.Analyzers.Lang.Dart` with tests mirroring fixtures.
|
||||
- Offline posture: never invoke `dart pub`; rely solely on provided lock/config; error clearly when missing lock.
|
||||
44
docs/modules/scanner/design/deno-analyzer-plan.md
Normal file
44
docs/modules/scanner/design/deno-analyzer-plan.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# Deno Analyzer Scope · SCANNER-ENG-0011 (2025-12-08)
|
||||
|
||||
## Goals
|
||||
- Deliver offline-safe Deno analyzer (lockfile/import graph/runtime signals) that matches Ruby/PHP parity bar.
|
||||
- Provide deterministic SBOM/inventory outputs and capability signals consumable by Policy/Surface.
|
||||
|
||||
## Inputs
|
||||
- `deno.json` / `deno.jsonc` (tasks, import map refs, npm bridging).
|
||||
- `deno.lock` v2/v3 (modules, npm section, integrity hashes).
|
||||
- Optional `import_map.json`; vendor/cache roots (`$DENO_DIR`, `vendor/`).
|
||||
- CLI flags via Surface.Env: `deno.disable_npm`, `deno.vendor`, `deno.lock_path`, `deno.import_map`.
|
||||
|
||||
## Pipeline (deterministic, offline)
|
||||
1) **Normalize config**: parse `deno.json`/jsonc; resolve `importMap` path; default to repo root import map if present. Sort keys.
|
||||
2) **Lock resolver**: read `deno.lock`; emit components:
|
||||
- `npm:` entries → PURL (`pkg:npm/<name>@<version>`) + integrity from `integrity`.
|
||||
- `specifiers` → source→target map for transitive graph.
|
||||
- `modules` (remote URLs) → canonical URL + content hash when present; mark `fetchSource: cache`.
|
||||
3) **Import map & vendor**:
|
||||
- Apply `imports`/`scopes` to rewrite edges before graph emission.
|
||||
- If `vendor/` exists, prefer vendored paths; emit `provenance: vendor`.
|
||||
4) **Graph builder**:
|
||||
- Build module graph from `specifiers` + import map rewrites; emit edges `(from -> to, kind: import|dynamic|npm)`.
|
||||
- Recognise `npm:` specifiers; map to npm package node.
|
||||
- Stable ordering: sort by `from, to`.
|
||||
5) **Runtime/capability signals**:
|
||||
- Detect permissions from `tasks` (`--allow-*` flags) and `deno.json` `unstable`/`no-check`.
|
||||
- Capture `nodeModulesDir` toggle to flag npm bridge.
|
||||
6) **Outputs**:
|
||||
- Inventory: npm components + remote module list (`digest`, `source`, `origin`).
|
||||
- Graph: edges with provenance (`lockfile`, `import_map`, `vendor`).
|
||||
- Signals: `deno.permissions[]`, `deno.node_compat`, `deno.unstable`.
|
||||
|
||||
## Tests & fixtures
|
||||
- Add fixtures under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Deno.Tests/Fixtures/`:
|
||||
- lockfile v2 + import map,
|
||||
- lockfile v3 with npm section,
|
||||
- vendorized project (`vendor/` present).
|
||||
- Determinism assertions: sorted edges, stable hash of inventory, no network calls (enforce via stubbed fetcher).
|
||||
|
||||
## Deliverables
|
||||
- Analyzer implementation + tests in `StellaOps.Scanner.Analyzers.Lang.Deno`.
|
||||
- Doc cross-link to `docs/modules/scanner/implementation_plan.md` and sprint log.
|
||||
- Offline posture: default `LIVE_FETCH=false` equivalent; rely solely on lock/import map/vendor.
|
||||
42
docs/modules/scanner/design/native-reachability-plan.md
Normal file
42
docs/modules/scanner/design/native-reachability-plan.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Native Reachability Graph Plan (Scanner · Signals Alignment)
|
||||
|
||||
## Goals
|
||||
- Extract native reachability graphs from ELF binaries across layers (stripped and unstripped), emitting:
|
||||
- Build IDs (`.note.gnu.build-id`) and code IDs per file.
|
||||
- Symbol digests (purl+symbol) and edges (callgraph) with deterministic ordering.
|
||||
- Synthetic roots for `_init`, `.init_array`, `.preinit_array`, entry points.
|
||||
- DSSE graph bundle per layer for Signals ingestion.
|
||||
- Offline-friendly, deterministic outputs (stable ordering, UTF-8, UTC).
|
||||
|
||||
## Inputs
|
||||
- Layered filesystem with ELF binaries and shared objects.
|
||||
- Layer metadata: digests from `scanner.rootfs.layers` and `scanner.layer.archives` (when provided).
|
||||
- Optional runtime proc snapshot for reconciliation (if available via Signals pipeline).
|
||||
|
||||
## Approach
|
||||
- **Discovery**: Walk layer directories; identify ELF binaries (`e_ident`, machine, class). Record per-layer path.
|
||||
- **Identifiers**: Capture build-id (hash of `.note.gnu.build-id`), fallback to SHA-256 of `.text` when absent; store code-id (PE/ELF-friendly string).
|
||||
- **Symbols**: Parse `.symtab`/`.dynsym`; compute stable symbol digests (e.g., SHA-256 over symbol bytes + name); include size/address for ordering.
|
||||
- **Edges**: Build callgraph from relocation/import tables and (when available) `.eh_frame`/`.plt` linkage; emit Unknown edges when target unresolved.
|
||||
- **Synthetic Roots**: Insert edges from synthetic root nodes (per binary) to `_start`, `_init`, `.init_array` entries.
|
||||
- **Layer Bundles**: Emit DSSE bundle per layer with edges, symbols, identifiers, and provenance (layer digest, path, sha256).
|
||||
- **Determinism**: Sort by layer digest, path, symbol name; normalize paths to POSIX separators; timestamps fixed to generation time in UTC ISO-8601.
|
||||
|
||||
## Deliverables
|
||||
- Library: `StellaOps.Scanner.Analyzers.Native` (new) with ELF reader and graph builder.
|
||||
- Tests: fixtures under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Native.Tests` using stripped/unstripped ELF samples (no network).
|
||||
- DSSE bundle schema: shared constants/types reused by Signals ingestion.
|
||||
- Sprint doc links: referenced from `SPRINT_0146_0001_0001_scanner_analyzer_gap_close.md`.
|
||||
|
||||
## Task Backlog (initial)
|
||||
1) Skeleton project `StellaOps.Scanner.Analyzers.Native` + plugin registration for scanner worker.
|
||||
2) ELF reader: header detection, build-id extraction, code-id calculation, section loader with deterministic sorting.
|
||||
3) Symbol digests: compute `sha256(name + addr + size + binding)`; emit per-symbol evidence and purl+symbol IDs.
|
||||
4) Callgraph builder: edges from PLT/relocs/imports; Unknown targets captured; synthetic roots for init arrays.
|
||||
5) Layer attribution: carry layer digest/source through evidence; emit DSSE bundle per layer with signatures stubbed for now.
|
||||
6) Tests/fixtures: stripped+unstripped ELF, shared objects, missing build-id, init array edges; golden JSON/NDJSON bundles.
|
||||
7) Signals alignment: finalize DSSE graph schema and bundle naming; hook into reachability ingestion contract.
|
||||
|
||||
## Open Questions
|
||||
- Final DSSE payload shape (Signals team) — currently assumed `graph.bundle` with edges, symbols, metadata.
|
||||
- Whether to include debugline info for coverage (could add optional module later).***
|
||||
@@ -0,0 +1,40 @@
|
||||
# Runtime Alignment (Scanner ↔ Zastava) · SCANNER-ENG-0014 (2025-12-08)
|
||||
|
||||
## Objective
|
||||
Align Kubernetes/VM target coverage between Scanner and Zastava so runtime signals, job orchestration, and evidence exports stay consistent across clusters and on-prem installs.
|
||||
|
||||
## Scope
|
||||
- Scanner: Worker runtime capture (EntryTrace), Surface.Env/FS detectors, analyzer job manifests, and policy predicates that rely on runtime/container metadata.
|
||||
- Zastava: runtime observation feeds (system call/ebpf), workload labeling, and admission hooks.
|
||||
|
||||
## Alignment Plan
|
||||
1) **Workload identity contract**
|
||||
- Standardize labels/annotations for scan jobs and Zastava monitors:
|
||||
- `stellaops.workload/id`, `tenant`, `project`, `component`, `channel`.
|
||||
- Container image digest required; tag optional.
|
||||
- Shared manifest snippet lives in `deploy/helm/stellaops` overlays; reuse in job templates.
|
||||
2) **Runtime evidence channels**
|
||||
- Scanner EntryTrace publishes `runtime.events` with fields: `workloadId`, `namespace`, `node`, `edgeType` (syscall/net/fs), `timestamp` (UTC, ISO-8601), `code_id` (when available).
|
||||
- Zastava observers mirror the same schema on `zastava.runtime.events`; controller stitches by `workloadId` and `imageDigest`.
|
||||
- Determinism: sort edge batches by `(workloadId, timestamp, edgeType)`.
|
||||
3) **Kubernetes defaults**
|
||||
- Namespace allowlist `scanner-runtime`/`zastava-runtime`; service accounts share RBAC for `pods/exec`, `pods/log`, `nodes/proxy` (read-only).
|
||||
- Feature flags: `scanner.runtime.capture.enabled` (default false), `zastava.attach.enabled` (default false) to keep sealed-mode/offline safe.
|
||||
4) **VM/bare-metal**
|
||||
- Use node agent mode: Scanner jobs emit host metadata `hostId`, `osRelease`; Zastava tailers tag events with same ids.
|
||||
- Shared log shipper config uses file socket paths under `/var/log/stellaops/runtime/*.ndjson`.
|
||||
5) **Evidence export**
|
||||
- Export Center receives combined runtime bundle with two streams: `scanner.entrytrace.ndjson`, `zastava.runtime.ndjson`; manifest includes hash of each and workload identity table.
|
||||
- Offline kit: bundle path `offline/runtime/<runId>/`; deterministic manifests/hashes.
|
||||
6) **SLOs & alerts**
|
||||
- Target: runtime event lag < 30s P95; drop rate < 0.5%.
|
||||
- Alerts wired via Prometheus: `stella_runtime_events_lag_seconds`, `stella_runtime_events_dropped_total`.
|
||||
|
||||
## Deliverables
|
||||
- Update job/observer templates (Helm/Compose) to include shared labels and feature flags.
|
||||
- Documented schema alignment (this note) referenced from sprint log.
|
||||
- Tests: determinism checks on merged runtime bundle; label presence asserted in integration harness.
|
||||
|
||||
## Next Steps
|
||||
- Wire labels/flags into `deploy/helm/stellaops` templates and Scanner Worker job manifests.
|
||||
- Add integration test to ensure EntryTrace and Zastava events with same workload id are coalesced without reordering.
|
||||
42
docs/modules/scanner/design/swiftpm-coverage-plan.md
Normal file
42
docs/modules/scanner/design/swiftpm-coverage-plan.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# SwiftPM Coverage Plan · SCANNER-ENG-0013 (2025-12-08)
|
||||
|
||||
## Goals
|
||||
- Plan Swift Package Manager coverage for Scanner: inventory, dependency graph, xcframework/binary target awareness, runtime hints.
|
||||
- Keep processing offline and deterministic; no `swift package` execution.
|
||||
|
||||
## Inputs
|
||||
- `Package.swift` (manifest) and `Package.resolved` (v2/v3 lockfile).
|
||||
- `.build/checkouts/**` (optional for checksum verification only).
|
||||
- Binary targets: `binaryTarget` entries, xcframeworks under `.xcframework/`.
|
||||
- Platform hints: `platforms`, `cLanguageStandard`, `cxxLanguageStandard`.
|
||||
|
||||
## Pipeline (deterministic, offline)
|
||||
1) **Resolve lockfile**:
|
||||
- Parse `Package.resolved`; emit packages with identity, version, repo URL, checksum.
|
||||
- PURL: `pkg:swift/<identity>@<version>`; include `vcs` metadata (git URL, revision).
|
||||
- Sort packages by identity.
|
||||
2) **Manifest signals**:
|
||||
- Parse `Package.swift` (static parse via tree-sitter Swift or manifest JSON dump if available) to extract:
|
||||
- products/targets (name, type library/test/executable).
|
||||
- binary targets (path/url, checksum).
|
||||
- platform minimum versions.
|
||||
3) **Graph builder**:
|
||||
- Edges from targets → dependencies; packages → transitive dependencies from lockfile pins.
|
||||
- Mark binary targets with `provenance: binary-target` and attach checksum if supplied.
|
||||
4) **Runtime hints**:
|
||||
- Collect `unsafeFlags`, linker settings, `swiftSettings`/`cSettings`/`cxxSettings` indicators (e.g., `-enable-library-evolution`).
|
||||
- Emit `xcframework` presence for Apple platform binaries.
|
||||
5) **Outputs**:
|
||||
- Inventory: Swift packages (PURL + checksum/vcs), binary targets (type=binary, checksum/path).
|
||||
- Graph: package dependency edges; target-to-target edges (optional).
|
||||
- Signals: platform minimums, binary target flags, unsafe flags presence.
|
||||
|
||||
## Tests & fixtures
|
||||
- Fixtures under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Native.Tests/Fixtures/SwiftPM/`:
|
||||
- Simple library/executable, binary target with checksum, mixed platform constraints.
|
||||
- Determinism: stable ordering, normalized checksums, no filesystem time dependency.
|
||||
|
||||
## Deliverables
|
||||
- Implementation to land under `StellaOps.Scanner.Analyzers.Native` (SwiftPM module).
|
||||
- Documentation cross-link to sprint log and `docs/modules/scanner/implementation_plan.md`.
|
||||
- Offline posture: never invoke `swift build`; rely solely on `Package.resolved`/manifest; error clearly when lockfile missing.
|
||||
10
docs/modules/scanner/dotnet-il.config.example.json
Normal file
10
docs/modules/scanner/dotnet-il.config.example.json
Normal file
@@ -0,0 +1,10 @@
|
||||
{
|
||||
// Enable IL/dependency edge emission and entrypoint export.
|
||||
"emitDependencyEdges": true,
|
||||
"includeEntrypoints": true,
|
||||
|
||||
// Optional runtime evidence merge (NDJSON lines with package/target/reason/confidence/source).
|
||||
// When provided, runtime edges are appended with prefix "edge.runtime".
|
||||
"runtimeEvidencePath": "runtime-evidence.ndjson",
|
||||
"runtimeEvidenceConfidence": "medium"
|
||||
}
|
||||
2
docs/modules/scanner/runtime-evidence.example.ndjson
Normal file
2
docs/modules/scanner/runtime-evidence.example.ndjson
Normal file
@@ -0,0 +1,2 @@
|
||||
{"package":"stellaops.toolkit","target":"native-lib","reason":"runtime-load","confidence":"medium","source":"trace"}
|
||||
{"package":"microsoft.extensions.logging","target":"microsoft.extensions.dependencyinjection","reason":"runtime-resolve","confidence":"medium","source":"probe"}
|
||||
Reference in New Issue
Block a user