up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Findings Ledger CI / build-test (push) Has been cancelled
Findings Ledger CI / migration-validation (push) Has been cancelled
Scanner Analyzers / Discover Analyzers (push) Has been cancelled
Signals Reachability Scoring & Events / reachability-smoke (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
cryptopro-linux-csp / build-and-test (push) Has been cancelled
Scanner Analyzers / Validate Test Fixtures (push) Has been cancelled
Signals CI & Image / signals-ci (push) Has been cancelled
sm-remote-ci / build-and-test (push) Has been cancelled
Findings Ledger CI / generate-manifest (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Scanner Analyzers / Build Analyzers (push) Has been cancelled
Scanner Analyzers / Test Language Analyzers (push) Has been cancelled
Scanner Analyzers / Verify Deterministic Output (push) Has been cancelled
Signals Reachability Scoring & Events / sign-and-upload (push) Has been cancelled

This commit is contained in:
StellaOps Bot
2025-12-09 09:38:09 +02:00
parent bc0762e97d
commit 108d1c64b3
193 changed files with 7265 additions and 13029 deletions

View File

@@ -25,21 +25,26 @@ Unlike Node.js, Bun may store packages entirely under `node_modules/.bun/` with
- Do not filter out hidden directories in container scans
- Verify evidence shows packages from both `node_modules/` and `node_modules/.bun/`
## 3. `bun.lockb` Migration Path
## 3. `bun.lockb` Policy (2025-12-09)
The binary lockfile (`bun.lockb`) format is undocumented and unstable. The analyzer treats it as **unsupported** and emits a remediation finding.
The binary lockfile (`bun.lockb`) remains **unsupported**. We will not parse it and will keep remediation-only handling until Bun publishes a stable, documented format.
**Migration command:**
**Posture:**
- Treat `bun.lockb` as unsupported input; do not attempt best-effort parsing.
- Emit a deterministic remediation finding instructing conversion to text.
- Skip package inventory when only `bun.lockb` is present to avoid nondeterministic/partial results.
**Migration command (required):**
```bash
bun install --save-text-lockfile
```
This generates `bun.lock` (JSONC text format) which the analyzer can parse.
This generates `bun.lock` (JSONC text format) which the analyzer parses.
**WebService response:** When only `bun.lockb` is present:
- The scan completes but reports unsupported status
- Remediation guidance is included in findings
- No package inventory is generated
**WebService response when only `bun.lockb` exists:**
- Scan completes with `unsupported` marker for the package manager.
- Remediation guidance is included in findings.
- No package inventory is generated until `bun.lock` is provided.
## 4. JSONC Lockfile Format

View File

@@ -0,0 +1,46 @@
# Dart & Swift Analyzer Scope Note (2025-12-09)
## Goals
- Define the initial analyzer scope for Dart (pub) and Swift (SwiftPM) with deterministic, offline-friendly behavior.
- Provide fixture/backlog list to unblock readiness tracking and align with Signals/Zastava expectations.
## Dart (pub)
- Inputs: `pubspec.yaml`, `pubspec.lock`, `.dart_tool/package_config.json`, and downloaded packages under `.dart_tool/pub`.
- Outputs:
- Inventory of `pkg:pub/<name>@<version>` with resolved source (hosted/path/git) and sha256 when present in lockfile.
- Dependency edges from `pubspec.lock`; dev dependencies emitted only when `include_dev=true`.
- Analyzer metadata: sdk constraint, null-safety flag, source type per package.
- Determinism:
- Sort packages and edges lexicographically.
- Normalize paths to POSIX; no network calls; rely only on lockfile/package_config on disk.
- Out of scope (v1):
- Flutter build graph, transitive runtime surface, and hosted index downloads.
- Git/path overrides beyond what is listed in lock/package_config.
- Fixtures/backlog:
- Hosted app with `pubspec.lock` and `.dart_tool/package_config.json` (dev deps included).
- Path dependency sample (relative and absolute).
- Git dependency sample with locked commit.
- Missing lockfile case (expect finding + no inventory).
## Swift (SwiftPM)
- Inputs: `Package.swift`, `Package.resolved` (v1/v2), `.build/` manifest cache when present.
- Outputs:
- Inventory of `pkg:swiftpm/<name>@<version>` with checksum from `Package.resolved` when available.
- Dependency edges from `Package.resolved` target graph; emit platforms/arch only when declared.
- Analyzer metadata: Swift tools version, resolution format, mirrors when specified.
- Determinism:
- Do not execute `swift package`; parse manifests/resolved files only.
- Stable ordering by package then target; normalize paths to POSIX.
- Out of scope (v1):
- Xcodeproj resolution, binary target downloads, and build artifacts hashing.
- Conditional target resolution beyond what `Package.resolved` records.
- Fixtures/backlog:
- Single-package app with `Package.resolved` v2 (checksum present).
- Nested target graph with products/targets/flexible platforms.
- Binary target entry (no download; expect metadata-only inventory).
- Missing `Package.resolved` case (emit finding, no inventory).
## Alignment & Next Steps
- Signals/Zastava: confirm package ID naming (`pkg:pub`, `pkg:swiftpm`) and dependency edge semantics for reachability ingestion.
- Add goldens/fixtures under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.DartSwift.Tests/Fixtures/**`.
- Update readiness checkpoints once fixtures and parsers land; current scope note unblocks backlog creation only.

View File

@@ -0,0 +1,40 @@
# Deno Analyzer Scope Note (2025-12-09)
## Goals
- Define deterministic, offline-friendly scope for the Deno analyzer to move readiness from “status mismatch” to planned execution.
- Enumerate fixtures and evidence needed to mark Amber→Green once implemented.
## Inputs
- `deno.json` / `deno.jsonc` (config and import maps).
- `deno.lock` (v2) with integrity hashes.
- Source tree for `import`/`export` graph; `node_modules/` when `npm:` specifiers are used (npm compatibility mode).
- Optional: cache dir (`~/.cache/deno`) when present in extracted images.
## Outputs
- Inventory of modules:
- `pkg:deno/<specifier>@<version>` for remote modules (normalize to URL without fragment).
- `pkg:npm/<name>@<version>` for `npm:` dependencies with lock hash.
- `pkg:file/<path>` for local modules (relative POSIX paths).
- Dependency edges:
- From importer to imported specifier with resolved path/URL.
- Include type (remote/local/npm), integrity (sha256 from lock), and media type when available.
- Metadata:
- Deno version (from lock/config if present).
- Import map path and hash.
- NPM compatibility flag + resolved registry scope when npm used.
## Determinism & Offline
- Never fetch network resources; rely solely on `deno.lock` + on-disk files.
- Normalize paths to POSIX; stable sorting (source path, then target).
- Hashes: prefer lock integrity; otherwise SHA-256 over file bytes for local modules.
## Fixtures / Backlog
1) Remote-only project with `deno.lock` (http imports) and import map.
2) Mixed project using `npm:` specifiers with `node_modules/` present.
3) Local-only project (relative imports) without lockfile → expect finding + no inventory.
4) Image/extracted cache with populated `~/.cache/deno` to verify offline reuse.
## Status & Next Steps
- Implement parser to ingest `deno.lock` v2 and import map; add graph builder over source files.
- Add fixtures under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Deno.Tests/Fixtures/**` with goldens; keep hashes stable.
- Update readiness checkpoints once fixtures land and TRX/binlogs captured.

View File

@@ -0,0 +1,45 @@
# .NET Analyzer Design · 11-001 Entrypoint Resolver (2025-12-09)
## Goals
- Resolve .NET entrypoints deterministically from project/publish artefacts and emit normalized identities (assembly name, MVID, TFM, RID, host kind, publish mode).
- Capture environment profiles (single-file, trimmed, self-contained vs framework-dependent, ALC hints) without executing payloads.
- Produce deterministic evidence aligned to `dotnet-il-metadata.schema.json` for downstream analyzers 11-002..005.
## Inputs
- `*.csproj`/`*.fsproj` metadata (TargetFrameworks, RuntimeIdentifiers, PublishSingleFile/Trim options).
- Publish outputs: apphost (`*.exe`), `*.dll`, `*.deps.json`, `*.runtimeconfig.json`, `*.targets` cache.
- RID graph from SDK (offline snapshot in repo), deterministic time provider.
## Outputs
- `entrypoints[]` records: `assembly`, `mvid`, `tfm`, `rid`, `hostKind` (apphost/framework-dependent/self-contained), `publishMode` (single-file/trimmed), `alcHints` (AssemblyLoadContext names), `probingPaths`, `nativeDeps` (apphost bundles).
- Evidence: `LanguageComponentEvidence` entries per entrypoint with locator = publish path, hash over file bytes for determinism.
- Diagnostics: missing deps/runtimeconfig, mixed RID publish, single-file without extractor support.
## Algorithm (deterministic)
1) Parse project: target frameworks, RIDs, publish flags; normalize to ordered sets.
2) Discover publish artefacts under `bin/<Configuration>/<TFM>/...` and `publish/` folders; prefer `*.runtimeconfig.json` when present.
3) Read `*.deps.json` to extract runtime targets and resolve primary entry assembly; fall back to `apphost` name.
4) Compute MVID from PE header; compute SHA-256 over `*.dll`/`*.exe` bytes; capture file size.
5) Classify host:
- `apphost` present -> `hostKind = apphost`; detect single-file bundle via marker sections.
- Framework-dependent -> `hostKind = framework-dependent`; use `runtimeconfig` probing paths.
6) Infer ALC hints: scan deps for `runtimeconfig.dev.json` probing paths and `additionalProbingPaths`; add known SDK paths.
7) Emit entrypoint with deterministic ordering: sort by assembly name, then RID, then TFM.
## Determinism & Offline
- No network access; relies solely on on-disk project/publish artefacts.
- Stable ordering and casing (`Ordinal` sort), UTC time provider.
- Hashes: SHA-256 over file bytes; no timestamps.
## Test & Fixture Plan
- Existing suite: `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.DotNet.Tests` (now green; TRX at `TestResults/dotnet/dotnet-tests.trx`).
- Fixtures to maintain:
- Framework-dependent app with deps/runtimeconfig.
- Self-contained single-file publish (bundle) with apphost.
- Trimmed publish with ALC hints.
- Multi-RID publish verifying RID selection and deterministic ordering.
- Add new fixtures under `...DotNet.Tests/Fixtures/` when new host kinds are supported; keep hashes stable.
## Next Steps
- Wire readiness checkpoints to mark 11-001 design+tests complete; keep CI runner validation optional (DEVOPS-SCANNER-CI-11-001) for reproducibility.
- Feed outputs into 11-002..005 analyzers once entrypoint metadata is consumed by downstream IL/reflection pipelines.

View File

@@ -0,0 +1,39 @@
# PHP Analyzer Autoload & Restore Design (2025-12-09)
## Goals
- Stabilize PHP analyzer pipeline (SCANNER-ENG-0010 / 27-001) by defining autoload graph handling, composer restore posture, and fixtures.
- Provide deterministic evidence suitable for CI and reachability alignment with Concelier/Signals.
## Inputs
- `composer.json` + `composer.lock`.
- `vendor/composer/*.php` autoload files (`autoload_psr4.php`, `autoload_classmap.php`, `autoload_files.php`, `autoload_static.php`).
- Installed vendor tree under `vendor/`.
- Optional: `composer.phar` version metadata for diagnostics (no execution).
## Outputs
- Package inventory: `pkg:composer/<name>@<version>` with source/dist hashes from lockfile.
- Autoload graph:
- PSR-4/PSR-0 mappings (namespace → path), classmap entries, files includes.
- Emit edges from package → file and namespace → path with deterministic ordering.
- Restore diagnostics:
- Detect missing vendor install vs lockfile drift; emit findings instead of network restore.
- Metadata:
- Composer version (from lock/platform field when present).
- Platform PHP extensions/version constraints.
## Determinism & Offline
- No composer install/updates; read-only parsing of lock/autoload/vendor.
- Stable ordering: sort packages, namespaces, classmap entries, files includes (ordinal, POSIX paths).
- Hashes: use lockfile dist/shasum when present; otherwise SHA-256 over on-disk file bytes for autoloaded files.
## Fixtures / Backlog
1) PSR-4 project with namespaced classes and classmap mix.
2) Project with `autoload_files.php` includes (functions/constants).
3) Lockfile present but vendor missing → expect finding, no inventory.
4) Path repo override + dist hash present.
## Implementation Steps
- Parser for composer.lock (packages + platform reqs) and autoload PHP arrays (psr-4, psr-0, classmap, files).
- Graph builder producing deterministic edges and evidence records.
- Findings for missing vendor, mismatched lock hash, or absent autoload files.
- Tests under `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Php.Tests` with goldens for each fixture; add TRX/binlogs to readiness once stable.

View File

@@ -0,0 +1,37 @@
# Runtime Parity Plan (Java / .NET / PHP) — Scanner Aú · Signals Alignment (2025-12-09)
## Objectives
- Close runtime parity gaps by pairing static analyzer hooks with runtime evidence for Java, .NET, and PHP.
- Produce deterministic artefacts (TRX/binlogs + NDJSON) that Signals can ingest for runtime reconciliation.
## Scope & Hooks
- **Java (21-005..011)**: jar/classpath resolution, `Main-Class`, module-info, shaded jars. Runtime hook: capture resolved classpath + main entry via proc snapshot or launcher args.
- **.NET (11-001..005)**: `.deps.json`, RID-graph, single-file/trimmed detection, `runtimeconfig.json`. Runtime hook: capture host command line + loaded assembly list via Signals proc trace.
- **PHP (27-001)**: composer autoload graph (`vendor/composer/autoload_*.php`), package metadata, runtime entry (fpm/cli). Runtime hook: map autoloaded files to runtime include graph when proc snapshot present.
## Evidence Plan
1) **Static**: ensure analyzers emit deterministically ordered inventories + edges with layer attribution (already enforced across analyzers).
2) **Runtime capture** (requires Signals):
- Provide proc snapshot schema to Scanner (cmdline, env, cwd, loaded modules/files).
- Export runtime observations as NDJSON with stable ordering (path, module, hash).
3) **Reconciliation**:
- Join static entries to runtime observations on normalized path + hash.
- Emit `runtime.match` / `runtime.miss` diagnostics with counts per analyzer.
4) **Artefacts**:
- CI: TRX/binlog per analyzer suite.
- NDJSON samples: runtime reconciliation outputs for each language (hosted under `src/Scanner/__Tests/.../Fixtures/RuntimeParity`).
## Task Backlog
- T1: Wire proc snapshot ingestion for Java/.NET/PHP analyzers (Signals contract).
- T2: Add runtime reconciliation step with deterministic ordering and diagnostics.
- T3: Author runtime fixtures (one per language) and goldens for reconciliation output.
- T4: Document runtime parity expectations in readiness checkpoints and surfaces guides.
## Constraints
- Offline-friendly: no network calls during reconciliation; rely solely on provided proc snapshot.
- Deterministic: stable sort (layer, path, name), UTC timestamps, no random seeds.
- Security: avoid executing payloads; treat proc snapshot as data only.
## Dependencies
- Signals to confirm proc snapshot schema and DSSE/NDJSON event shape for runtime observations.
- Dedicated CI runner (DEVOPS-SCANNER-CI-11-001) to record TRX/binlogs for Java/.NET suites.

View File

@@ -7,13 +7,13 @@
## Phase Readiness
| Phase / Sprint | Status | Evidence | Gaps / Actions |
| --- | --- | --- | --- |
| Phase II · Sprint 0131 (Deno/Java/.NET bootstrap) | Amber/Red | Deno runtime capture shipped and tested; Java chain 21-005..011 blocked on Concelier build + CI runner; .NET Lang 11-001 blocked awaiting clean runner; PHP VFS 27-001 blocked pending bootstrap spec. | Need CI slice (DEVOPS-SCANNER-CI-11-001) for Java/.NET; define PHP bootstrap spec and fixtures to unblock 27-001. |
| Phase II · Sprint 0131 (Deno/Java/.NET bootstrap) | Amber/Red | Deno runtime capture shipped and tested; Java chain 21-005..011 still blocked on runner; .NET Lang 11-001 design/tests completed locally (TRX `TestResults/dotnet/dotnet-tests.trx`, design at `docs/modules/scanner/design/dotnet-analyzer-11-001.md`); PHP VFS 27-001 blocked pending bootstrap spec. | Need CI slice (DEVOPS-SCANNER-CI-11-001) for Java rerun; finalize PHP bootstrap spec and fixtures to unblock 27-001; publish Deno fixtures. |
| Phase III · Sprint 0132 (Native + Node foundations) | Amber | Native analyzers 20-001..010 shipped with tests; Node 22-001..005 shipped; Node isolated/CI tests pending due to build graph bloat; .NET Lang 11-002..005 blocked on upstream design 11-001 outputs. | Trim Node test graph or run on clean runner to record pass; unblock .NET analyzer design to proceed with runtime/export/fixtures. |
| Phase IV · Sprint 0133 (Node bundle/source-map) | Amber | Phase22 bundle/native/WASM observation implemented and fixtures hashed; validation tests pending (SDK resolver cancels build on current runner). | Execute `scripts/run-node-phase22-smoke.sh` on clean runner; capture TRX/binlog to close. |
| Phase V · Sprint 0134 (PHP fixtures/runtime/package) | Green | PHP analyzer fixtures, runtime evidence, and packaging shipped; docs updated. | Keep fixture hashes stable; rerun benchmarks when dependencies change. |
| Phase VI · Sprint 0135 (Python container + Ruby VFS/edges) | Green | Python container/zipapp adapters shipped; Ruby VFS/dependency edges/observations/runtime capture packaged; EntryTrace 18-502/503 delivered. | Maintain determinism; re-run EntryTrace suite in CI. |
| Phase VII · Sprint 0136 (EntryTrace surface/CLI) | Green | EntryTrace phase VII tasks 18-504/505/506 completed; CLI/WebService surfaces show best-terminal metadata and confidence. | Keep NDJSON schema stable; rerun worker payload tests in CI. |
| Sprint 0138 (Ruby parity & future analyzers) | Amber/Red | Ruby parity shipped; Mongo package inventory live. PHP pipeline SCANNER-ENG-0010 blocked on composer/autoload design + restore stability; Deno/Dart/Swift analyzer scopes blocked awaiting design; Kubernetes/VM roadmap pending. | Resolve PHP restore/design, produce Deno/Dart/Swift scopes, schedule Zastava/Runtime alignment. |
| Sprint 0138 (Ruby parity & future analyzers) | Amber/Red | Ruby parity shipped; Mongo package inventory live. PHP pipeline SCANNER-ENG-0010 blocked on composer/autoload design + restore stability (design at `docs/modules/scanner/design/php-autoload-design.md`); Deno scope drafted (`docs/modules/scanner/design/deno-analyzer-scope.md`); Dart/Swift scope drafted (`docs/modules/scanner/design/dart-swift-analyzer-scope.md`); Kubernetes/VM roadmap pending. | Implement PHP autoload parser/fixtures per design; add Deno fixtures and validation evidence; align with Zastava/Runtime and update readiness once fixtures land. |
## Overall
- Green areas: native analyzers, PHP fixtures/runtime packaging, Ruby analyzer, Python container adapters, EntryTrace phases VIVII.
@@ -22,5 +22,7 @@
## Recommended Next Actions
1) Secure clean CI slice for Java/.NET and Node Phase22 smoke tests; store binlogs/TRX.
2) Finalise PHP analyzer design (composer/autoload graph) and stabilise restore pipeline to unblock SCANNER-ENG-0010/27-001.
3) Publish Deno/Dart/Swift analyzer scopes with fixtures to unblock 0138 tasks and roadmap alignment with Zastava/Runtime.
4) Re-run EntryTrace and Native suites in CI to lock deterministic hashes before downstream release.
3) Publish Deno/Dart/Swift analyzer scopes with fixtures to unblock 0138 tasks and roadmap alignment with Zastava/Runtime (scope note added at `docs/modules/scanner/design/dart-swift-analyzer-scope.md`; fixtures pending).
4) Lock bun.lockb posture as remediation-only (doc updated at `docs/modules/scanner/bun-analyzer-gotchas.md`); no parser work planned unless format stabilises.
5) Draft runtime parity plan for Java/.NET/PHP and align with Signals proc snapshot schema (plan at `docs/modules/scanner/design/runtime-parity-plan.md`); add reconciliation fixtures once schema confirmed.
6) Re-run EntryTrace and Native suites in CI to lock deterministic hashes before downstream release.