stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 1 Releases Wiki Activity
Files
14617e9c3b6b21ea3e88e50f2e2f8a2333554824
git.stella-ops.org/src/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.md
master 799f787de2 Add Policy DSL Validator, Schema Exporter, and Simulation Smoke tools 
- Implemented PolicyDslValidator with command-line options for strict mode and JSON output.
- Created PolicySchemaExporter to generate JSON schemas for policy-related models.
- Developed PolicySimulationSmoke tool to validate policy simulations against expected outcomes.
- Added project files and necessary dependencies for each tool.
- Ensured proper error handling and usage instructions across tools.
2025-10-27 08:00:11 +02:00

7.6 KiB
Raw Blame History

Python Analyzer Task Flow

Seq ID Status Depends on Description Exit Criteria
1 SCANNER-ANALYZERS-LANG-10-303A DONE (2025-10-21) SCANNER-ANALYZERS-LANG-10-307 STREAM-based parser for *.dist-info (METADATA, WHEEL, entry_points.txt) with normalization + evidence capture. Parser handles CPython 3.8–3.12 metadata variations; fixtures confirm canonical ordering and UTF-8 handling.
2 SCANNER-ANALYZERS-LANG-10-303B DONE (2025-10-21) SCANNER-ANALYZERS-LANG-10-303A RECORD hash verifier with chunked hashing, Zip64 support, and mismatch diagnostics. Verifier processes 5 GB RECORD fixture without allocations >2 MB; mismatches produce deterministic evidence records.
3 SCANNER-ANALYZERS-LANG-10-303C DONE (2025-10-21) SCANNER-ANALYZERS-LANG-10-303B Editable install + pip cache detection; integrate EntryTrace hints for runtime usage flags. Editable installs resolved to source path; usage flags propagated; regression tests cover mixed editable + wheel installs.
4 SCANNER-ANALYZERS-LANG-10-307P DONE (2025-10-23) SCANNER-ANALYZERS-LANG-10-303C Shared helper integration (license metadata, quiet provenance, component merging). Shared helpers reused; analyzer-specific metadata minimal; deterministic merge tests pass.
5 SCANNER-ANALYZERS-LANG-10-308P DONE (2025-10-23) SCANNER-ANALYZERS-LANG-10-307P Golden fixtures + determinism harness for Python analyzer; add benchmark and hash throughput reporting. Fixtures under Fixtures/lang/python/; determinism CI guard; benchmark CSV added with threshold alerts.
6 SCANNER-ANALYZERS-LANG-10-309P DONE (2025-10-23) SCANNER-ANALYZERS-LANG-10-308P Package plug-in (manifest, DI registration) and document Offline Kit bundling of Python stdlib metadata if needed. Manifest copied to plugins/scanner/analyzers/lang/; Worker loads analyzer; Offline Kit doc updated.

Python Entry-Point Analyzer (Sprint 43)

Imposed rule: work of this type or tasks of this type on this component — and everywhere else it should be applied.

ID Status Owner(s) Depends on Description Exit Criteria
SCANNER-ANALYZERS-PYTHON-23-001 TODO Python Analyzer Guild SCANNER-ANALYZERS-LANG-10-309P Build input normalizer & virtual filesystem for wheels, sdists, editable installs, zipapps, site-packages trees, and container roots. Detect Python version targets (pyproject.toml, runtime.txt, Dockerfile) + virtualenv layout deterministically. Normalizer ingests fixtures (venv, wheel, sdist, zipapp, container layer) without extraction; records python_version, root metadata, and namespace resolution hints; determinism harness updated.
SCANNER-ANALYZERS-PYTHON-23-002 TODO Python Analyzer Guild SCANNER-ANALYZERS-PYTHON-23-001 Entrypoint discovery: module __main__, console_scripts entry points, scripts, zipapp main, manage.py/gunicorn/celery patterns. Capture invocation context (module vs package, argv wrappers). Fixtures produce entrypoint list with kind (console, module, package, zipapp, framework) and deterministic ordering; warnings for missing targets recorded.
SCANNER-ANALYZERS-PYTHON-23-003 TODO Python Analyzer Guild SCANNER-ANALYZERS-PYTHON-23-001 Static import graph builder using AST and bytecode fallback. Support import, from ... import, relative imports, importlib.import_module, __import__ with literal args, pkgutil.extend_path. AST scanner emits edges for explicit imports; literal importlib calls covered; unresolved/dynamic patterns yield dynamic-import warnings with candidate prefixes; regression fixtures pass.
SCANNER-ANALYZERS-PYTHON-23-004 TODO Python Analyzer Guild SCANNER-ANALYZERS-PYTHON-23-003 Python resolver engine (importlib semantics) handling namespace packages (PEP 420), package discovery order, .pth files, sys.path composition, zipimport, and site-packages precedence across virtualenv/container roots. Resolver reproduces importlib behaviour on fixture matrix (namespace pkg, zipimport, multi-site-dir); includes explain traces; determinism tests for path ordering succeed.
SCANNER-ANALYZERS-PYTHON-23-005 TODO Python Analyzer Guild SCANNER-ANALYZERS-PYTHON-23-004 Packaging adapters: pip editable (.egg-link), Poetry/Flit layout, Conda prefix, .dist-info/RECORD cross-check, container layer overlays. Adapters resolve editable links, conda pkgs, layered site-packages; edges capture provider path + metadata; warnings emitted for missing RECORD entries.
SCANNER-ANALYZERS-PYTHON-23-006 TODO Python Analyzer Guild SCANNER-ANALYZERS-PYTHON-23-003 Detect native extensions (*.so, *.pyd), CFFI modules, ctypes loaders, embedded WASM, and runtime capability signals (subprocess, multiprocessing, ctypes, eval). Fixtures with native/CFFI/ctypes emit native-extension, cffi, ctypes hints; capability flags recorded; metadata captures ABI/platform info.
SCANNER-ANALYZERS-PYTHON-23-007 TODO Python Analyzer Guild SCANNER-ANALYZERS-PYTHON-23-002 Framework/config heuristics: Django, Flask, FastAPI, Celery, AWS Lambda handlers, Gunicorn, Click/Typer CLIs, logging configs, pyproject optional dependencies. Tagged as hints only. Framework fixtures produce hint records with source files (settings.py, pyproject extras, celery app); no resolver impact; determinism maintained.

Python Observation & Runtime (Sprint 44)

ID Status Owner(s) Depends on Description Exit Criteria
SCANNER-ANALYZERS-PYTHON-23-008 TODO Python Analyzer Guild SCANNER-ANALYZERS-PYTHON-23-004 Produce AOC-compliant observations: entrypoints, components (modules/packages/native), edges (import, namespace, dynamic-hint, native-extension) with reason codes/confidence and resolver traces. Observation JSON for fixtures deterministic; includes explain trace per edge and namespace resolution metadata; passes AOC compliance lint.
SCANNER-ANALYZERS-PYTHON-23-009 TODO Python Analyzer Guild, QA Guild SCANNER-ANALYZERS-PYTHON-23-008 Fixture suite + perf benchmarks covering virtualenv, namespace packages, zipapp, editable installs, containers, lambda handler. Fixture set committed under fixtures/lang/python/ep; determinism CI and perf (<250ms medium project) gates enabled.
SCANNER-ANALYZERS-PYTHON-23-010 TODO Python Analyzer Guild, Signals Guild SCANNER-ANALYZERS-PYTHON-23-008 Optional runtime evidence: import hook capturing module load events with path scrubbing, optional bytecode instrumentation for importlib hooks, multiprocessing tracer. Runtime harness records module loads for sample app; paths hashed; runtime edges merge without altering resolver precedence; privacy doc updated.
SCANNER-ANALYZERS-PYTHON-23-011 TODO Python Analyzer Guild, DevOps Guild SCANNER-ANALYZERS-PYTHON-23-008 Package analyzer plug-in, add CLI commands (`stella python inspect resolve
SCANNER-ANALYZERS-PYTHON-23-012 TODO Python Analyzer Guild SCANNER-ANALYZERS-PYTHON-23-001 Container/zipapp adapter enhancements: parse OCI layers for Python runtime, detect PYTHONPATH/PYTHONHOME env, record warnings for sitecustomize/startup hooks. Container fixtures output runtime metadata (python binary, env vars) and warnings for startup hooks; zipapp fixture resolves internal modules; determinism retained.