up
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
This commit is contained in:
StellaOps Bot
@@ -41,6 +41,48 @@ completely isolated network:
|
||||
|
||||
The PHP analyzer parses `composer.lock` for Composer dependencies and supports optional runtime evidence via the `stella-trace.php` shim; set `STELLA_PHP_OPCACHE=1` to enable opcache statistics collection.
|
||||
|
||||
**Python analyzer features:**
|
||||
- **Wheel/sdist/editable** parsing with dependency edges from `METADATA`, `PKG-INFO`, `requirements.txt`, and `pyproject.toml`
|
||||
- **Virtual environment** support for virtualenv, venv, and conda prefix layouts
|
||||
- **PEP 420 namespace packages** with proper `importlib` resolution semantics across `sys.path`
|
||||
- **Python version detection** via `pyproject.toml`, `runtime.txt`, Dockerfile `FROM python:*`, `.python-version`
|
||||
- **Native extension detection** for `.so`, `.pyd`, CFFI modules, ctypes loaders, and embedded WASM
|
||||
- **Framework/config heuristics** for Django, Flask, FastAPI, Celery, AWS Lambda, Gunicorn, Click/Typer CLIs
|
||||
- **AOC-compliant observations**: entrypoints (module `__main__`, console_scripts, zipapp), components (modules/packages/native), edges (import, namespace, dynamic-hint, native-extension) with resolver traces
|
||||
- **Optional runtime evidence** via import hook; the bundled `stellaops_trace.py` module captures module load events with SHA-256 path hashing for secure evidence correlation
|
||||
- **CLI inspection**: run `stella python inspect --root /path/to/app` to analyze a Python workspace locally
|
||||
|
||||
**Surface.Env configuration:** Scanner Worker and WebService components use the Surface.Env library for configuration discovery. In air-gapped deployments, configure the following environment variables (see `docs/modules/scanner/design/surface-env.md` for details):
|
||||
|
||||
| Variable | Description | Air-gap Default |
|
||||
|----------|-------------|-----------------|
|
||||
| `SCANNER_SURFACE_FS_ENDPOINT` | Base URI for Surface.FS / RustFS storage | `http://rustfs:8080` |
|
||||
| `SCANNER_SURFACE_FS_BUCKET` | Bucket for manifests/artefacts | `surface-cache` |
|
||||
| `SCANNER_SURFACE_CACHE_ROOT` | Local cache directory | `/var/lib/stellaops/surface` |
|
||||
| `SCANNER_SURFACE_CACHE_QUOTA_MB` | Cache quota in MB (64-262144) | `4096` |
|
||||
| `SCANNER_SURFACE_PREFETCH_ENABLED` | Enable manifest prefetch | `false` |
|
||||
| `SCANNER_SURFACE_TENANT` | Tenant namespace | `default` |
|
||||
| `SCANNER_SURFACE_SECRETS_PROVIDER` | Secrets provider (`file`, `kubernetes`) | `file` |
|
||||
| `SCANNER_SURFACE_SECRETS_ROOT` | Root path for file provider | `/etc/stellaops/secrets` |
|
||||
| `SCANNER_SURFACE_SECRETS_ALLOW_INLINE` | Allow inline secrets | `false` |
|
||||
|
||||
For Helm deployments, configure via `values.yaml`:
|
||||
```yaml
|
||||
surface:
|
||||
fs:
|
||||
endpoint: "http://rustfs:8080"
|
||||
bucket: "surface-cache"
|
||||
cache:
|
||||
root: "/var/lib/stellaops/surface"
|
||||
quotaMb: 4096
|
||||
tenant: "default"
|
||||
secrets:
|
||||
provider: "file"
|
||||
root: "/etc/stellaops/secrets"
|
||||
```
|
||||
|
||||
For Docker Compose, these variables are pre-configured in `docker-compose.airgap.yaml` with sensible defaults.
|
||||
|
||||
**Advisory AI volume primer:** ship a tarball containing empty `queue/`, `plans/`, and `outputs/` directories plus their ownership metadata. During import, extract it onto the RWX volume used by `advisory-ai-web` and `advisory-ai-worker` so pods start with the expected directory tree even on air-gapped nodes.
|
||||
|
||||
*Scanner core:* C# 12 on **.NET {{ dotnet }}**.
|
||||
|
||||
@@ -34,8 +34,8 @@
|
||||
| 8 | CONCELIER-LNM-21-004 | DONE (2025-11-27) | Completed: AOC write guards + tests + docs | Concelier Core Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Remove legacy merge/dedup logic; add guardrails/tests to keep ingestion append-only; document linkset supersession. |
|
||||
| 9 | CONCELIER-LNM-21-005 | DONE (2025-11-27) | Completed: Event contract + publisher interfaces + tests + docs | Concelier Core Guild · Platform Events Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Emit `advisory.linkset.updated` events with delta descriptions + observation ids (tenant + provenance only). |
|
||||
| 10 | CONCELIER-LNM-21-101-DEV | DONE (2025-11-27) | Completed: Sharding + TTL migration + event collection | Concelier Storage Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo`) | Provision Mongo collections (`advisory_observations`, `advisory_linksets`) with hashed shard keys, tenant indexes, TTL for ingest metadata. |
|
||||
| 11 | CONCELIER-LNM-21-102-DEV | TODO | Unblocked by 21-101-DEV completion; CI runner available for migrations. | Concelier Storage Guild · DevOps Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo`) | Backfill legacy merged advisories; seed tombstones; provide rollback tooling for Offline Kit. |
|
||||
| 12 | CONCELIER-LNM-21-103-DEV | BLOCKED (awaits 21-102-DEV) | Requires 21-102-DEV completion; CI runner available for object-store bootstrap tests. | Concelier Storage Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo`) | Move large raw payloads to object storage with deterministic pointers; update bootstrapper/offline seeds; preserve provenance metadata. |
|
||||
| 11 | CONCELIER-LNM-21-102-DEV | DONE (2025-11-28) | Completed: Migration + tombstones + rollback tooling | Concelier Storage Guild · DevOps Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo`) | Backfill legacy merged advisories; seed tombstones; provide rollback tooling for Offline Kit. |
|
||||
| 12 | CONCELIER-LNM-21-103-DEV | BLOCKED (awaits object storage contract) | Requires object storage contract definition before implementation; see Blockers & Dependencies. | Concelier Storage Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo`) | Move large raw payloads to object storage with deterministic pointers; update bootstrapper/offline seeds; preserve provenance metadata. |
|
||||
| 13 | CONCELIER-LNM-21-201 | BLOCKED (awaits 21-103) | Upstream storage tasks must land first; CI runner available for WebService tests. | Concelier WebService Guild · BE-Base Platform Guild (`src/Concelier/StellaOps.Concelier.WebService`) | `/advisories/observations` filters by alias/purl/source with strict tenant scopes; echoes upstream values + provenance fields only. |
|
||||
| 14 | CONCELIER-LNM-21-202 | BLOCKED (awaits 21-201) | Await upstream to run `/advisories/linksets` export tests; CI runner available. | Concelier WebService Guild (`src/Concelier/StellaOps.Concelier.WebService`) | `/advisories/linksets`/`export`/`evidence` endpoints surface correlation + conflict payloads and `ERR_AGG_*` mapping; no synthesis/merge. |
|
||||
| 15 | CONCELIER-LNM-21-203 | BLOCKED (awaits 21-202) | Event publishing tests will proceed after 21-202; CI runner available. | Concelier WebService Guild · Platform Events Guild (`src/Concelier/StellaOps.Concelier.WebService`) | Publish idempotent NATS/Redis events for new observations/linksets with documented schemas; include tenant + provenance references only. |
|
||||
@@ -46,6 +46,8 @@
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-11-28 | CONCELIER-LNM-21-103-DEV BLOCKED: Object storage contract for raw payloads not yet defined. Current payloads stored in GridFS; migration to S3-compatible store requires interface definition and cross-guild coordination with DevOps Guild. Marked task blocked and documented in Decisions & Risks. | Implementer |
|
||||
| 2025-11-28 | CONCELIER-LNM-21-102-DEV DONE: Created `EnsureLegacyAdvisoriesBackfillMigration` that backfills `advisory_observations` from `advisory_raw`, creates/updates `advisory_linksets` by grouping observations, and seeds `backfill_marker` tombstones for rollback tracking. Added rollback script at `ops/devops/scripts/rollback-lnm-backfill.js` for Offline Kit. Updated MIGRATIONS.md with migration entry and operator runbook. Build passed. | Implementer |
|
||||
| 2025-11-27 | CONCELIER-LNM-21-101-DEV DONE: Created `EnsureLinkNotMergeShardingAndTtlMigration` adding hashed shard key indexes on `tenantId` for horizontal scaling, optional TTL indexes for `ObservationRetention`/`LinksetRetention`/`EventRetention` options, and `advisory_linkset_events` collection for linkset event outbox. Updated `MongoStorageOptions` with retention properties. Registered both `EnsureLinkNotMergeCollectionsMigration` and new sharding/TTL migration in DI. | Implementer |
|
||||
| 2025-11-27 | CONCELIER-LNM-21-005 DONE: Implemented `advisory.linkset.updated@1` event infrastructure (`AdvisoryLinksetUpdatedEvent`, `IAdvisoryLinksetEventPublisher`, `IAdvisoryLinksetEventOutbox`, `AdvisoryLinksetEventPublisherOptions`). Added 9 unit tests covering delta computation, conflict summaries, and provenance mapping. Documented event contract at `docs/modules/concelier/events/advisory.linkset.updated@1.md`. | Implementer |
|
||||
| 2025-11-27 | CONCELIER-LNM-21-004 DONE: Implemented AOC write guard infrastructure (`IAdvisoryObservationWriteGuard`, `AdvisoryObservationWriteGuard`, `AppendOnlyViolationException`). Added 13 unit tests covering Proceed/SkipIdentical/RejectMutation dispositions. Documented AOC and linkset supersession model in `docs/modules/concelier/link-not-merge-schema.md`. Legacy merge logic already deprecated with `[Obsolete]` and gated by `NoMergeEnabled` flag (defaults true). | Implementer |
|
||||
@@ -115,6 +117,7 @@
|
||||
- CONCELIER-GRAPH-28-102 implemented: contract lives at `docs/modules/concelier/api/evidence-batch.md`; integration test covers empty-match path. Ensure consumers align on tenant header + limits before rollout.
|
||||
- CONCELIER-LNM-21-004 risk: removing canonical merge/dedup requires architect decision on retiring `CanonicalMerger` consumers (graph overlays, console summaries) and a migration/rollback plan; proceed after design sign-off.
|
||||
- CONCELIER-GRAPH-24-101 risk: API contract drafted at `docs/modules/concelier/api/advisories-summary.md`; implementation pending WebService wiring and consumer alignment.
|
||||
- CONCELIER-LNM-21-103-DEV blocked: Object storage contract for raw payloads not yet defined. Requires cross-guild coordination between Storage Guild and DevOps Guild. Current payloads stored in GridFS; migration to S3-compatible store (MinIO) requires interface definition, migration strategy, bootstrapper updates, and offline seed support.
|
||||
|
||||
## Next Checkpoints
|
||||
- Next LNM schema review: align with CARTO-GRAPH/LNM owners (date TBD); unblock tasks 1–2 and 5–15.
|
||||
|
||||
@@ -44,7 +44,8 @@
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-11-25 | Runner disk is full (“No space left on device”); orchestrator WebService tests cannot be re-run. Free bin/obj/TestResults and `ops/devops/artifacts/ci-110` before continuing ORCH-32/33/34. | Concelier Core |
|
||||
| 2025-11-28 | Disk space issue resolved (56GB available). Fixed `InitializeMongoAsync` to skip in testing mode. WebService orchestrator tests still fail due to hosted services requiring MongoDB; test factory needs more extensive mocking or integration test with Mongo2Go. ORCH tasks remain BLOCKED pending test infrastructure fix. | Implementer |
|
||||
| 2025-11-25 | Runner disk is full ("No space left on device"); orchestrator WebService tests cannot be re-run. Free bin/obj/TestResults and `ops/devops/artifacts/ci-110` before continuing ORCH-32/33/34. | Concelier Core |
|
||||
| 2025-11-25 | Storage.Mongo job-store slice executed locally: `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Storage.Mongo.Tests/StellaOps.Concelier.Storage.Mongo.Tests.csproj -c Debug --no-restore --no-build --filter FullyQualifiedName~MongoJobStore` (3/3 pass). TRX: `ops/devops/artifacts/ci-110/20251125T034529Z/trx/concelier-storage-jobstore.trx`. Broader suite still pending CI. | Concelier Core |
|
||||
| 2025-11-25 | WebService orchestrator filter run (`dotnet test ...WebService.Tests.csproj --filter FullyQualifiedName~Orchestrator`) produced no matching tests; TRX recorded at `ops/devops/artifacts/ci-110/20251125T040900Z/trx/concelier-web-orch.trx`. Need to add orchestrator WebService tests before closing ORCH-32/33/34. | Concelier Core |
|
||||
| 2025-11-25 | Attempted to add WebService orchestrator tests with Mongo bypass; repo disk is full (`No space left on device`), preventing further builds/tests. Cleanup of bin/obj/TestResults and ops/devops artifacts required before rerunning orchestrator test slice. | Concelier Core |
|
||||
|
||||
@@ -26,22 +26,29 @@
|
||||
| P3 | PREP-CONCELIER-VULN-29-001 | DONE (2025-11-19) | Bridge contract published at `docs/modules/concelier/bridges/vuln-29-001.md`; sample fixture location noted. | Concelier WebService Guild · Vuln Explorer Guild (`src/Concelier/StellaOps.Concelier.WebService`) | Provide Concelier/Vuln bridge contract (advisory keys, search params, sample responses) that VEX Lens + Vuln Explorer rely on; publish OpenAPI excerpt and fixtures. |
|
||||
| 0 | POLICY-AUTH-SIGNALS-LIB-115 | DONE (2025-11-19) | Package `StellaOps.Policy.AuthSignals` 0.1.0-alpha published to `local-nugets/`; schema/fixtures at `docs/policy/*`. | Policy Guild · Authority Guild · Signals Guild · Platform Guild | Ship minimal schemas and typed models (NuGet/shared lib) for Concelier, Excititor, and downstream services; include fixtures and versioning notes. |
|
||||
| 1 | CONCELIER-POLICY-20-002 | DONE (2025-11-20) | Vendor alias + SemVer range normalization landed; tests green. | Concelier Core Guild · Policy Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Expand linkset builders with vendor equivalence, NEVRA/PURL normalization, version-range parsing so policy joins are accurate without prioritizing sources. |
|
||||
| 2 | CONCELIER-POLICY-20-003 | BLOCKED | Upstream POLICY-20-001 outputs missing; 20-002 complete. | Concelier Storage Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo`) | Advisory selection cursors + change-stream checkpoints for deterministic policy deltas; include offline migration scripts. |
|
||||
| 3 | CONCELIER-POLICY-23-001 | BLOCKED | Depends on 20-003 (blocked). | Concelier Core Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Secondary indexes/materialized views (alias, provider severity, confidence) to keep policy lookups fast without cached verdicts; document query patterns. |
|
||||
| 4 | CONCELIER-POLICY-23-002 | BLOCKED | Depends on 23-001 (blocked). | Concelier Core Guild · Platform Events Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Ensure `advisory.linkset.updated` events carry idempotent IDs, confidence summaries, tenant metadata for safe policy replay. |
|
||||
| 5 | CONCELIER-RISK-66-001 | BLOCKED | Blocked on POLICY-AUTH-SIGNALS-LIB-115 and POLICY chain. | Concelier Core Guild · Risk Engine Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Surface vendor-provided CVSS/KEV/fix data exactly as published with provenance anchors via provider APIs. |
|
||||
| 6 | CONCELIER-RISK-66-002 | BLOCKED | Blocked on POLICY-AUTH-SIGNALS-LIB-115 and 66-001. | Concelier Core Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Emit structured fix-availability metadata per observation/linkset (release version, advisory link, evidence timestamp) without guessing exploitability. |
|
||||
| 7 | CONCELIER-RISK-67-001 | BLOCKED | Blocked on POLICY-AUTH-SIGNALS-LIB-115 and 66-001. | Concelier Core Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Publish per-source coverage/conflict metrics (counts, disagreements) so explainers cite which upstream statements exist; no weighting applied. |
|
||||
| 8 | CONCELIER-RISK-68-001 | BLOCKED | Blocked on POLICY-AUTH-SIGNALS-LIB-115 and POLICY-RISK-68-001. | Concelier Core Guild · Policy Studio Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Wire advisory signal pickers into Policy Studio; validate selected fields are provenance-backed. |
|
||||
| 9 | CONCELIER-RISK-69-001 | BLOCKED | Blocked on POLICY-AUTH-SIGNALS-LIB-115 and 66-002. | Concelier Core Guild · Notifications Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Emit notifications on upstream advisory field changes (e.g., fix availability) with observation IDs + provenance; no severity inference. |
|
||||
| 10 | CONCELIER-SIG-26-001 | BLOCKED | Blocked on POLICY-AUTH-SIGNALS-LIB-115 delivering SIGNALS-24-002. | Concelier Core Guild · Signals Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Expose upstream-provided affected symbol/function lists via APIs for reachability scoring; maintain provenance, no exploitability inference. |
|
||||
| 2 | CONCELIER-POLICY-20-003 | DONE (2025-11-28) | Implemented `PolicyDeltaCheckpoint` model, `IPolicyDeltaCheckpointStore` interface, MongoDB store + migration `20251128_policy_delta_checkpoints`. | Concelier Storage Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo`) | Advisory selection cursors + change-stream checkpoints for deterministic policy deltas; include offline migration scripts. |
|
||||
| 3 | CONCELIER-POLICY-23-001 | DONE (2025-11-28) | Implemented migration `20251128_policy_lookup_indexes` with alias multikey, confidence, and severity indexes. Query patterns documented in migration XML docs. | Concelier Core Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Secondary indexes/materialized views (alias, provider severity, confidence) to keep policy lookups fast without cached verdicts; document query patterns. |
|
||||
| 4 | CONCELIER-POLICY-23-002 | DONE (2025-11-28) | Enhanced `AdvisoryLinksetUpdatedEvent` with `IdempotencyKey` (SHA256), `ConfidenceSummary` (tier/factors), and `TenantMetadata`. | Concelier Core Guild · Platform Events Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Ensure `advisory.linkset.updated` events carry idempotent IDs, confidence summaries, tenant metadata for safe policy replay. |
|
||||
| 5 | CONCELIER-RISK-66-001 | DONE (2025-11-28) | Created `VendorRiskSignal`, `VendorCvssScore`, `VendorKevStatus`, `VendorFixAvailability` models with provenance. Extractor parses OSV/NVD formats. | Concelier Core Guild · Risk Engine Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Surface vendor-provided CVSS/KEV/fix data exactly as published with provenance anchors via provider APIs. |
|
||||
| 6 | CONCELIER-RISK-66-002 | TODO | Upstream 66-001 DONE. Ready to emit fix-availability metadata. | Concelier Core Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Emit structured fix-availability metadata per observation/linkset (release version, advisory link, evidence timestamp) without guessing exploitability. |
|
||||
| 7 | CONCELIER-RISK-67-001 | TODO | Upstream 66-001 DONE. Ready to publish coverage/conflict metrics. | Concelier Core Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Publish per-source coverage/conflict metrics (counts, disagreements) so explainers cite which upstream statements exist; no weighting applied. |
|
||||
| 8 | CONCELIER-RISK-68-001 | BLOCKED | Blocked on POLICY-RISK-68-001. | Concelier Core Guild · Policy Studio Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Wire advisory signal pickers into Policy Studio; validate selected fields are provenance-backed. |
|
||||
| 9 | CONCELIER-RISK-69-001 | BLOCKED | Blocked on 66-002. | Concelier Core Guild · Notifications Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Emit notifications on upstream advisory field changes (e.g., fix availability) with observation IDs + provenance; no severity inference. |
|
||||
| 10 | CONCELIER-SIG-26-001 | BLOCKED | Blocked on SIGNALS-24-002. | Concelier Core Guild · Signals Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Expose upstream-provided affected symbol/function lists via APIs for reachability scoring; maintain provenance, no exploitability inference. |
|
||||
| 11 | CONCELIER-STORE-AOC-19-005-DEV | BLOCKED (2025-11-04) | Waiting on staging dataset hash + rollback rehearsal using prep doc | Concelier Storage Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo`) | Execute raw-linkset backfill/rollback plan so Mongo reflects Link-Not-Merge data; rehearse rollback (dev/staging). |
|
||||
| 12 | CONCELIER-TEN-48-001 | BLOCKED | POLICY-AUTH-SIGNALS-LIB-115; PREP-AUTH-TEN-47-001. | Concelier Core Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Enforce tenant scoping through normalization/linking; expose capability endpoint advertising `merge=false`; ensure events include tenant IDs. |
|
||||
| 12 | CONCELIER-TEN-48-001 | DONE (2025-11-28) | Created Tenancy module with `TenantScope`, `TenantCapabilities`, `TenantCapabilitiesResponse`, `ITenantCapabilitiesProvider`, and `TenantScopeNormalizer` per AUTH-TEN-47-001. | Concelier Core Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Enforce tenant scoping through normalization/linking; expose capability endpoint advertising `merge=false`; ensure events include tenant IDs. |
|
||||
| 13 | CONCELIER-VEXLENS-30-001 | BLOCKED | PREP-CONCELIER-VULN-29-001; VEXLENS-30-005 | Concelier WebService Guild · VEX Lens Guild (`src/Concelier/StellaOps.Concelier.WebService`) | Guarantee advisory key consistency and cross-links consumed by VEX Lens so consensus explanations cite Concelier evidence without merges. |
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-11-28 | Completed CONCELIER-TEN-48-001: created Tenancy module with `TenantScope`, `TenantCapabilities`, `TenantCapabilitiesResponse`, `ITenantCapabilitiesProvider`, `LinkNotMergeTenantCapabilitiesProvider`, and `TenantScopeNormalizer`. Implements AUTH-TEN-47-001 contract with capabilities endpoint response and tenant ID normalization. Build green. | Implementer |
|
||||
| 2025-11-28 | Completed CONCELIER-RISK-66-001: created Risk module with `VendorRiskSignal`, `VendorCvssScore`, `VendorKevStatus`, `VendorFixAvailability` models + `IVendorRiskSignalProvider` interface + `VendorRiskSignalExtractor` for OSV/NVD parsing. All with provenance anchors. Build green. Tasks 6 and 7 now TODO. | Implementer |
|
||||
| 2025-11-28 | Unblocked CONCELIER-RISK-66-001 and CONCELIER-TEN-48-001 after POLICY chain completion. Tasks 5 and 12 moved to TODO. | Implementer |
|
||||
| 2025-11-28 | Completed CONCELIER-POLICY-23-002: enhanced `AdvisoryLinksetUpdatedEvent` with `IdempotencyKey` (SHA256 of linkset identity + content), `ConfidenceSummary` (tier classification: high/medium/low/very-low/unknown + contributing factors), and `TenantMetadata` (URN + namespace extraction). Build green. POLICY chain (20-002/003, 23-001/002) now complete. | Implementer |
|
||||
| 2025-11-28 | Completed CONCELIER-POLICY-23-001: implemented migration `20251128_policy_lookup_indexes` with indexes for alias lookups (multikey on `linkset.aliases`), confidence filtering (`confidence`, `tenantId+confidence`), severity queries (`normalized.severities.system/score`), and pagination (`tenantId+createdAt`). Query patterns documented in XML docs. Build green; MIGRATIONS.md updated. Task 4 (23-002) now TODO. | Implementer |
|
||||
| 2025-11-28 | Completed CONCELIER-POLICY-20-003: implemented `PolicyDeltaCheckpoint` model + `IPolicyDeltaCheckpointStore` interface in Core/Linksets; MongoDB document, store, and migration (`20251128_policy_delta_checkpoints`) in Storage.Mongo. Indexes on tenantId, consumerId, compound, and updatedAt. Build green; MIGRATIONS.md updated. Task 3 (23-001) now TODO. | Implementer |
|
||||
| 2025-11-28 | Unblocked CONCELIER-POLICY-20-003 after verifying POLICY-20-001 DONE in Sprint 0114. Task moved to TODO; ready for implementation. | Implementer |
|
||||
| 2025-11-25 | Synced status with tasks-all: RISK-66/68/69, SIG-26-001, TEN-48-001, VEXLENS-30-001 remain BLOCKED despite signals library shipping; blockers are POLICY-20-001 outputs, AUTH-TEN-47-001, SIGNALS-24-002, VEXLENS-30-005. | Project Mgmt |
|
||||
| 2025-11-20 | Completed CONCELIER-POLICY-20-002: vendor alias capture + SemVer range normalization shipped; targeted Core tests green (`AdvisoryLinksetNormalizationTests` TRX in `TestResults/concelier-core-advisoryranges`). | Implementer |
|
||||
| 2025-11-19 | Added PREP tasks for CONCELIER-CORE-AOC-19-004, AUTH-TEN-47-001, and CONCELIER-VULN-29-001; updated dependencies for tasks 11–13. | Project Mgmt |
|
||||
|
||||
@@ -27,15 +27,17 @@
|
||||
| 6 | POLICY-ENGINE-20-006 | BLOCKED (2025-11-27) | Depends on 20-005. | Policy · Scheduler Worker Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
| 7 | POLICY-ENGINE-20-007 | BLOCKED (2025-11-27) | Depends on 20-006. | Policy · Observability Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
| 8 | POLICY-ENGINE-20-008 | BLOCKED (2025-11-27) | Depends on 20-007. | Policy · QA Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
| 9 | POLICY-ENGINE-20-009 | BLOCKED (2025-11-27) | Depends on 20-008. | Policy · Storage Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
| 10 | POLICY-ENGINE-27-001 | BLOCKED (2025-11-27) | Depends on 20-009. | Policy Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
| 11 | POLICY-ENGINE-27-002 | BLOCKED (2025-11-27) | Depends on 27-001. | Policy · Observability Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
| 9 | POLICY-ENGINE-20-009 | DONE (2025-11-28) | MongoDB schemas/indexes for policies, policy_runs, effective_finding_* with migrations and tenant enforcement. | Policy · Storage Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
| 10 | POLICY-ENGINE-27-001 | DONE (2025-11-28) | Extended compile outputs with symbol table, rule index, documentation, coverage metadata, and deterministic hashes. | Policy Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
| 11 | POLICY-ENGINE-27-002 | DONE (2025-11-28) | Enhanced simulate endpoints with rule firing counts, heatmap aggregates, sampled explain traces with deterministic ordering, and delta summaries. | Policy · Observability Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
| 12 | POLICY-ENGINE-29-001 | BLOCKED (2025-11-27) | Depends on 27-004. | Policy Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
| 13 | POLICY-ENGINE-29-002 | DONE (2025-11-23) | Contract published at `docs/modules/policy/contracts/29-002-streaming-simulation.md`. | Policy · Findings Ledger Guild / `src/Policy/StellaOps.Policy.Engine` |
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-11-28 | Completed POLICY-ENGINE-27-002: Enhanced simulation analytics with SimulationAnalytics models (RuleFiringCounts, SimulationHeatmap, SampledExplainTraces, SimulationDeltaSummary) and SimulationAnalyticsService. Integrated into RiskSimulationResult. 15 new unit tests. | Policy Guild |
|
||||
| 2025-11-28 | Completed POLICY-ENGINE-20-009: MongoDB schemas/indexes with migration infrastructure (PolicyEngineMongoContext, migrations, TenantFilterBuilder). Completed POLICY-ENGINE-27-001: Extended compile outputs with PolicyCompileMetadata (symbol table, rule index, documentation, coverage metadata, deterministic hashes) via PolicyMetadataExtractor. 16 new unit tests. | Policy Guild |
|
||||
| 2025-11-27 | Marked POLICY-CONSOLE-23-002 and POLICY-ENGINE-20-003..29-001 BLOCKED due to unmet upstream contracts (POLICY-CONSOLE-23-001, deterministic evaluator 20-002 chain). | Policy Guild |
|
||||
| 2025-11-23 | Published POLICY-ENGINE-29-002 streaming simulation contract (`docs/modules/policy/contracts/29-002-streaming-simulation.md`); marked task 13 DONE. | Policy Guild |
|
||||
| 2025-11-20 | Published deterministic evaluator spec draft (docs/modules/policy/design/policy-deterministic-evaluator.md); moved PREP-POLICY-ENGINE-20-002 to DOING. | Project Mgmt |
|
||||
|
||||
@@ -26,14 +26,15 @@
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| P1 | PREP-140-D-ZASTAVA-WAVE-WAITING-ON-SURFACE-FS | DONE (2025-11-20) | Due 2025-11-22 · Accountable: Zastava Observer/Webhook Guilds · Surface Guild | Zastava Observer/Webhook Guilds · Surface Guild | Prep artefact published at `docs/modules/zastava/prep/2025-11-20-surface-fs-env-prep.md` (cache drop cadence, env helper ownership, DSSE requirements). |
|
||||
| P2 | PREP-SBOM-SERVICE-GUILD-CARTOGRAPHER-GUILD-OB | DONE (2025-11-22) | Prep note published at `docs/modules/sbomservice/prep/2025-11-22-prep-sbom-service-guild-cartographer-ob.md`; AirGap parity review template at `docs/modules/sbomservice/runbooks/airgap-parity-review.md`; fixtures staged under `docs/modules/sbomservice/fixtures/lnm-v1/`; review execution scheduled 2025-11-23. | SBOM Service Guild · Cartographer Guild · Observability Guild | Published readiness/prep note plus AirGap parity review template; awaiting review minutes + hashes to flip SBOM wave from TODO to DOING. |
|
||||
| 1 | 140.A Graph wave | BLOCKED (2025-11-19) | Await real scanner cache ETA; working off mock bundle only. | Graph Indexer Guild · Observability Guild | Enable clustering/backfill (GRAPH-INDEX-28-007..010) against mock bundle; revalidate once real cache lands. |
|
||||
| 2 | 140.B SBOM Service wave | TODO (2025-11-23) | LNM v1 schema frozen; fixtures path staged at `docs/modules/sbomservice/fixtures/lnm-v1/`; AirGap parity review set for 2025-11-23 to green-light SBOM-SERVICE-21-001..004. | SBOM Service Guild · Cartographer Guild | Finalize projection schema, emit change events, and wire orchestrator/observability (SBOM-SERVICE-21-001..004, SBOM-AIAI-31-001/002). |
|
||||
| 3 | 140.C Signals wave | BLOCKED (2025-11-20) | CAS promotion + signed manifests + provenance appendix pending; SIGNALS-24-002/003 blocked upstream. TRACTORS: see `docs/signals/cas-promotion-24-002.md` and `docs/signals/provenance-24-003.md`. | Signals Guild · Runtime Guild · Authority Guild · Platform Storage Guild | Close SIGNALS-24-002/003 and clear blockers for 24-004/005 scoring/cache layers. |
|
||||
| 4 | 140.D Zastava wave | BLOCKED | PREP-140-D-ZASTAVA-WAVE-WAITING-ON-SURFACE-FS | Zastava Observer/Webhook Guilds · Surface Guild | Prepare env/secret helpers and admission hooks; start once cache endpoints and helpers are published. |
|
||||
| 1 | 140.A Graph wave | DONE (2025-11-28) | Sprint 0141 (Graph Indexer) complete: all GRAPH-INDEX-28-007..010 tasks DONE. | Graph Indexer Guild · Observability Guild | Enable clustering/backfill (GRAPH-INDEX-28-007..010) against mock bundle; revalidate once real cache lands. |
|
||||
| 2 | 140.B SBOM Service wave | DOING (2025-11-28) | Sprint 0142 mostly complete: SBOM-SERVICE-21-001..004, SBOM-AIAI-31-001/002, SBOM-ORCH-32/33/34-001, SBOM-VULN-29-001/002 all DONE. Only SBOM-CONSOLE-23-001/002 remain BLOCKED. | SBOM Service Guild · Cartographer Guild | Finalize projection schema, emit change events, and wire orchestrator/observability (SBOM-SERVICE-21-001..004, SBOM-AIAI-31-001/002). |
|
||||
| 3 | 140.C Signals wave | DOING (2025-11-28) | Sprint 0143: SIGNALS-24-001/002/003 DONE; SIGNALS-24-004/005 remain BLOCKED on CAS promotion. | Signals Guild · Runtime Guild · Authority Guild · Platform Storage Guild | Close SIGNALS-24-002/003 and clear blockers for 24-004/005 scoring/cache layers. |
|
||||
| 4 | 140.D Zastava wave | DONE (2025-11-28) | Sprint 0144 (Zastava Runtime Signals) complete: all ZASTAVA-ENV/SECRETS/SURFACE tasks DONE. | Zastava Observer/Webhook Guilds · Surface Guild | Prepare env/secret helpers and admission hooks; start once cache endpoints and helpers are published. |
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-11-28 | Synced wave status with downstream sprints: 140.A Graph (DONE per Sprint 0141); 140.B SBOM (DOING, mostly complete per Sprint 0142); 140.C Signals (DOING, 3/5 done per Sprint 0143); 140.D Zastava (DONE per Sprint 0144). Updated Delivery Tracker and unblocked Sprint 0150 dependencies. | Implementer |
|
||||
| 2025-11-20 | Completed PREP-140-D-ZASTAVA-WAVE-WAITING-ON-SURFACE-FS: published cache/env helper prep at `docs/modules/zastava/prep/2025-11-20-surface-fs-env-prep.md`; status set to DONE. | Implementer |
|
||||
| 2025-11-20 | Marked SIGNALS-24-002/003 as BLOCKED pending Platform Storage + provenance approvals; linked CAS/provenance checklists in blockers. | Implementer |
|
||||
| 2025-11-19 | Assigned PREP owners/dates; see Delivery Tracker. | Planning |
|
||||
@@ -93,17 +94,17 @@ This file now only tracks the runtime & signals status snapshot. Active backlog
|
||||
|
||||
| Wave | Guild owners | Shared prerequisites | Status | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 140.A Graph | Graph Indexer Guild · Observability Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner (phase I tracked under `docs/implplan/SPRINT_130_scanner_surface.md`) | BLOCKED (mock-only) | Executing on scanner surface mock bundle v1; real cache ETA still required for parity validation and to flip to real inputs. |
|
||||
| 140.B SbomService | SBOM Service Guild · Cartographer Guild · Observability Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | PREP-SBOM-SERVICE-GUILD-CARTOGRAPHER-GUILD-OB | Prep note published 2025-11-22 at `docs/modules/sbomservice/prep/2025-11-22-prep-sbom-service-guild-cartographer-ob.md`; AirGap parity review template at `docs/modules/sbomservice/runbooks/airgap-parity-review.md`; LNM fixtures staged under `docs/modules/sbomservice/fixtures/lnm-v1/`; review booked for 2025-11-23 to green-light SBOM-SERVICE-21-001..004. |
|
||||
| 140.C Signals | Signals Guild · Authority Guild (for scopes) · Runtime Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | BLOCKED (red) | CAS checklist + provenance appendix overdue; callgraph retrieval live but artifacts not trusted until CAS/signing lands. |
|
||||
| 140.D Zastava | Zastava Observer/Webhook Guilds · Security Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | PREP-SBOM-SERVICE-GUILD-CARTOGRAPHER-GUILD-OB | Surface.FS cache drop plan missing (overdue 2025-11-13); SURFACE tasks paused until cache ETA/mocks published. |
|
||||
| 140.A Graph | Graph Indexer Guild · Observability Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner (phase I tracked under `docs/implplan/SPRINT_130_scanner_surface.md`) | DONE (2025-11-28) | Sprint 0141 complete: GRAPH-INDEX-28-007..010 all DONE. |
|
||||
| 140.B SbomService | SBOM Service Guild · Cartographer Guild · Observability Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | DOING (2025-11-28) | Sprint 0142 mostly complete: SBOM-SERVICE-21-001..004, SBOM-AIAI-31-001/002, SBOM-ORCH-32/33/34-001, SBOM-VULN-29-001/002 DONE. SBOM-CONSOLE-23-001/002 remain BLOCKED. |
|
||||
| 140.C Signals | Signals Guild · Authority Guild (for scopes) · Runtime Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | DOING (2025-11-28) | Sprint 0143: SIGNALS-24-001/002/003 DONE; SIGNALS-24-004/005 remain BLOCKED on CAS promotion. |
|
||||
| 140.D Zastava | Zastava Observer/Webhook Guilds · Security Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | DONE (2025-11-28) | Sprint 0144 complete: ZASTAVA-ENV/SECRETS/SURFACE all DONE. |
|
||||
|
||||
# Status snapshot (2025-11-18)
|
||||
# Status snapshot (2025-11-28)
|
||||
|
||||
- **140.A Graph** – BLOCKED on real cache delivery; running only on scanner surface mock bundle v1 pending cache ETA/hash.
|
||||
- **140.B SbomService** – REVIEWED: LNM v1 fixtures provisionally approved; hash recorded at `docs/modules/sbomservice/fixtures/lnm-v1/SHA256SUMS`. Minutes: `docs/modules/sbomservice/reviews/2025-11-23-airgap-parity.md`. SBOM-SERVICE-21-001 is DOING; 21-002..004 next in sequence.
|
||||
- **140.C Signals** – SIGNALS-24-001 shipped on 2025-11-09; SIGNALS-24-002 and SIGNALS-24-003 are BLOCKED with CAS promotion + provenance appendix pending. Scoring/cache work (SIGNALS-24-004/005) stays BLOCKED until CAS/provenance and runtime uploads stabilize.
|
||||
- **140.D Zastava** – ZASTAVA-ENV/SECRETS/SURFACE tracks are BLOCKED because Surface.FS cache outputs from Scanner are still unavailable; guilds continue prepping Surface.Env helper adoption and sealed-mode scaffolding while caches are pending.
|
||||
- **140.A Graph** – DONE. Sprint 0141 complete: GRAPH-INDEX-28-007..010 all shipped.
|
||||
- **140.B SbomService** – DOING. Sprint 0142 mostly complete: SBOM-SERVICE-21-001..004, SBOM-AIAI-31-001/002, SBOM-ORCH-32/33/34-001, SBOM-VULN-29-001/002 all DONE. Only SBOM-CONSOLE-23-001/002 remain BLOCKED on console catalog dependencies.
|
||||
- **140.C Signals** – DOING. Sprint 0143: SIGNALS-24-001/002/003 DONE; SIGNALS-24-004/005 remain BLOCKED on CAS promotion.
|
||||
- **140.D Zastava** – DONE. Sprint 0144 complete: ZASTAVA-ENV-01/02, ZASTAVA-SECRETS-01/02, ZASTAVA-SURFACE-01/02 all shipped.
|
||||
|
||||
## Wave task tracker (refreshed 2025-11-18)
|
||||
|
||||
|
||||
@@ -21,27 +21,42 @@
|
||||
## Delivery Tracker
|
||||
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | 150.A-Orchestrator | TODO | Wait for Sprint 0120.A/0130.A/0140.A readiness; verify Scanner surface artifacts | Orchestrator Service Guild · AirGap Policy/Controller Guilds · Observability Guild | Kick off orchestration scheduling/telemetry baseline for automation epic. |
|
||||
| 1 | 150.A-Orchestrator | TODO | 0140.A (Graph) ✅ DONE, 0140.D (Zastava) ✅ DONE. Remaining blockers: 0120.A AirGap staleness + 0130.A Scanner surface | Orchestrator Service Guild · AirGap Policy/Controller Guilds · Observability Guild | Kick off orchestration scheduling/telemetry baseline for automation epic. |
|
||||
| 2 | 150.B-PacksRegistry | TODO | 150.A must reach DOING; confirm tenancy scaffolding from Orchestrator | Packs Registry Guild · Exporter Guild · Security Guild | Packs registry automation stream staged; start after Orchestrator scaffolding. |
|
||||
| 3 | 150.C-Scheduler | TODO | Hold until 0140.A Graph overlays and 0130.A Scanner surface green | Scheduler WebService/Worker Guilds · Findings Ledger Guild · Observability Guild | Scheduler impact index improvements gated on Graph overlays. |
|
||||
| 3 | 150.C-Scheduler | TODO | 0140.A Graph ✅ DONE. Remaining blocker: 0130.A Scanner surface | Scheduler WebService/Worker Guilds · Findings Ledger Guild · Observability Guild | Scheduler impact index improvements gated on Graph overlays. |
|
||||
| 4 | 150.D-TaskRunner | TODO | Requires Orchestrator/Scheduler telemetry baselines (150.A/150.C) | Task Runner Guild · AirGap Guilds · Evidence Locker Guild | Execution engine upgrades and evidence integration to start post-baselines. |
|
||||
|
||||
## Wave Coordination Snapshot
|
||||
| Wave | Guild owners | Shared prerequisites | Status | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 150.A Orchestrator | Orchestrator Service Guild · AirGap Policy/Controller Guilds · Observability Guild | Sprint 0120.A – AirGap; Sprint 0130.A – Scanner; Sprint 0140.A – Graph | TODO | Pending confirmation that Scanner surface artifacts are ready; keep job telemetry work prepped for fast start. |
|
||||
| 150.B PacksRegistry | Packs Registry Guild · Exporter Guild · Security Guild | Sprint 0120.A – AirGap; Sprint 0130.A – Scanner; Sprint 0140.A – Graph | TODO | Blocked on Orchestrator tenancy scaffolding; specs are ready once 150.A flips to DOING. |
|
||||
| 150.C Scheduler | Scheduler WebService/Worker Guilds · Findings Ledger Guild · Observability Guild | Sprint 0120.A – AirGap; Sprint 0130.A – Scanner; Sprint 0140.A – Graph | TODO | Impact index improvements need Graph overlays; hold until 0140.A status improves. |
|
||||
| 150.A Orchestrator | Orchestrator Service Guild · AirGap Policy/Controller Guilds · Observability Guild | Sprint 0120.A – AirGap; Sprint 0130.A – Scanner; Sprint 0140.A – Graph | TODO | Graph (0140.A) and Zastava (0140.D) now DONE. AirGap staleness (0120.A 56-002/57/58) and Scanner surface (0130.A) remain blockers. Approaching readiness. |
|
||||
| 150.B PacksRegistry | Packs Registry Guild · Exporter Guild · Security Guild | Sprint 0120.A – AirGap; Sprint 0130.A – Scanner; Sprint 0140.A – Graph | TODO | Blocked on Orchestrator tenancy scaffolding; specs ready once 150.A flips to DOING. |
|
||||
| 150.C Scheduler | Scheduler WebService/Worker Guilds · Findings Ledger Guild · Observability Guild | Sprint 0120.A – AirGap; Sprint 0130.A – Scanner; Sprint 0140.A – Graph | TODO | Graph overlays (0140.A) now DONE. Scheduler impact index work can proceed once Scanner surface (0130.A) clears. |
|
||||
| 150.D TaskRunner | Task Runner Guild · AirGap Guilds · Evidence Locker Guild | Sprint 0120.A – AirGap; Sprint 0130.A – Scanner; Sprint 0140.A – Graph | TODO | Execution engine upgrades staged; start once Orchestrator/Scheduler telemetry baselines exist. |
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-11-28 | Synced with downstream sprints: Sprint 0141 (Graph) DONE, Sprint 0142 (SBOM) mostly DONE, Sprint 0143 (Signals) 3/5 DONE, Sprint 0144 (Zastava) DONE. Updated Sprint 0140 tracker and revised 150.* upstream dependency status. 150.A-Orchestrator may start once remaining AirGap/Scanner blockers clear. | Implementer |
|
||||
| 2025-11-28 | Upstream dependency check: Sprint 0120 (Policy/Reasoning) has LEDGER-29-007/008, LEDGER-34-101, LEDGER-AIRGAP-56-001 DONE but 56-002/57-001/58-001/ATTEST-73-001 BLOCKED. Sprint 0140 (Runtime/Signals) has all waves BLOCKED except SBOM (TODO). No Sprint 0130.A file found. All 150.* tasks remain TODO pending upstream readiness. | Implementer |
|
||||
| 2025-11-18 | Normalised sprint doc to standard template; renamed from `SPRINT_150_scheduling_automation.md`. | Planning |
|
||||
|
||||
## Upstream Dependency Status (as of 2025-11-28)
|
||||
| Upstream Sprint | Key Deliverable | Status | Impact on 150.* |
|
||||
| --- | --- | --- | --- |
|
||||
| Sprint 0120.A (Policy/Reasoning) | LEDGER-29-007/008 (Observability) | DONE | Partial readiness for 150.A |
|
||||
| Sprint 0120.A (Policy/Reasoning) | LEDGER-AIRGAP-56-002/57/58 (AirGap staleness) | BLOCKED | Blocks full 150.A readiness |
|
||||
| Sprint 0130.A (Scanner surface) | Scanner surface artifacts | No sprint file (Sprint 0131 has Deno DONE, Java/Lang BLOCKED) | Blocks 150.A, 150.C verification |
|
||||
| Sprint 0140.A (Graph overlays) | 140.A Graph wave | **DONE** (Sprint 0141 complete) | Unblocks 150.C Scheduler graph deps |
|
||||
| Sprint 0140.A (Graph overlays) | 140.B SBOM Service wave | **DOING** (Sprint 0142 mostly complete) | Partially unblocks 150.A/150.C |
|
||||
| Sprint 0140.A (Graph overlays) | 140.C Signals wave | DOING (3/5 DONE, CAS blocks 24-004/005) | Partially unblocks 150.A telemetry |
|
||||
| Sprint 0140.A (Graph overlays) | 140.D Zastava wave | **DONE** (Sprint 0144 complete) | Unblocks 150.A surface deps |
|
||||
|
||||
## Decisions & Risks
|
||||
- All waves remain gated on upstream AirGap/Scanner/Graph readiness; no new intra-decade dependencies introduced.
|
||||
- **Progress (2025-11-28):** Graph (0140.A) and Zastava (0140.D) waves now DONE; SBOM Service (0140.B) and Signals (0140.C) waves DOING. Main remaining blockers are 0120.A AirGap staleness tasks and 0130.A Scanner surface artifacts.
|
||||
- 150.A Orchestrator and 150.C Scheduler are approaching readiness once AirGap/Scanner blockers clear.
|
||||
- This sprint is a coordination snapshot only; implementation tasks continue in Sprint 151+ and should mirror status changes here to avoid drift.
|
||||
- Sprint 0130.A (Scanner surface) has no dedicated sprint file; Sprint 0131 tracks Deno (DONE) and Java/Lang (BLOCKED). Coordinate with Scanner Guild to finalize.
|
||||
|
||||
## Next Checkpoints
|
||||
- None scheduled; add next scheduling/automation sync once upstream readiness dates are confirmed.
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
| 3 | WEB-AIAI-31-003 | BLOCKED (2025-11-22) | Blocked by WEB-AIAI-31-002; telemetry targets depend on routing/batching contract. | BE-Base Platform Guild; Observability Guild | Telemetry + audit for advisory AI, guardrail block visibility. |
|
||||
| 4 | WEB-AOC-19-002 | TODO | Depends on WEB-AOC-19-001; align DSSE/CMS helper APIs. | BE-Base Platform Guild | Ship `ProvenanceBuilder`, checksum utilities, signature verification helper with tests. |
|
||||
| 5 | WEB-AOC-19-003 | TODO | Depends on WEB-AOC-19-002; confirm Roslyn analyzer rules. | QA Guild; BE-Base Platform Guild | Analyzer to prevent forbidden key writes; shared guard-validation fixtures. |
|
||||
| 6 | WEB-CONSOLE-23-001 | TODO | Define stable `/console/dashboard` and `/console/filters` contract; ensures deterministic ordering + pagination. | BE-Base Platform Guild; Product Analytics Guild | Tenant-scoped aggregates for findings, VEX overrides, advisory deltas, run health, policy change log. |
|
||||
| 6 | WEB-CONSOLE-23-001 | DONE (2025-11-28) | `/console/dashboard` and `/console/filters` endpoints implemented with tenant-scoped aggregates. | BE-Base Platform Guild; Product Analytics Guild | Tenant-scoped aggregates for findings, VEX overrides, advisory deltas, run health, policy change log. |
|
||||
| 7 | CONSOLE-VULN-29-001 | BLOCKED (2025-11-19) | Blocked on WEB-CONSOLE-23-001 contract and Concelier graph schema freeze. | Console Guild; BE-Base Platform Guild | `/console/vuln/*` workspace endpoints with filters/reachability badges and DTOs once schemas stabilize. |
|
||||
| 8 | CONSOLE-VEX-30-001 | BLOCKED (2025-11-19) | Blocked on WEB-CONSOLE-23-001 and Excititor console contract (SSE payload validation). | Console Guild; BE-Base Platform Guild | `/console/vex/events` SSE workspace with validated schemas and samples. |
|
||||
| 9 | WEB-CONSOLE-23-002 | TODO | Depends on WEB-CONSOLE-23-001; design heartbeat/backoff + auth scopes. | BE-Base Platform Guild; Scheduler Guild | `/console/status` polling and `/console/runs/{id}/stream` SSE/WebSocket proxy with queue lag metrics. |
|
||||
@@ -79,3 +79,4 @@
|
||||
| 2025-11-22 | Synced `docs/implplan/tasks-all.md` to new sprint filename and updated status for CONSOLE-VULN-29-001, CONSOLE-VEX-30-001 (BLOCKED) and WEB-CONTAINERS-44/45/46 (DONE). | Planning |
|
||||
| 2025-11-22 | Added completion dates in `tasks-all` for WEB-CONTAINERS-44/45/46 and aligned BLOCKED dates for VULN-29-001/VEX-30-001. | Planning |
|
||||
| 2025-11-22 | Harmonized all `CONTAINERS-44/45/46` rows in `tasks-all` to DONE with dates to match sprint status. | Planning |
|
||||
| 2025-11-28 | Completed WEB-CONSOLE-23-001: Implemented `/console/dashboard` and `/console/filters` endpoints in Authority module. Dashboard returns tenant-scoped aggregates (findings summary, VEX overrides, advisory deltas, run health, policy change log) with 30-day trend data. Filters endpoint returns deterministic filter categories with counts and cache-validation hash. Added 8 unit tests for dashboard/filters endpoints. Implementation in `src/Authority/StellaOps.Authority/StellaOps.Authority/Console/`. | Policy Guild |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Sprint 124 - Policy & Reasoning
|
||||
|
||||
_Last updated: November 8, 2025. Implementation order is DOING → TODO → BLOCKED._
|
||||
_Last updated: November 28, 2025. Implementation order is DOING → TODO → BLOCKED._
|
||||
|
||||
Focus areas below were split out of the previous combined sprint; execute sections in order unless noted.
|
||||
|
||||
@@ -18,8 +18,8 @@ Focus: Policy & Reasoning focus on Policy (phase II).
|
||||
| 5 | POLICY-ENGINE-20-005 | DONE (2025-11-27) | Determinism guard implemented in `src/Policy/StellaOps.Policy.Engine/DeterminismGuard/` with static analyzer (`ProhibitedPatternAnalyzer`), runtime sandbox (`DeterminismGuardService`, `EvaluationScope`), and guarded evaluator integration (`GuardedPolicyEvaluator`) | Policy Guild, Security Engineering / src/Policy/StellaOps.Policy.Engine |
|
||||
| 6 | POLICY-ENGINE-20-006 | DONE (2025-11-27) | Incremental orchestrator implemented in `src/Policy/StellaOps.Policy.Engine/IncrementalOrchestrator/` with `PolicyChangeEvent` models (advisory/VEX/SBOM change types), `IncrementalPolicyOrchestrator` (batching, deduplication, retry logic), and `IncrementalOrchestratorBackgroundService` (continuous processing, metrics) | Policy Guild, Scheduler Worker Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 7 | POLICY-ENGINE-20-007 | DONE (2025-11-27) | Structured traces implemented in `src/Policy/StellaOps.Policy.Engine/Telemetry/` with `RuleHitTrace.cs` (trace models, statistics), `RuleHitTraceCollector.cs` (sampling controls, exporters), and `ExplainTraceExport.cs` (JSON/NDJSON/Text/Markdown export formats) | Policy Guild, Observability Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 8 | POLICY-ENGINE-20-008 | TODO | Add unit/property/golden/perf suites covering policy compilation, evaluation correctness, determinism, and SLA targets (Deps: POLICY-ENGINE-20-007) | Policy Guild, QA Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 9 | POLICY-ENGINE-20-009 | TODO | Define Mongo schemas/indexes for `policies`, `policy_runs`, and `effective_finding_*`; implement migrations and tenant enforcement (Deps: POLICY-ENGINE-20-008) | Policy Guild, Storage Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 8 | POLICY-ENGINE-20-008 | DONE (2025-11-28) | Unit test suites added in `src/Policy/__Tests/StellaOps.Policy.Engine.Tests/` for DeterminismGuard, SelectionJoin, IncrementalOrchestrator, Materialization, and Telemetry components (99 tests passing) | Policy Guild, QA Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 9 | POLICY-ENGINE-20-009 | DONE (2025-11-28) | MongoDB schemas implemented in `src/Policy/StellaOps.Policy.Engine/Storage/Mongo/` with document classes (`PolicyDocuments.cs`, `PolicyRunDocument.cs`, `EffectiveFindingDocument.cs`, `PolicyAuditDocument.cs`), options (`PolicyEngineMongoOptions.cs`), context (`PolicyEngineMongoContext.cs`), migrations (`EnsurePolicyCollectionsMigration.cs`, `EnsurePolicyIndexesMigration.cs`, `EffectiveFindingCollectionInitializer.cs`), migration runner, and tenant enforcement (`TenantFilterBuilder.cs`) | Policy Guild, Storage Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 10 | POLICY-ENGINE-27-001 | TODO | Extend compile outputs to include rule coverage metadata, symbol table, inline documentation, and rule index for editor autocomplete; persist deterministic hashes (Deps: POLICY-ENGINE-20-009) | Policy Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 11 | POLICY-ENGINE-27-002 | TODO | Enhance simulate endpoints to emit rule firing counts, heatmap aggregates, sampled explain traces with deterministic ordering, and delta summaries for quick/batch sims (Deps: POLICY-ENGINE-27-001) | Policy Guild, Observability Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 12 | POLICY-ENGINE-29-001 | TODO | Implement batch evaluation endpoint (`POST /policy/eval/batch`) returning determinations + rationale chain for sets of `(artifact,purl,version,advisory)` tuples; support pagination and cost budgets (Deps: POLICY-ENGINE-27-004) | Policy Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
@@ -29,6 +29,9 @@ Focus: Policy & Reasoning focus on Policy (phase II).
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2025-11-28 | POLICY-ENGINE-20-009: Completed MongoDB storage layer - document schemas for policies, policy_revisions, policy_bundles, policy_runs, effective_finding_*, effective_finding_history_*, and policy_audit collections. Created `PolicyEngineMongoOptions.cs` (connection/collection configuration with TTL settings), `PolicyEngineMongoContext.cs` (database access with read/write concerns), migration infrastructure (`IPolicyEngineMongoMigration`, `PolicyEngineMigrationRunner`, `PolicyEngineMongoInitializer`), `EnsurePolicyCollectionsMigration.cs` (creates base collections), `EnsurePolicyIndexesMigration.cs` (indexes for policies, revisions, bundles, runs, audit), `EffectiveFindingCollectionInitializer.cs` (dynamic per-policy collection creation with indexes), `TenantFilterBuilder.cs` (tenant enforcement utilities), and `ServiceCollectionExtensions.cs` (DI registration). Status → DONE. | Implementer |
|
||||
| 2025-11-28 | POLICY-ENGINE-20-008: Completed unit test suites - `DeterminismGuardTests.cs` (static analyzer, runtime sandbox, guarded evaluator), `SelectionJoinTests.cs` (PURL equivalence, tuple resolution, VEX overlay), `IncrementalOrchestratorTests.cs` (event processing, deduplication, priority batching), `MaterializationTests.cs` (deterministic IDs, content hashing), `TelemetryTests.cs` (trace factory, statistics, sampling). 99 tests passing. Status → DONE. | Implementer |
|
||||
| 2025-11-27 | POLICY-ENGINE-20-007: Completed structured traces - `RuleHitTrace.cs` (trace models, factory, statistics aggregation), `RuleHitTraceCollector.cs` (sampling controls with VEX/severity-aware rates, incident mode, exporters), `ExplainTraceExport.cs` (JSON/NDJSON/Text/Markdown formats, builder pattern). Status → DONE. | Implementer |
|
||||
| 2025-11-27 | POLICY-ENGINE-20-006: Completed incremental orchestrator - `PolicyChangeEvent.cs` (change event models with factory for advisory/VEX/SBOM changes, deterministic content hashing, batching), `IncrementalPolicyOrchestrator.cs` (event processing with idempotency, retry logic, priority-based batching), `IncrementalOrchestratorBackgroundService.cs` (continuous processing with metrics). Status → DONE. | Implementer |
|
||||
| 2025-11-27 | POLICY-ENGINE-20-005: Completed determinism guard - `DeterminismViolation.cs` (violation models/options), `ProhibitedPatternAnalyzer.cs` (static analysis with regex patterns for DateTime.Now, Random, Guid.NewGuid, HttpClient, File.Read, etc.), `DeterminismGuardService.cs` (runtime sandbox with EvaluationScope, DeterministicTimeProvider), `GuardedPolicyEvaluator.cs` (integration layer). Status → DONE. | Implementer |
|
||||
| 2025-11-27 | POLICY-ENGINE-20-004: Completed materialization writer - `EffectiveFindingModels.cs` (document schema), `EffectiveFindingWriter.cs` (upsert + append-only history). Tenant-scoped collections, trace references, content hash deduplication. Status → DONE. | Implementer |
|
||||
|
||||
@@ -12,16 +12,16 @@ Focus: Policy & Reasoning focus on Policy (phase IV).
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 1 | POLICY-ENGINE-40-003 | DONE | Provide API/SDK utilities for consumers (Web Scanner, Graph Explorer) to request policy decisions with source evidence summaries (top severity sources, conflict counts) (Deps: POLICY-ENGINE-40-002) | Policy Guild, Web Scanner Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 2 | POLICY-ENGINE-50-001 | DONE | Implement SPL compiler: validate YAML, canonicalize, produce signed bundle, store artifact in object storage, write `policy_revisions` with AOC metadata (Deps: POLICY-ENGINE-40-003) | Policy Guild, Platform Security / src/Policy/StellaOps.Policy.Engine |
|
||||
| 3 | POLICY-ENGINE-50-002 | TODO | Build runtime evaluator executing compiled plans over advisory/vex linksets + SBOM asset metadata with deterministic caching (Redis) and fallback path (Deps: POLICY-ENGINE-50-001) | Policy Guild, Runtime Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 4 | POLICY-ENGINE-50-003 | TODO | Implement evaluation/compilation metrics, tracing, and structured logs (`policy_eval_seconds`, `policy_compiles_total`, explanation sampling) (Deps: POLICY-ENGINE-50-002) | Policy Guild, Observability Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 5 | POLICY-ENGINE-50-004 | TODO | Build event pipeline: subscribe to linkset/SBOM updates, schedule re-eval jobs, emit `policy.effective.updated` events with diff metadata (Deps: POLICY-ENGINE-50-003) | Policy Guild, Platform Events Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 6 | POLICY-ENGINE-50-005 | TODO | Design and implement `policy_packs`, `policy_revisions`, `policy_runs`, `policy_artifacts` collections with indexes, TTL, and tenant scoping (Deps: POLICY-ENGINE-50-004) | Policy Guild, Storage Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 7 | POLICY-ENGINE-50-006 | TODO | Implement explainer persistence + retrieval APIs linking decisions to explanation tree and AOC chain (Deps: POLICY-ENGINE-50-005) | Policy Guild, QA Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 8 | POLICY-ENGINE-50-007 | TODO | Provide evaluation worker host/DI wiring and job orchestration hooks for batch re-evaluations after policy activation (Deps: POLICY-ENGINE-50-006) | Policy Guild, Scheduler Worker Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 9 | POLICY-ENGINE-60-001 | TODO | Maintain Redis effective decision maps per asset/snapshot for Graph overlays; implement versioning and eviction strategy (Deps: POLICY-ENGINE-50-007) | Policy Guild, SBOM Service Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 10 | POLICY-ENGINE-60-002 | TODO | Expose simulation bridge for Graph What-if APIs, supporting hypothetical SBOM diffs and draft policies without persisting results (Deps: POLICY-ENGINE-60-001) | Policy Guild, BE-Base Platform Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 11 | POLICY-ENGINE-70-002 | TODO | Design and create Mongo collections (`exceptions`, `exception_reviews`, `exception_bindings`) with indexes and migrations; expose repository APIs (Deps: POLICY-ENGINE-60-002) | Policy Guild, Storage Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 12 | POLICY-ENGINE-70-003 | TODO | Build Redis exception decision cache (`exceptions_effective_map`) with warm/invalidation logic reacting to `exception.*` events (Deps: POLICY-ENGINE-70-002) | Policy Guild, Runtime Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 3 | POLICY-ENGINE-50-002 | DONE | Build runtime evaluator executing compiled plans over advisory/vex linksets + SBOM asset metadata with deterministic caching (Redis) and fallback path (Deps: POLICY-ENGINE-50-001) | Policy Guild, Runtime Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 4 | POLICY-ENGINE-50-003 | DONE | Implement evaluation/compilation metrics, tracing, and structured logs (`policy_eval_seconds`, `policy_compiles_total`, explanation sampling) (Deps: POLICY-ENGINE-50-002) | Policy Guild, Observability Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 5 | POLICY-ENGINE-50-004 | DONE | Build event pipeline: subscribe to linkset/SBOM updates, schedule re-eval jobs, emit `policy.effective.updated` events with diff metadata (Deps: POLICY-ENGINE-50-003) | Policy Guild, Platform Events Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 6 | POLICY-ENGINE-50-005 | DONE | Design and implement `policy_packs`, `policy_revisions`, `policy_runs`, `policy_artifacts` collections with indexes, TTL, and tenant scoping (Deps: POLICY-ENGINE-50-004) | Policy Guild, Storage Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 7 | POLICY-ENGINE-50-006 | DONE | Implement explainer persistence + retrieval APIs linking decisions to explanation tree and AOC chain (Deps: POLICY-ENGINE-50-005) | Policy Guild, QA Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 8 | POLICY-ENGINE-50-007 | DONE | Provide evaluation worker host/DI wiring and job orchestration hooks for batch re-evaluations after policy activation (Deps: POLICY-ENGINE-50-006) | Policy Guild, Scheduler Worker Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 9 | POLICY-ENGINE-60-001 | DONE | Maintain Redis effective decision maps per asset/snapshot for Graph overlays; implement versioning and eviction strategy (Deps: POLICY-ENGINE-50-007) | Policy Guild, SBOM Service Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 10 | POLICY-ENGINE-60-002 | DONE | Expose simulation bridge for Graph What-if APIs, supporting hypothetical SBOM diffs and draft policies without persisting results (Deps: POLICY-ENGINE-60-001) | Policy Guild, BE-Base Platform Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 11 | POLICY-ENGINE-70-002 | DONE | Design and create Mongo collections (`exceptions`, `exception_reviews`, `exception_bindings`) with indexes and migrations; expose repository APIs (Deps: POLICY-ENGINE-60-002) | Policy Guild, Storage Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 12 | POLICY-ENGINE-70-003 | DONE | Build Redis exception decision cache (`exceptions_effective_map`) with warm/invalidation logic reacting to `exception.*` events (Deps: POLICY-ENGINE-70-002) | Policy Guild, Runtime Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 13 | POLICY-ENGINE-70-004 | TODO | Extend metrics/tracing/logging for exception application (latency, counts, expiring events) and include AOC references in logs (Deps: POLICY-ENGINE-70-003) | Policy Guild, Observability Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 14 | POLICY-ENGINE-70-005 | TODO | Provide APIs/workers hook for exception activation/expiry (auto start/end) and event emission (`exception.activated/expired`) (Deps: POLICY-ENGINE-70-004) | Policy Guild, Scheduler Worker Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
| 15 | POLICY-ENGINE-80-001 | TODO | Integrate reachability/exploitability inputs into evaluation pipeline (state/score/confidence) with caching and explain support (Deps: POLICY-ENGINE-70-005) | Policy Guild, Signals Guild / src/Policy/StellaOps.Policy.Engine |
|
||||
@@ -48,3 +48,13 @@ Focus: Policy & Reasoning focus on Policy (phase IV).
|
||||
| 2025-11-27 | Started POLICY-ENGINE-40-003; implemented PolicyDecisionService, PolicyDecisionEndpoint, PolicyDecisionModels, tests. Blocked by pre-existing build issues in Telemetry.Core and RiskProfile projects. | Implementer |
|
||||
| 2025-11-27 | Fixed pre-existing build issues (TelemetryContext API mismatch, JsonSchema.Net v5 API changes, OpenTelemetry Meter API changes, test project missing packages/namespaces). All 9 PolicyDecisionServiceTests pass. POLICY-ENGINE-40-003 marked DONE. | Implementer |
|
||||
| 2025-11-27 | Implemented POLICY-ENGINE-50-001: Extended SPL compiler with AOC metadata support. Added PolicyAocMetadata, PolicyProvenance, PolicyAttestationRef models. Updated PolicyBundleService to capture compilation metadata, source/artifact digests, complexity metrics, provenance info. Added 4 new tests (all pass). POLICY-ENGINE-50-001 marked DONE. | Implementer |
|
||||
| 2025-11-28 | Implemented POLICY-ENGINE-50-002: Built runtime evaluator with deterministic caching. Created `IPolicyEvaluationCache` interface, `InMemoryPolicyEvaluationCache` implementation with TTL/eviction, `PolicyRuntimeEvaluationService` with batch evaluation support, cache key generation using SHA256 digests (policy, subject, context). Extended `PolicyBundleRecord` to store compiled `PolicyIrDocument`. Added 8 tests (all pass). POLICY-ENGINE-50-002 marked DONE. | Implementer |
|
||||
| 2025-11-28 | Implemented POLICY-ENGINE-50-003: Integrated telemetry into PolicyCompilationService and PolicyRuntimeEvaluationService. Added OpenTelemetry Activity tracing for `policy.compile`, `policy.evaluate`, and `policy.evaluate_batch` operations. Integrated existing metrics (RecordCompilation, RecordEvaluation, RecordEvaluationLatency, RecordRuleFired, RecordError, RecordEvaluationFailure). Added structured logging with context (duration, rule counts, complexity, cache hits). All 23 core tests pass. POLICY-ENGINE-50-003 marked DONE. | Implementer |
|
||||
| 2025-11-28 | Implemented POLICY-ENGINE-50-004: Built event pipeline for linkset/SBOM updates. Created `PolicyEffectiveEventModels.cs` with event types (`policy.effective.updated`, `policy.effective.added`, `policy.effective.removed`, `policy.effective.batch_completed`), `PolicyDecisionDiff` for diff metadata. Created `PolicyEventProcessor.cs` with `IPolicyEffectiveEventPublisher`, `IReEvaluationJobScheduler` interfaces. Processor handles PolicyChangeEvents, schedules re-evaluation jobs, and emits effective events with diffs. Added 3 new telemetry counters. Build succeeds. POLICY-ENGINE-50-004 marked DONE. | Implementer |
|
||||
| 2025-11-28 | Implemented POLICY-ENGINE-50-005: MongoDB collections with tenant scoping and indexes. Pre-existing infrastructure includes: `PolicyDocument`, `PolicyRevisionDocument`, `PolicyBundleDocument`, `PolicyRunDocument` classes in Documents folder; `EnsurePolicyIndexesMigration` with TTL indexes for policy_runs collection; `PolicyEngineMongoOptions` for configuration. Created `MongoPolicyPackRepository.cs` implementing `IPolicyPackRepository` with tenant-scoped CRUD operations for policy packs, revisions, bundles; approval workflow; activation tracking. Fixed pre-existing bug in `PolicyMetadataExtractor.cs` (string comparisons for enum operators). All 11 core tests pass. POLICY-ENGINE-50-005 marked DONE. | Implementer |
|
||||
| 2025-11-28 | Implemented POLICY-ENGINE-50-006: Explainer persistence and retrieval APIs with AOC chain linkage. Created `PolicyExplainDocument.cs` with MongoDB documents for explain traces including `ExplainInputContextDocument`, `ExplainRuleStepDocument`, `ExplainVexEvidenceDocument`, `ExplainStatisticsDocument`, `ExplainAocChainDocument`. Created `PolicyExplainerService.cs` with `IExplainTraceRepository` interface, `StoredExplainTrace`/`ExplainAocChain` records, `ExplainQueryOptions` for filtering/pagination, `AocChainValidationResult` for verifying attestation chain integrity. Service links explain traces to policy bundle AOC metadata (compilation ID, source/artifact digests, attestation references). Added `policy_explain_traces_stored_total` telemetry counter. Added `PolicyExplainsCollection` and `ExplainTraceRetention` to options. Added indexes for `policy_explains` collection (tenant_runId, tenant_policy_evaluatedAt_desc, tenant_subjectHash, aocChain_compilationId, expiresAt_ttl). All 11 core tests pass. POLICY-ENGINE-50-006 marked DONE. | Implementer |
|
||||
| 2025-11-28 | Implemented POLICY-ENGINE-50-007: Evaluation worker host and DI wiring for job orchestration. Created `PolicyEvaluationWorkerService.cs` integrating with existing `PolicyEventProcessor.DequeueJob()` for job scheduling, with `EvaluationJobResult` record for tracking job outcomes. Created `PolicyEvaluationWorkerHost.cs` as BackgroundService with configurable concurrency from `PolicyEngineWorkerOptions`. Created `PolicyEngineServiceCollectionExtensions.cs` with `AddPolicyEngineCore()`, `AddPolicyEngineEventPipeline()`, `AddPolicyEngineWorker()`, `AddPolicyEngineExplainer()`, and combined `AddPolicyEngine()` extension methods. Worker integrates with existing `IPolicyEffectiveEventPublisher` and `IReEvaluationJobScheduler` interfaces. Added `ScheduleActivationReEvalAsync()` hook for triggering re-evaluations after policy activation. All 182 tests pass. POLICY-ENGINE-50-007 marked DONE. | Implementer |
|
||||
| 2025-11-28 | Implemented POLICY-ENGINE-60-001: Redis effective decision maps for Graph overlays. Added StackExchange.Redis package. Created `EffectiveDecisionMap/EffectiveDecisionModels.cs` with `EffectiveDecisionEntry`, `EffectiveDecisionQueryResult`, `EffectiveDecisionSummary`, `EffectiveDecisionFilter` records for storing/querying policy decisions per asset/snapshot. Created `EffectiveDecisionMap/IEffectiveDecisionMap.cs` interface with Set/Get/Query/Invalidate operations plus versioning (`GetVersionAsync`, `IncrementVersionAsync`). Created `EffectiveDecisionMap/RedisEffectiveDecisionMap.cs` with TTL-based eviction using Redis key structure `stellaops:edm:{tenant}:{snapshot}:e:{asset}` for entries, `:idx` sorted sets for indexing, `:v` for version counters. Added `EffectiveDecisionMapOptions` to `PolicyEngineOptions`. Added `policy_effective_decision_map_operations_total` telemetry counter. Added `AddEffectiveDecisionMap()` and `AddPolicyEngineRedis()` DI extensions. All 182 tests pass. POLICY-ENGINE-60-001 marked DONE. | Implementer |
|
||||
| 2025-11-28 | Implemented POLICY-ENGINE-60-002: What-If simulation bridge for Graph APIs. Created `WhatIfSimulation/WhatIfSimulationModels.cs` with comprehensive request/response models (`WhatIfSimulationRequest`, `WhatIfSimulationResponse`, `WhatIfDraftPolicy`, `WhatIfSbomDiff`, `WhatIfDecisionChange`, `WhatIfDecision`, `WhatIfExplanation`, `WhatIfSummary`, `WhatIfImpact`, `WhatIfPolicyRef`). Created `WhatIfSimulation/WhatIfSimulationService.cs` supporting: hypothetical SBOM diffs (add/remove/upgrade/downgrade operations), draft policy comparison, baseline decision lookup from effective decision map, simulated decision computation considering VEX status and reachability, change detection and diff computation, impact assessment with risk delta recommendations. Service integrates with `IEffectiveDecisionMap` for baseline lookups, `IPolicyPackRepository` for policy retrieval, `PolicyCompilationService` for potential on-the-fly compilation. Added `AddWhatIfSimulation()` DI extension. Telemetry via existing `RecordSimulation()` counter. All 181 core tests pass. POLICY-ENGINE-60-002 marked DONE. | Implementer |
|
||||
| 2025-11-28 | Implemented POLICY-ENGINE-70-002: MongoDB collections for policy exceptions with indexes and repository APIs. Created `Storage/Mongo/Documents/PolicyExceptionDocuments.cs` with `PolicyExceptionDocument` (exceptions with scope, risk assessment, compensating controls, workflow states), `ExceptionScopeDocument` (advisory/CVE/PURL/asset targeting), `ExceptionRiskAssessmentDocument` (risk levels, justification), `ExceptionReviewDocument` (multi-reviewer approval workflow), `ReviewDecisionDocument` (individual decisions with conditions), `ExceptionBindingDocument` (asset-specific bindings with time ranges). Created `Storage/Mongo/Repositories/IExceptionRepository.cs` interface with CRUD operations for exceptions, reviews, and bindings; query options for filtering/pagination; methods for finding applicable exceptions, pending activations, expiring exceptions. Created `Storage/Mongo/Repositories/MongoExceptionRepository.cs` MongoDB implementation with tenant scoping. Added collection names to `PolicyEngineMongoOptions` (exceptions, exception_reviews, exception_bindings). Created `Storage/Mongo/Migrations/EnsureExceptionIndexesMigration.cs` with comprehensive indexes: tenant+status, tenant+type+status, tenant+created, tenant+tags, scope.advisoryIds, scope.assetIds, scope.cveIds, expiry tracking, reviewer queues, binding lookups. Added `policy_exception_operations_total` telemetry counter with `RecordExceptionOperation()` method. Registered migration and repository in `ServiceCollectionExtensions`. All 196 core tests pass. POLICY-ENGINE-70-002 marked DONE. | Implementer |
|
||||
| 2025-11-28 | Implemented POLICY-ENGINE-70-003: Redis exception decision cache with warm/invalidation logic. Created `ExceptionCache/ExceptionCacheModels.cs` with `ExceptionCacheEntry` (cached exception for fast lookup with priority, decision override, expiry), `ExceptionCacheQueryResult` (query results with cache metadata), `ExceptionCacheSummary` (tenant summary with counts by type/decision), `ExceptionCacheOptions` (TTL, auto-warm, max entries), `ExceptionCacheStats` (hit/miss counts, memory usage). Created `ExceptionCache/IExceptionEffectiveCache.cs` interface with `GetForAssetAsync`, `GetBatchAsync`, `SetAsync`, `SetBatchAsync`, `InvalidateExceptionAsync`, `InvalidateAssetAsync`, `InvalidateTenantAsync`, `WarmAsync`, `HandleExceptionEventAsync` for event-driven invalidation; `ExceptionEvent` record for exception lifecycle events (activated, expired, revoked, updated, created, deleted). Created `ExceptionCache/RedisExceptionEffectiveCache.cs` Redis implementation with key structure: `stellaops:exc:{tenant}:a:{asset}:{advisory}` for asset entries, `stellaops:exc:{tenant}:idx:e:{exceptionId}` for exception-to-asset index, `stellaops:exc:{tenant}:v` for version counter. Warm logic loads from `IExceptionRepository` for active/pending exceptions. Invalidation reacts to exception events. Added `ExceptionCacheOptions` to `PolicyEngineOptions`. Added `policy_exception_cache_operations_total` telemetry counter with `RecordExceptionCacheOperation()` method. Added `AddExceptionEffectiveCache()` DI extension. All 197 core tests pass. POLICY-ENGINE-70-003 marked DONE. | Implementer |
|
||||
|
||||
@@ -17,9 +17,9 @@ Dependency: Sprint 133 - 4. Scanner.IV — Scanner & Surface focus on Scanner (p
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-003` | DONE | Static import graph builder using AST and bytecode fallback. Support `import`, `from ... import`, relative imports, `importlib.import_module`, `__import__` with literal args, `pkgutil.extend_path`. | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-002 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-004` | DONE | Python resolver engine (importlib semantics) handling namespace packages (PEP 420), package discovery order, `.pth` files, `sys.path` composition, zipimport, and site-packages precedence across virtualenv/container roots. | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-003 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-005` | DONE | Packaging adapters: pip editable (`.egg-link`), Poetry/Flit layout, Conda prefix, `.dist-info/RECORD` cross-check, container layer overlays. | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-004 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-006` | TODO | Detect native extensions (`*.so`, `*.pyd`), CFFI modules, ctypes loaders, embedded WASM, and runtime capability signals (subprocess, multiprocessing, ctypes, eval). | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-005 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-007` | TODO | Framework/config heuristics: Django, Flask, FastAPI, Celery, AWS Lambda handlers, Gunicorn, Click/Typer CLIs, logging configs, pyproject optional dependencies. Tagged as hints only. | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-006 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-008` | TODO | Produce AOC-compliant observations: entrypoints, components (modules/packages/native), edges (import, namespace, dynamic-hint, native-extension) with reason codes/confidence and resolver traces. | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-007 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-009` | TODO | Fixture suite + perf benchmarks covering virtualenv, namespace packages, zipapp, editable installs, containers, lambda handler. | Python Analyzer Guild, QA Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-008 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-010` | TODO | Optional runtime evidence: import hook capturing module load events with path scrubbing, optional bytecode instrumentation for `importlib` hooks, multiprocessing tracer. | Python Analyzer Guild, Signals Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-009 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-011` | TODO | Package analyzer plug-in, add CLI commands (`stella python inspect`), refresh Offline Kit documentation. | Python Analyzer Guild, DevOps Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-010 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-006` | DONE | Detect native extensions (`*.so`, `*.pyd`), CFFI modules, ctypes loaders, embedded WASM, and runtime capability signals (subprocess, multiprocessing, ctypes, eval). | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-005 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-007` | DONE | Framework/config heuristics: Django, Flask, FastAPI, Celery, AWS Lambda handlers, Gunicorn, Click/Typer CLIs, logging configs, pyproject optional dependencies. Tagged as hints only. | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-006 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-008` | DONE | Produce AOC-compliant observations: entrypoints, components (modules/packages/native), edges (import, namespace, dynamic-hint, native-extension) with reason codes/confidence and resolver traces. | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-007 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-009` | DONE | Fixture suite + perf benchmarks covering virtualenv, namespace packages, zipapp, editable installs, containers, lambda handler. | Python Analyzer Guild, QA Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-008 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-010` | DONE | Optional runtime evidence: import hook capturing module load events with path scrubbing, optional bytecode instrumentation for `importlib` hooks, multiprocessing tracer. | Python Analyzer Guild, Signals Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-009 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-011` | DONE | Package analyzer plug-in, add CLI commands (`stella python inspect`), refresh Offline Kit documentation. | Python Analyzer Guild, DevOps Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-010 |
|
||||
|
||||
@@ -7,7 +7,7 @@ Dependency: Sprint 134 - 5. Scanner.V — Scanner & Surface focus on Scanner (ph
|
||||
|
||||
| Task ID | State | Summary | Owner / Source | Depends On |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-012` | TODO | Container/zipapp adapter enhancements: parse OCI layers for Python runtime, detect `PYTHONPATH`/`PYTHONHOME` env, record warnings for sitecustomize/startup hooks. | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-011 |
|
||||
| `SCANNER-ANALYZERS-PYTHON-23-012` | DONE | Container/zipapp adapter enhancements: parse OCI layers for Python runtime, detect `PYTHONPATH`/`PYTHONHOME` env, record warnings for sitecustomize/startup hooks. | Python Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Python) | SCANNER-ANALYZERS-PYTHON-23-011 |
|
||||
| `SCANNER-ANALYZERS-RUBY-28-001` | DONE | Build input normalizer & VFS for Ruby projects: merge source trees, Gemfile/Gemfile.lock, vendor/bundle, .gem archives, `.bundle/config`, Rack configs, containers. Detect framework/job fingerprints deterministically. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | — |
|
||||
| `SCANNER-ANALYZERS-RUBY-28-002` | DONE | Gem & Bundler analyzer: parse Gemfile/Gemfile.lock, vendor specs, .gem archives, produce package nodes (PURLs), dependency edges, bin scripts, Bundler group metadata. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ANALYZERS-RUBY-28-001 |
|
||||
| `SCANNER-ANALYZERS-RUBY-28-003` | DONE | Require/autoload graph builder: resolve static/dynamic require, require_relative, load; infer Zeitwerk autoload paths and Rack boot chain. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ANALYZERS-RUBY-28-002 |
|
||||
|
||||
@@ -17,7 +17,7 @@ Dependency: Sprint 135 - 6. Scanner.VI — Scanner & Surface focus on Scanner (p
|
||||
| `SURFACE-ENV-02` | DONE (2025-11-18) | Strongly-typed env accessors implemented; validation covers required endpoint, bounds, TLS cert path; regression tests passing. | Scanner Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Env) | SURFACE-ENV-01 |
|
||||
| `SURFACE-ENV-03` | DONE (2025-11-27) | Adopt the env helper across Scanner Worker/WebService/BuildX plug-ins. | Scanner Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Env) | SURFACE-ENV-02 |
|
||||
| `SURFACE-ENV-04` | DONE (2025-11-27) | Wire env helper into Zastava Observer/Webhook containers. | Zastava Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Env) | SURFACE-ENV-02 |
|
||||
| `SURFACE-ENV-05` | TODO | Update Helm/Compose/offline kit templates with new env knobs and documentation. | Ops Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Env) | SURFACE-ENV-03, SURFACE-ENV-04 |
|
||||
| `SURFACE-ENV-05` | DONE | Update Helm/Compose/offline kit templates with new env knobs and documentation. | Ops Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Env) | SURFACE-ENV-03, SURFACE-ENV-04 |
|
||||
| `SCANNER-EVENTS-16-301` | BLOCKED (2025-10-26) | Emit orchestrator-compatible envelopes (`scanner.event.*`) and update integration tests to verify Notifier ingestion (no Redis queue coupling). | Scanner WebService Guild (src/Scanner/StellaOps.Scanner.WebService) | — |
|
||||
| `SCANNER-GRAPH-21-001` | DONE (2025-11-27) | Provide webhook/REST endpoint for Cartographer to request policy overlays and runtime evidence for graph nodes, ensuring determinism and tenant scoping. | Scanner WebService Guild, Cartographer Guild (src/Scanner/StellaOps.Scanner.WebService) | — |
|
||||
| `SCANNER-LNM-21-001` | BLOCKED (2025-11-27) | Update `/reports` and `/policy/runtime` payloads to consume advisory/vex linksets, exposing source severity arrays and conflict summaries alongside effective verdicts. Blocked: requires Concelier HTTP client integration or shared library; no existing Concelier dependency in Scanner WebService. | Scanner WebService Guild, Policy Guild (src/Scanner/StellaOps.Scanner.WebService) | — |
|
||||
@@ -29,8 +29,8 @@ Dependency: Sprint 135 - 6. Scanner.VI — Scanner & Surface focus on Scanner (p
|
||||
| `SURFACE-SECRETS-04` | DONE (2025-11-27) | Integrate Surface.Secrets into Scanner Worker/WebService/BuildX for registry + CAS creds. | Scanner Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Secrets) | SURFACE-SECRETS-02 |
|
||||
| `SURFACE-SECRETS-05` | DONE (2025-11-27) | Invoke Surface.Secrets from Zastava Observer/Webhook for CAS & attestation secrets. | Zastava Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Secrets) | SURFACE-SECRETS-02 |
|
||||
| `SURFACE-SECRETS-06` | BLOCKED (2025-11-27) | Update deployment manifests/offline kit bundles to provision secret references instead of raw values. Requires Ops Guild input on Helm/Compose patterns for Surface.Secrets provider configuration. | Ops Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Secrets) | SURFACE-SECRETS-03 |
|
||||
| `SCANNER-ENG-0020` | TODO | Implement Homebrew collector & fragment mapper per `design/macos-analyzer.md` §3.1. | Scanner Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0021` | TODO | Implement pkgutil receipt collector per `design/macos-analyzer.md` §3.2. | Scanner Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0020` | DONE (2025-11-28) | Implement Homebrew collector & fragment mapper per `design/macos-analyzer.md` §3.1. | Scanner Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0021` | DONE (2025-11-28) | Implement pkgutil receipt collector per `design/macos-analyzer.md` §3.2. | Scanner Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0022` | TODO | Implement macOS bundle inspector & capability overlays per `design/macos-analyzer.md` §3.3. | Scanner Guild, Policy Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0023` | TODO | Deliver macOS policy/offline integration per `design/macos-analyzer.md` §5–6. | Scanner Guild, Offline Kit Guild, Policy Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0024` | TODO | Implement Windows MSI collector per `design/windows-analyzer.md` §3.1. | Scanner Guild (docs/modules/scanner) | — |
|
||||
@@ -42,7 +42,7 @@ Dependency: Sprint 135 - 6. Scanner.VI — Scanner & Surface focus on Scanner (p
|
||||
| `SURFACE-FS-03` | DONE (2025-11-27) | Integrate Surface.FS writer into Scanner Worker analyzer pipeline to persist layer + entry-trace fragments. | Scanner Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.FS) | SURFACE-FS-02 |
|
||||
| `SURFACE-FS-04` | DONE (2025-11-27) | Integrate Surface.FS reader into Zastava Observer runtime drift loop. | Zastava Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.FS) | SURFACE-FS-02 |
|
||||
| `SURFACE-FS-05` | DONE (2025-11-27) | Expose Surface.FS pointers via Scanner WebService reports and coordinate rescan planning with Scheduler. | Scanner Guild, Scheduler Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.FS) | SURFACE-FS-03 |
|
||||
| `SURFACE-FS-06` | TODO | Update scanner-engine guide and offline kit docs with Surface.FS workflow. | Docs Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.FS) | SURFACE-FS-02..05 |
|
||||
| `SURFACE-FS-06` | DONE (2025-11-28) | Update scanner-engine guide and offline kit docs with Surface.FS workflow. | Docs Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.FS) | SURFACE-FS-02..05 |
|
||||
| `SCANNER-SURFACE-04` | TODO | DSSE-sign every `layer.fragments` payload, emit `_composition.json`, and persist DSSE envelopes so offline kits can replay deterministically (see `docs/modules/scanner/deterministic-sbom-compose.md` §2.1). | Scanner Worker Guild (src/Scanner/StellaOps.Scanner.Worker) | SCANNER-SURFACE-01, SURFACE-FS-03 |
|
||||
| `SURFACE-FS-07` | TODO | Extend Surface.FS manifest schema with `composition.recipe`, fragment attestation metadata, and verification helpers per deterministic SBOM spec. | Scanner Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.FS) | SCANNER-SURFACE-04 |
|
||||
| `SCANNER-EMIT-15-001` | TODO | Enforce canonical JSON (`stella.contentHash`, Merkle root metadata, zero timestamps) for fragments and composed CycloneDX inventory/usage BOMs. Documented in `docs/modules/scanner/deterministic-sbom-compose.md` §2.2. | Scanner Emit Guild (src/Scanner/__Libraries/StellaOps.Scanner.Emit) | SCANNER-SURFACE-04 |
|
||||
@@ -51,7 +51,7 @@ Dependency: Sprint 135 - 6. Scanner.VI — Scanner & Surface focus on Scanner (p
|
||||
| `SURFACE-VAL-02` | DONE (2025-11-23) | Validation library now enforces secrets schema, fallback/provider checks, and inline/file guardrails; tests added. | Scanner Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Validation) | SURFACE-VAL-01, SURFACE-ENV-02, SURFACE-FS-02 |
|
||||
| `SURFACE-VAL-03` | DONE (2025-11-23) | Validation runner wired into Worker/WebService startup and pre-analyzer paths (OS, language, EntryTrace). | Scanner Guild, Analyzer Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Validation) | SURFACE-VAL-02 |
|
||||
| `SURFACE-VAL-04` | DONE (2025-11-27) | Expose validation helpers to Zastava and other runtime consumers for preflight checks. | Scanner Guild, Zastava Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Validation) | SURFACE-VAL-02 |
|
||||
| `SURFACE-VAL-05` | TODO | Document validation extensibility, registration, and customization in scanner-engine guides. | Docs Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Validation) | SURFACE-VAL-02 |
|
||||
| `SURFACE-VAL-05` | DONE | Document validation extensibility, registration, and customization in scanner-engine guides. | Docs Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.Validation) | SURFACE-VAL-02 |
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
@@ -86,3 +86,7 @@ Dependency: Sprint 135 - 6. Scanner.VI — Scanner & Surface focus on Scanner (p
|
||||
| 2025-11-18 | Attempted `dotnet test` for Worker Surface manifest configurator; restore failed fetching StackExchange.Redis from nuget.org (network timeout); tests still pending CI. | Implementer |
|
||||
| 2025-11-18 | SCANNER-ENV-03 started: BuildX plugin now loads Surface.Env defaults (SCANNER/SURFACE prefixes) for cache root/bucket/tenant when args/env missing; tests not yet added. | Implementer |
|
||||
| 2025-11-19 | Marked SCANNER-ENV-03, SURFACE-SECRETS-01/02, and SURFACE-VAL-01 BLOCKED pending Security/Surface schema approvals and published env/secrets artifacts; move back to TODO once upstream contracts land. | Implementer |
|
||||
| 2025-11-28 | Created `docs/modules/scanner/guides/surface-validation-extensibility.md` covering custom validators, reporters, configuration, and testing; SURFACE-VAL-05 DONE. | Implementer |
|
||||
| 2025-11-28 | Created `docs/modules/scanner/guides/surface-fs-workflow.md` with end-to-end workflow including artefact generation, storage layout, consumption, and offline kit handling; SURFACE-FS-06 DONE. | Implementer |
|
||||
| 2025-11-28 | Created `StellaOps.Scanner.Analyzers.OS.Homebrew` library with `HomebrewReceiptParser` (INSTALL_RECEIPT.json parsing), `HomebrewPackageAnalyzer` (Cellar discovery for Intel/Apple Silicon), and `HomebrewAnalyzerPlugin`; added `BuildHomebrew` PURL builder, `HomebrewCellar` evidence source; 23 tests passing. SCANNER-ENG-0020 DONE. | Implementer |
|
||||
| 2025-11-28 | Created `StellaOps.Scanner.Analyzers.OS.Pkgutil` library with `PkgutilReceiptParser` (plist parsing), `BomParser` (BOM file enumeration), `PkgutilPackageAnalyzer` (receipt discovery from /var/db/receipts), and `PkgutilAnalyzerPlugin`; added `BuildPkgutil` PURL builder, `PkgutilReceipt` evidence source; 9 tests passing. SCANNER-ENG-0021 DONE. | Implementer |
|
||||
|
||||
@@ -7,15 +7,15 @@ Depends on: Sprint 180.A - Cli.IV
|
||||
Summary: Experience & SDKs focus on Cli (phase V).
|
||||
Task ID | State | Task description | Owners (Source)
|
||||
--- | --- | --- | ---
|
||||
CLI-TEN-47-001 | DOING | Implement `stella login`, `whoami`, `tenants list`, persistent profiles, secure token storage, and `--tenant` override with validation. Partial: `auth login`, `auth whoami` already exist; `tenants list` implemented. Remaining: persistent profiles, secure token storage enhancements, `--tenant` override validation. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-TEN-49-001 | TODO | Add service account token minting, delegation (`stella token delegate`), impersonation banner, and audit-friendly logging. Dependencies: CLI-TEN-47-001. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VEX-30-001 | TODO | Implement `stella vex consensus list` with filters, paging, policy selection, `--json/--csv`. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VEX-30-002 | TODO | Implement `stella vex consensus show` displaying quorum, evidence, rationale, signature status. Dependencies: CLI-VEX-30-001. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VEX-30-003 | TODO | Implement `stella vex simulate` for trust/threshold overrides with JSON diff output. Dependencies: CLI-VEX-30-002. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VEX-30-004 | TODO | Implement `stella vex export` for consensus NDJSON bundles with signature verification helper. Dependencies: CLI-VEX-30-003. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-001 | TODO | Implement `stella vuln list` with grouping, paging, filters, `--json/--csv`, and policy selection. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-002 | TODO | Implement `stella vuln show` displaying evidence, policy rationale, paths, ledger summary; support `--json` for automation. Dependencies: CLI-VULN-29-001. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-003 | TODO | Add workflow commands (`assign`, `comment`, `accept-risk`, `verify-fix`, `target-fix`, `reopen`) with filter selection (`--filter`) and idempotent retries. Dependencies: CLI-VULN-29-002. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-004 | TODO | Implement `stella vuln simulate` producing delta summaries and optional Markdown report for CI. Dependencies: CLI-VULN-29-003. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-005 | TODO | Add `stella vuln export` and `stella vuln bundle verify` commands to trigger/download evidence bundles and verify signatures. Dependencies: CLI-VULN-29-004. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-006 | TODO | Update CLI docs/examples for Vulnerability Explorer with compliance checklist and CI snippets. Dependencies: CLI-VULN-29-005. | DevEx/CLI Guild, Docs Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-TEN-47-001 | DONE | Implement `stella login`, `whoami`, `tenants list`, persistent profiles, secure token storage, and `--tenant` override with validation. Completed: `auth login`/`auth whoami` existed; `tenants list`/`use`/`current`/`clear` commands added; TenantProfileStore for persistent profiles at ~/.stellaops/profile.json; global `--tenant` option with profile fallback; tenant validation against Authority when available. Token storage uses existing file cache at ~/.stellaops/tokens/. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-TEN-49-001 | DONE | Add service account token minting, delegation (`stella token delegate`), impersonation banner, and audit-friendly logging. Completed: `auth token mint` and `auth token delegate` commands; TokenMint/DelegateRequest/Response models; AuthorityConsoleClient extended with MintTokenAsync, DelegateTokenAsync, IntrospectTokenAsync; CheckAndDisplayImpersonationBannerAsync helper for audit-aware impersonation notices. Note: Authority service endpoints (POST /console/token/mint, /delegate, /introspect) need backend implementation. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VEX-30-001 | DONE | Implement `stella vex consensus list` with filters, paging, policy selection, `--json/--csv`. Completed: VexModels.cs with request/response records; IBackendOperationsClient.ListVexConsensusAsync; BackendOperationsClient implementation calling GET /api/vex/consensus; BuildVexCommand in CommandFactory.cs with `vex consensus list` subcommand; HandleVexConsensusListAsync handler with table/JSON/CSV output, tenant resolution via TenantProfileStore, pagination support. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VEX-30-002 | DONE | Implement `stella vex consensus show` displaying quorum, evidence, rationale, signature status. Dependencies: CLI-VEX-30-001. Completed: VexConsensusDetailResponse with quorum/rationale/signature/evidence models; IBackendOperationsClient.GetVexConsensusAsync; BackendOperationsClient implementation calling GET /api/vex/consensus/{vulnId}/{productKey}; `vex consensus show` subcommand in CommandFactory.cs; HandleVexConsensusShowAsync handler with rich Spectre.Console formatted output including panels and tables for all sections. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VEX-30-003 | DONE | Implement `stella vex simulate` for trust/threshold overrides with JSON diff output. Dependencies: CLI-VEX-30-002. Completed: VexSimulationRequest/Response models with TrustOverrides, ThresholdOverride, QuorumOverride, ExcludeProviders; SimulateVexConsensusAsync interface and implementation calling POST /api/vex/consensus/simulate; `vex simulate` command with --trust provider=weight, --threshold, --quorum, --exclude, --include-only, --changed-only options; HandleVexSimulateAsync handler with before/after diff table and summary panel. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VEX-30-004 | DONE | Implement `stella vex export` for consensus NDJSON bundles with signature verification helper. Dependencies: CLI-VEX-30-003. Completed: VexExportRequest/Response models with format, signed, filter options; VexExportVerifyRequest/Result for local verification; IBackendOperationsClient.ExportVexConsensusAsync (POST /api/vex/consensus/export) and DownloadVexExportAsync (GET /api/vex/consensus/export/{exportId}); `vex export` command with --vuln-id, --product-key, --purl, --status, --output, --unsigned filters; `vex export verify` subcommand with --expected-digest and --public-key for local digest/signature verification; HandleVexExportAsync handler with download and progress display; HandleVexVerifyAsync for offline verification with SHA-256 digest calculation. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-001 | DONE | Implement `stella vuln list` with grouping, paging, filters, `--json/--csv`, and policy selection. Completed: VulnModels.cs with VulnListRequest/Response, VulnItem, VulnSeverityInfo, VulnAffectedPackage, VulnGroupingInfo, VulnGroup and all models for CLI-VULN-29-002 through CLI-VULN-29-005; IBackendOperationsClient extended with ListVulnerabilitiesAsync, GetVulnerabilityAsync, ExecuteVulnWorkflowAsync, SimulateVulnerabilitiesAsync, ExportVulnerabilitiesAsync, DownloadVulnExportAsync; BackendOperationsClient HTTP implementations calling GET/POST /api/vuln/*; `vuln list` command with --vuln-id, --severity, --status, --purl, --cpe, --sbom-id, --policy-id, --policy-version, --group-by, --limit, --offset, --cursor, --tenant, --json, --csv options; HandleVulnListAsync handler with grouped and individual table output, CSV output, color-coded severity/status display. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-002 | DONE | Implement `stella vuln show` displaying evidence, policy rationale, paths, ledger summary; support `--json` for automation. Dependencies: CLI-VULN-29-001. Completed: `vuln show` subcommand with vulnerability-id argument, --tenant, --json, --verbose options; HandleVulnShowAsync handler; RenderVulnDetail helper with Spectre.Console panels and tables for: header (ID, status, severity, VEX, aliases, assignee, dates), description, affected packages table, policy rationale panel with rules, evidence table, dependency paths, workflow ledger history table, references list. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-003 | DONE | Add workflow commands (`assign`, `comment`, `accept-risk`, `verify-fix`, `target-fix`, `reopen`) with filter selection (`--filter`) and idempotent retries. Dependencies: CLI-VULN-29-002. Completed: Six workflow subcommands under `vuln` command: `assign <assignee>`, `comment <text>`, `accept-risk <justification> [--due-date]`, `verify-fix <fix-version>`, `target-fix <version> [--due-date]`, `reopen <comment>`. All commands share common options: --vuln-id (multi-value), --filter-severity, --filter-status, --filter-purl, --filter-sbom for bulk operations; --tenant, --idempotency-key for retries, --json for automation. HandleVulnWorkflowAsync handler builds VulnWorkflowRequest with action-specific fields, calls ExecuteVulnWorkflowAsync (POST /api/vuln/workflow), renders success/error table with affected counts. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-004 | DONE | Implement `stella vuln simulate` producing delta summaries and optional Markdown report for CI. Dependencies: CLI-VULN-29-003. Completed: `vuln simulate` subcommand with --policy-id, --policy-version, --vex-override vulnId=status (multi), --severity-threshold, --sbom-id (multi), --markdown, --changed-only, --output (file), --tenant, --json options; HandleVulnSimulateAsync handler parsing VEX overrides into Dictionary, building VulnSimulationRequest, calling SimulateVulnerabilitiesAsync (POST /api/vuln/simulate); output includes simulation summary panel (total/changed/upgrades/downgrades/nochange), delta table with before/after status and change indicator (UPGRADE/DOWNGRADE), optional Markdown report to file or console for CI integration. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-005 | DONE | Add `stella vuln export` and `stella vuln bundle verify` commands to trigger/download evidence bundles and verify signatures. Dependencies: CLI-VULN-29-004. Completed: `vuln export` command with --vuln-id (multi), --sbom-id (multi), --policy-id, --format (ndjson/json), --include-evidence, --include-ledger, --signed (defaults true), --output (required), --tenant options; HandleVulnExportAsync handler calling ExportVulnerabilitiesAsync (POST /api/vuln/export) and DownloadVulnExportAsync to stream bundle to file; output displays item count, format, signature info, digest; `vuln export verify` subcommand with file argument, --expected-digest, --public-key options; HandleVulnExportVerifyAsync performs SHA-256 digest calculation, optional signature file detection (.sig), renders verification panel with pass/fail status. | DevEx/CLI Guild (src/Cli/StellaOps.Cli)
|
||||
CLI-VULN-29-006 | DONE | Update CLI docs/examples for Vulnerability Explorer with compliance checklist and CI snippets. Dependencies: CLI-VULN-29-005. Completed: Created docs/modules/cli/guides/vuln-explorer-cli.md with comprehensive documentation covering: Prerequisites (scopes, connectivity); vuln list with filters, grouping, pagination, --json/--csv; vuln show with all output sections; Workflow commands (assign, comment, accept-risk, verify-fix, target-fix, reopen) with idempotency support; vuln simulate for policy/VEX delta analysis with CI Markdown output; vuln export and export verify for compliance bundles; Exit codes table; Compliance checklist (inventory, SLA, risk acceptance audit, evidence bundles); CI pipeline snippets for GitHub Actions, GitLab CI, Jenkins; Offline operation guidance. | DevEx/CLI Guild, Docs Guild (src/Cli/StellaOps.Cli)
|
||||
502
docs/modules/cli/guides/vuln-explorer-cli.md
Normal file
502
docs/modules/cli/guides/vuln-explorer-cli.md
Normal file
@@ -0,0 +1,502 @@
|
||||
# CLI Vulnerability Explorer Commands Reference
|
||||
|
||||
> **Audience:** DevEx engineers, security operators, and CI authors managing vulnerabilities through the `stella` CLI.
|
||||
> **Scope:** Command synopsis, options, exit codes, and CI integration patterns for `stella vuln` commands as introduced in Sprint 205.
|
||||
|
||||
The Vulnerability Explorer CLI enables comprehensive vulnerability management including listing, inspection, workflow operations, policy simulation, and export. All commands support multi-tenant environments and integrate with StellaOps Authority for authentication.
|
||||
|
||||
---
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
- CLI version: `stella` >= 0.21.0 (Vulnerability Explorer feature gate enabled).
|
||||
- Required scopes (DPoP-bound):
|
||||
- `vuln:view` for listing and viewing vulnerabilities.
|
||||
- `vuln:workflow` for workflow operations (assign, comment, accept-risk, etc.).
|
||||
- `vuln:simulate` for policy simulation.
|
||||
- `vuln:export` for exporting evidence bundles.
|
||||
- `tenant:select` if using tenant switching.
|
||||
- Connectivity: direct access to Backend APIs or configured backend URL.
|
||||
- Environment: set `STELLAOPS_BACKEND_URL`, `STELLA_TENANT`, and authenticate via `stella auth login`.
|
||||
|
||||
---
|
||||
|
||||
## 2. `stella vuln list`
|
||||
|
||||
### 2.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella vuln list \
|
||||
[--vuln-id <id>] \
|
||||
[--severity critical|high|medium|low] \
|
||||
[--status open|triaged|accepted|fixed|risk_accepted] \
|
||||
[--purl <package-url>] \
|
||||
[--cpe <cpe>] \
|
||||
[--sbom-id <sbom-id>] \
|
||||
[--policy-id <policy-id>] \
|
||||
[--policy-version <version>] \
|
||||
[--group-by severity|status|sbom|policy] \
|
||||
[--limit <n>] [--offset <n>] [--cursor <token>] \
|
||||
[--tenant <tenant-id>] \
|
||||
[--json] [--csv] [--verbose]
|
||||
```
|
||||
|
||||
### 2.2 Description
|
||||
|
||||
Lists vulnerabilities matching the specified filters with pagination support. Supports grouped summaries for reporting and machine-readable output for automation.
|
||||
|
||||
### 2.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--vuln-id <id>` | Filter by vulnerability ID (e.g., CVE-2024-1234). |
|
||||
| `--severity <level>` | Filter by severity (critical, high, medium, low). |
|
||||
| `--status <status>` | Filter by workflow status. |
|
||||
| `--purl <package-url>` | Filter by Package URL pattern. |
|
||||
| `--cpe <cpe>` | Filter by CPE pattern. |
|
||||
| `--sbom-id <sbom-id>` | Filter by SBOM identifier. |
|
||||
| `--policy-id <policy-id>` | Filter by policy ID. |
|
||||
| `--policy-version <version>` | Filter by policy version. |
|
||||
| `--group-by <field>` | Group results by field (shows summary counts). |
|
||||
| `--limit <n>` | Maximum results to return (default 50). |
|
||||
| `--offset <n>` | Number of results to skip. |
|
||||
| `--cursor <token>` | Pagination cursor from previous response. |
|
||||
| `--tenant <tenant-id>` | Override tenant for multi-tenant deployments. |
|
||||
| `--json` | Output as JSON for automation. |
|
||||
| `--csv` | Output as CSV for spreadsheet import. |
|
||||
| `--verbose` | Enable debug logging. |
|
||||
|
||||
### 2.4 Examples
|
||||
|
||||
List critical vulnerabilities:
|
||||
|
||||
```bash
|
||||
stella vuln list --severity critical
|
||||
```
|
||||
|
||||
Group by status for reporting:
|
||||
|
||||
```bash
|
||||
stella vuln list --group-by status --json > status-summary.json
|
||||
```
|
||||
|
||||
Export CSV for compliance audit:
|
||||
|
||||
```bash
|
||||
stella vuln list --severity critical --severity high --csv > critical-vulns.csv
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. `stella vuln show`
|
||||
|
||||
### 3.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella vuln show <vulnerability-id> \
|
||||
[--tenant <tenant-id>] \
|
||||
[--json] [--verbose]
|
||||
```
|
||||
|
||||
### 3.2 Description
|
||||
|
||||
Displays detailed information about a specific vulnerability including severity, affected packages, policy rationale, evidence, dependency paths, and workflow history.
|
||||
|
||||
### 3.3 Output Sections
|
||||
|
||||
- **Header:** Vulnerability ID, status, severity, VEX status, aliases, assignee, dates.
|
||||
- **Description:** Full vulnerability description.
|
||||
- **Affected Packages:** Table of affected packages with versions and fix status.
|
||||
- **Policy Rationale:** Active policy rules and their evaluation results.
|
||||
- **Evidence:** Timeline of evidence collected.
|
||||
- **Dependency Paths:** Transitive dependency chains leading to vulnerability.
|
||||
- **Workflow History:** Audit ledger of all workflow actions.
|
||||
- **References:** Links to advisories, patches, and documentation.
|
||||
|
||||
### 3.4 Example
|
||||
|
||||
```bash
|
||||
stella vuln show CVE-2024-1234 --json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Workflow Commands
|
||||
|
||||
All workflow commands support bulk operations via `--vuln-id` (repeatable) or filter options.
|
||||
|
||||
### 4.1 `stella vuln assign`
|
||||
|
||||
Assign vulnerabilities to a team member.
|
||||
|
||||
```bash
|
||||
stella vuln assign <assignee> \
|
||||
[--vuln-id <id>]... \
|
||||
[--filter-severity <level>] \
|
||||
[--filter-status <status>] \
|
||||
[--filter-purl <pattern>] \
|
||||
[--filter-sbom <sbom-id>] \
|
||||
[--tenant <tenant-id>] \
|
||||
[--idempotency-key <key>] \
|
||||
[--json] [--verbose]
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
stella vuln assign security-team \
|
||||
--filter-severity critical \
|
||||
--filter-status open
|
||||
```
|
||||
|
||||
### 4.2 `stella vuln comment`
|
||||
|
||||
Add a comment to vulnerabilities.
|
||||
|
||||
```bash
|
||||
stella vuln comment "<text>" \
|
||||
--vuln-id CVE-2024-1234 \
|
||||
[--json]
|
||||
```
|
||||
|
||||
### 4.3 `stella vuln accept-risk`
|
||||
|
||||
Accept risk for vulnerabilities with documented justification.
|
||||
|
||||
```bash
|
||||
stella vuln accept-risk "<justification>" \
|
||||
--vuln-id CVE-2024-1234 \
|
||||
[--due-date 2025-12-31] \
|
||||
[--json]
|
||||
```
|
||||
|
||||
### 4.4 `stella vuln verify-fix`
|
||||
|
||||
Mark vulnerabilities as fixed and verified.
|
||||
|
||||
```bash
|
||||
stella vuln verify-fix <fix-version> \
|
||||
--vuln-id CVE-2024-1234 \
|
||||
[--json]
|
||||
```
|
||||
|
||||
### 4.5 `stella vuln target-fix`
|
||||
|
||||
Set target fix version and due date.
|
||||
|
||||
```bash
|
||||
stella vuln target-fix <version> \
|
||||
--vuln-id CVE-2024-1234 \
|
||||
[--due-date 2025-06-30] \
|
||||
[--json]
|
||||
```
|
||||
|
||||
### 4.6 `stella vuln reopen`
|
||||
|
||||
Reopen previously closed vulnerabilities.
|
||||
|
||||
```bash
|
||||
stella vuln reopen "<reason>" \
|
||||
--vuln-id CVE-2024-1234 \
|
||||
[--json]
|
||||
```
|
||||
|
||||
### 4.7 Idempotency
|
||||
|
||||
All workflow commands support `--idempotency-key` for safe retries in CI pipelines:
|
||||
|
||||
```bash
|
||||
stella vuln assign security-team \
|
||||
--vuln-id CVE-2024-1234 \
|
||||
--idempotency-key "assign-cve-2024-1234-$(date +%Y%m%d)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. `stella vuln simulate`
|
||||
|
||||
### 5.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella vuln simulate \
|
||||
[--policy-id <id>] \
|
||||
[--policy-version <version>] \
|
||||
[--vex-override <vulnId>=<status>]... \
|
||||
[--severity-threshold <level>] \
|
||||
[--sbom-id <id>]... \
|
||||
[--markdown] \
|
||||
[--changed-only] \
|
||||
[--output <file>] \
|
||||
[--tenant <tenant-id>] \
|
||||
[--json] [--verbose]
|
||||
```
|
||||
|
||||
### 5.2 Description
|
||||
|
||||
Simulates the impact of policy or VEX changes without modifying data. Produces delta summaries showing which vulnerabilities would change status, useful for policy review and CI gates.
|
||||
|
||||
### 5.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--policy-id <id>` | Policy ID to simulate. |
|
||||
| `--policy-version <version>` | Policy version to simulate against. |
|
||||
| `--vex-override <vulnId>=<status>` | Override VEX status for simulation (repeatable). |
|
||||
| `--severity-threshold <level>` | Minimum severity to include. |
|
||||
| `--sbom-id <id>` | Limit simulation to specific SBOMs (repeatable). |
|
||||
| `--markdown` | Include Markdown report for CI. |
|
||||
| `--changed-only` | Only show items that would change. |
|
||||
| `--output <file>` | Write Markdown report to file. |
|
||||
| `--json` | Output full simulation results as JSON. |
|
||||
|
||||
### 5.4 Output
|
||||
|
||||
The command displays:
|
||||
- **Summary Panel:** Total evaluated, changed, upgrades, downgrades.
|
||||
- **Delta Table:** Before/after status comparison with UPGRADE/DOWNGRADE indicators.
|
||||
- **Markdown Report:** Optional CI-friendly report.
|
||||
|
||||
### 5.5 CI Integration Example
|
||||
|
||||
```bash
|
||||
# Run simulation and fail if any downgrades
|
||||
stella vuln simulate \
|
||||
--policy-id prod-policy \
|
||||
--changed-only \
|
||||
--markdown \
|
||||
--output simulation-report.md
|
||||
|
||||
# Check exit code
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "Simulation found issues - see simulation-report.md"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. `stella vuln export`
|
||||
|
||||
### 6.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella vuln export \
|
||||
--output <file> \
|
||||
[--vuln-id <id>]... \
|
||||
[--sbom-id <id>]... \
|
||||
[--policy-id <id>] \
|
||||
[--format ndjson|json] \
|
||||
[--include-evidence] \
|
||||
[--include-ledger] \
|
||||
[--signed] \
|
||||
[--tenant <tenant-id>] \
|
||||
[--verbose]
|
||||
```
|
||||
|
||||
### 6.2 Description
|
||||
|
||||
Exports vulnerability evidence bundles for compliance documentation, audits, or offline analysis. Bundles can be cryptographically signed for integrity verification.
|
||||
|
||||
### 6.3 Options
|
||||
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--output <file>` | Output file path (required). |
|
||||
| `--vuln-id <id>` | Vulnerability IDs to include (repeatable). |
|
||||
| `--sbom-id <id>` | SBOM IDs to scope export (repeatable). |
|
||||
| `--policy-id <id>` | Policy ID for filtering. |
|
||||
| `--format <fmt>` | Output format: `ndjson` (default) or `json`. |
|
||||
| `--include-evidence` | Include evidence data (default: true). |
|
||||
| `--include-ledger` | Include workflow ledger (default: true). |
|
||||
| `--signed` | Request signed bundle (default: true). |
|
||||
|
||||
### 6.4 Example
|
||||
|
||||
```bash
|
||||
stella vuln export \
|
||||
--output compliance-bundle.ndjson \
|
||||
--sbom-id prod-app-sbom \
|
||||
--signed
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. `stella vuln export verify`
|
||||
|
||||
### 7.1 Synopsis
|
||||
|
||||
```bash
|
||||
stella vuln export verify <file> \
|
||||
[--expected-digest <sha256:hex>] \
|
||||
[--public-key <key-file>] \
|
||||
[--verbose]
|
||||
```
|
||||
|
||||
### 7.2 Description
|
||||
|
||||
Verifies the integrity and optional signature of an exported vulnerability bundle. Use this to validate bundles received from external sources or stored archives.
|
||||
|
||||
### 7.3 Example
|
||||
|
||||
```bash
|
||||
stella vuln export verify compliance-bundle.ndjson \
|
||||
--expected-digest sha256:abc123... \
|
||||
--public-key /path/to/public.pem
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Exit Codes
|
||||
|
||||
| Exit Code | Meaning |
|
||||
|-----------|---------|
|
||||
| `0` | Command completed successfully. |
|
||||
| `1` | General error (see error message). |
|
||||
| `130` | Operation cancelled by user (Ctrl+C). |
|
||||
|
||||
---
|
||||
|
||||
## 9. Compliance Checklist
|
||||
|
||||
Use these commands to demonstrate vulnerability management compliance:
|
||||
|
||||
### 9.1 Vulnerability Inventory
|
||||
|
||||
```bash
|
||||
# Generate complete vulnerability inventory
|
||||
stella vuln list --json > inventory.json
|
||||
|
||||
# Summary by severity
|
||||
stella vuln list --group-by severity --json > severity-summary.json
|
||||
```
|
||||
|
||||
### 9.2 SLA Compliance
|
||||
|
||||
```bash
|
||||
# Find critical vulns older than 30 days
|
||||
stella vuln list \
|
||||
--severity critical \
|
||||
--status open \
|
||||
--json | jq '.items[] | select(.updatedAt < (now - 2592000 | todate))'
|
||||
```
|
||||
|
||||
### 9.3 Risk Acceptance Audit
|
||||
|
||||
```bash
|
||||
# Export all risk-accepted vulnerabilities with justifications
|
||||
stella vuln list --status risk_accepted --json > risk-accepted.json
|
||||
```
|
||||
|
||||
### 9.4 Evidence Bundle for Audit
|
||||
|
||||
```bash
|
||||
# Export signed evidence bundle
|
||||
stella vuln export \
|
||||
--output audit-$(date +%Y%m%d).ndjson \
|
||||
--signed
|
||||
|
||||
# Verify bundle integrity
|
||||
stella vuln export verify audit-$(date +%Y%m%d).ndjson
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. CI Pipeline Snippets
|
||||
|
||||
### 10.1 GitHub Actions
|
||||
|
||||
```yaml
|
||||
- name: Check Critical Vulnerabilities
|
||||
run: |
|
||||
count=$(stella vuln list --severity critical --status open --json | jq '.total')
|
||||
if [ "$count" -gt 0 ]; then
|
||||
echo "::error::Found $count critical open vulnerabilities"
|
||||
stella vuln list --severity critical --status open
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Policy Simulation Gate
|
||||
run: |
|
||||
stella vuln simulate \
|
||||
--policy-id ${{ env.POLICY_ID }} \
|
||||
--changed-only \
|
||||
--markdown \
|
||||
--output ${{ github.workspace }}/simulation.md
|
||||
|
||||
cat ${{ github.workspace }}/simulation.md >> $GITHUB_STEP_SUMMARY
|
||||
```
|
||||
|
||||
### 10.2 GitLab CI
|
||||
|
||||
```yaml
|
||||
vuln-check:
|
||||
script:
|
||||
- stella auth login --token $STELLA_TOKEN
|
||||
- |
|
||||
if stella vuln list --severity critical --status open --json | jq -e '.total > 0'; then
|
||||
echo "Critical vulnerabilities found!"
|
||||
stella vuln list --severity critical --status open
|
||||
exit 1
|
||||
fi
|
||||
artifacts:
|
||||
reports:
|
||||
dotenv: vuln-status.env
|
||||
```
|
||||
|
||||
### 10.3 Jenkins Pipeline
|
||||
|
||||
```groovy
|
||||
stage('Vulnerability Check') {
|
||||
steps {
|
||||
sh '''
|
||||
stella vuln list \
|
||||
--severity critical \
|
||||
--severity high \
|
||||
--status open \
|
||||
--csv > vulns.csv
|
||||
'''
|
||||
archiveArtifacts artifacts: 'vulns.csv'
|
||||
|
||||
script {
|
||||
def count = sh(
|
||||
script: "stella vuln list --severity critical --status open --json | jq '.total'",
|
||||
returnStdout: true
|
||||
).trim().toInteger()
|
||||
|
||||
if (count > 0) {
|
||||
error("Found ${count} critical vulnerabilities")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Offline Operation
|
||||
|
||||
When operating in air-gapped environments:
|
||||
|
||||
1. Export vulnerability data before going offline:
|
||||
```bash
|
||||
stella vuln export --output vuln-bundle.ndjson --signed
|
||||
```
|
||||
|
||||
2. Transfer bundle to air-gapped system.
|
||||
|
||||
3. Verify bundle integrity:
|
||||
```bash
|
||||
stella vuln export verify vuln-bundle.ndjson \
|
||||
--expected-digest sha256:...
|
||||
```
|
||||
|
||||
For full offline kit support, see the [Offline Kit documentation](../../../24_OFFLINE_KIT.md).
|
||||
|
||||
---
|
||||
|
||||
## 12. Related Documentation
|
||||
|
||||
- [VEX Consensus CLI](./vex-cli.md) - VEX status management
|
||||
- [Policy Simulation](../../policy/guides/simulation.md) - Policy testing
|
||||
- [Authentication Guide](./auth-cli.md) - Token management
|
||||
- [API Reference](../../../09_API_CLI_REFERENCE.md) - Full API documentation
|
||||
414
docs/modules/scanner/guides/surface-fs-workflow.md
Normal file
414
docs/modules/scanner/guides/surface-fs-workflow.md
Normal file
@@ -0,0 +1,414 @@
|
||||
# Surface.FS Workflow Guide
|
||||
|
||||
> **Version:** 1.0 (2025-11-28)
|
||||
>
|
||||
> **Audience:** Scanner Worker/WebService integrators, Zastava operators, Offline Kit builders
|
||||
|
||||
## Overview
|
||||
|
||||
Surface.FS provides a content-addressable storage layer for Scanner-derived artefacts. This guide covers the end-to-end workflow from artefact generation to consumption, including offline bundle handling.
|
||||
|
||||
## Workflow Stages
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Scanner Worker │───▶│ Surface.FS │───▶│ Consumers │
|
||||
│ - Scan image │ │ - Store manifest│ │ - WebService │
|
||||
│ - Generate │ │ - Store payload │ │ - Zastava │
|
||||
│ artefacts │ │ - Local cache │ │ - CLI │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Generate: │ │ Store: │ │ Consume: │
|
||||
│ - Layer frags │ │ - RustFS/S3 │ │ - Report API │
|
||||
│ - EntryTrace │ │ - Local disk │ │ - Drift detect │
|
||||
│ - SBOM frags │ │ - Offline kit │ │ - Rescan plan │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
## Stage 1: Artefact Generation (Scanner Worker)
|
||||
|
||||
### 1.1 Configure Surface.FS
|
||||
|
||||
```csharp
|
||||
// In Scanner Worker startup
|
||||
builder.Services.AddSurfaceFileCache();
|
||||
builder.Services.AddSurfaceManifestStore();
|
||||
```
|
||||
|
||||
Environment variables (see [Surface.Env guide](../design/surface-env.md)):
|
||||
```bash
|
||||
SCANNER_SURFACE_FS_ENDPOINT=http://rustfs:8080
|
||||
SCANNER_SURFACE_FS_BUCKET=surface-cache
|
||||
SCANNER_SURFACE_CACHE_ROOT=/var/lib/stellaops/surface
|
||||
SCANNER_SURFACE_TENANT=default
|
||||
```
|
||||
|
||||
### 1.2 Generate and Publish Artefacts
|
||||
|
||||
```csharp
|
||||
public async Task<ScanResult> ExecuteScanAsync(ScanJob job, CancellationToken ct)
|
||||
{
|
||||
// 1. Run analyzers to generate artefacts
|
||||
var layerFragments = await AnalyzeLayersAsync(job.Image, ct);
|
||||
var entryTrace = await AnalyzeEntryPointsAsync(job.Image, ct);
|
||||
var sbomFragments = await GenerateSbomAsync(job.Image, ct);
|
||||
|
||||
// 2. Create manifest document
|
||||
var manifest = new SurfaceManifestDocument
|
||||
{
|
||||
Schema = "stellaops.surface.manifest@1",
|
||||
Tenant = _environment.Settings.Tenant,
|
||||
ImageDigest = job.Image.Digest,
|
||||
ScanId = job.Id,
|
||||
GeneratedAt = DateTimeOffset.UtcNow,
|
||||
Source = new SurfaceManifestSource
|
||||
{
|
||||
Component = "scanner.worker",
|
||||
Version = _version,
|
||||
WorkerInstance = Environment.MachineName,
|
||||
Attempt = job.Attempt
|
||||
},
|
||||
Artifacts = new List<SurfaceManifestArtifact>()
|
||||
};
|
||||
|
||||
// 3. Add artefacts to manifest
|
||||
foreach (var fragment in layerFragments)
|
||||
{
|
||||
var payloadUri = await _manifestWriter.StorePayloadAsync(
|
||||
fragment.Content,
|
||||
"layer.fragments",
|
||||
ct);
|
||||
|
||||
manifest.Artifacts.Add(new SurfaceManifestArtifact
|
||||
{
|
||||
Kind = "layer.fragments",
|
||||
Uri = payloadUri,
|
||||
Digest = fragment.Digest,
|
||||
MediaType = "application/vnd.stellaops.layer-fragments+json",
|
||||
Format = "json",
|
||||
SizeBytes = fragment.Content.Length
|
||||
});
|
||||
}
|
||||
|
||||
// 4. Publish manifest
|
||||
var result = await _manifestWriter.PublishAsync(manifest, ct);
|
||||
|
||||
_logger.LogInformation(
|
||||
"Published manifest {Digest} with {Count} artefacts",
|
||||
result.Digest,
|
||||
manifest.Artifacts.Count);
|
||||
|
||||
return new ScanResult
|
||||
{
|
||||
ManifestUri = result.Uri,
|
||||
ManifestDigest = result.Digest
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### 1.3 Cache EntryTrace Results
|
||||
|
||||
```csharp
|
||||
public async Task<EntryTraceGraph?> GetOrComputeEntryTraceAsync(
|
||||
ImageReference image,
|
||||
EntryTraceOptions options,
|
||||
CancellationToken ct)
|
||||
{
|
||||
// Create deterministic cache key
|
||||
var cacheKey = new SurfaceCacheKey(
|
||||
@namespace: "entrytrace.graph",
|
||||
tenant: _environment.Settings.Tenant,
|
||||
digest: ComputeOptionsHash(options, image.Digest));
|
||||
|
||||
// Try cache first
|
||||
var cached = await _cache.TryGetAsync<EntryTraceGraph>(cacheKey, ct);
|
||||
if (cached is not null)
|
||||
{
|
||||
_logger.LogDebug("EntryTrace cache hit for {Key}", cacheKey);
|
||||
return cached;
|
||||
}
|
||||
|
||||
// Compute and cache
|
||||
var graph = await ComputeEntryTraceAsync(image, options, ct);
|
||||
await _cache.SetAsync(cacheKey, graph, ct);
|
||||
|
||||
return graph;
|
||||
}
|
||||
```
|
||||
|
||||
## Stage 2: Storage (Surface.FS)
|
||||
|
||||
### 2.1 Manifest Storage Layout
|
||||
|
||||
```
|
||||
<bucket>/
|
||||
├── manifests/
|
||||
│ └── <tenant>/
|
||||
│ └── <digest[0..1]>/
|
||||
│ └── <digest[2..3]>/
|
||||
│ └── <digest>.json
|
||||
└── payloads/
|
||||
└── <tenant>/
|
||||
└── <kind>/
|
||||
└── sha256/
|
||||
└── <digest[0..1]>/
|
||||
└── <digest[2..3]>/
|
||||
└── <digest>.json.zst
|
||||
```
|
||||
|
||||
### 2.2 Local Cache Layout
|
||||
|
||||
```
|
||||
<cache_root>/
|
||||
├── manifests/ # Manifest JSON files
|
||||
│ └── <tenant>/...
|
||||
├── cache/ # Hot artefacts
|
||||
│ └── <namespace>/
|
||||
│ └── <tenant>/
|
||||
│ └── <digest>
|
||||
└── temp/ # In-progress writes
|
||||
```
|
||||
|
||||
### 2.3 Manifest URI Format
|
||||
|
||||
```
|
||||
cas://<bucket>/<prefix>/<tenant>/<digest[0..1]>/<digest[2..3]>/<digest>.json
|
||||
```
|
||||
|
||||
Example:
|
||||
```
|
||||
cas://surface-cache/manifests/acme/ab/cd/abcdef0123456789...json
|
||||
```
|
||||
|
||||
## Stage 3: Consumption
|
||||
|
||||
### 3.1 WebService API
|
||||
|
||||
```http
|
||||
GET /api/v1/scans/{id}
|
||||
```
|
||||
|
||||
Response includes Surface manifest pointer:
|
||||
```json
|
||||
{
|
||||
"id": "scan-1234",
|
||||
"status": "completed",
|
||||
"surface": {
|
||||
"manifestUri": "cas://surface-cache/manifests/acme/ab/cd/...",
|
||||
"manifestDigest": "sha256:abcdef...",
|
||||
"artifacts": [
|
||||
{
|
||||
"kind": "layer.fragments",
|
||||
"uri": "cas://surface-cache/payloads/acme/layer.fragments/...",
|
||||
"digest": "sha256:123456...",
|
||||
"mediaType": "application/vnd.stellaops.layer-fragments+json"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3.2 Zastava Drift Detection
|
||||
|
||||
```csharp
|
||||
public async Task<DriftResult> DetectDriftAsync(
|
||||
string imageDigest,
|
||||
CancellationToken ct)
|
||||
{
|
||||
// 1. Fetch baseline manifest
|
||||
var manifestUri = await _surfacePointerService.GetManifestUriAsync(imageDigest, ct);
|
||||
var manifest = await _manifestReader.TryGetByUriAsync(manifestUri, ct);
|
||||
|
||||
if (manifest is null)
|
||||
{
|
||||
return DriftResult.NoBaseline();
|
||||
}
|
||||
|
||||
// 2. Get EntryTrace artefact
|
||||
var entryTraceArtifact = manifest.Artifacts
|
||||
.FirstOrDefault(a => a.Kind == "entrytrace.graph");
|
||||
|
||||
if (entryTraceArtifact is null)
|
||||
{
|
||||
return DriftResult.NoEntryTrace();
|
||||
}
|
||||
|
||||
// 3. Compare with runtime
|
||||
var baseline = await _payloadStore.GetAsync<EntryTraceGraph>(
|
||||
entryTraceArtifact.Uri, ct);
|
||||
|
||||
var runtime = await _runtimeCollector.CollectAsync(ct);
|
||||
|
||||
return CompareGraphs(baseline, runtime);
|
||||
}
|
||||
```
|
||||
|
||||
### 3.3 Scheduler Rescan Planning
|
||||
|
||||
```csharp
|
||||
public async Task<RescanPlan> CreateRescanPlanAsync(
|
||||
string imageDigest,
|
||||
CancellationToken ct)
|
||||
{
|
||||
// 1. Read manifest to understand what was scanned
|
||||
var manifest = await _manifestReader.TryGetByDigestAsync(imageDigest, ct);
|
||||
|
||||
if (manifest is null || IsExpired(manifest))
|
||||
{
|
||||
return RescanPlan.FullRescan();
|
||||
}
|
||||
|
||||
// 2. Check for layer changes
|
||||
var layerArtifact = manifest.Artifacts
|
||||
.FirstOrDefault(a => a.Kind == "layer.fragments");
|
||||
|
||||
if (layerArtifact is not null)
|
||||
{
|
||||
var layers = await _payloadStore.GetAsync<LayerFragments>(
|
||||
layerArtifact.Uri, ct);
|
||||
|
||||
var changedLayers = await DetectChangedLayersAsync(layers, ct);
|
||||
|
||||
if (changedLayers.Any())
|
||||
{
|
||||
return RescanPlan.IncrementalRescan(changedLayers);
|
||||
}
|
||||
}
|
||||
|
||||
return RescanPlan.NoRescanNeeded();
|
||||
}
|
||||
```
|
||||
|
||||
## Offline Kit Workflow
|
||||
|
||||
### Export (Online Environment)
|
||||
|
||||
```bash
|
||||
# 1. Build offline kit with Surface manifests
|
||||
python ops/offline-kit/build_offline_kit.py \
|
||||
--version 2025.10.0 \
|
||||
--include-surface-manifests \
|
||||
--output-dir out/offline-kit
|
||||
|
||||
# 2. Kit structure includes:
|
||||
# offline/
|
||||
# surface/
|
||||
# manifests/
|
||||
# <tenant>/<digest[0..1]>/<digest[2..3]>/<digest>.json
|
||||
# payloads/
|
||||
# <tenant>/<kind>/sha256/<digest[0..1]>/<digest[2..3]>/<digest>.json.zst
|
||||
# manifest-index.json
|
||||
```
|
||||
|
||||
### Import (Air-Gapped Environment)
|
||||
|
||||
```csharp
|
||||
public async Task ImportOfflineKitAsync(
|
||||
string kitPath,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var surfacePath = Path.Combine(kitPath, "surface");
|
||||
var indexPath = Path.Combine(surfacePath, "manifest-index.json");
|
||||
|
||||
var index = await LoadIndexAsync(indexPath, ct);
|
||||
|
||||
foreach (var entry in index.Manifests)
|
||||
{
|
||||
// 1. Load and verify manifest
|
||||
var manifestPath = Path.Combine(surfacePath, entry.RelativePath);
|
||||
var manifest = await LoadManifestAsync(manifestPath, ct);
|
||||
|
||||
// 2. Verify digest
|
||||
var computedDigest = ComputeDigest(manifest);
|
||||
if (computedDigest != entry.Digest)
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"Manifest digest mismatch: expected {entry.Digest}, got {computedDigest}");
|
||||
}
|
||||
|
||||
// 3. Import via Surface.FS API
|
||||
await _manifestWriter.PublishAsync(manifest, ct);
|
||||
|
||||
_logger.LogInformation(
|
||||
"Imported manifest {Digest} for image {Image}",
|
||||
entry.Digest,
|
||||
manifest.ImageDigest);
|
||||
}
|
||||
|
||||
// 4. Import payloads
|
||||
foreach (var payload in index.Payloads)
|
||||
{
|
||||
var payloadPath = Path.Combine(surfacePath, payload.RelativePath);
|
||||
await _payloadStore.ImportAsync(payloadPath, payload.Uri, ct);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Offline Operation
|
||||
|
||||
Once imported, Surface.FS consumers operate normally:
|
||||
|
||||
```csharp
|
||||
// Same code works online and offline
|
||||
var manifest = await _manifestReader.TryGetByUriAsync(manifestUri, ct);
|
||||
var payload = await _payloadStore.GetAsync(artifact.Uri, ct);
|
||||
```
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### SurfaceManifestStoreOptions
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `Bucket` | `surface-cache` | Object store bucket |
|
||||
| `ManifestPrefix` | `manifests` | Prefix for manifest objects |
|
||||
| `PayloadPrefix` | `payloads` | Prefix for payload objects |
|
||||
| `LocalManifestRoot` | `<cache>/manifests` | Local manifest directory |
|
||||
|
||||
### SurfaceCacheOptions
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `Root` | `<temp>/stellaops/surface` | Cache root directory |
|
||||
| `QuotaMegabytes` | `4096` | Cache size limit |
|
||||
| `EvictionThreshold` | `0.9` | Trigger eviction at 90% quota |
|
||||
|
||||
## Metrics
|
||||
|
||||
| Metric | Labels | Description |
|
||||
|--------|--------|-------------|
|
||||
| `surface_manifest_published_total` | `tenant`, `kind` | Manifests published |
|
||||
| `surface_manifest_cache_hit_total` | `namespace`, `tenant` | Cache hits |
|
||||
| `surface_manifest_publish_duration_ms` | `tenant` | Publish latency |
|
||||
| `surface_payload_persisted_total` | `kind` | Payloads stored |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Manifest Not Found
|
||||
|
||||
1. Check tenant matches between writer and reader
|
||||
2. Verify Surface.FS endpoint is reachable
|
||||
3. Check bucket permissions
|
||||
4. Review `surface_manifest_published_total` metric
|
||||
|
||||
### Cache Miss Despite Expected Hit
|
||||
|
||||
1. Verify cache key components match (namespace, tenant, digest)
|
||||
2. Check cache quota - eviction may have occurred
|
||||
3. Review `surface_manifest_cache_hit_total` metric
|
||||
|
||||
### Offline Import Failures
|
||||
|
||||
1. Verify manifest digest matches index
|
||||
2. Check file permissions on import path
|
||||
3. Ensure Surface.FS endpoint is writable
|
||||
4. Review import logs for specific errors
|
||||
|
||||
## References
|
||||
|
||||
- [Surface.FS Design](../design/surface-fs.md)
|
||||
- [Surface.Env Design](../design/surface-env.md)
|
||||
- [Surface.Validation Guide](./surface-validation-extensibility.md)
|
||||
- [Offline Kit Documentation](../../../../24_OFFLINE_KIT.md)
|
||||
455
docs/modules/scanner/guides/surface-validation-extensibility.md
Normal file
455
docs/modules/scanner/guides/surface-validation-extensibility.md
Normal file
@@ -0,0 +1,455 @@
|
||||
# Surface.Validation Extensibility Guide
|
||||
|
||||
> **Version:** 1.0 (2025-11-28)
|
||||
>
|
||||
> **Audience:** Scanner Worker/WebService integrators, custom analyzer developers, Zastava contributors
|
||||
|
||||
## Overview
|
||||
|
||||
Surface.Validation provides a pluggable validator framework for ensuring configuration and data preconditions before performing scanner work. This guide covers how to extend the validation system with custom validators, customize reporting, and integrate validation into your components.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Basic Registration
|
||||
|
||||
```csharp
|
||||
// In Program.cs or your DI configuration
|
||||
builder.Services.AddSurfaceValidation();
|
||||
```
|
||||
|
||||
This registers the default validators:
|
||||
- `SurfaceEndpointValidator` - Validates Surface.FS endpoint and bucket
|
||||
- `SurfaceCacheValidator` - Validates cache directory writability and quota
|
||||
- `SurfaceSecretsValidator` - Validates secrets provider configuration
|
||||
|
||||
### Adding Custom Validators
|
||||
|
||||
```csharp
|
||||
builder.Services.AddSurfaceValidation(builder =>
|
||||
{
|
||||
builder.AddValidator<MyCustomValidator>();
|
||||
builder.AddValidator<AnotherValidator>();
|
||||
});
|
||||
```
|
||||
|
||||
## Writing Custom Validators
|
||||
|
||||
### Validator Interface
|
||||
|
||||
Implement `ISurfaceValidator` to create a custom validator:
|
||||
|
||||
```csharp
|
||||
public interface ISurfaceValidator
|
||||
{
|
||||
ValueTask<SurfaceValidationResult> ValidateAsync(
|
||||
SurfaceValidationContext context,
|
||||
CancellationToken cancellationToken = default);
|
||||
}
|
||||
```
|
||||
|
||||
### Example: Registry Credentials Validator
|
||||
|
||||
```csharp
|
||||
public sealed class RegistryCredentialsValidator : ISurfaceValidator
|
||||
{
|
||||
private readonly IHttpClientFactory _httpClientFactory;
|
||||
|
||||
public RegistryCredentialsValidator(IHttpClientFactory httpClientFactory)
|
||||
{
|
||||
_httpClientFactory = httpClientFactory;
|
||||
}
|
||||
|
||||
public async ValueTask<SurfaceValidationResult> ValidateAsync(
|
||||
SurfaceValidationContext context,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(context);
|
||||
|
||||
var issues = new List<SurfaceValidationIssue>();
|
||||
|
||||
// Access secrets configuration from context
|
||||
var secrets = context.Environment.Secrets;
|
||||
if (secrets.Provider == "file" && string.IsNullOrEmpty(secrets.Root))
|
||||
{
|
||||
issues.Add(SurfaceValidationIssue.Error(
|
||||
"REGISTRY_SECRETS_ROOT_MISSING",
|
||||
"Registry secrets root path is not configured.",
|
||||
"Set SCANNER_SURFACE_SECRETS_ROOT to the secrets directory."));
|
||||
}
|
||||
|
||||
// Access custom properties passed during validation
|
||||
if (context.Properties.TryGetValue("registryEndpoint", out var endpoint))
|
||||
{
|
||||
var reachable = await CheckEndpointAsync(endpoint?.ToString(), cancellationToken);
|
||||
if (!reachable)
|
||||
{
|
||||
issues.Add(SurfaceValidationIssue.Warning(
|
||||
"REGISTRY_ENDPOINT_UNREACHABLE",
|
||||
$"Registry endpoint {endpoint} is not reachable.",
|
||||
"Verify network connectivity to the container registry."));
|
||||
}
|
||||
}
|
||||
|
||||
return issues.Count == 0
|
||||
? SurfaceValidationResult.Success()
|
||||
: SurfaceValidationResult.FromIssues(issues);
|
||||
}
|
||||
|
||||
private async Task<bool> CheckEndpointAsync(string? endpoint, CancellationToken ct)
|
||||
{
|
||||
if (string.IsNullOrEmpty(endpoint)) return true;
|
||||
|
||||
try
|
||||
{
|
||||
var client = _httpClientFactory.CreateClient();
|
||||
client.Timeout = TimeSpan.FromMilliseconds(500); // Keep validations fast
|
||||
var response = await client.GetAsync(endpoint, ct);
|
||||
return response.IsSuccessStatusCode;
|
||||
}
|
||||
catch
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Best Practices for Validators
|
||||
|
||||
1. **Keep validations fast** - Target < 500ms per validator to avoid blocking startup
|
||||
2. **Use appropriate severity levels**:
|
||||
- `Error` - Fatal misconfiguration that prevents operation
|
||||
- `Warning` - Suboptimal configuration that may cause issues
|
||||
- `Info` - Informational notices
|
||||
3. **Provide actionable hints** - Include remediation steps in the hint parameter
|
||||
4. **Access services via context** - Use `context.Services.GetService<T>()` for DI
|
||||
5. **Check cancellation tokens** - Honor cancellation for async operations
|
||||
|
||||
## Validation Context
|
||||
|
||||
### Creating Context with Properties
|
||||
|
||||
```csharp
|
||||
var context = SurfaceValidationContext.Create(
|
||||
serviceProvider,
|
||||
componentName: "Scanner.Worker",
|
||||
environment: surfaceEnvironment,
|
||||
properties: new Dictionary<string, object?>
|
||||
{
|
||||
["jobId"] = currentJob.Id,
|
||||
["imageDigest"] = image.Digest,
|
||||
["configPath"] = "/etc/scanner/config.yaml"
|
||||
});
|
||||
```
|
||||
|
||||
### Accessing Context in Validators
|
||||
|
||||
```csharp
|
||||
public ValueTask<SurfaceValidationResult> ValidateAsync(
|
||||
SurfaceValidationContext context,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
// Access environment settings
|
||||
var endpoint = context.Environment.SurfaceFsEndpoint;
|
||||
var bucket = context.Environment.SurfaceFsBucket;
|
||||
var tenant = context.Environment.Tenant;
|
||||
|
||||
// Access custom properties
|
||||
if (context.Properties.TryGetValue("imageDigest", out var digest))
|
||||
{
|
||||
// Validate specific to this image
|
||||
}
|
||||
|
||||
// Access DI services
|
||||
var logger = context.Services.GetService<ILogger<MyValidator>>();
|
||||
}
|
||||
```
|
||||
|
||||
## Running Validators
|
||||
|
||||
### Using the Validator Runner
|
||||
|
||||
```csharp
|
||||
public class MyService
|
||||
{
|
||||
private readonly ISurfaceValidatorRunner _runner;
|
||||
private readonly ISurfaceEnvironment _environment;
|
||||
|
||||
public MyService(ISurfaceValidatorRunner runner, ISurfaceEnvironment environment)
|
||||
{
|
||||
_runner = runner;
|
||||
_environment = environment;
|
||||
}
|
||||
|
||||
public async Task ExecuteAsync(CancellationToken ct)
|
||||
{
|
||||
var context = SurfaceValidationContext.Create(
|
||||
_serviceProvider,
|
||||
"MyService",
|
||||
_environment.Settings);
|
||||
|
||||
// Option 1: Get results and handle manually
|
||||
var result = await _runner.RunAllAsync(context, ct);
|
||||
if (!result.IsSuccess)
|
||||
{
|
||||
foreach (var issue in result.Issues.Where(i => i.Severity == SurfaceValidationSeverity.Error))
|
||||
{
|
||||
_logger.LogError("Validation failed: {Code} - {Message}", issue.Code, issue.Message);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Option 2: Throw on failure (respects options)
|
||||
await _runner.EnsureAsync(context, ct);
|
||||
|
||||
// Continue with work...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Custom Reporting
|
||||
|
||||
### Implementing a Reporter
|
||||
|
||||
```csharp
|
||||
public sealed class MetricsSurfaceValidationReporter : ISurfaceValidationReporter
|
||||
{
|
||||
private readonly IMetricsFactory _metrics;
|
||||
|
||||
public MetricsSurfaceValidationReporter(IMetricsFactory metrics)
|
||||
{
|
||||
_metrics = metrics;
|
||||
}
|
||||
|
||||
public void Report(SurfaceValidationContext context, SurfaceValidationResult result)
|
||||
{
|
||||
var counter = _metrics.CreateCounter<long>("surface_validation_issues_total");
|
||||
|
||||
foreach (var issue in result.Issues)
|
||||
{
|
||||
counter.Add(1, new KeyValuePair<string, object?>[]
|
||||
{
|
||||
new("code", issue.Code),
|
||||
new("severity", issue.Severity.ToString().ToLowerInvariant()),
|
||||
new("component", context.ComponentName)
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Registering Custom Reporters
|
||||
|
||||
```csharp
|
||||
// Replace default reporter
|
||||
builder.Services.AddSingleton<ISurfaceValidationReporter, MetricsSurfaceValidationReporter>();
|
||||
|
||||
// Or add alongside default (using composite pattern)
|
||||
builder.Services.Decorate<ISurfaceValidationReporter>((inner, sp) =>
|
||||
new CompositeSurfaceValidationReporter(
|
||||
inner,
|
||||
sp.GetRequiredService<MetricsSurfaceValidationReporter>()));
|
||||
```
|
||||
|
||||
## Configuration Options
|
||||
|
||||
### SurfaceValidationOptions
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `ThrowOnFailure` | `true` | Whether `EnsureAsync()` throws on validation failure |
|
||||
| `ContinueOnError` | `false` | Whether to continue running validators after first error |
|
||||
|
||||
Configure via `IConfiguration`:
|
||||
|
||||
```json
|
||||
{
|
||||
"Surface": {
|
||||
"Validation": {
|
||||
"ThrowOnFailure": true,
|
||||
"ContinueOnError": false
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Or programmatically:
|
||||
|
||||
```csharp
|
||||
builder.Services.Configure<SurfaceValidationOptions>(options =>
|
||||
{
|
||||
options.ThrowOnFailure = true;
|
||||
options.ContinueOnError = true; // Useful for diagnostics
|
||||
});
|
||||
```
|
||||
|
||||
## Issue Codes
|
||||
|
||||
### Standard Codes
|
||||
|
||||
| Code | Severity | Validator |
|
||||
|------|----------|-----------|
|
||||
| `SURFACE_ENV_MISSING_ENDPOINT` | Error | SurfaceEndpointValidator |
|
||||
| `SURFACE_FS_BUCKET_MISSING` | Error | SurfaceEndpointValidator |
|
||||
| `SURFACE_ENV_CACHE_DIR_UNWRITABLE` | Error | SurfaceCacheValidator |
|
||||
| `SURFACE_ENV_CACHE_QUOTA_INVALID` | Error | SurfaceCacheValidator |
|
||||
| `SURFACE_SECRET_PROVIDER_UNKNOWN` | Error | SurfaceSecretsValidator |
|
||||
| `SURFACE_SECRET_CONFIGURATION_MISSING` | Error | SurfaceSecretsValidator |
|
||||
| `SURFACE_ENV_TENANT_MISSING` | Error | SurfaceSecretsValidator |
|
||||
|
||||
### Custom Issue Codes
|
||||
|
||||
Follow the naming convention: `<SUBSYSTEM>_<COMPONENT>_<ISSUE>`
|
||||
|
||||
```csharp
|
||||
public static class MyValidationCodes
|
||||
{
|
||||
public const string RegistrySecretsRootMissing = "REGISTRY_SECRETS_ROOT_MISSING";
|
||||
public const string RegistryEndpointUnreachable = "REGISTRY_ENDPOINT_UNREACHABLE";
|
||||
public const string CacheWarmupFailed = "CACHE_WARMUP_FAILED";
|
||||
}
|
||||
```
|
||||
|
||||
## Integration Examples
|
||||
|
||||
### Scanner Worker Startup
|
||||
|
||||
```csharp
|
||||
// In hosted service
|
||||
public async Task StartAsync(CancellationToken ct)
|
||||
{
|
||||
var context = SurfaceValidationContext.Create(
|
||||
_services,
|
||||
"Scanner.Worker",
|
||||
_surfaceEnv.Settings);
|
||||
|
||||
try
|
||||
{
|
||||
await _validatorRunner.EnsureAsync(context, ct);
|
||||
_logger.LogInformation("Surface validation passed");
|
||||
}
|
||||
catch (SurfaceValidationException ex)
|
||||
{
|
||||
_logger.LogCritical(ex, "Surface validation failed; worker cannot start");
|
||||
throw;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Per-Scan Validation
|
||||
|
||||
```csharp
|
||||
public async Task<ScanResult> ScanImageAsync(ImageReference image, CancellationToken ct)
|
||||
{
|
||||
var context = SurfaceValidationContext.Create(
|
||||
_services,
|
||||
"Scanner.Analyzer",
|
||||
_surfaceEnv.Settings,
|
||||
new Dictionary<string, object?>
|
||||
{
|
||||
["imageDigest"] = image.Digest,
|
||||
["imageReference"] = image.Reference
|
||||
});
|
||||
|
||||
var result = await _validatorRunner.RunAllAsync(context, ct);
|
||||
|
||||
if (result.HasErrors)
|
||||
{
|
||||
return ScanResult.Failed(result.Issues.Select(i => i.Message));
|
||||
}
|
||||
|
||||
// Proceed with scan...
|
||||
}
|
||||
```
|
||||
|
||||
### Zastava Webhook Readiness
|
||||
|
||||
```csharp
|
||||
app.MapGet("/readyz", async (ISurfaceValidatorRunner runner, ISurfaceEnvironment env) =>
|
||||
{
|
||||
var context = SurfaceValidationContext.Create(
|
||||
app.Services,
|
||||
"Zastava.Webhook",
|
||||
env.Settings);
|
||||
|
||||
var result = await runner.RunAllAsync(context);
|
||||
|
||||
if (!result.IsSuccess)
|
||||
{
|
||||
return Results.Json(new
|
||||
{
|
||||
status = "unhealthy",
|
||||
issues = result.Issues.Select(i => new { i.Code, i.Message, i.Hint })
|
||||
}, statusCode: 503);
|
||||
}
|
||||
|
||||
return Results.Ok(new { status = "healthy" });
|
||||
});
|
||||
```
|
||||
|
||||
## Testing Validators
|
||||
|
||||
### Unit Testing
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task Validator_MissingEndpoint_ReturnsError()
|
||||
{
|
||||
// Arrange
|
||||
var settings = new SurfaceEnvironmentSettings(
|
||||
SurfaceFsEndpoint: new Uri("https://surface.invalid"),
|
||||
SurfaceFsBucket: "",
|
||||
// ... other settings
|
||||
);
|
||||
|
||||
var context = SurfaceValidationContext.Create(
|
||||
new ServiceCollection().BuildServiceProvider(),
|
||||
"Test",
|
||||
settings);
|
||||
|
||||
var validator = new SurfaceEndpointValidator();
|
||||
|
||||
// Act
|
||||
var result = await validator.ValidateAsync(context);
|
||||
|
||||
// Assert
|
||||
Assert.False(result.IsSuccess);
|
||||
Assert.Contains(result.Issues, i => i.Code == SurfaceValidationIssueCodes.SurfaceEndpointMissing);
|
||||
}
|
||||
```
|
||||
|
||||
### Integration Testing
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task ValidationRunner_AllValidatorsExecute()
|
||||
{
|
||||
// Arrange
|
||||
var services = new ServiceCollection();
|
||||
services.AddSurfaceValidation(builder =>
|
||||
{
|
||||
builder.AddValidator<TestValidator1>();
|
||||
builder.AddValidator<TestValidator2>();
|
||||
});
|
||||
|
||||
var provider = services.BuildServiceProvider();
|
||||
var runner = provider.GetRequiredService<ISurfaceValidatorRunner>();
|
||||
|
||||
var context = SurfaceValidationContext.Create(
|
||||
provider,
|
||||
"IntegrationTest",
|
||||
CreateValidSettings());
|
||||
|
||||
// Act
|
||||
var result = await runner.RunAllAsync(context);
|
||||
|
||||
// Assert
|
||||
Assert.True(result.IsSuccess);
|
||||
}
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- [Surface.Validation Design](../design/surface-validation.md)
|
||||
- [Surface.Env Design](../design/surface-env.md)
|
||||
- [Surface.Secrets Schema](../design/surface-secrets-schema.md)
|
||||
Reference in New Issue
Block a user