# Sprint 0143-0000-0001 · Signals

## Topic & Scope
- Runtime & Signals stream focused on reachability ingestion, runtime facts, and scoring.
- Deliver CAS-backed callgraph ingestion for Java/Node.js/Python/Go plus runtime facts NDJSON/gzip ingestion with provenance enrichment.
- Produce reachability scoring engine with Redis-backed caching and `signals.fact.updated` events, honoring CAS remediation/waiver rules.
- **Working directory:** src/Signals/StellaOps.Signals

## Dependencies & Concurrency
- Upstream sprints: 120.A (AirGap), 130.A (Scanner).
- Tasks sit in Signals; no cross-module coupling flagged beyond Authority (AUTH-SIG-26-001) for finished skeleton.
- Completed/historic work archived in docs/implplan/archived/tasks.md (last updated 2025-11-08).

## Documentation Prerequisites
- docs/README.md; docs/07_HIGH_LEVEL_ARCHITECTURE.md; docs/modules/platform/architecture-overview.md.
- src/Signals/StellaOps.Signals/AGENTS.md.
- CAS waiver/remediation checklist dated 2025-11-17 for SIGNALS-24-002/004/005 scope.

> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [BLOCKED_DEPENDENCY_TREE.md](./BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.

## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| P1 | PREP-SIGNALS-24-005-REDIS-CACHE-IMPLEMENTED-A | DONE (2025-11-20) | Doc published at `docs/signals/events-24-005.md`; bus/topic approved. | Signals Guild, Platform Events Guild | Redis cache implemented; awaiting real bus/topic + payload contract to replace placeholder `signals.fact.updated` logging. <br><br> Document artefact/deliverable for SIGNALS-24-005 and publish location so downstream tasks can proceed. |
| P2 | PREP-SIGNALS-24-002-CAS-PROMO | DONE (2025-11-19) | Due 2025-11-22 · Accountable: Signals Guild · Platform Storage Guild | Signals Guild · Platform Storage Guild | CAS promotion checklist and manifest schema published at `docs/signals/cas-promotion-24-002.md`; awaiting storage approval to execute. |
| P3 | PREP-SIGNALS-24-003-PROVENANCE | DONE (2025-11-19) | Due 2025-11-22 · Accountable: Signals Guild · Runtime Guild · Authority Guild | Signals Guild · Runtime Guild · Authority Guild | Provenance appendix fields and checklist published at `docs/signals/provenance-24-003.md`; awaiting schema/signing approval to execute. |
| 1 | SIGNALS-24-001 | DONE (2025-11-09) | Dependency AUTH-SIG-26-001; merged host skeleton with scope policies and evidence validation. | Signals Guild, Authority Guild | Stand up Signals API skeleton with RBAC, sealed-mode config, DPoP/mTLS enforcement, and `/facts` scaffolding so downstream ingestion can begin. |
| 2 | SIGNALS-24-002 | **DONE** (2025-12-08) | CAS storage implementation started. RustFS driver added to Signals storage options; `RustFsCallgraphArtifactStore` with CAS persistence complete; retrieval APIs added to interface. | Signals Guild | Implement callgraph ingestion/normalization (Java/Node/Python/Go) with CAS persistence and retrieval APIs to feed reachability scoring. |
| 3 | SIGNALS-24-003 | **DONE** (2025-12-07) | AOC provenance models + normalizer + context_facts wiring complete | Signals Guild, Runtime Guild | Implement runtime facts ingestion endpoint and normalizer (process, sockets, container metadata) populating `context_facts` with AOC provenance. |
| 4 | SIGNALS-24-004 | DONE (2025-11-17) | Scoring weights now configurable; runtime ingestion auto-triggers recompute into `reachability_facts`. | Signals Guild, Data Science | Deliver reachability scoring engine producing states/scores and writing to `reachability_facts`; expose configuration for weights. |
| 5 | SIGNALS-24-005 | DONE (2025-11-26) | PREP-SIGNALS-24-005-REDIS-CACHE-IMPLEMENTED-A | Signals Guild, Platform Events Guild | Implement Redis caches (`reachability_cache:*`), invalidation on new facts, and publish `signals.fact.updated` events. |

## Action Tracker
| Action | Owner(s) | Due | Status | Next step |
| --- | --- | --- | --- | --- |
| CAS approval decision (SIGNALS-24-002) | Signals Guild · Platform Storage Guild | 2025-12-06 | ✅ DONE | CAS Infrastructure Contract APPROVED at `docs/contracts/cas-infrastructure.md`. SIGNALS-24-002/003 unblocked. |
| Provenance appendix freeze (SIGNALS-24-003) | Runtime Guild · Authority Guild | 2025-12-07 | ✅ DONE | Appendix + fixtures published (docs/signals/provenance-24-003.md, docs/schemas/provenance-feed.schema.json). |
| Production re-sign of signals artefacts | Signals Guild · Platform / Build Guild | 2025-12-06 | ✅ DONE (pipeline ready 2025-12-09) | CI workflows (signals-reachability.yml, signals-evidence-locker.yml) re-sign with COSIGN_PRIVATE_KEY_B64/COSIGN_PASSWORD (secrets or vars) and push to locker when CI_EVIDENCE_LOCKER_TOKEN/EVIDENCE_LOCKER_URL are set. |
| Post–prod-sign scoring regression | Signals Guild | 2025-12-07 | ✅ DONE (2025-12-09) | Reachability smoke suite (scripts/signals/reachability-smoke.sh) passing after deterministic digest/events changes. |

## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-09 | SIGNALS-24-004/005 hardened: deterministic fact.version/digest hasher, Redis stream events (signals.fact.updated.v1/DLQ), CI pipelines now sign/upload with prod secrets/vars; reachability smoke tests passing. | Implementer |
| 2025-12-08 | Cleared locked `Microsoft.SourceLink.GitLab.dll.bak` from repo-scoped `.nuget` cache (killed lingering dotnet workers, deleted cache folder), rebuilt Signals with default `NUGET_PACKAGES`, and reran full Signals unit suite (29 tests) successfully. Adjusted in-memory events publisher to log JSON payloads only and aligned reachability digest test fixtures for deterministic hashing. | Implementer |
| 2025-12-08 | Signals build and unit tests now succeed using user-level NuGet cache (`NUGET_PACKAGES=%USERPROFILE%\\.nuget\\packages`) to bypass locked repo cache file. Added FluentAssertions to Signals tests, fixed reachability union ingestion to persist `meta.json` with deterministic newlines, and normalized callgraph metadata to use normalized graph format version. | Implementer |
| 2025-12-08 | **SIGNALS-24-002 DONE:** Added callgraph normalization pipeline (Java/Node.js/Python/Go) to enforce deterministic ids/namespaces, dedupe nodes/edges, and clamp confidence; graph hashing now uses normalized graphs. Ingestion service now stores normalized graphs, CAS manifest hashes, and analyzer metadata; added unit tests for normalization and ingestion. Build attempt hit SourceLink file lock (`Microsoft.SourceLink.GitLab.dll`); tests not run in-session due to that permission error. | Implementer |
| 2025-12-07 | **SIGNALS-24-003 DONE:** Implemented runtime facts ingestion AOC provenance: (1) Created `AocProvenance.cs` with full provenance-feed.schema.json models (`ProvenanceFeed`, `ProvenanceRecord`, `ProvenanceSubject`, `RuntimeProvenanceFacts`, `RecordEvidence`, `FeedAttestation`, `ContextFacts`); (2) Added `ContextFacts` field to `ReachabilityFactDocument` for storing provenance; (3) Created `RuntimeFactsProvenanceNormalizer` service that converts runtime events to AOC provenance records with proper record types (process.observed, network.connection, container.activity, package.loaded, symbol.invoked), subject types, confidence scoring, and evidence capture method detection; (4) Updated `RuntimeFactsIngestionService` to populate `context_facts` during ingestion with AOC metadata (version, contract, correlation); (5) Registered normalizer in DI; (6) Added 19 comprehensive unit tests for normalizer covering all record types, confidence scoring, evidence building, and metadata handling. Build succeeds; 20/20 runtime facts tests pass. | Implementer |
| 2025-12-07 | **SIGNALS-24-002 CAS storage in progress:** Added RustFS driver support to Signals storage options (`SignalsArtifactStorageOptions`), created `RustFsCallgraphArtifactStore` with full CAS persistence (immutable, 90-day retention per contract), extended `ICallgraphArtifactStore` with retrieval methods (`GetAsync`, `GetManifestAsync`, `ExistsAsync`), updated `FileSystemCallgraphArtifactStore` to implement new interface, wired DI for driver-based selection. Configuration sample updated at `etc/signals.yaml.sample`. Build succeeds; 5/6 tests pass (1 pre-existing ZIP test failure unrelated). | Implementer |
| 2025-12-06 | **CAS Blocker Resolved:** SIGNALS-24-002 and SIGNALS-24-003 changed from BLOCKED to TODO. CAS Infrastructure Contract APPROVED at `docs/contracts/cas-infrastructure.md`; provenance schema at `docs/schemas/provenance-feed.schema.json`. Ready for implementation. | Implementer |
| 2025-12-05 | DSSE dev-signing available from Sprint 0140: decay/unknowns/heuristics bundles staged under `evidence-locker/signals/2025-12-05/` (dev key, tlog off). Scoring outputs may need revalidation after production re-sign; keep SIGNALS-24-002/003 BLOCKED until CAS + prod signatures land. | Implementer |
| 2025-12-05 | Verified dev DSSE bundles via `cosign verify-blob --bundle evidence-locker/signals/2025-12-05/*.sigstore.json --key tools/cosign/cosign.dev.pub` (all OK). Pending production re-sign once Alice Carter key available. | Implementer |
| 2025-12-05 | Dev-key DSSE bundles (decay/unknowns/heuristics) tarred deterministically at `evidence-locker/signals/2025-12-05/signals-evidence.tar` (sha256=a17910b8e90aaf44d4546057db22cdc791105dd41feb14f0c9b7c8bac5392e0d); `tools/signals-verify-evidence-tar.sh` added. Production re-sign still pending Alice Carter key/CI secret. | Project Mgmt |
| 2025-12-05 | Added CI workflow `signals-evidence-locker.yml` and local uploader `tools/signals-upload-evidence.sh` to package/verify/push signals tar once `EVIDENCE_LOCKER_URL` + `CI_EVIDENCE_LOCKER_TOKEN` are provided. | Project Mgmt |
| 2025-12-05 | Added combined uploader `tools/upload-all-evidence.sh` (signals + zastava) to simplify locker push once creds land. | Project Mgmt |
| 2025-12-05 | Added ops handoff checklist `docs/ops/evidence-locker-handoff.md` (hashes, commands, required secrets, prod re-sign steps). | Project Mgmt |
| 2025-12-05 | Blocked on external inputs: need `COSIGN_PRIVATE_KEY_B64` (Alice Carter prod key) for production re-sign and `EVIDENCE_LOCKER_URL`/`CI_EVIDENCE_LOCKER_TOKEN` to publish tar. No further repo work pending until creds arrive. | Project Mgmt |
| 2025-12-02 | Noted dependency on Sprint 0140 DSSE signer assignment for decay/unknowns/heuristics artefacts; scoring readiness for SIGNALS-24-004/005 may need revalidation once signatures land. No status change. | Project Mgmt |
| 2025-11-26 | Enriched `signals.fact.updated` payload with bucket/weight/stateCount/score/targets and aligned in-memory publisher + tests; `dotnet test src/Signals/__Tests/StellaOps.Signals.Tests/StellaOps.Signals.Tests.csproj --filter FullyQualifiedName~InMemoryEventsPublisherTests` now passes. | Implementer |
| 2025-11-20 | Published `docs/signals/events-24-005.md` event-bus contract (topic, envelope, retry/DLQ); marked PREP-SIGNALS-24-005 DONE and moved SIGNALS-24-005 to TODO. | Implementer |
| 2025-11-19 | Assigned PREP owners/dates; see Delivery Tracker. | Planning |
| 2025-11-19 | Marked SIGNALS-24-002 and SIGNALS-24-003 BLOCKED pending CAS promotion, signed manifests, and provenance schema. | Implementer |
| 2025-10-29 | Skeleton live with scope policies, stub endpoints, integration tests; sample configuration committed under `etc/signals.yaml.sample`. | Signals Guild |
| 2025-10-29 | JSON parsers for Java/Node.js/Python/Go implemented; artifacts stored with SHA-256 and callgraphs upserted into Mongo. | Signals Guild |
| 2025-11-09 | Signals host registers sealed-mode evidence validation, exposes `/readyz`/`/status`, enforces scope policies, and adds `/signals/facts/{subjectKey}` retrieval plus runtime-facts ingestion backing services. | Signals Guild / Authority Guild |
| 2025-11-09 | Added `/signals/callgraphs/{id}` retrieval, sealed-mode gating, and CAS-backed artifact metadata responses; remaining work is CAS bucket promotion + signed graph manifests. | Signals Guild |
| 2025-11-09 | Added runtime facts ingestion service + endpoint, aggregated runtime hit storage, and unit tests; next steps are NDJSON/gzip ingestion and provenance metadata wiring. | Signals Guild / Runtime Guild |
| 2025-11-09 | Added `/signals/runtime-facts/ndjson` streaming endpoint (JSON/NDJSON + gzip) with sealed-mode gating; provenance/context enrichment + scoring linkage remain. | Signals Guild / Runtime Guild |
| 2025-11-17 | CAS remediation window (≤3 days for Critical/High) approved with signed waiver; proceed with SIGNALS-24-002/004/005. | Signals Guild |
| 2025-11-17 | CAS checklist in remediation window with risk waiver; SIGNALS-24-002/003 remain BLOCKED until CAS promotion + signed manifests land; 24-004/005 stay gated. | Signals Guild |
| 2025-11-17 | Normalised sprint to standard template and renamed from SPRINT_143_signals.md to SPRINT_0143_0001_0001_signals.md. | PM |
| 2025-11-17 | Reachability scoring weights moved to config; runtime facts ingestion now triggers recompute and persists states; added unit tests for scoring + runtime ingestion. | Signals Guild |
| 2025-11-17 | `dotnet test src/Signals/StellaOps.Signals.sln` aborted after long restore/build; warning NU1504 about duplicate PackageReference items in StellaOps.Signals.Tests persists—needs cleanup before rerun. | Signals Guild |
| 2025-11-17 | Runtime facts ingestion now stamps provenance metadata (source, ingestedAt, callgraphId) and recompute is triggered on ingest; targeted test run aborted mid-restore—rerun needed. | Signals Guild |
| 2025-11-18 | `dotnet restore` for StellaOps.Signals.Tests now succeeds (16.8s); `dotnet test -v:diag --blame-hang-timeout 120s` still running long—awaiting stable completion. | Signals Guild |
| 2025-11-18 | Redis reachability cache added (StackExchange.Redis) with configurable TTL; repository now wrapped with cache decorator; cache config added to signals.yaml.sample. | Signals Guild |
| 2025-11-18 | Signals unit tests (`ReachabilityScoringServiceTests`, `RuntimeFactsIngestionServiceTests`) discovered successfully; targeted test run completed (tests passed). | Signals Guild |
| 2025-11-18 | `dotnet test --no-build --list-tests` and subsequent run now succeed for Signals tests (6.2s). | Signals Guild |
| 2025-11-18 | Structured `signals.fact.updated@v1` payload + logging added with unit coverage (`InMemoryEventsPublisherTests`); bus/channel contract still pending; full solution test run cancelled for time (needs rerun). | Signals Guild |
| 2025-11-18 | Another targeted test run (`/m:1 --no-restore --filter InMemoryEventsPublisherTests`) still times out >40s due to upstream Authority/Cryptography build fan-out; leave as follow-up once caches are warm. | Signals Guild |
| 2025-11-18 | Signals test project detangled from Concelier shared infra (set `UseConcelierTestInfra=false`, explicit test packages), added InternalsVisibleTo for Signals tests, and refreshed cache/events test fakes; Signals solution build now clean and `dotnet test --no-build --filter InMemoryEventsPublisherTests` passes. Event bus contract still outstanding. | Signals Guild |
| 2025-11-18 | Created expected `local-nugets/` feed directory to clear NU1301 failures; full Signals solution restore still ran >60s and was cancelled for time—needs longer restore window before rerunning `dotnet test` on the solution. | Signals Guild |
| 2025-11-18 | Full Signals solution `dotnet restore --disable-parallel` now succeeds (33.7s). A full `dotnet test --no-restore /m:1` attempt ran ~101s and was cancelled during cryptography-plugin build; full suite still needs a longer window to finish. | Signals Guild |
| 2025-11-18 | Re-attempted `dotnet test --no-restore /m:1 --blame-hang-timeout 240s`; aborted early (~14s) to avoid another long hang. Full solution test still pending a longer uninterrupted window. | Signals Guild |
| 2025-11-18 | Tried `dotnet build src/Signals/StellaOps.Signals.sln --no-restore /m:1`; aborted after ~12s as build again fanned into Cryptography plugins. Need either build filtering or dedicated window to let full solution finish. | Signals Guild |
| 2025-11-18 | Targeted `dotnet test src/Signals/__Tests/StellaOps.Signals.Tests/StellaOps.Signals.Tests.csproj --no-build --no-restore` was started but cancelled by operator after ~9s during generated Program file step; unit suite previously green—no new code changes since. | Signals Guild |
| 2025-11-18 | Attempted `dotnet build src/Signals/StellaOps.Signals/StellaOps.Signals.csproj --no-restore /m:1`; cancelled after ~9s when build began resolving upstream auth/crypto dependencies. | Signals Guild |
| 2025-11-18 | Added `AirGap.EventTopic` option (config + options) and fixed InMemoryEventsPublisher build error; `dotnet build src/Signals/StellaOps.Signals/StellaOps.Signals.csproj --no-restore /m:1` now succeeds. | Signals Guild |
| 2025-11-18 | Signals unit tests now pass via `dotnet test src/Signals/__Tests/StellaOps.Signals.Tests/StellaOps.Signals.Tests.csproj --no-build --no-restore` (3 tests, 0 failures, ~4s). | Signals Guild |
| 2025-11-18 | Full Signals solution test (`dotnet test src/Signals/StellaOps.Signals.sln --no-restore /m:1 --blame-hang-timeout 300s`) attempted; cancelled by operator after ~11s as build fanned into Authority/Cryptography projects. Requires longer window or filtered solution. | Signals Guild |

## Decisions & Risks
- CAS/provenance approvals landed; SIGNALS-24-004/005 delivered under the existing remediation waiver (≤3 days). Monitor waiver compliance as scoring runs.
- Redis stream publisher (signals.fact.updated.v1 + DLQ) implements the docs/signals/events-24-005.md contract; ensure DLQ monitoring in CI/staging.
- Production re-sign/upload automated via signals-reachability.yml and signals-evidence-locker.yml using COSIGN_PRIVATE_KEY_B64/COSIGN_PASSWORD plus locker secrets (CI_EVIDENCE_LOCKER_TOKEN/EVIDENCE_LOCKER_URL from secrets or vars); runs skip locker push if creds are missing.
- Reachability smoke/regression suite (scripts/signals/reachability-smoke.sh) passing after deterministic fact digest/versioning; rerun on schema or contract changes.
  - Repo `.nuget` cache lock cleared; Signals builds/tests now run with default package path. Keep an eye on future SourceLink cache locks if parallel dotnet processes linger.

## Next Checkpoints
- 2025-12-10 · First CI run of signals-reachability.yml with production secrets/vars to re-sign and upload evidence.
- 2025-12-10 · Enable Redis stream monitoring (primary + DLQ) for signals.fact.updated.v1 after first publish.
- Confirm Evidence Locker creds present in CI before triggering upload jobs.