Files
git.stella-ops.org/docs/product-advisories/archive
master 8bbfe4d2d2 feat(rate-limiting): Implement core rate limiting functionality with configuration, decision-making, metrics, middleware, and service registration
- Add RateLimitConfig for configuration management with YAML binding support.
- Introduce RateLimitDecision to encapsulate the result of rate limit checks.
- Implement RateLimitMetrics for OpenTelemetry metrics tracking.
- Create RateLimitMiddleware for enforcing rate limits on incoming requests.
- Develop RateLimitService to orchestrate instance and environment rate limit checks.
- Add RateLimitServiceCollectionExtensions for dependency injection registration.
2025-12-17 18:02:37 +02:00

1029 lines
30 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Heres a compact, practical blueprint to fuse **SmartDiff** with a **CallStack Reachability** engine and emit **DSSEsigned diff attestations** that carry a **weighted impact index**—so StellaOps can prove not just “what changed,” but “how risky the change is in runtimereachable code.”
---
# What this does (in plain words)
* **SmartDiff**: computes semantic diffs between two artifact states (SBOMs, lockfiles, symbols, call maps).
* **Reachability**: measures whether a changed function/package is actually on a path that executes in your services (based on call graph + entrypoints).
* **Weighted Impact Index (WII)**: one number (0100) that rises when the change lands on short, highlyused, externallyfacing, or privileged call paths—and when known vulns become more “deterministic” (exploitably reachable).
* **DSSE Attestation**: a signed, portable JSON (intoto/DSSE) that binds the diff + WII to build/run evidence.
---
# Signals that feed the Weighted Impact Index
**Perdelta features** (each contributes a weight; defaults in brackets):
* `Δreach_len` change in **shortest reachable path length** to an entrypoint (−∞..+∞) [w=0.25]
* `Δlib_depth` change in **library call depth** (indirection layers) [w=0.1]
* `exposure` whether the touched symbol is **public/externalfacing** (API, RPC, HTTP route) [w=0.15]
* `privilege` whether path crosses **privileged sinks** (deserialization, shell, fs/net, crypto) [w=0.15]
* `hot_path` historical runtime evidence (pprof, APM, eBPF) showing **frequent execution** [w=0.15]
* `cvss_v4` normalized CVSS v4.0 severity for affected CVEs (010 → 01) [w=0.1]
* `epss_v4` exploit probability (01) [w=0.1]
* `guard_coverage` presence of sanitizers/validations on the path (reduces score) [w=0.1]
**Determinism nudge**
If `reachability == true` and `(cvss_v4 > 0.7 || epss_v4 > 0.5)`, add a +5 bonus to reflect “evidencelinked determinism.”
**Final WII**
```
WII = clamp01( Σ (w_i * feature_i_normalized) ) * 100
```
---
# Minimal data you need in the engines
## 1) SmartDiff (inputs/outputs)
**Inputs:** SBOM(CycloneDX), symbol graph (perlang indexers), lockfiles, route maps.
**Outputs:** `DiffUnit[]` with:
```json
{
"unitId": "pkg:npm/lodash@4.17.21#function:merge",
"change": "modified|added|removed",
"before": {"hash":"...", "attrs": {...}},
"after": {"hash":"...", "attrs": {...}}
}
```
## 2) Reachability Engine
**Inputs:** call graph (nodes: symbols; edges: calls), entrypoints (HTTP routes, jobs, message handlers), runtime heat (optional).
**Queries:** `isReachable(symbol)`, `shortestPathLen(symbol)`, `libCallDepth(symbol)`, `hasPrivilegedSink(path)`, `hasGuards(path)`.
---
# Putting it together (pipeline)
1. **Collect**: For image/artifact A→B, build call graph, import SBOMs, CVE map, EPSS/CVSS data, routes, runtime heat.
2. **Diff**: Run SmartDiff → `DiffUnit[]`.
3. **Enrich per DiffUnit** using Reachability:
* `reachable = isReachable(unit.symbol)`
* `Δreach_len = shortestPathLen_B - shortestPathLen_A`
* `Δlib_depth = libCallDepth_B - libCallDepth_A`
* `exposure/privilege/hot_path/guard_coverage` booleans from path analysis
* `cvss_v4/epss_v4` from Feed (Concelier) + Excititor
4. **Score**: Compute `WII` per unit; also compute **artifactlevel WII** as:
* `max(WII_unit)` and `p95(WII_unit)` for “spike” vs “broad” impact.
5. **Attest**: Emit DSSE statement with diff + scores + evidence URIs (SBOM digest, callgraph digest, logs).
6. **Publish/Store**: Rekor(v2) mirror (ProofMarket Ledger), and PostgreSQL as systemofrecord.
---
# DSSE statement (example)
```json
{
"_type": "https://in-toto.io/Statement/v1",
"subject": [{"name":"ghcr.io/acme/payments:1.9.3","digest":{"sha256":"..."} }],
"predicateType": "https://stella-ops.org/attestations/smart-diff-wii@v1",
"predicate": {
"artifactBefore": {"digest":{"sha256":"..."}},
"artifactAfter": {"digest":{"sha256":"..."}},
"evidence": {
"sbomBefore":{"mediaType":"application/vnd.cdx+json","digest":{"sha256":"..." }},
"sbomAfter": {"mediaType":"application/vnd.cdx+json","digest":{"sha256":"..." }},
"callGraph": {"mediaType":"application/vnd.stella.callgraph+json","digest":{"sha256":"..."}},
"runtimeHeat": {"mediaType":"application/json","optional":true,"digest":{"sha256":"..."}}
},
"units": [{
"unitId":"pkg:nuget/Newtonsoft.Json@13.0.3#type:JsonSerializer",
"change":"modified",
"features":{
"reachable":true,
"deltaReachLen":-2,
"deltaLibDepth":-1,
"exposure":true,
"privilege":true,
"hotPath":true,
"guardCoverage":false,
"cvssV4":0.84,
"epssV4":0.61
},
"wii": 78.4,
"paths":[
{"entry":"HTTP POST /api/import","shortestLen":3,"privSinks":["fs.write"] }
]
}],
"aggregate": {"maxWii": 78.4, "p95Wii": 42.1}
},
"dsse": {"alg":"ed25519","keyid":"stella-authority:kid:abc123","sig":"..."}
}
```
---
# .NET 10 integration (skeletal but endtoend)
## Contracts
```csharp
public record DiffUnit(string UnitId, ChangeKind Change, Attr? Before, Attr? After);
public interface IReachabilityService {
bool IsReachable(SymbolId s);
int? ShortestPathLen(SymbolId s);
int LibCallDepth(SymbolId s);
bool PathHasPrivilegedSinks(SymbolId s);
bool PathHasGuards(SymbolId s);
bool IsHotPath(SymbolId s);
}
public sealed class WiiScorer {
public double Score(WiiFeatures f) {
double sum =
0.25 * NormalizeDelta(f.DeltaReachLen) +
0.10 * NormalizeDelta(f.DeltaLibDepth) +
0.15 * Bool(f.Exposure) +
0.15 * Bool(f.Privilege) +
0.15 * Bool(f.HotPath) +
0.10 * Clamp01(f.CvssV4) +
0.10 * Clamp01(f.EpssV4) -
0.10 * Bool(f.GuardCoverage);
if (f.Reachable && (f.CvssV4 > 0.7 || f.EpssV4 > 0.5)) sum += 0.05;
return Math.Round(Clamp01(sum) * 100, 1);
}
// helper normalizers (Δ capped to ±5 for stability)
static double NormalizeDelta(int? d) => Clamp01(((d ?? 0) + 5) / 10.0);
static double Bool(bool b) => b ? 1.0 : 0.0;
static double Clamp01(double x) => Math.Min(1, Math.Max(0, x));
}
```
## Orchestrator (Scanner.WebService or Scheduled.Worker)
```csharp
public async Task<SmartDiffAttestation> RunAsync(Artifact before, Artifact after) {
var diffs = await _smartDiff.ComputeAsync(before, after);
var scorer = new WiiScorer();
var units = new List<AttestedUnit>();
foreach (var d in diffs) {
var s = SymbolId.Parse(d.UnitId);
var feat = new WiiFeatures {
Reachable = _reach.IsReachable(s),
DeltaReachLen = SafeDelta(_reach.ShortestPathLen(s), _baseline.ShortestPathLen(s)),
DeltaLibDepth = _reach.LibCallDepth(s) - _baseline.LibCallDepth(s),
Exposure = _exposure.IsExternalFacing(s),
Privilege = _reach.PathHasPrivilegedSinks(s),
HotPath = _reach.IsHotPath(s),
GuardCoverage = _reach.PathHasGuards(s),
CvssV4 = _vuln.CvssV4For(s),
EpssV4 = _vuln.EpssV4For(s)
};
var wii = scorer.Score(feat);
units.Add(new AttestedUnit(d.UnitId, d.Change, feat, wii, _reach.PathPreview(s)));
}
var agg = new {
maxWii = units.Max(u => u.Wii),
p95Wii = Percentile(units.Select(u => u.Wii), 0.95)
};
var stmt = _attestor.BuildDsse(before, after, units, agg, _evidence.Hashes());
return await _attestor.SignAsync(stmt);
}
```
---
# Where it lives in StellaOps
* **Concelier** (feeds): CVE → CVSS v4.0 and EPSS v4 hydration.
* **Excititor** (VEX): accepts WII + reachability to mark *Affected/Not Affected/Under Investigation* with evidence.
* **Scanner.WebService & Scanner.Workers**: build call graphs, compute diffs, ask Concelier/Excititor for scores, produce attestations.
* **Notify.WebService**: triggers when `aggregate.maxWii >= threshold` or when `reachable && epss_v4 > X`.
---
# Developer checklist (DoD)
* [ ] Perlanguage callgraph adapters: .NET, Java, Node, Python (symbol → entrypoint reach; shortest path).
* [ ] SmartDiff emits `unitId` at **function/method granularity** (fall back to package).
* [ ] Concelier endpoints to fetch **CVSS v4** and **EPSS v4** per `purl`/symbol.
* [ ] WiiScorer unit tests with frozen fixtures (golden files).
* [ ] DSSE attestation schema + JSON Schema validation.
* [ ] Rekor(v2) publish & local verification; offline bundle export.
* [ ] Policy gates: block deploy if `maxWii ≥ 70` **and** `reachable == true`.
* [ ] UI panel: show **top 5 WII units**, path preview, “why this score” explainer.
* [ ] CLI: `stella attest diff --before A --after B --emit dsse.json`.
---
# Why this strengthens the moat
* Turns **reachability** into a **determinism signal**, not just “found a CVE.”
* Binds **evidence** (graphs, SBOMs, metrics) into **signed, portable** proofs.
* Produces a **single, comparable index** that procurement/compliance and SREs can act on in seconds.
If you want, I can drop in a readytorun `.NET 10` sample project (Scanner plugin + WiiScorer tests) and a JSONSchema for the DSSE predicate.
Below is a **full, endtoend implementation plan** to ship “SmartDiff + CallStack Reachability + Weighted Impact Index + DSSEsigned diff attestations” into production inside Stella Ops (Scanner / Concelier / Excititor / Notify), with clear sequencing, deliverables, and the concrete engineering work breakdown.
---
## 1) Target outcomes and success criteria
### Outcomes
1. **For any artifact upgrade (A → B)**, produce a **DSSEsigned intoto Statement** that includes:
* What changed (SmartDiff)
* What is runtime reachable (CallStack Reachability)
* A **perchange Weighted Impact Index (WII)** and artifact aggregates (max, p95)
* Evidence digests (SBOMs, call graph, runtime heat, vuln feeds) bound to the attestation
2. **Policy gates** can block or require approval based on:
* `maxWII`, `reachable == true`, `epss/cvss thresholds`, “privileged path” flags, etc.
3. **Operators can explain the score**: top changes, entrypoints, shortest path, sinks, guards, heat.
### Success criteria (Definition of Done)
* ✅ Deterministic diff attestations (same inputs → same statement bytes before signing)
* ✅ Signature verification succeeds offline using published key(s)
* ✅ Correct reachability on representative services (route hits match predicted reachable set)
* ✅ Low noise: “unreachable changes” do not page
* ✅ Scoring is configurable and versioned (weights tracked)
* ✅ Works in CI and/or cluster scanning pipeline
* ✅ Stored in ledger + queryable DB + searchable UI
---
## 2) System architecture and data flow
### High-level data flow
```
Artifact A/B (OCI digest, build metadata)
├─ SBOM A/B (CycloneDX)
├─ Symbol Index A/B (function/type identifiers)
├─ Call Graph A/B (nodes=symbols, edges=calls, entrypoints)
├─ Runtime Heat (optional; APM/eBPF)
├─ Vuln Intelligence (CVSS v4, EPSS) from Concelier
└─ Smart-Diff (A vs B) -> DiffUnit[]
└─ Reachability enrich (A/B) -> features per unit
└─ WII scoring
└─ DSSE in-toto Statement (predicate)
└─ Sign (KMS/HSM/Key vault) -> DSSE envelope
├─ Publish to Ledger (Rekor-like)
├─ Store in Postgres + Object store
└─ Notify/Policy evaluation
```
### Services / modules to implement or extend
* **Scanner.Workers**: build evidence (SBOM, call graph), compute diff, compute reachability features, compute WII
* **Scanner.WebService**: APIs to request attestations, query results, verify
* **Concelier**: CVE → package/symbol mapping, CVSS v4 + EPSS hydration
* **Excititor**: produce/ingest VEX decisions using WII + reachability evidence
* **Notify**: alerting rules and policy gates for CI/CD and runtime
---
## 3) Core data contracts (must come first)
### 3.1 Stable identifiers
You need **stable IDs** for everything so diffs and reachability join correctly:
* **Artifact ID**: OCI digest (sha256) + image name + tag (tag is not trusted as primary)
* **Package ID**: PURL (package-url standard)
* **Symbol ID**: language-specific but normalized:
* `.NET`: `assembly::namespace.type::method(signature)` or `pdb mvid + token`
* Java: `class#method(desc)`
* Node: `module:path#exportName` (best-effort)
* Python: `module:function` (best-effort)
Rule: **Symbol IDs must remain stable across rebuilds** where possible (prefer token-based or fully-qualified signature).
### 3.2 Predicate schema v1
Lock the predicate shape early:
* `artifactBefore`, `artifactAfter` digests
* `evidence` digests/URIs (sbom, callGraph, runtimeHeat, vulnSnapshot)
* `units[]` (diff units with features, paths, wii)
* `aggregate` (max/p95 etc.)
* `scoring` (weights + version + normalizer caps)
* `generator` info (scanner version, build id)
Deliverables:
* JSON Schema for predicate `smart-diff-wii@v1`
* Protobuf (optional) for internal transport
* Golden-file fixtures for serialization determinism
---
## 4) Phase plan (sequenced deliverables, no guessy timelines)
### Milestone M0 — Foundations (must be completed before “real scoring”)
**Goal:** You can sign/verify attestations and store evidence.
**Work items**
1. **DSSE + in-toto Statement implementation**
* Choose signing primitive: Ed25519 or ECDSA P256
* Implement:
* Statement builder (canonical JSON)
* DSSE envelope wrapper
* Signature verify endpoint + CLI
* Add key rotation fields: `keyid`, `predicateType version`, `scanner build version`
2. **Evidence store**
* Object storage bucket layout:
* `/sbom/{artifactDigest}.cdx.json`
* `/callgraph/{artifactDigest}.cg.json`
* `/runtimeheat/{service}/{date}.json`
* `/vuln-snapshot/{date}.json`
* Every evidence object has digest recorded in DB
3. **Database schema**
* `artifacts(id, digest, name, createdAt, buildMetaJson)`
* `evidence(id, artifactDigest, kind, digest, uri, createdAt)`
* `attestations(id, subjectDigest, beforeDigest, afterDigest, predicateType, dsseDigest, createdAt, aggregateJson)`
* `attestation_units(attestationId, unitId, changeKind, reachable, wii, featuresJson, pathsJson)`
4. **Ledger integration**
* Minimal: append-only table + hash chaining (if you want quickly)
* Full: publish to Rekor-like transparency log if already present in your ecosystem
**DoD**
* `stella verify dsse.json` returns OK
* Stored attestations can be fetched by subject digest
* Evidence digests validate
---
### Milestone M1 — SmartDiff v1 (package + file level)
**Goal:** Produce a signed attestation that captures “what changed” even before reachability.
**Work items**
1. **SBOM ingestion & normalization**
* Parse CycloneDX SBOM
* Normalize component identity to PURL
* Extract versions, hashes, scopes, dependencies edges
2. **Diff engine (SBOM-level)**
* Added/removed/updated packages
* Transitive dependency changes
* License changes (optional)
* Output `DiffUnit[]` at package granularity first
3. **Attestation emitter**
* Populate predicate with:
* units (packages)
* aggregate metrics: number of changed packages; “risk placeholders” (no reachability yet)
**DoD**
* For a dependency bump PR, the system emits DSSE attestation with package diffs
---
### Milestone M2 — Call graph & reachability for .NET (first “real value”)
**Goal:** For .NET services, determine whether changed symbols/packages are reachable from entrypoints.
**Work items**
1. **.NET call graph builder**
* Implement Roslyn-based static analysis for:
* method invocations
* virtual calls (best-effort: conservative edges)
* async/await (capture call relations)
* Capture:
* nodes: methods/functions
* edges: caller → callee
* metadata: assembly, namespace, accessibility, source location (if available)
2. **Entrypoint extractor (.NET)**
* ASP.NET Core minimal APIs:
* `MapGet/MapPost/...` route mapping to delegate targets
* MVC/WebAPI:
* controller action methods
* gRPC endpoints
* Background workers:
* `IHostedService`, Hangfire jobs, Quartz jobs
* Message handlers:
* MassTransit / Kafka consumers (pattern match + config hooks)
3. **Reachability index**
* Store adjacency lists
* For each entrypoint, compute:
* reachable set
* shortest path length to each reachable node (BFS on unweighted graph)
* path witness (store 13 representative paths for explainability)
* Store:
* `distToNearestEntrypoint[node]`
* `nearestEntrypoint[node]`
* (optional) `countEntrypointsReaching[node]`
4. **Join SmartDiff with reachability**
* If you only have package diffs at this stage:
* Map package → symbol set using symbol index
* Mark unit reachable if any symbol reachable
* If you already have symbol diffs:
* Directly query reachability per symbol
**DoD**
* For a PR that changes a controller path or core code, top diffs show reachable paths
* For a PR that only touches unreachable code (dead feature flags), system marks unreachable
---
### Milestone M3 — SmartDiff v2 (symbol-level diffs)
**Goal:** Move from “package changed” to “what functions/types changed.”
**Work items**
1. **Symbol indexer (.NET)**
* Extract public/internal symbols
* Map symbol → file/module + hash of IL/body (or semantic hash)
* Record signature + accessibility + attributes
2. **Symbol-level diff**
* Added/removed/modified methods/types
* Semantic hashing to avoid noise from non-functional rebuild differences
* Generate unit IDs like:
* `pkg:nuget/Newtonsoft.Json@13.0.3#method:Namespace.Type::Method(args)`
3. **Unit grouping**
* Group symbol deltas under:
* package delta
* assembly delta
* “API surface delta” (public symbol changes) for exposure
**DoD**
* Attestation units list individual changed symbols with reachable evidence
---
### Milestone M4 — Feature extraction for WII
**Goal:** Compute the features that make WII meaningful and explainable.
**Work items**
1. **Exposure classification**
* `exposure=true` if symbol is:
* directly an entrypoint method
* in the shortest path to an entrypoint
* part of public API surface changes
* Store explanation: “reachable from HTTP POST /x”
2. **Privilege sink detection**
* Maintain a versioned sink catalog:
* deserialization entrypoints
* process execution
* filesystem writes
* network dial / SSRF primitives
* crypto key handling
* dynamic code evaluation
* Mark if any witness path crosses sinks
* Store sink list in `paths[]`
3. **Guard coverage detection**
* Catalog of guard functions:
* input validation, sanitizers, authz checks, schema validators
* Heuristic: on witness path, detect guard call before sink
* `guardCoverage=true` reduces WII
4. **Library depth**
* Compute “lib call depth” heuristics:
* number of frames from entrypoint to unit
* number of boundary crossings (app → lib → lib)
* Use in scoring normalization
5. **Runtime heat integration (optional but high impact)**
* Ingest APM sampling / pprof / eBPF:
* `(symbolId → invocationCount or CPU%)`
* Normalize to 0..1 `hotPath`
* Add mapping strategy:
* route names → controller action symbols
**DoD**
* Every unit has a features object with enough fields to justify its score
---
### Milestone M5 — Vulnerability intelligence + determinism linkage
**Goal:** Use Concelier data to raise score when reachable changes align with exploitable vulns.
**Work items**
1. **Vuln snapshot service (Concelier)**
* Provide API:
* `GET /vuln/by-purl?purl=...` → CVEs + CVSS v4 + EPSS
* `GET /vuln/snapshot/{date}` for reproducibility
2. **Package ↔ symbol ↔ vuln mapping**
* Map CVE affected package versions to `DiffUnit` packages
* (Optional advanced) map to symbols if your feed provides function-level info
3. **Determinism rule**
* If `reachable=true` AND (cvss>threshold OR epss>threshold) add bonus
* Record “why” in unit explanation metadata
**DoD**
* A dependency bump that introduces/removes a CVE changes WII accordingly
* Attestation includes vuln snapshot digest
---
### Milestone M6 — WII scoring engine v1 + calibration
**Goal:** Produce a stable numeric index and calibrate thresholds.
**Work items**
1. **Scoring engine**
* Implement WII as:
* weighted sum of normalized features
* clamp to 0..100
* Make scoring config **external + versioned**:
* weights
* normalization caps (e.g., delta path len capped at ±5)
* determinism bonus amounts
* Include `scoring.version` and config hash in predicate
2. **Golden tests**
* Fixture diffs with expected WII
* Regression tests for scoring changes (if weights change, version bumps)
3. **Calibration workflow**
* Backtest on historical PRs/incidents:
* correlate WII with incidents / rollbacks / sev tickets
* Produce recommended initial gate thresholds:
* block: `maxWII >= X` and reachable and privileged
* warn: `p95WII >= Y`
* Store calibration report as an artifact (so you can justify policy)
**DoD**
* Score doesnt oscillate due to minor code movement
* Thresholds are defensible and adjustable
---
### Milestone M7 — Policy engine + CI/CD integration
**Goal:** Make it enforceable.
**Work items**
1. **Policy evaluation component**
* Inputs:
* DSSE attestation
* verification result
* environment context (prod/stage)
* Output:
* allow / warn / block + reason
2. **CI integration**
* Pipeline step:
* build artifact
* generate evidence
* compute diff against deployed baseline
* emit + sign attestation
* run policy gate
* Attach attestation to build metadata / OCI registry (as referrers if supported in your ecosystem)
3. **Deployment integration**
* Admission controller / deploy-time check:
* verify signature
* enforce policy
**DoD**
* A deployment with `reachable && maxWII >= threshold` is blocked or requires approval
---
### Milestone M8 — UI/UX and operator experience
**Goal:** People can understand and act quickly.
**Work items**
1. **Diff attestation viewer**
* Show:
* aggregate WII (max/p95)
* top units by WII
* per unit: features + witness path(s)
* sinks/guards
* vuln evidence (CVSS/EPSS) with snapshot date
2. **Explainability**
* “Why this score” breakdown:
* weights * feature values
* determinism bonus
* Link to evidence objects (SBOM digest, call graph digest)
3. **Notifications**
* Rules:
* page if `maxWII >= hard` and reachable and privileged
* slack/email if `maxWII >= warn`
* Include the top 3 units with paths
**DoD**
* Operators can make a decision within ~12 minutes reading the UI (no digging through logs)
---
### Milestone M9 — Multi-language expansion + runtime reachability improvements
**Goal:** Expand coverage beyond .NET and reduce static-analysis blind spots.
**Work items**
1. **Language adapters**
* Java: bytecode analyzer (ASM/Soot-like approach), Spring entrypoints
* Node: TypeScript compiler graph, Express/Koa routes (heuristics)
* Python: AST + import graph; Django/FastAPI routes (heuristics)
2. **Dynamic call handling**
* Reflection / DI / dynamic dispatch:
* conservative edges in static
* supplement with runtime traces to confirm
3. **Distributed reachability**
* Cross-service edges inferred from:
* OpenTelemetry traces (service A → service B endpoint)
* Build “service-level call graph” overlay:
* entrypoints + downstream calls
**DoD**
* Coverage reaches your top N services and languages
* False negatives reduced by runtime evidence
---
## 5) Detailed engineering work breakdown (by component)
### A) SmartDiff engine
**Deliverables**
* `ISmartDiff` interface with pluggable diff sources:
* SBOM diff
* lockfile diff
* symbol diff
* route diff (entrypoints changed)
**Key implementation tasks**
* Normalization layer (PURL, symbol IDs)
* Diff computation:
* add/remove/update
* semantic hash comparison
* Output:
* stable `DiffUnit` list
* deterministic ordering (sort by unitId)
**Risk controls**
* Deterministic hashing and ordering to keep DSSE stable
* “Noise filters” for rebuild-only diffs
---
### B) Call graph builder
**Deliverables**
* `CallGraph` object:
* nodes, edges
* node metadata
* entrypoints list
**Key tasks**
* Static analysis per language
* Entrypoint extraction (routes/jobs/consumers)
* Graph serialization format:
* versioned
* compressed adjacency lists
**Risk controls**
* Expect incomplete graphs; never treat as perfect truth
* Maintain confidence score per edge if desired
---
### C) Reachability service
**Deliverables**
* `IReachabilityService` with:
* `IsReachable(symbol)`
* `ShortestPathLen(symbol)`
* `PathPreview(symbol)` (witness)
* `LibCallDepth(symbol)`
* `PathHasPrivilegedSinks(symbol)`
* `PathHasGuards(symbol)`
**Key tasks**
* BFS from entrypoints
* Store distances and witnesses
* Cache per artifact digest
* Incremental updates:
* recompute only impacted parts when call graph changes (optional optimization)
---
### D) Feature extraction + WII scorer
**Deliverables**
* `WiiFeatures` + `WiiScorer`
* Versioned `ScoringConfig` (weights/normalizers)
**Key tasks**
* Normalization functions (caps and monotonic transforms)
* Determinism bonus logic
* Aggregation (max, p95, counts by changeKind)
**Risk controls**
* Scoring changes require a version bump
* Golden tests + backtests
---
### E) Attestation service
**Deliverables**
* `BuildStatement(...)`
* `SignDsse(...)`
* `VerifyDsse(...)`
**Key tasks**
* Canonical JSON serialization (avoid map-order randomness)
* Key management:
* key IDs
* rotation and revocation list handling
* Attach evidence digest set
**Risk controls**
* Sign only canonical bytes
* Record scanner version and config hash
---
### F) Persistence + ledger
**Deliverables**
* DB migrations
* Object store client
* Ledger publish/verify integration
**Key tasks**
* Store DSSE envelope bytes and computed digest
* Index by:
* subject digest
* before/after
* maxWII
* reachable count
* Retention policies
---
### G) Policy + Notifications
**Deliverables**
* Policy rules (OPA/Rego or internal DSL)
* CI step and deploy-time verifier
* Notify workflows
**Key tasks**
* Verification must be mandatory before policy evaluation
* Provide human-readable reasons
---
## 6) Testing strategy (ship safely)
### Unit tests
* SmartDiff normalization and diff correctness
* Reachability BFS correctness
* WII scoring determinism
* Predicate schema validation
* DSSE sign/verify roundtrip
### Integration tests
* Build sample .NET service → generate call graph → diff two versions → attest
* Concelier mocked responses for CVSS/EPSS
### End-to-end tests
* In CI: build → attest → store → verify → policy gate
* In deployment: admission check verifies signature and policy
### Performance tests
* Large call graph (100k+ nodes) BFS time and memory
* Batch scoring of thousands of diff units
### Security tests
* Tampered evidence digest detection
* Signature replay attempts (wrong subject digest)
* Key rotation tests
---
## 7) Rollout plan and operational guardrails
### Rollout stages
1. **Observe-only mode**
* Generate attestations, no gates
* Tune scoring weights and thresholds
2. **Warn mode**
* Notify only for high WII or reachable vuln combos
3. **Enforce mode**
* Block only on clear high-risk conditions (reachable + privileged + high WII)
* Add “break glass” path with audit logging
### Operational metrics
* Attestation generation success rate
* Verification failure rate
* Reachability coverage (% units with reachable computation)
* False positive/negative reports (human feedback)
* Policy gate blocks over time
### Playbooks
* What to do when:
* call graph generation fails
* Concelier feed unavailable
* signature verification fails
* scoring config mismatch
---
## 8) Concrete “first 10 commits” checklist
1. Add predicate JSON schema + canonical JSON serializer
2. Implement DSSE sign/verify library + CLI command
3. Create DB schema + evidence storage plumbing
4. Implement SBOM ingestion + SBOM diff -> `DiffUnit[]`
5. Emit DSSE attestation for SBOM diffs only
6. Implement .NET entrypoint extraction (minimal API + controllers)
7. Implement .NET call graph builder (basic invocations)
8. Implement reachability BFS + path witness extraction
9. Add WII scoring with config + golden tests
10. Add CI policy step (verify + evaluate thresholds) in warn-only mode
---
## 9) Deliverables bundle (what you should end up with)
* **Code**
* SmartDiff engine + plugins
* Call graph builders (starting with .NET)
* Reachability service + caching
* WII scoring service + config
* Attestation builder + DSSE signer/verifier
* Policy evaluation step
* UI endpoints + viewer
* **Schemas and specs**
* `smart-diff-wii@v1` JSON schema
* Evidence media types and versioning rules
* Scoring config format + versioning policy
* **Ops**
* Playbooks and runbooks
* Metrics dashboards
* Key rotation procedure
* Backtest/calibration report
---
If you want the plan converted into a **Jira-ready epic/story breakdown** (with each story having acceptance criteria and dependencies), tell me whether youre implementing **only .NET first** or **multi-language from day one**—and Ill output the backlog in that format.