docs consolidation work
This commit is contained in:
StellaOps Bot
@@ -18,7 +18,7 @@ Policy Engine compiles and evaluates Stella DSL policies deterministically, prod
|
||||
4. Coordinate cross-module changes in the main /AGENTS.md description and through the sprint plan.
|
||||
|
||||
## Guardrails
|
||||
- Honour the Aggregation-Only Contract where applicable (see ../../ingestion/aggregation-only-contract.md).
|
||||
- Honour the Aggregation-Only Contract where applicable (see ../../aoc/aggregation-only-contract.md).
|
||||
- Preserve determinism: sort outputs, normalise timestamps (UTC ISO-8601), and avoid machine-specific artefacts.
|
||||
- Keep Offline Kit parity in mind—document air-gapped workflows for any new feature.
|
||||
- Update runbooks/observability assets when operational characteristics change.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
> **Ownership:** Policy Guild • Platform Guild
|
||||
> **Services:** `StellaOps.Policy.Engine` (Minimal API + worker host)
|
||||
> **Data Stores:** PostgreSQL (`policy.*` schemas for packs, runs, exceptions, receipts), Object storage (explain bundles), optional queue
|
||||
> **Related docs:** [Policy overview](../../policy/overview.md), [DSL](../../policy/dsl.md), [SPL v1](../../policy/spl-v1.md), [Lifecycle](../../policy/lifecycle.md), [Runtime](../../policy/runtime.md), [Governance](../../policy/governance.md), [REST API](../../policy/api.md), [Policy CLI](../cli/guides/policy.md), [Architecture overview](../platform/architecture-overview.md), [AOC reference](../../ingestion/aggregation-only-contract.md)
|
||||
> **Related docs:** [Policy overview](../../policy/overview.md), [DSL](../../policy/dsl.md), [SPL v1](../../policy/spl-v1.md), [Lifecycle](../../policy/lifecycle.md), [Runtime](../../policy/runtime.md), [Governance](../../policy/governance.md), [REST API](../../policy/api.md), [Policy CLI](../cli/guides/policy.md), [Architecture overview](../platform/architecture-overview.md), [AOC reference](../../aoc/aggregation-only-contract.md)
|
||||
|
||||
This dossier describes the internal structure of the Policy Engine service delivered in Epic 2. It focuses on module boundaries, deterministic evaluation, orchestration, and integration contracts with Concelier, Excititor, SBOM Service, Authority, Scheduler, and Observability stacks.
|
||||
|
||||
@@ -111,7 +111,7 @@ Key notes:
|
||||
| **Authority Client** (`Authority/`) | Acquire tokens, enforce scopes, perform DPoP key rotation. | Only service identity uses `effective:write`. |
|
||||
| **DSL Compiler** (`Dsl/`) | Parse, canonicalise, IR generation, checksum caching. | Uses Roslyn-like pipeline; caches by `policyId+version+hash`. |
|
||||
| **Selection Layer** (`Selection/`) | Batch SBOM ↔ advisory ↔ VEX joiners; apply equivalence tables; support incremental cursors. | Deterministic ordering (SBOM → advisory → VEX). |
|
||||
| **Evaluator** (`Evaluation/`) | Execute IR with first-match semantics, compute severity/trust/reachability weights, record rule hits, and emit a unified confidence score with factor breakdown (reachability/runtime/VEX/provenance/policy). | Stateless; all inputs provided by selection layer. |
|
||||
| **Evaluator** (`Evaluation/`) | Execute IR with first-match semantics, compute severity/trust/reachability weights, record rule hits, and emit a unified confidence score with factor breakdown (reachability/runtime/VEX/provenance/policy). | Stateless; all inputs provided by selection layer. |
|
||||
| **Signals** (`Signals/`) | Normalizes reachability, trust, entropy, uncertainty, runtime hits into a single dictionary passed to Evaluator; supplies default `unknown` values when signals missing. Entropy penalties are derived from Scanner `layer_summary.json`/`entropy.report.json` (K=0.5, cap=0.3, block at image opaque ratio > 0.15 w/ unknown provenance) and exported via `policy_entropy_penalty_value` / `policy_entropy_image_opaque_ratio`; SPL scope `entropy.*` exposes `penalty`, `image_opaque_ratio`, `blocked`, `warned`, `capped`, `top_file_opaque_ratio`. | Aligns with `signals.*` namespace in DSL. |
|
||||
| **Materialiser** (`Materialization/`) | Upsert effective findings, append history, manage explain bundle exports. | PostgreSQL transactions per SBOM chunk. |
|
||||
| **Orchestrator** (`Runs/`) | Change-stream ingestion, fairness, retry/backoff, queue writer. | Works with Scheduler Models DTOs. |
|
||||
@@ -203,150 +203,150 @@ Determinism guard instrumentation wraps the evaluator, rejecting access to forbi
|
||||
|
||||
All payloads are immutable and include analyzer fingerprints (`scanner.native@sha256:...`, `policyEngine@sha256:...`) so replay tooling can recompute identical digests. Determinism tests cover both the OpenVEX JSON and the DSSE payload bytes.
|
||||
|
||||
|
||||
---
|
||||
|
||||
### 6.2 · Trust Lattice Policy Gates
|
||||
|
||||
The Policy Engine evaluates Trust Lattice gates after claim score merging to enforce trust-based constraints on VEX verdicts.
|
||||
|
||||
#### Gate Interface
|
||||
|
||||
```csharp
|
||||
public interface IPolicyGate
|
||||
{
|
||||
Task<GateResult> EvaluateAsync(
|
||||
MergeResult mergeResult,
|
||||
PolicyGateContext context,
|
||||
CancellationToken ct = default);
|
||||
}
|
||||
|
||||
public sealed record GateResult
|
||||
{
|
||||
public required string GateName { get; init; }
|
||||
public required bool Passed { get; init; }
|
||||
public string? Reason { get; init; }
|
||||
public ImmutableDictionary<string, object> Details { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
#### Available Gates
|
||||
|
||||
| Gate | Purpose | Configuration Key |
|
||||
|------|---------|-------------------|
|
||||
| **MinimumConfidenceGate** | Reject verdicts below confidence threshold per environment | `gates.minimumConfidence` |
|
||||
| **UnknownsBudgetGate** | Fail scan if unknowns exceed budget | `gates.unknownsBudget` |
|
||||
| **SourceQuotaGate** | Prevent single-source dominance without corroboration | `gates.sourceQuota` |
|
||||
| **ReachabilityRequirementGate** | Require reachability proof for critical CVEs | `gates.reachabilityRequirement` |
|
||||
| **EvidenceFreshnessGate** | Reject stale evidence below freshness threshold | `gates.evidenceFreshness` |
|
||||
|
||||
#### MinimumConfidenceGate
|
||||
|
||||
Requires minimum confidence threshold for suppression verdicts:
|
||||
|
||||
```yaml
|
||||
gates:
|
||||
minimumConfidence:
|
||||
enabled: true
|
||||
thresholds:
|
||||
production: 0.75 # High bar for production
|
||||
staging: 0.60 # Moderate for staging
|
||||
development: 0.40 # Permissive for dev
|
||||
applyToStatuses:
|
||||
- not_affected
|
||||
- fixed
|
||||
```
|
||||
|
||||
- **Behavior**: `affected` status bypasses this gate (conservative default).
|
||||
- **Result**: `confidence_below_threshold` when verdict confidence < environment threshold.
|
||||
|
||||
#### UnknownsBudgetGate
|
||||
|
||||
Limits exposure to unknown/unscored dependencies:
|
||||
|
||||
```yaml
|
||||
gates:
|
||||
unknownsBudget:
|
||||
enabled: true
|
||||
maxUnknownCount: 5
|
||||
maxCumulativeUncertainty: 2.0
|
||||
escalateOnExceed: true
|
||||
```
|
||||
|
||||
- **Behavior**: Fails when unknowns exceed count limit OR cumulative uncertainty exceeds budget.
|
||||
- **Cumulative uncertainty**: `sum(1 - ClaimScore)` across all verdicts.
|
||||
|
||||
#### SourceQuotaGate
|
||||
|
||||
Prevents single-source verdicts without corroboration:
|
||||
|
||||
```yaml
|
||||
gates:
|
||||
sourceQuota:
|
||||
enabled: true
|
||||
maxInfluencePercent: 60
|
||||
corroborationDelta: 0.10
|
||||
requireCorroboration: true
|
||||
```
|
||||
|
||||
- **Behavior**: Fails when single source provides > 60% of verdict weight AND no second source is within delta (0.10).
|
||||
- **Rationale**: Ensures critical decisions have multi-source validation.
|
||||
|
||||
#### ReachabilityRequirementGate
|
||||
|
||||
Requires reachability proof for high-severity vulnerabilities:
|
||||
|
||||
```yaml
|
||||
gates:
|
||||
reachabilityRequirement:
|
||||
enabled: true
|
||||
applySeverities:
|
||||
- critical
|
||||
- high
|
||||
exemptStatuses:
|
||||
- not_affected
|
||||
bypassReasons:
|
||||
- component_not_present
|
||||
```
|
||||
|
||||
- **Behavior**: Fails when CRITICAL/HIGH CVE marked `not_affected` lacks reachability proof (unless bypass reason applies).
|
||||
|
||||
#### Gate Registry
|
||||
|
||||
Gates are registered via DI and evaluated in sequence:
|
||||
|
||||
```csharp
|
||||
public interface IPolicyGateRegistry
|
||||
{
|
||||
IEnumerable<IPolicyGate> GetEnabledGates(string environment);
|
||||
Task<GateEvaluationResult> EvaluateAllAsync(
|
||||
MergeResult mergeResult,
|
||||
PolicyGateContext context,
|
||||
CancellationToken ct = default);
|
||||
}
|
||||
```
|
||||
|
||||
#### Gate Metrics
|
||||
|
||||
- `policy_gate_evaluations_total{gate,result}` — Count of gate evaluations by outcome
|
||||
- `policy_gate_failures_total{gate,reason}` — Count of gate failures by reason
|
||||
- `policy_gate_latency_seconds{gate}` — Gate evaluation latency histogram
|
||||
|
||||
#### Gate Implementation Reference
|
||||
|
||||
| Gate | Source File |
|
||||
|------|-------------|
|
||||
| MinimumConfidenceGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/MinimumConfidenceGate.cs` |
|
||||
| UnknownsBudgetGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/UnknownsBudgetGate.cs` |
|
||||
| SourceQuotaGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/SourceQuotaGate.cs` |
|
||||
| ReachabilityRequirementGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/ReachabilityRequirementGate.cs` |
|
||||
| EvidenceFreshnessGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/EvidenceFreshnessGate.cs` |
|
||||
|
||||
See `etc/policy-gates.yaml.sample` for complete gate configuration options.
|
||||
|
||||
**Related Documentation:**
|
||||
- [Trust Lattice Specification](../excititor/trust-lattice.md)
|
||||
- [Verdict Manifest Specification](../authority/verdict-manifest.md)
|
||||
|
||||
---
|
||||
|
||||
### 6.2 · Trust Lattice Policy Gates
|
||||
|
||||
The Policy Engine evaluates Trust Lattice gates after claim score merging to enforce trust-based constraints on VEX verdicts.
|
||||
|
||||
#### Gate Interface
|
||||
|
||||
```csharp
|
||||
public interface IPolicyGate
|
||||
{
|
||||
Task<GateResult> EvaluateAsync(
|
||||
MergeResult mergeResult,
|
||||
PolicyGateContext context,
|
||||
CancellationToken ct = default);
|
||||
}
|
||||
|
||||
public sealed record GateResult
|
||||
{
|
||||
public required string GateName { get; init; }
|
||||
public required bool Passed { get; init; }
|
||||
public string? Reason { get; init; }
|
||||
public ImmutableDictionary<string, object> Details { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
#### Available Gates
|
||||
|
||||
| Gate | Purpose | Configuration Key |
|
||||
|------|---------|-------------------|
|
||||
| **MinimumConfidenceGate** | Reject verdicts below confidence threshold per environment | `gates.minimumConfidence` |
|
||||
| **UnknownsBudgetGate** | Fail scan if unknowns exceed budget | `gates.unknownsBudget` |
|
||||
| **SourceQuotaGate** | Prevent single-source dominance without corroboration | `gates.sourceQuota` |
|
||||
| **ReachabilityRequirementGate** | Require reachability proof for critical CVEs | `gates.reachabilityRequirement` |
|
||||
| **EvidenceFreshnessGate** | Reject stale evidence below freshness threshold | `gates.evidenceFreshness` |
|
||||
|
||||
#### MinimumConfidenceGate
|
||||
|
||||
Requires minimum confidence threshold for suppression verdicts:
|
||||
|
||||
```yaml
|
||||
gates:
|
||||
minimumConfidence:
|
||||
enabled: true
|
||||
thresholds:
|
||||
production: 0.75 # High bar for production
|
||||
staging: 0.60 # Moderate for staging
|
||||
development: 0.40 # Permissive for dev
|
||||
applyToStatuses:
|
||||
- not_affected
|
||||
- fixed
|
||||
```
|
||||
|
||||
- **Behavior**: `affected` status bypasses this gate (conservative default).
|
||||
- **Result**: `confidence_below_threshold` when verdict confidence < environment threshold.
|
||||
|
||||
#### UnknownsBudgetGate
|
||||
|
||||
Limits exposure to unknown/unscored dependencies:
|
||||
|
||||
```yaml
|
||||
gates:
|
||||
unknownsBudget:
|
||||
enabled: true
|
||||
maxUnknownCount: 5
|
||||
maxCumulativeUncertainty: 2.0
|
||||
escalateOnExceed: true
|
||||
```
|
||||
|
||||
- **Behavior**: Fails when unknowns exceed count limit OR cumulative uncertainty exceeds budget.
|
||||
- **Cumulative uncertainty**: `sum(1 - ClaimScore)` across all verdicts.
|
||||
|
||||
#### SourceQuotaGate
|
||||
|
||||
Prevents single-source verdicts without corroboration:
|
||||
|
||||
```yaml
|
||||
gates:
|
||||
sourceQuota:
|
||||
enabled: true
|
||||
maxInfluencePercent: 60
|
||||
corroborationDelta: 0.10
|
||||
requireCorroboration: true
|
||||
```
|
||||
|
||||
- **Behavior**: Fails when single source provides > 60% of verdict weight AND no second source is within delta (0.10).
|
||||
- **Rationale**: Ensures critical decisions have multi-source validation.
|
||||
|
||||
#### ReachabilityRequirementGate
|
||||
|
||||
Requires reachability proof for high-severity vulnerabilities:
|
||||
|
||||
```yaml
|
||||
gates:
|
||||
reachabilityRequirement:
|
||||
enabled: true
|
||||
applySeverities:
|
||||
- critical
|
||||
- high
|
||||
exemptStatuses:
|
||||
- not_affected
|
||||
bypassReasons:
|
||||
- component_not_present
|
||||
```
|
||||
|
||||
- **Behavior**: Fails when CRITICAL/HIGH CVE marked `not_affected` lacks reachability proof (unless bypass reason applies).
|
||||
|
||||
#### Gate Registry
|
||||
|
||||
Gates are registered via DI and evaluated in sequence:
|
||||
|
||||
```csharp
|
||||
public interface IPolicyGateRegistry
|
||||
{
|
||||
IEnumerable<IPolicyGate> GetEnabledGates(string environment);
|
||||
Task<GateEvaluationResult> EvaluateAllAsync(
|
||||
MergeResult mergeResult,
|
||||
PolicyGateContext context,
|
||||
CancellationToken ct = default);
|
||||
}
|
||||
```
|
||||
|
||||
#### Gate Metrics
|
||||
|
||||
- `policy_gate_evaluations_total{gate,result}` — Count of gate evaluations by outcome
|
||||
- `policy_gate_failures_total{gate,reason}` — Count of gate failures by reason
|
||||
- `policy_gate_latency_seconds{gate}` — Gate evaluation latency histogram
|
||||
|
||||
#### Gate Implementation Reference
|
||||
|
||||
| Gate | Source File |
|
||||
|------|-------------|
|
||||
| MinimumConfidenceGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/MinimumConfidenceGate.cs` |
|
||||
| UnknownsBudgetGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/UnknownsBudgetGate.cs` |
|
||||
| SourceQuotaGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/SourceQuotaGate.cs` |
|
||||
| ReachabilityRequirementGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/ReachabilityRequirementGate.cs` |
|
||||
| EvidenceFreshnessGate | `src/Policy/__Libraries/StellaOps.Policy/Gates/EvidenceFreshnessGate.cs` |
|
||||
|
||||
See `etc/policy-gates.yaml.sample` for complete gate configuration options.
|
||||
|
||||
**Related Documentation:**
|
||||
- [Trust Lattice Specification](../excititor/trust-lattice.md)
|
||||
- [Verdict Manifest Specification](../authority/verdict-manifest.md)
|
||||
---
|
||||
|
||||
## 7 · Security & Tenancy
|
||||
|
||||
Reference in New Issue
Block a user