stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 3 Releases Wiki Activity

save progress

Browse Source
This commit is contained in:
StellaOps Bot
2026-01-04 14:54:52 +02:00
parent c49b03a254
commit 3098e84de4
132 changed files with 19783 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

540
docs/implplan/SPRINT_20260104_002_SCANNER_secret_leak_detection_core.md Normal file
View File

@@ -0,0 +1,540 @@
# Sprint 20260104_002_SCANNER - Secret Leak Detection Core Analyzer
## Topic & Scope
Implement the core `StellaOps.Scanner.Analyzers.Secrets` plugin that detects accidentally committed secrets in container layers during scans. This is the foundational sprint for secret leak detection capability.
**Key deliverables:**
1. **Secrets Analyzer Plugin**: Core analyzer that executes regex/entropy-based detection rules
2. **Rule Engine**: Rule definition models, matching logic, and deterministic execution
3. **Masking Engine**: Payload masking to ensure secrets never leak in outputs
4. **Evidence Emission**: `secret.leak` evidence type integration with ScanAnalysisStore
5. **Feature Flag**: Experimental toggle for gradual rollout
**Working directory:** `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/`
## Dependencies & Concurrency
- **Depends on**: Surface.Secrets, Surface.Validation, Surface.Env (already implemented)
- **Required by**: Sprint 20260104_003 (Rule Bundle Infrastructure), Sprint 20260104_004 (Policy DSL)
- **Parallel work**: Tasks SLD-001 through SLD-008 can be developed concurrently
- **Integration tasks** (SLD-009+) require prior tasks complete
## Documentation Prerequisites
- docs/README.md
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
- docs/modules/scanner/architecture.md
- docs/modules/scanner/design/surface-secrets.md
- docs/modules/scanner/operations/secret-leak-detection.md (target spec)
- CLAUDE.md (especially Section 8: Code Quality & Determinism Rules)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | SLD-001 | TODO | None | Scanner Guild | Create project structure and csproj |
| 2 | SLD-002 | TODO | None | Scanner Guild | Define SecretRule and SecretRuleset models |
| 3 | SLD-003 | TODO | None | Scanner Guild | Implement ISecretDetector interface and RegexDetector |
| 4 | SLD-004 | TODO | None | Scanner Guild | Implement EntropyDetector for high-entropy string detection |
| 5 | SLD-005 | TODO | None | Scanner Guild | Implement PayloadMasker with configurable masking strategies |
| 6 | SLD-006 | TODO | None | Scanner Guild | Define SecretLeakEvidence record and finding model |
| 7 | SLD-007 | TODO | SLD-002 | Scanner Guild | Implement RulesetLoader with JSON parsing |
| 8 | SLD-008 | TODO | None | Scanner Guild | Add SecretsAnalyzerOptions with feature flag support |
| 9 | SLD-009 | TODO | SLD-003,SLD-004 | Scanner Guild | Implement CompositeSecretDetector combining regex and entropy |
| 10 | SLD-010 | TODO | SLD-006,SLD-009 | Scanner Guild | Implement SecretsAnalyzer (ILanguageAnalyzer) |
| 11 | SLD-011 | TODO | SLD-010 | Scanner Guild | Add SecretsAnalyzerHost for plugin lifecycle |
| 12 | SLD-012 | TODO | SLD-011 | Scanner Guild | Integrate with Scanner Worker pipeline |
| 13 | SLD-013 | TODO | SLD-010 | Scanner Guild | Add DI registration in ServiceCollectionExtensions |
| 14 | SLD-014 | TODO | All | Scanner Guild | Add comprehensive unit tests |
| 15 | SLD-015 | TODO | SLD-014 | Scanner Guild | Add integration tests with test fixtures |
| 16 | SLD-016 | TODO | All | Scanner Guild | Create AGENTS.md for module |
## Task Details
### SLD-001: Project Structure
Create the project skeleton following Scanner conventions:
```
src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/
├── StellaOps.Scanner.Analyzers.Secrets.csproj
├── AGENTS.md
├── AssemblyInfo.cs
├── Detectors/
│ ├── ISecretDetector.cs
│ ├── RegexDetector.cs
│ ├── EntropyDetector.cs
│ └── CompositeSecretDetector.cs
├── Rules/
│ ├── SecretRule.cs
│ ├── SecretRuleset.cs
│ └── RulesetLoader.cs
├── Masking/
│ ├── IPayloadMasker.cs
│ └── PayloadMasker.cs
├── Evidence/
│ ├── SecretLeakEvidence.cs
│ └── SecretFinding.cs
├── SecretsAnalyzer.cs
├── SecretsAnalyzerHost.cs
├── SecretsAnalyzerOptions.cs
└── ServiceCollectionExtensions.cs
```
csproj should reference:
- StellaOps.Scanner.Core
- StellaOps.Scanner.Surface
- StellaOps.Evidence.Core
### SLD-002: Rule Models
Define the rule structure for secret detection:
```csharp
/// <summary>
/// A single secret detection rule.
/// </summary>
public sealed record SecretRule
{
public required string Id { get; init; } // e.g., "stellaops.secrets.aws-access-key"
public required string Version { get; init; } // e.g., "2025.11.0"
public required string Name { get; init; } // Human-readable name
public required string Description { get; init; }
public required SecretRuleType Type { get; init; } // Regex, Entropy, Composite
public required string Pattern { get; init; } // Regex pattern or entropy config
public required SecretSeverity Severity { get; init; }
public required SecretConfidence Confidence { get; init; }
public string? MaskingHint { get; init; } // e.g., "prefix:4,suffix:2"
public ImmutableArray<string> Keywords { get; init; } // Pre-filter keywords
public ImmutableArray<string> FilePatterns { get; init; } // Glob patterns for file filtering
public bool Enabled { get; init; } = true;
}
public enum SecretRuleType { Regex, Entropy, Composite }
public enum SecretSeverity { Low, Medium, High, Critical }
public enum SecretConfidence { Low, Medium, High }
/// <summary>
/// A versioned collection of secret detection rules.
/// </summary>
public sealed record SecretRuleset
{
public required string Id { get; init; } // e.g., "secrets.ruleset"
public required string Version { get; init; } // e.g., "2025.11"
public required DateTimeOffset CreatedAt { get; init; }
public required ImmutableArray<SecretRule> Rules { get; init; }
public string? Sha256Digest { get; init; } // Integrity hash
}
```
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Rules/`
### SLD-003: Regex Detector
Implement regex-based secret detection:
```csharp
public interface ISecretDetector
{
string DetectorId { get; }
ValueTask<IReadOnlyList<SecretMatch>> DetectAsync(
ReadOnlyMemory<byte> content,
string filePath,
SecretRule rule,
CancellationToken ct = default);
}
public sealed record SecretMatch(
SecretRule Rule,
string FilePath,
int LineNumber,
int ColumnStart,
int ColumnEnd,
ReadOnlyMemory<byte> RawMatch, // For masking
double ConfidenceScore);
public sealed class RegexDetector : ISecretDetector
{
public string DetectorId => "regex";
// Implementation notes:
// - Use compiled regex for performance
// - Apply keyword pre-filter before regex matching
// - Respect file pattern filters
// - Track line/column for precise location
// - Never log raw match content
}
```
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Detectors/`
### SLD-004: Entropy Detector
Implement Shannon entropy-based detection for high-entropy strings:
```csharp
public sealed class EntropyDetector : ISecretDetector
{
public string DetectorId => "entropy";
// Implementation notes:
// - Calculate Shannon entropy for candidate strings
// - Default threshold: 4.5 bits per character
// - Minimum length: 16 characters
// - Skip common high-entropy non-secrets (UUIDs, hashes in comments)
// - Apply charset detection (base64, hex, alphanumeric)
}
public static class EntropyCalculator
{
/// <summary>
/// Calculates Shannon entropy in bits per character.
/// </summary>
public static double Calculate(ReadOnlySpan<byte> data)
{
// Use CultureInfo.InvariantCulture for all formatting
// Return 0.0 for empty input
}
}
```
### SLD-005: Payload Masker
Implement secure payload masking:
```csharp
public interface IPayloadMasker
{
/// <summary>
/// Masks a secret payload preserving prefix/suffix for identification.
/// </summary>
/// <param name="payload">The raw secret bytes</param>
/// <param name="hint">Optional masking hint from rule (e.g., "prefix:4,suffix:2")</param>
/// <returns>Masked string (e.g., "AKIA****B7")</returns>
string Mask(ReadOnlySpan<byte> payload, string? hint = null);
}
public sealed class PayloadMasker : IPayloadMasker
{
// Default: preserve first 4 and last 2 characters
// Replace middle with asterisks (max 8 asterisks)
// Minimum output length: 8 characters
// Never expose more than 6 characters total
public const int DefaultPrefixLength = 4;
public const int DefaultSuffixLength = 2;
public const int MaxMaskLength = 8;
public const char MaskChar = '*';
}
```
### SLD-006: Evidence Models
Define the evidence structure for policy integration:
```csharp
/// <summary>
/// Evidence record for a detected secret leak.
/// </summary>
public sealed record SecretLeakEvidence
{
public required string EvidenceType => "secret.leak";
public required string RuleId { get; init; }
public required string RuleVersion { get; init; }
public required SecretSeverity Severity { get; init; }
public required SecretConfidence Confidence { get; init; }
public required string FilePath { get; init; }
public required int LineNumber { get; init; }
public required string Mask { get; init; } // Masked payload
public required string BundleId { get; init; }
public required string BundleVersion { get; init; }
public required DateTimeOffset DetectedAt { get; init; }
public ImmutableDictionary<string, string>? Metadata { get; init; }
}
/// <summary>
/// Aggregated finding for a single secret match.
/// </summary>
public sealed record SecretFinding
{
public required Guid Id { get; init; }
public required SecretLeakEvidence Evidence { get; init; }
public required string ScanId { get; init; }
public required string TenantId { get; init; }
public required string ArtifactDigest { get; init; }
}
```
### SLD-007: Ruleset Loader
Implement deterministic ruleset loading:
```csharp
public interface IRulesetLoader
{
ValueTask<SecretRuleset> LoadAsync(
string rulesetPath,
CancellationToken ct = default);
ValueTask<SecretRuleset> LoadFromJsonlAsync(
Stream rulesStream,
string bundleId,
string bundleVersion,
CancellationToken ct = default);
}
public sealed class RulesetLoader : IRulesetLoader
{
// Implementation notes:
// - Parse secrets.ruleset.rules.jsonl (NDJSON format)
// - Validate rule schema on load
// - Sort rules by ID for deterministic ordering
// - Calculate and verify SHA-256 digest
// - Use CultureInfo.InvariantCulture for all parsing
// - Log bundle version on successful load
}
```
### SLD-008: Analyzer Options
Configuration options with feature flag:
```csharp
public sealed class SecretsAnalyzerOptions
{
/// <summary>
/// Enable secret leak detection (experimental feature).
/// </summary>
public bool Enabled { get; set; } = false;
/// <summary>
/// Path to the ruleset bundle directory.
/// </summary>
public string RulesetPath { get; set; } = "/opt/stellaops/plugins/scanner/analyzers/secrets";
/// <summary>
/// Minimum confidence level to report findings.
/// </summary>
public SecretConfidence MinConfidence { get; set; } = SecretConfidence.Medium;
/// <summary>
/// Maximum findings per scan (circuit breaker).
/// </summary>
public int MaxFindingsPerScan { get; set; } = 1000;
/// <summary>
/// File size limit for scanning (bytes).
/// </summary>
public long MaxFileSizeBytes { get; set; } = 10 * 1024 * 1024; // 10MB
/// <summary>
/// Enable entropy-based detection.
/// </summary>
public bool EnableEntropyDetection { get; set; } = true;
/// <summary>
/// Entropy threshold (bits per character).
/// </summary>
public double EntropyThreshold { get; set; } = 4.5;
}
```
### SLD-009: Composite Detector
Combine multiple detection strategies:
```csharp
public sealed class CompositeSecretDetector : ISecretDetector
{
private readonly IReadOnlyList<ISecretDetector> _detectors;
private readonly ILogger<CompositeSecretDetector> _logger;
public string DetectorId => "composite";
// Implementation notes:
// - Execute detectors in parallel where possible
// - Deduplicate overlapping matches
// - Merge confidence scores for overlapping detections
// - Respect per-rule detector type preference
}
```
### SLD-010: Secrets Analyzer
Main analyzer implementation:
```csharp
public sealed class SecretsAnalyzer : ILayerAnalyzer
{
public string AnalyzerId => "secrets";
public string DisplayName => "Secret Leak Detector";
// Implementation notes:
// - Check feature flag before processing
// - Load ruleset once at startup (cached)
// - Apply file pattern filters efficiently
// - Execute detection on text files only
// - Emit SecretLeakEvidence for each finding
// - Apply masking before any output
// - Track metrics: scanner.secret.finding_total
// - Add tracing span: scanner.secrets.scan
}
```
### SLD-011: Analyzer Host
Lifecycle management for the analyzer:
```csharp
public sealed class SecretsAnalyzerHost : IHostedService
{
// Implementation notes:
// - Load and validate ruleset on startup
// - Log bundle version and rule count
// - Verify DSSE signature if available
// - Graceful shutdown with finding flush
// - Emit startup log: "SecretsAnalyzerHost: Loaded bundle {version} with {count} rules"
}
```
### SLD-012: Worker Integration
Integrate with Scanner Worker pipeline:
```csharp
// In Scanner.Worker processing pipeline:
// 1. Add SecretsAnalyzer to analyzer chain (after language analyzers)
// 2. Gate execution on feature flag
// 3. Store findings in ScanAnalysisStore
// 4. Include in scan completion event
```
### SLD-013: DI Registration
```csharp
public static class ServiceCollectionExtensions
{
public static IServiceCollection AddSecretsAnalyzer(
this IServiceCollection services,
IConfiguration configuration)
{
services.AddOptions<SecretsAnalyzerOptions>()
.Bind(configuration.GetSection("Scanner:Analyzers:Secrets"))
.ValidateDataAnnotations()
.ValidateOnStart();
services.AddSingleton<IPayloadMasker, PayloadMasker>();
services.AddSingleton<IRulesetLoader, RulesetLoader>();
services.AddSingleton<ISecretDetector, RegexDetector>();
services.AddSingleton<ISecretDetector, EntropyDetector>();
services.AddSingleton<ISecretDetector, CompositeSecretDetector>();
services.AddSingleton<SecretsAnalyzer>();
services.AddHostedService<SecretsAnalyzerHost>();
return services;
}
}
```
### SLD-014: Unit Tests
Required test coverage in `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Secrets.Tests/`:
```
├── Detectors/
│ ├── RegexDetectorTests.cs
│ ├── EntropyDetectorTests.cs
│ ├── EntropyCalculatorTests.cs
│ └── CompositeSecretDetectorTests.cs
├── Rules/
│ ├── SecretRuleTests.cs
│ └── RulesetLoaderTests.cs
├── Masking/
│ └── PayloadMaskerTests.cs
├── Evidence/
│ └── SecretLeakEvidenceTests.cs
├── SecretsAnalyzerTests.cs
└── Fixtures/
├── aws-access-key.txt
├── github-token.txt
├── private-key.pem
└── test-ruleset.jsonl
```
Test requirements:
- All tests must be deterministic
- Use `[Trait("Category", "Unit")]` for unit tests
- Test masking never exposes full secrets
- Test entropy calculation with known inputs
- Test regex patterns match expected secrets
### SLD-015: Integration Tests
Integration tests with Scanner Worker:
```
├── SecretsAnalyzerIntegrationTests.cs
│ - Test full scan with secrets embedded
│ - Verify findings in ScanAnalysisStore
│ - Verify masking in output
│ - Test feature flag disables analyzer
├── RulesetLoadingTests.cs
│ - Test loading from file system
│ - Test invalid ruleset handling
│ - Test missing bundle handling
```
### SLD-016: Module AGENTS.md
Create `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/AGENTS.md` with:
- Mission statement
- Scope definition
- Required reading list
- Working agreements
- Security considerations
## Built-in Rule Examples
Initial rules to include in default bundle:
| Rule ID | Pattern Type | Description |
|---------|--------------|-------------|
| `stellaops.secrets.aws-access-key` | Regex | AWS Access Key ID (AKIA...) |
| `stellaops.secrets.aws-secret-key` | Regex + Entropy | AWS Secret Access Key |
| `stellaops.secrets.github-pat` | Regex | GitHub Personal Access Token |
| `stellaops.secrets.github-app` | Regex | GitHub App Token (ghs_, ghp_) |
| `stellaops.secrets.gitlab-pat` | Regex | GitLab Personal Access Token |
| `stellaops.secrets.private-key-rsa` | Regex | RSA Private Key (PEM) |
| `stellaops.secrets.private-key-ec` | Regex | EC Private Key (PEM) |
| `stellaops.secrets.jwt` | Regex + Entropy | JSON Web Token |
| `stellaops.secrets.basic-auth` | Regex | Basic Auth credentials in URLs |
| `stellaops.secrets.generic-api-key` | Entropy | High-entropy API key patterns |
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| Use NDJSON for rule format | Line-based parsing, easy streaming, git-friendly diffs |
| Mask before any persistence | Defense in depth - secrets never stored |
| Feature flag default off | Safe rollout, tenant opt-in required |
| Entropy threshold 4.5 bits | Balance between false positives and detection rate |
| Max 1000 findings per scan | Circuit breaker prevents DoS on noisy images |
| Text files only | Binary secret detection deferred to future sprint |
## Metrics & Observability
| Metric | Type | Labels |
|--------|------|--------|
| `scanner.secret.finding_total` | Counter | tenant, ruleId, severity, confidence |
| `scanner.secret.scan_duration_seconds` | Histogram | tenant |
| `scanner.secret.rules_loaded` | Gauge | bundleVersion |
| `scanner.secret.files_scanned` | Counter | tenant |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Based on gap analysis of secrets scanning support |

451
docs/implplan/SPRINT_20260104_003_SCANNER_secret_rule_bundles.md Normal file
View File

@@ -0,0 +1,451 @@
# Sprint 20260104_003_SCANNER - Secret Detection Rule Bundle Infrastructure
## Topic & Scope
Implement the DSSE-signed rule bundle infrastructure for secret leak detection. This sprint delivers the signing, verification, and distribution pipeline for deterministic rule bundles.
**Key deliverables:**
1. **Bundle Schema**: Formal JSON schema for rule bundles
2. **Bundle Builder**: CLI tool to create and sign rule bundles
3. **DSSE Integration**: Signing and verification using existing Signer/Attestor modules
4. **Bundle Verification**: Runtime verification of bundle integrity and provenance
5. **Default Bundle**: Initial set of secret detection rules
**Working directory:** `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/`, `src/Cli/`, `offline/rules/secrets/`
## Dependencies & Concurrency
- **Depends on**: Sprint 20260104_002 (Core Analyzer), StellaOps.Attestor, StellaOps.Signer
- **Required by**: Sprint 20260104_005 (Offline Kit Integration)
- **Parallel work**: Tasks RB-001 through RB-005 can be developed concurrently
## Documentation Prerequisites
- docs/modules/scanner/operations/secret-leak-detection.md
- docs/modules/attestor/architecture.md
- docs/modules/signer/architecture.md
- docs/ci/dsse-build-flow.md
- CLAUDE.md Section 8.6 (DSSE PAE Consistency)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | RB-001 | DONE | None | Scanner Guild | Define bundle manifest JSON schema |
| 2 | RB-002 | DONE | None | Scanner Guild | Define rules JSONL schema and validation |
| 3 | RB-003 | DONE | RB-001,RB-002 | Scanner Guild | Create BundleBuilder class for bundle creation |
| 4 | RB-004 | DONE | RB-003 | Scanner Guild | Add DSSE signing integration |
| 5 | RB-005 | DONE | RB-004 | Scanner Guild | Implement BundleVerifier with Attestor integration |
| 6 | RB-006 | DONE | RB-005 | CLI Guild | Add `stella secrets bundle create` CLI command |
| 7 | RB-007 | DONE | RB-005 | CLI Guild | Add `stella secrets bundle verify` CLI command |
| 8 | RB-008 | DONE | RB-005 | Scanner Guild | Integrate verification into SecretsAnalyzerHost |
| 9 | RB-009 | DONE | RB-002 | Scanner Guild | Create default rule definitions |
| 10 | RB-010 | DONE | RB-009 | Scanner Guild | Build and sign initial bundle (2026.01) |
| 11 | RB-011 | DONE | All | Scanner Guild | Add unit and integration tests |
| 12 | RB-012 | DONE | RB-010 | Docs Guild | Document bundle lifecycle and rotation |
## Task Details
### RB-001: Bundle Manifest Schema
Define the manifest schema (`secrets.ruleset.manifest.json`):
```json
{
"$schema": "https://stellaops.io/schemas/secrets-ruleset-manifest-v1.json",
"schemaVersion": "1.0",
"id": "secrets.ruleset",
"version": "2026.01",
"createdAt": "2026-01-04T00:00:00Z",
"description": "StellaOps Secret Detection Rules",
"rules": [
{
"id": "stellaops.secrets.aws-access-key",
"version": "1.0.0",
"severity": "high",
"enabled": true
}
],
"integrity": {
"rulesFile": "secrets.ruleset.rules.jsonl",
"rulesSha256": "abc123...",
"totalRules": 15,
"enabledRules": 15
},
"signatures": {
"dsseEnvelope": "secrets.ruleset.dsse.json"
}
}
```
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Bundles/Schemas/`
### RB-002: Rules JSONL Schema
Define the rule entry schema for NDJSON format:
```json
{
"id": "stellaops.secrets.aws-access-key",
"version": "1.0.0",
"name": "AWS Access Key ID",
"description": "Detects AWS Access Key IDs in source code and configuration files",
"type": "regex",
"pattern": "(?:A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}",
"severity": "high",
"confidence": "high",
"keywords": ["AKIA", "ASIA", "AIDA", "aws"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.conf", "*.config"],
"maskingHint": "prefix:4,suffix:2",
"metadata": {
"category": "cloud-credentials",
"provider": "aws",
"references": ["https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html"]
}
}
```
Validation rules:
- `id` must be namespaced (e.g., `stellaops.secrets.*`)
- `version` must be valid SemVer
- `pattern` must be valid regex (compile-time validation)
- `severity` must be one of: low, medium, high, critical
- `confidence` must be one of: low, medium, high
### RB-003: Bundle Builder
Implement the bundle creation logic:
```csharp
public interface IBundleBuilder
{
/// <summary>
/// Creates a bundle from individual rule files.
/// </summary>
Task<BundleArtifact> BuildAsync(
BundleBuildOptions options,
CancellationToken ct = default);
}
public sealed record BundleBuildOptions
{
public required string OutputDirectory { get; init; }
public required string BundleId { get; init; }
public required string Version { get; init; }
public required IReadOnlyList<string> RuleFiles { get; init; }
public TimeProvider? TimeProvider { get; init; }
}
public sealed record BundleArtifact
{
public required string ManifestPath { get; init; }
public required string RulesPath { get; init; }
public required string RulesSha256 { get; init; }
public required int TotalRules { get; init; }
}
public sealed class BundleBuilder : IBundleBuilder
{
// Implementation notes:
// - Validate each rule on load
// - Sort rules by ID for deterministic output
// - Compute SHA-256 of rules file
// - Generate manifest with integrity info
// - Use TimeProvider for timestamps (determinism)
}
```
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Bundles/`
### RB-004: DSSE Signing Integration
Integrate with Signer module for bundle signing:
```csharp
public interface IBundleSigner
{
/// <summary>
/// Signs a bundle artifact producing a DSSE envelope.
/// </summary>
Task<DsseEnvelope> SignAsync(
BundleArtifact artifact,
SigningOptions options,
CancellationToken ct = default);
}
public sealed record SigningOptions
{
public required string KeyId { get; init; }
public string PayloadType { get; init; } = "application/vnd.stellaops.secrets-ruleset+json";
public bool IncludeCertificateChain { get; init; } = true;
}
public sealed class BundleSigner : IBundleSigner
{
private readonly ISigner _signer;
// Implementation notes:
// - Use existing Signer infrastructure
// - Payload is the manifest JSON (not rules file)
// - Include rules file digest in signed payload
// - Support multiple signature algorithms
}
```
### RB-005: Bundle Verifier
Implement verification with Attestor integration:
```csharp
public interface IBundleVerifier
{
/// <summary>
/// Verifies a bundle's DSSE signature and integrity.
/// </summary>
Task<BundleVerificationResult> VerifyAsync(
string bundleDirectory,
VerificationOptions options,
CancellationToken ct = default);
}
public sealed record VerificationOptions
{
public string? AttestorUrl { get; init; }
public bool RequireRekorProof { get; init; } = false;
public IReadOnlyList<string>? TrustedKeyIds { get; init; }
}
public sealed record BundleVerificationResult
{
public required bool IsValid { get; init; }
public required string BundleVersion { get; init; }
public required DateTimeOffset SignedAt { get; init; }
public required string SignerKeyId { get; init; }
public string? RekorLogId { get; init; }
public IReadOnlyList<string>? ValidationErrors { get; init; }
}
public sealed class BundleVerifier : IBundleVerifier
{
private readonly IAttestorClient _attestorClient;
// Implementation notes:
// - Verify DSSE envelope signature
// - Verify rules file SHA-256 matches manifest
// - Optionally verify Rekor transparency log entry
// - Support offline verification (no network calls)
}
```
### RB-006: CLI Bundle Create Command
Add CLI command for bundle creation:
```bash
stella secrets bundle create \
--output ./bundles/2026.01 \
--bundle-id secrets.ruleset \
--version 2026.01 \
--rules ./rules/*.json \
--sign \
--key-id stellaops-secrets-signer
```
Implementation in `src/Cli/StellaOps.Cli/Commands/Secrets/`:
```csharp
[Command("secrets bundle create")]
public class BundleCreateCommand
{
[Option("--output", Required = true)]
public string OutputDirectory { get; set; }
[Option("--bundle-id", Required = true)]
public string BundleId { get; set; }
[Option("--version", Required = true)]
public string Version { get; set; }
[Option("--rules", Required = true)]
public IReadOnlyList<string> RuleFiles { get; set; }
[Option("--sign")]
public bool Sign { get; set; }
[Option("--key-id")]
public string? KeyId { get; set; }
}
```
### RB-007: CLI Bundle Verify Command
Add CLI command for bundle verification:
```bash
stella secrets bundle verify \
--bundle ./bundles/2026.01 \
--attestor-url http://attestor.local \
--require-rekor
```
```csharp
[Command("secrets bundle verify")]
public class BundleVerifyCommand
{
[Option("--bundle", Required = true)]
public string BundleDirectory { get; set; }
[Option("--attestor-url")]
public string? AttestorUrl { get; set; }
[Option("--require-rekor")]
public bool RequireRekor { get; set; }
}
```
### RB-008: Analyzer Host Integration
Update SecretsAnalyzerHost to verify bundles on startup:
```csharp
public sealed class SecretsAnalyzerHost : IHostedService
{
public async Task StartAsync(CancellationToken ct)
{
var bundlePath = _options.RulesetPath;
// Verify bundle integrity
var verification = await _verifier.VerifyAsync(bundlePath, new VerificationOptions
{
RequireRekorProof = _options.RequireSignatureVerification
}, ct);
if (!verification.IsValid)
{
_logger.LogError("Bundle verification failed: {Errors}",
string.Join(", ", verification.ValidationErrors ?? []));
if (_options.FailOnInvalidBundle)
throw new InvalidOperationException("Secret detection bundle verification failed");
return; // Analyzer disabled
}
_logger.LogInformation(
"SecretsAnalyzerHost: Loaded bundle {Version} signed by {KeyId} with {Count} rules",
verification.BundleVersion,
verification.SignerKeyId,
_ruleset.Rules.Length);
}
}
```
### RB-009: Default Rule Definitions
Create initial rule set in `offline/rules/secrets/sources/`:
| File | Rule ID | Description |
|------|---------|-------------|
| `aws-access-key.json` | `stellaops.secrets.aws-access-key` | AWS Access Key ID |
| `aws-secret-key.json` | `stellaops.secrets.aws-secret-key` | AWS Secret Access Key |
| `github-pat.json` | `stellaops.secrets.github-pat` | GitHub Personal Access Token |
| `github-app.json` | `stellaops.secrets.github-app` | GitHub App Token (ghs_, ghp_) |
| `gitlab-pat.json` | `stellaops.secrets.gitlab-pat` | GitLab Personal Access Token |
| `azure-storage.json` | `stellaops.secrets.azure-storage-key` | Azure Storage Account Key |
| `gcp-service-account.json` | `stellaops.secrets.gcp-service-account` | GCP Service Account JSON |
| `private-key-rsa.json` | `stellaops.secrets.private-key-rsa` | RSA Private Key (PEM) |
| `private-key-ec.json` | `stellaops.secrets.private-key-ec` | EC Private Key (PEM) |
| `private-key-openssh.json` | `stellaops.secrets.private-key-openssh` | OpenSSH Private Key |
| `jwt.json` | `stellaops.secrets.jwt` | JSON Web Token |
| `slack-token.json` | `stellaops.secrets.slack-token` | Slack API Token |
| `stripe-key.json` | `stellaops.secrets.stripe-key` | Stripe API Key |
| `sendgrid-key.json` | `stellaops.secrets.sendgrid-key` | SendGrid API Key |
| `generic-api-key.json` | `stellaops.secrets.generic-api-key` | Generic high-entropy API key |
### RB-010: Initial Bundle Build
Create the signed 2026.01 bundle:
```
offline/rules/secrets/2026.01/
├── secrets.ruleset.manifest.json
├── secrets.ruleset.rules.jsonl
└── secrets.ruleset.dsse.json
```
Build script in `.gitea/scripts/build/build-secrets-bundle.sh`:
```bash
#!/bin/bash
set -euo pipefail
VERSION="${1:-2026.01}"
OUTPUT_DIR="offline/rules/secrets/${VERSION}"
stella secrets bundle create \
--output "${OUTPUT_DIR}" \
--bundle-id secrets.ruleset \
--version "${VERSION}" \
--rules offline/rules/secrets/sources/*.json \
--sign \
--key-id stellaops-secrets-signer
```
### RB-011: Tests
Unit tests:
- `BundleBuilderTests.cs` - Bundle creation and validation
- `BundleVerifierTests.cs` - Signature verification
- `RuleValidatorTests.cs` - Rule schema validation
Integration tests:
- `BundleRoundtripTests.cs` - Create, sign, verify cycle
- `CliCommandTests.cs` - CLI command execution
### RB-012: Documentation
Update `docs/modules/scanner/operations/secret-leak-detection.md`:
- Add bundle creation workflow
- Document verification process
- Add troubleshooting for signature failures
Create `docs/modules/scanner/operations/secrets-bundle-rotation.md`:
- Bundle versioning strategy
- Rotation procedures
- Rollback instructions
## Bundle Directory Structure
```
offline/rules/secrets/
├── sources/ # Source rule definitions (not distributed)
│ ├── aws-access-key.json
│ ├── aws-secret-key.json
│ └── ...
├── 2026.01/ # Signed release bundle
│ ├── secrets.ruleset.manifest.json
│ ├── secrets.ruleset.rules.jsonl
│ └── secrets.ruleset.dsse.json
└── latest -> 2026.01 # Symlink to latest stable
```
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| NDJSON for rules | Streaming parse, git-friendly, easy validation |
| Sign manifest (not rules) | Manifest includes rules digest; smaller signature payload |
| Optional Rekor verification | Supports air-gapped deployments |
| Symlink for latest | Simple upgrade path, atomic switch |
| Source rules in repo | Version control, review process for rule changes |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Part of secret leak detection implementation |
| 2026-01-04 | RB-001 to RB-010 | Implemented bundle infrastructure, signing, verification, CLI commands |
| 2026-01-04 | RB-011 | Fixed and validated 37 unit tests for bundle system |
| 2026-01-04 | RB-012 | Updated secret-leak-detection.md, created secrets-bundle-rotation.md |
| 2026-01-04 | Sprint completed | All 12 tasks DONE |

543
docs/implplan/SPRINT_20260104_004_POLICY_secret_dsl_integration.md Normal file
View File

@@ -0,0 +1,543 @@
# Sprint 20260104_004_POLICY - Secret Leak Detection Policy DSL Integration
## Topic & Scope
Extend the Policy Engine and stella-dsl with `secret.*` predicates to enable policy-driven decisions on secret leak findings. This sprint delivers the policy integration layer.
**Key deliverables:**
1. **Policy Predicates**: `secret.hasFinding()`, `secret.bundle.version()`, `secret.match.count()`, `secret.mask.applied`
2. **Evidence Binding**: Connect SecretLeakEvidence to policy evaluation context
3. **Example Policies**: Sample policies for common secret blocking/warning scenarios
4. **Policy Validation**: Schema updates for secret-related predicates
**Working directory:** `src/Policy/`, `src/PolicyDsl/`
## Dependencies & Concurrency
- **Depends on**: Sprint 20260104_002 (Core Analyzer - SecretLeakEvidence model)
- **Parallel with**: Sprint 20260104_003 (Rule Bundles)
- **Required by**: Sprint 20260104_005 (Offline Kit Integration)
## Documentation Prerequisites
- docs/modules/policy/architecture.md
- docs/policy/dsl.md
- docs/modules/policy/secret-leak-detection-readiness.md
- CLAUDE.md Section 8 (Code Quality)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PSD-001 | TODO | None | Policy Guild | Define ISecretEvidenceProvider interface |
| 2 | PSD-002 | TODO | PSD-001 | Policy Guild | Implement SecretEvidenceContext binding |
| 3 | PSD-003 | TODO | None | Policy Guild | Add secret.hasFinding() predicate |
| 4 | PSD-004 | TODO | None | Policy Guild | Add secret.bundle.version() predicate |
| 5 | PSD-005 | TODO | None | Policy Guild | Add secret.match.count() predicate |
| 6 | PSD-006 | TODO | None | Policy Guild | Add secret.mask.applied predicate |
| 7 | PSD-007 | TODO | None | Policy Guild | Add secret.path.allowlist() predicate |
| 8 | PSD-008 | TODO | PSD-003-007 | Policy Guild | Register predicates in PolicyDslRegistry |
| 9 | PSD-009 | TODO | PSD-008 | Policy Guild | Update DSL schema validation |
| 10 | PSD-010 | TODO | PSD-008 | Policy Guild | Create example policy templates |
| 11 | PSD-011 | TODO | All | Policy Guild | Add unit and integration tests |
| 12 | PSD-012 | TODO | All | Docs Guild | Update policy/dsl.md documentation |
## Task Details
### PSD-001: Secret Evidence Provider Interface
Define the interface for secret evidence access:
```csharp
/// <summary>
/// Provides secret leak evidence for policy evaluation.
/// </summary>
public interface ISecretEvidenceProvider
{
/// <summary>
/// Gets all secret findings for the current evaluation context.
/// </summary>
IReadOnlyList<SecretLeakEvidence> GetFindings();
/// <summary>
/// Gets the active rule bundle metadata.
/// </summary>
SecretBundleMetadata? GetBundleMetadata();
/// <summary>
/// Checks if masking was successfully applied to all findings.
/// </summary>
bool IsMaskingApplied();
}
public sealed record SecretBundleMetadata(
string BundleId,
string Version,
DateTimeOffset SignedAt,
int RuleCount);
```
Location: `src/Policy/__Libraries/StellaOps.Policy/Secrets/`
### PSD-002: Evidence Context Binding
Bind secret evidence to the policy evaluation context:
```csharp
public sealed class SecretEvidenceContext
{
private readonly ISecretEvidenceProvider _provider;
public SecretEvidenceContext(ISecretEvidenceProvider provider)
{
_provider = provider;
}
public IReadOnlyList<SecretLeakEvidence> Findings => _provider.GetFindings();
public SecretBundleMetadata? Bundle => _provider.GetBundleMetadata();
public bool MaskingApplied => _provider.IsMaskingApplied();
}
// Integration with PolicyEvaluationContext
public sealed class PolicyEvaluationContext
{
// ... existing properties ...
public SecretEvidenceContext? Secrets { get; init; }
}
```
### PSD-003: hasFinding Predicate
Implement the `secret.hasFinding()` predicate:
```csharp
/// <summary>
/// Returns true if any secret finding matches the filter criteria.
/// </summary>
/// <example>
/// secret.hasFinding() // Any finding
/// secret.hasFinding(severity: "high") // High severity
/// secret.hasFinding(ruleId: "stellaops.secrets.aws-*") // AWS rules (glob)
/// secret.hasFinding(severity: "high", confidence: "high") // Both filters
/// </example>
[DslPredicate("secret.hasFinding")]
public sealed class SecretHasFindingPredicate : IPolicyPredicate
{
public bool Evaluate(
PolicyEvaluationContext context,
IReadOnlyDictionary<string, object?>? args)
{
var findings = context.Secrets?.Findings ?? [];
if (findings.Count == 0) return false;
var ruleIdPattern = args?.GetValueOrDefault("ruleId") as string;
var severity = args?.GetValueOrDefault("severity") as string;
var confidence = args?.GetValueOrDefault("confidence") as string;
return findings.Any(f =>
MatchesRuleId(f.RuleId, ruleIdPattern) &&
MatchesSeverity(f.Severity, severity) &&
MatchesConfidence(f.Confidence, confidence));
}
private static bool MatchesRuleId(string ruleId, string? pattern)
{
if (string.IsNullOrEmpty(pattern)) return true;
if (pattern.EndsWith("*"))
return ruleId.StartsWith(pattern[..^1], StringComparison.Ordinal);
return ruleId.Equals(pattern, StringComparison.Ordinal);
}
}
```
Location: `src/Policy/__Libraries/StellaOps.Policy/Predicates/Secret/`
### PSD-004: bundle.version Predicate
Implement the `secret.bundle.version()` predicate:
```csharp
/// <summary>
/// Returns true if the active bundle meets or exceeds the required version.
/// </summary>
/// <example>
/// secret.bundle.version("2026.01") // At least 2026.01
/// </example>
[DslPredicate("secret.bundle.version")]
public sealed class SecretBundleVersionPredicate : IPolicyPredicate
{
public bool Evaluate(
PolicyEvaluationContext context,
IReadOnlyDictionary<string, object?>? args)
{
var requiredVersion = args?.GetValueOrDefault("requiredVersion") as string
?? throw new PolicyEvaluationException("secret.bundle.version requires requiredVersion argument");
var bundle = context.Secrets?.Bundle;
if (bundle == null) return false;
return CompareVersions(bundle.Version, requiredVersion) >= 0;
}
private static int CompareVersions(string current, string required)
{
// Simple calendar version comparison (YYYY.MM format)
return string.Compare(current, required, StringComparison.Ordinal);
}
}
```
### PSD-005: match.count Predicate
Implement the `secret.match.count()` predicate:
```csharp
/// <summary>
/// Returns the count of findings matching the filter criteria.
/// </summary>
/// <example>
/// secret.match.count() > 0 // Any findings
/// secret.match.count(ruleId: "stellaops.secrets.aws-*") >= 5 // Many AWS findings
/// </example>
[DslPredicate("secret.match.count")]
public sealed class SecretMatchCountPredicate : IPolicyPredicate<int>
{
public int Evaluate(
PolicyEvaluationContext context,
IReadOnlyDictionary<string, object?>? args)
{
var findings = context.Secrets?.Findings ?? [];
var ruleIdPattern = args?.GetValueOrDefault("ruleId") as string;
return findings.Count(f => MatchesRuleId(f.RuleId, ruleIdPattern));
}
}
```
### PSD-006: mask.applied Predicate
Implement the `secret.mask.applied` predicate:
```csharp
/// <summary>
/// Returns true if masking was successfully applied to all findings.
/// </summary>
/// <example>
/// secret.mask.applied // Verify masking succeeded
/// </example>
[DslPredicate("secret.mask.applied")]
public sealed class SecretMaskAppliedPredicate : IPolicyPredicate
{
public bool Evaluate(
PolicyEvaluationContext context,
IReadOnlyDictionary<string, object?>? args)
{
return context.Secrets?.MaskingApplied ?? true; // Default true if no findings
}
}
```
### PSD-007: path.allowlist Predicate
Implement the `secret.path.allowlist()` predicate for false positive suppression:
```csharp
/// <summary>
/// Returns true if all findings are in paths matching the allowlist patterns.
/// </summary>
/// <example>
/// secret.path.allowlist(["**/test/**", "**/fixtures/**"]) // Ignore test files
/// </example>
[DslPredicate("secret.path.allowlist")]
public sealed class SecretPathAllowlistPredicate : IPolicyPredicate
{
public bool Evaluate(
PolicyEvaluationContext context,
IReadOnlyDictionary<string, object?>? args)
{
var patterns = args?.GetValueOrDefault("patterns") as IReadOnlyList<string>;
if (patterns == null || patterns.Count == 0)
throw new PolicyEvaluationException("secret.path.allowlist requires patterns argument");
var findings = context.Secrets?.Findings ?? [];
if (findings.Count == 0) return true;
return findings.All(f => patterns.Any(p => GlobMatcher.IsMatch(f.FilePath, p)));
}
}
```
### PSD-008: Predicate Registration
Register predicates in the DSL registry:
```csharp
public static class SecretPredicateRegistration
{
public static void RegisterSecretPredicates(this PolicyDslRegistry registry)
{
registry.RegisterPredicate<SecretHasFindingPredicate>("secret.hasFinding");
registry.RegisterPredicate<SecretBundleVersionPredicate>("secret.bundle.version");
registry.RegisterPredicate<SecretMatchCountPredicate>("secret.match.count");
registry.RegisterPredicate<SecretMaskAppliedPredicate>("secret.mask.applied");
registry.RegisterPredicate<SecretPathAllowlistPredicate>("secret.path.allowlist");
}
}
```
### PSD-009: DSL Schema Validation
Update the DSL schema to include secret predicates:
```json
{
"predicates": {
"secret.hasFinding": {
"description": "Returns true if any secret finding matches the filter",
"arguments": {
"ruleId": { "type": "string", "optional": true, "description": "Rule ID pattern (supports * glob)" },
"severity": { "type": "string", "optional": true, "enum": ["low", "medium", "high", "critical"] },
"confidence": { "type": "string", "optional": true, "enum": ["low", "medium", "high"] }
},
"returns": "boolean"
},
"secret.bundle.version": {
"description": "Returns true if the active bundle meets or exceeds the required version",
"arguments": {
"requiredVersion": { "type": "string", "required": true, "description": "Minimum required version (YYYY.MM format)" }
},
"returns": "boolean"
},
"secret.match.count": {
"description": "Returns the count of findings matching the filter",
"arguments": {
"ruleId": { "type": "string", "optional": true, "description": "Rule ID pattern (supports * glob)" }
},
"returns": "integer"
},
"secret.mask.applied": {
"description": "Returns true if masking was successfully applied to all findings",
"arguments": {},
"returns": "boolean"
},
"secret.path.allowlist": {
"description": "Returns true if all findings are in paths matching the allowlist",
"arguments": {
"patterns": { "type": "array", "items": { "type": "string" }, "required": true }
},
"returns": "boolean"
}
}
}
```
### PSD-010: Example Policy Templates
Create example policies in `docs/modules/policy/examples/`:
**secret-blocker.stella** - Block high-severity secrets:
```dsl
policy "Secret Leak Guard" syntax "stella-dsl@1" {
metadata {
description = "Block high-confidence secret leaks in production scans"
tags = ["secrets", "compliance", "security"]
}
rule block_critical priority 100 {
when secret.hasFinding(severity: "critical")
then escalate to "block";
because "Critical secret leak detected - deployment blocked";
}
rule block_high_confidence priority 90 {
when secret.hasFinding(severity: "high", confidence: "high")
then escalate to "block";
because "High severity secret leak with high confidence detected";
}
rule warn_medium priority 50 {
when secret.hasFinding(severity: "medium")
then warn message "Medium severity secret detected - review required";
}
rule require_current_bundle priority 10 {
when not secret.bundle.version("2026.01")
then warn message "Secret detection bundle is out of date";
}
}
```
**secret-allowlist.stella** - Suppress test file findings:
```dsl
policy "Secret Allowlist" syntax "stella-dsl@1" {
metadata {
description = "Suppress false positives in test fixtures"
tags = ["secrets", "testing"]
}
rule allow_test_fixtures priority 200 {
when secret.path.allowlist([
"**/test/**",
"**/tests/**",
"**/fixtures/**",
"**/__fixtures__/**",
"**/testdata/**"
])
then annotate decision.notes := "Findings in test paths - suppressed";
else continue;
}
rule allow_examples priority 190 {
when secret.path.allowlist([
"**/examples/**",
"**/samples/**",
"**/docs/**"
])
and secret.hasFinding(confidence: "low")
then annotate decision.notes := "Low confidence findings in example paths";
else continue;
}
}
```
**secret-threshold.stella** - Threshold-based blocking:
```dsl
policy "Secret Threshold Guard" syntax "stella-dsl@1" {
metadata {
description = "Block scans exceeding secret finding thresholds"
tags = ["secrets", "thresholds"]
}
rule excessive_secrets priority 80 {
when secret.match.count() > 50
then escalate to "block";
because "Excessive number of secret findings (>50) - likely misconfigured scan";
}
rule many_aws_secrets priority 70 {
when secret.match.count(ruleId: "stellaops.secrets.aws-*") > 10
then escalate to "review";
because "Multiple AWS credentials detected - security review required";
}
}
```
### PSD-011: Tests
Unit tests in `src/Policy/__Tests/StellaOps.Policy.Tests/Predicates/Secret/`:
```
├── SecretHasFindingPredicateTests.cs
│ - Test empty findings returns false
│ - Test matching severity filter
│ - Test matching confidence filter
│ - Test ruleId glob matching
│ - Test combined filters
├── SecretBundleVersionPredicateTests.cs
│ - Test version comparison
│ - Test missing bundle returns false
│ - Test exact version match
├── SecretMatchCountPredicateTests.cs
│ - Test empty findings returns 0
│ - Test count with filter
│ - Test count without filter
├── SecretMaskAppliedPredicateTests.cs
│ - Test masking applied
│ - Test masking not applied
│ - Test default for no findings
├── SecretPathAllowlistPredicateTests.cs
│ - Test glob pattern matching
│ - Test multiple patterns
│ - Test no matching patterns
└── PolicyEvaluationIntegrationTests.cs
- Test full policy evaluation with secrets
- Test policy chaining
- Test decision propagation
```
### PSD-012: Documentation
Update `docs/policy/dsl.md` with new predicates section:
```markdown
## Secret Leak Detection Predicates
The following predicates are available for secret leak detection policy rules:
### secret.hasFinding(ruleId?, severity?, confidence?)
Returns `true` if any secret finding matches the specified filters.
**Arguments:**
- `ruleId` (string, optional): Rule ID pattern with optional `*` glob suffix
- `severity` (string, optional): One of `low`, `medium`, `high`, `critical`
- `confidence` (string, optional): One of `low`, `medium`, `high`
**Example:**
```dsl
when secret.hasFinding(severity: "high", confidence: "high")
```
### secret.bundle.version(requiredVersion)
Returns `true` if the active rule bundle version meets or exceeds the required version.
**Arguments:**
- `requiredVersion` (string, required): Minimum version in `YYYY.MM` format
**Example:**
```dsl
when not secret.bundle.version("2026.01")
then warn message "Bundle out of date";
```
### secret.match.count(ruleId?)
Returns the integer count of findings matching the optional rule ID filter.
**Example:**
```dsl
when secret.match.count() > 50
```
### secret.mask.applied
Returns `true` if payload masking was successfully applied to all findings.
**Example:**
```dsl
when not secret.mask.applied
then escalate to "block";
because "Masking failed - secrets may be exposed";
```
### secret.path.allowlist(patterns)
Returns `true` if all findings are in file paths matching at least one allowlist pattern.
**Arguments:**
- `patterns` (array of strings, required): Glob patterns for allowed paths
**Example:**
```dsl
when secret.path.allowlist(["**/test/**", "**/fixtures/**"])
```
```
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| Glob patterns for ruleId | Simple, familiar syntax for rule filtering |
| YYYY.MM version format | Matches bundle versioning convention |
| Default true for mask.applied with no findings | Conservative - don't fail on clean scans |
| Path allowlist as AND | All findings must be in allowed paths to pass |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Part of secret leak detection implementation |

591
docs/implplan/SPRINT_20260104_005_AIRGAP_secret_offline_kit.md Normal file
View File

@@ -0,0 +1,591 @@
# Sprint 20260104_005_AIRGAP - Secret Detection Offline Kit Integration
## Topic & Scope
Integrate secret detection rule bundles with the Offline Kit infrastructure for air-gapped deployments. This sprint ensures complete offline parity for secret leak detection.
**Key deliverables:**
1. **Bundle Distribution**: Include signed bundles in Offline Kit exports
2. **Import Workflow**: Bundle import and verification scripts
3. **Attestor Mirror**: Local verification support without internet
4. **CI/CD Integration**: Automated bundle inclusion in releases
5. **Upgrade Path**: Bundle rotation procedures for offline environments
**Working directory:** `src/AirGap/`, `devops/offline/`, `offline/rules/secrets/`
## Dependencies & Concurrency
- **Depends on**: Sprint 20260104_002 (Core Analyzer), Sprint 20260104_003 (Rule Bundles)
- **Parallel with**: Sprint 20260104_004 (Policy DSL)
- **Blocks**: Production deployment of secret leak detection
## Documentation Prerequisites
- docs/24_OFFLINE_KIT.md
- docs/modules/airgap/airgap-mode.md
- docs/modules/scanner/operations/secret-leak-detection.md
- CLAUDE.md Section 8 (Determinism)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | OKS-001 | TODO | None | AirGap Guild | Update Offline Kit manifest schema for rules |
| 2 | OKS-002 | TODO | OKS-001 | AirGap Guild | Add secrets bundle to BundleBuilder |
| 3 | OKS-003 | TODO | OKS-002 | AirGap Guild | Create bundle verification in Importer |
| 4 | OKS-004 | TODO | None | AirGap Guild | Add Attestor mirror support for bundle verification |
| 5 | OKS-005 | TODO | OKS-003 | AirGap Guild | Create bundle installation script |
| 6 | OKS-006 | TODO | OKS-005 | AirGap Guild | Add bundle rotation/upgrade workflow |
| 7 | OKS-007 | TODO | None | CI/CD Guild | Add bundle to release workflow |
| 8 | OKS-008 | TODO | All | AirGap Guild | Add integration tests for offline flow |
| 9 | OKS-009 | TODO | All | Docs Guild | Update offline kit documentation |
| 10 | OKS-010 | TODO | All | DevOps Guild | Update Helm charts for bundle mounting |
## Task Details
### OKS-001: Manifest Schema Update
Update the Offline Kit manifest to include rule bundles:
```json
{
"version": "2.1",
"created": "2026-01-04T00:00:00Z",
"components": {
"advisory": { ... },
"policy": { ... },
"vex": { ... },
"rules": {
"secrets": {
"bundleId": "secrets.ruleset",
"version": "2026.01",
"path": "rules/secrets/2026.01",
"files": [
{
"name": "secrets.ruleset.manifest.json",
"sha256": "abc123..."
},
{
"name": "secrets.ruleset.rules.jsonl",
"sha256": "def456..."
},
{
"name": "secrets.ruleset.dsse.json",
"sha256": "ghi789..."
}
],
"signature": {
"keyId": "stellaops-secrets-signer",
"verifiedAt": "2026-01-04T00:00:00Z"
}
}
}
}
}
```
Location: `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Schemas/`
### OKS-002: Bundle Builder Extension
Extend BundleBuilder to include secrets rule bundles:
```csharp
public sealed class SnapshotBundleBuilder
{
// Add secrets bundle extraction
public async Task<BundleResult> BuildAsync(
BundleBuildContext context,
CancellationToken ct = default)
{
// ... existing extractors ...
// Add secrets rules extractor
await ExtractSecretsRulesAsync(context, ct);
return result;
}
private async Task ExtractSecretsRulesAsync(
BundleBuildContext context,
CancellationToken ct)
{
var sourcePath = _options.SecretsRulesBundlePath;
if (string.IsNullOrEmpty(sourcePath) || !Directory.Exists(sourcePath))
{
_logger.LogWarning("Secrets rules bundle not found at {Path}", sourcePath);
return;
}
var targetPath = Path.Combine(context.OutputPath, "rules", "secrets");
Directory.CreateDirectory(targetPath);
// Copy bundle files
foreach (var file in Directory.GetFiles(sourcePath, "secrets.ruleset.*"))
{
var targetFile = Path.Combine(targetPath, Path.GetFileName(file));
await CopyWithIntegrityAsync(file, targetFile, ct);
}
// Add to manifest
context.Manifest.Rules["secrets"] = new RuleBundleManifest
{
BundleId = "secrets.ruleset",
Version = await ReadBundleVersionAsync(sourcePath, ct),
Path = "rules/secrets"
};
}
}
```
Location: `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Services/`
### OKS-003: Importer Verification
Add bundle verification to the Offline Kit importer:
```csharp
public sealed class OfflineKitImporter
{
public async Task<ImportResult> ImportAsync(
string kitPath,
ImportOptions options,
CancellationToken ct = default)
{
// ... existing imports ...
// Import and verify secrets rules
if (manifest.Rules.TryGetValue("secrets", out var secretsBundle))
{
await ImportSecretsRulesAsync(kitPath, secretsBundle, options, ct);
}
return result;
}
private async Task ImportSecretsRulesAsync(
string kitPath,
RuleBundleManifest bundle,
ImportOptions options,
CancellationToken ct)
{
var sourcePath = Path.Combine(kitPath, bundle.Path);
var targetPath = _options.SecretsRulesInstallPath;
// Verify bundle integrity
var verifier = _serviceProvider.GetRequiredService<IBundleVerifier>();
var verification = await verifier.VerifyAsync(sourcePath, new VerificationOptions
{
AttestorUrl = options.AttestorMirrorUrl,
RequireRekorProof = options.RequireRekorProof
}, ct);
if (!verification.IsValid)
{
throw new ImportException($"Secrets bundle verification failed: {string.Join(", ", verification.ValidationErrors ?? [])}");
}
// Install bundle
await InstallBundleAsync(sourcePath, targetPath, ct);
_logger.LogInformation(
"Installed secrets bundle {Version} signed by {KeyId}",
verification.BundleVersion,
verification.SignerKeyId);
}
}
```
### OKS-004: Attestor Mirror Support
Configure Attestor mirror for offline bundle verification:
```csharp
public sealed class OfflineAttestorClient : IAttestorClient
{
private readonly string _mirrorPath;
public OfflineAttestorClient(string mirrorPath)
{
_mirrorPath = mirrorPath;
}
public async Task<VerificationResult> VerifyDsseAsync(
DsseEnvelope envelope,
VerifyOptions options,
CancellationToken ct = default)
{
// Load mirrored certificate chain
var chainPath = Path.Combine(_mirrorPath, "certs", options.KeyId + ".pem");
if (!File.Exists(chainPath))
{
return VerificationResult.Failed($"Certificate not found in mirror: {options.KeyId}");
}
var chain = await LoadCertificateChainAsync(chainPath, ct);
// Verify signature locally
var result = await _dsseVerifier.VerifyAsync(envelope, chain, ct);
// Optionally verify against mirrored Rekor entries
if (options.RequireRekorProof)
{
var rekorPath = Path.Combine(_mirrorPath, "rekor", envelope.PayloadDigest + ".json");
if (!File.Exists(rekorPath))
{
return VerificationResult.Failed("Rekor entry not found in mirror");
}
result = result.WithRekorEntry(await LoadRekorEntryAsync(rekorPath, ct));
}
return result;
}
}
```
### OKS-005: Installation Script
Create bundle installation script for operators:
**`devops/offline/scripts/install-secrets-bundle.sh`:**
```bash
#!/bin/bash
set -euo pipefail
BUNDLE_PATH="${1:-/mnt/offline-kit/rules/secrets}"
INSTALL_PATH="${2:-/opt/stellaops/plugins/scanner/analyzers/secrets}"
ATTESTOR_MIRROR="${3:-/mnt/offline-kit/attestor-mirror}"
echo "Installing secrets bundle from ${BUNDLE_PATH}"
# Verify bundle before installation
export STELLA_ATTESTOR_URL="file://${ATTESTOR_MIRROR}"
if ! stella secrets bundle verify --bundle "${BUNDLE_PATH}" --require-rekor; then
echo "ERROR: Bundle verification failed" >&2
exit 1
fi
# Create installation directory
mkdir -p "${INSTALL_PATH}"
# Install bundle files
cp -v "${BUNDLE_PATH}"/secrets.ruleset.* "${INSTALL_PATH}/"
# Set permissions
chmod 640 "${INSTALL_PATH}"/secrets.ruleset.*
chown stellaops:stellaops "${INSTALL_PATH}"/secrets.ruleset.*
# Verify installation
INSTALLED_VERSION=$(jq -r '.version' "${INSTALL_PATH}/secrets.ruleset.manifest.json")
echo "Successfully installed secrets bundle version ${INSTALLED_VERSION}"
echo ""
echo "To activate, restart Scanner Worker:"
echo " systemctl restart stellaops-scanner-worker"
echo ""
echo "Or with Kubernetes:"
echo " kubectl rollout restart deployment/scanner-worker"
```
### OKS-006: Bundle Rotation Workflow
Document and implement bundle upgrade procedure:
**Upgrade Workflow:**
1. **Pre-upgrade Verification**
```bash
# Verify new bundle
stella secrets bundle verify --bundle /path/to/new-bundle
# Compare with current
CURRENT=$(jq -r '.version' /opt/stellaops/.../secrets.ruleset.manifest.json)
NEW=$(jq -r '.version' /path/to/new-bundle/secrets.ruleset.manifest.json)
echo "Upgrading from ${CURRENT} to ${NEW}"
```
2. **Backup Current Bundle**
```bash
BACKUP_DIR="/opt/stellaops/backups/secrets-bundles/$(date +%Y%m%d)"
mkdir -p "${BACKUP_DIR}"
cp -a /opt/stellaops/plugins/scanner/analyzers/secrets/* "${BACKUP_DIR}/"
```
3. **Install New Bundle**
```bash
./install-secrets-bundle.sh /path/to/new-bundle
```
4. **Rolling Restart**
```bash
# Kubernetes
kubectl rollout restart deployment/scanner-worker --namespace stellaops
# Systemd
systemctl restart stellaops-scanner-worker
```
5. **Verify Upgrade**
```bash
# Check logs for new version
kubectl logs -l app=scanner-worker --tail=100 | grep "SecretsAnalyzerHost"
```
**Rollback Procedure:**
```bash
# Restore backup
cp -a "${BACKUP_DIR}"/* /opt/stellaops/plugins/scanner/analyzers/secrets/
# Restart workers
kubectl rollout restart deployment/scanner-worker
```
### OKS-007: Release Workflow Integration
Add secrets bundle to CI/CD release pipeline:
**`.gitea/workflows/release-offline-kit.yml`:**
```yaml
jobs:
build-offline-kit:
steps:
- name: Build secrets bundle
run: |
stella secrets bundle create \
--output ./offline-kit/rules/secrets/${VERSION} \
--bundle-id secrets.ruleset \
--version ${VERSION} \
--rules ./offline/rules/secrets/sources/*.json \
--sign \
--key-id ${SECRETS_SIGNER_KEY_ID}
- name: Include in offline kit
run: |
# Bundle is automatically included via BundleBuilder
- name: Verify bundle in kit
run: |
stella secrets bundle verify \
--bundle ./offline-kit/rules/secrets/${VERSION}
```
**Add to `.gitea/scripts/build/build-offline-kit.sh`:**
```bash
# Build and sign secrets bundle
echo "Building secrets rule bundle..."
stella secrets bundle create \
--output "${OUTPUT_DIR}/rules/secrets/${BUNDLE_VERSION}" \
--bundle-id secrets.ruleset \
--version "${BUNDLE_VERSION}" \
--rules offline/rules/secrets/sources/*.json \
--sign \
--key-id "${SECRETS_SIGNER_KEY_ID}"
```
### OKS-008: Integration Tests
Create integration tests for offline flow:
```csharp
[Trait("Category", "Integration")]
public class OfflineSecretsIntegrationTests : IClassFixture<OfflineKitFixture>
{
[Fact]
public async Task OfflineKit_IncludesSecretsBundleWithValidSignature()
{
// Arrange
var kit = await _fixture.BuildOfflineKitAsync();
// Act
var bundlePath = Path.Combine(kit.Path, "rules", "secrets");
var verifier = new BundleVerifier(_attestorMirror);
var result = await verifier.VerifyAsync(bundlePath, new VerificationOptions());
// Assert
result.IsValid.Should().BeTrue();
result.BundleVersion.Should().NotBeEmpty();
}
[Fact]
public async Task Importer_InstallsAndVerifiesBundle()
{
// Arrange
var kit = await _fixture.BuildOfflineKitAsync();
var importer = new OfflineKitImporter(_options);
// Act
var result = await importer.ImportAsync(kit.Path, new ImportOptions
{
AttestorMirrorUrl = _attestorMirrorUrl
});
// Assert
result.Success.Should().BeTrue();
Directory.Exists(_installPath).Should().BeTrue();
File.Exists(Path.Combine(_installPath, "secrets.ruleset.manifest.json")).Should().BeTrue();
}
[Fact]
public async Task Scanner_LoadsBundleFromOfflineInstallation()
{
// Arrange
await ImportOfflineKitAsync();
// Act
var host = new SecretsAnalyzerHost(_options, _logger);
await host.StartAsync(CancellationToken.None);
// Assert
host.IsEnabled.Should().BeTrue();
host.BundleVersion.Should().Be(_expectedVersion);
}
}
```
### OKS-009: Documentation Updates
Update `docs/24_OFFLINE_KIT.md`:
```markdown
## Secret Detection Rules
The Offline Kit includes DSSE-signed rule bundles for secret leak detection.
### Bundle Contents
```
offline-kit/
├── rules/
│ └── secrets/
│ └── 2026.01/
│ ├── secrets.ruleset.manifest.json # Rule metadata
│ ├── secrets.ruleset.rules.jsonl # Rule definitions
│ └── secrets.ruleset.dsse.json # DSSE signature
└── attestor-mirror/
├── certs/
│ └── stellaops-secrets-signer.pem # Signing certificate
└── rekor/
└── <digest>.json # Transparency log entry
```
### Installation
1. **Verify Bundle**
```bash
export STELLA_ATTESTOR_URL="file:///mnt/offline-kit/attestor-mirror"
stella secrets bundle verify --bundle /mnt/offline-kit/rules/secrets/2026.01
```
2. **Install Bundle**
```bash
./devops/offline/scripts/install-secrets-bundle.sh \
/mnt/offline-kit/rules/secrets/2026.01 \
/opt/stellaops/plugins/scanner/analyzers/secrets
```
3. **Enable Feature**
```yaml
scanner:
features:
experimental:
secret-leak-detection: true
```
4. **Restart Workers**
```bash
kubectl rollout restart deployment/scanner-worker
```
### Verification
Check that the bundle is loaded:
```bash
kubectl logs -l app=scanner-worker --tail=100 | grep SecretsAnalyzerHost
# Expected: SecretsAnalyzerHost: Loaded bundle 2026.01 signed by stellaops-secrets-signer with N rules
```
### Bundle Rotation
See [Secret Bundle Rotation](./modules/scanner/operations/secrets-bundle-rotation.md) for upgrade procedures.
```
### OKS-010: Helm Chart Updates
Update Helm charts for bundle mounting:
**`devops/helm/stellaops/templates/scanner-worker-deployment.yaml`:**
```yaml
spec:
template:
spec:
volumes:
# Existing volumes...
- name: secrets-rules
{{- if .Values.scanner.secretsRules.persistentVolumeClaim }}
persistentVolumeClaim:
claimName: {{ .Values.scanner.secretsRules.persistentVolumeClaim }}
{{- else }}
configMap:
name: {{ include "stellaops.fullname" . }}-secrets-rules
{{- end }}
containers:
- name: scanner-worker
volumeMounts:
# Existing mounts...
- name: secrets-rules
mountPath: /opt/stellaops/plugins/scanner/analyzers/secrets
readOnly: true
```
**`devops/helm/stellaops/values.yaml`:**
```yaml
scanner:
features:
experimental:
secretLeakDetection: false # Enable via override
secretsRules:
# Use PVC for air-gapped installations
persistentVolumeClaim: ""
# Or use ConfigMap for simple deployments
bundleVersion: "2026.01"
```
## Directory Structure
```
offline/rules/secrets/
├── sources/ # Source rule JSON files (not in kit)
│ ├── aws-access-key.json
│ └── ...
├── 2026.01/ # Signed bundle (in kit)
│ ├── secrets.ruleset.manifest.json
│ ├── secrets.ruleset.rules.jsonl
│ └── secrets.ruleset.dsse.json
└── latest -> 2026.01 # Symlink
devops/offline/
├── scripts/
│ ├── install-secrets-bundle.sh # Installation script
│ └── rotate-secrets-bundle.sh # Rotation script
└── templates/
└── secrets-bundle-pvc.yaml # PVC template for air-gap
```
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| Include Attestor mirror in kit | Enables fully offline verification |
| File:// URL for offline Attestor | Simple, no network required |
| ConfigMap fallback | Simpler for non-air-gapped deployments |
| Symlink for latest | Atomic version switching |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Part of secret leak detection implementation |

30
docs/implplan/SPRINT_20260104_001_BE_adaptive_noise_gating.md → docs/implplan/archived/2026-01-04-completed-sprints/SPRINT_20260104_001_BE_adaptive_noise_gating.md
View File

@@ -28,17 +28,17 @@ Implement adaptive noise-gating for vulnerability graphs to reduce alert fatigue
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | NG-001 | TODO | None | Guild | Add ProofStrength enum to StellaOps.Evidence.Core |
| 2 | NG-002 | TODO | NG-001 | Guild | Add ProofStrength field to EvidenceRecord |
| 3 | NG-003 | TODO | None | Guild | Create EdgeSemanticKey and deduplication logic in ReachGraph |
| 4 | NG-004 | TODO | None | Guild | Add StabilityDampingGate to Policy.Engine.Gates |
| 5 | NG-005 | TODO | NG-004 | Guild | Add StabilityDampingOptions with configurable thresholds |
| 6 | NG-006 | TODO | None | Guild | Create DeltaSection enum in VexLens |
| 7 | NG-007 | TODO | NG-006 | Guild | Extend VexDelta with section categorization |
| 8 | NG-008 | TODO | NG-001,NG-003,NG-004,NG-006 | Guild | Create INoiseGate interface and NoiseGateService |
| 9 | NG-009 | TODO | NG-008 | Guild | Add DI registration in VexLensServiceCollectionExtensions |
| 10 | NG-010 | TODO | All | Guild | Add unit tests for all new components |
| 11 | NG-011 | TODO | NG-010 | Guild | Update module AGENTS.md files |
| 1 | NG-001 | DONE | None | Guild | Add ProofStrength enum to StellaOps.Evidence.Core |
| 2 | NG-002 | DONE | NG-001 | Guild | Add ProofStrength field to EvidenceRecord |
| 3 | NG-003 | DONE | None | Guild | Create EdgeSemanticKey and deduplication logic in ReachGraph |
| 4 | NG-004 | DONE | None | Guild | Add StabilityDampingGate to Policy.Engine.Gates |
| 5 | NG-005 | DONE | NG-004 | Guild | Add StabilityDampingOptions with configurable thresholds |
| 6 | NG-006 | DONE | None | Guild | Create DeltaSection enum in VexLens |
| 7 | NG-007 | DONE | NG-006 | Guild | Extend VexDelta with section categorization |
| 8 | NG-008 | DONE | NG-001,NG-003,NG-004,NG-006 | Guild | Create INoiseGate interface and NoiseGateService |
| 9 | NG-009 | DONE | NG-008 | Guild | Add DI registration in VexLensServiceCollectionExtensions |
| 10 | NG-010 | DONE | All | Guild | Add unit tests for all new components |
| 11 | NG-011 | DONE | NG-010 | Guild | Update module AGENTS.md files |
## Task Details
@@ -184,4 +184,12 @@ Update module documentation:
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Based on product advisory review |
| 2026-01-04 | NG-001,NG-002 | Created ProofStrength enum, ProofStrengthExtensions, ProofRecord in StellaOps.Evidence.Models |
| 2026-01-04 | NG-003 | Created EdgeSemanticKey, DeduplicatedEdge, EdgeDeduplicator in StellaOps.ReachGraph.Deduplication |
| 2026-01-04 | NG-004,NG-005 | Created StabilityDampingGate, StabilityDampingOptions in StellaOps.Policy.Engine.Gates |
| 2026-01-04 | NG-006,NG-007 | Created DeltaSection, DeltaEntry, DeltaReport, DeltaReportBuilder in StellaOps.VexLens.Delta |
| 2026-01-04 | NG-008,NG-009 | Created INoiseGate, NoiseGateService, NoiseGateOptions; registered DI in VexLensServiceCollectionExtensions |
| 2026-01-04 | NG-010 | Added StabilityDampingGateTests, NoiseGateServiceTests, DeltaReportBuilderTests |
| 2026-01-04 | NG-011 | Updated VexLens and Policy.Engine AGENTS.md files |
| 2026-01-04 | Sprint complete | All 11 tasks DONE |

223
docs/implplan/archived/SPRINT_20260104_002_FE_noise_gating_delta_ui.md Normal file
View File

@@ -0,0 +1,223 @@
# Sprint 20260104_002_FE - Noise-Gating Delta Report UI
## Topic & Scope
Implement frontend components to display noise-gating delta reports from the VexLens backend. This sprint composes existing Angular components to minimize new code while providing a complete UI for:
1. **Delta Report Display**: Show changes between vulnerability graph snapshots
2. **Section-Based Navigation**: Tabs for New, Resolved, ConfidenceUp/Down, PolicyImpact, Damped sections
3. **Gating Statistics**: Edge deduplication rates and verdict damping metrics
4. **Backend API Endpoints**: Expose DeltaReport via VexLens WebService
**Working directories:**
- `src/Web/StellaOps.Web/src/app/` (frontend)
- `src/VexLens/StellaOps.VexLens.WebService/` (backend API)
## Dependencies & Concurrency
- Builds on completed Sprint 20260104_001_BE (backend NoiseGate implementation)
- Reuses existing components: `DeltaSummaryStripComponent`, `TabsComponent`, `GatingExplainerComponent`
- Tasks NG-FE-001 through NG-FE-003 (backend + models) must complete before NG-FE-004+
## Existing Components to Reuse
| Component | Location | Usage |
|-----------|----------|-------|
| `DeltaSummaryStripComponent` | `features/compare/components/` | Overview stats display |
| `TabsComponent` / `TabPanelDirective` | `shared/components/tabs/` | Section navigation |
| `GatingExplainerComponent` | `features/triage/components/gating-explainer/` | Per-entry explanations |
| `DeltaComputeService` patterns | `features/compare/services/` | Signal-based state management |
| `GatingReason`, `DeltaSummary` | `features/triage/models/gating.model.ts` | Existing delta/gating types |
| `VexStatementStatus` | `core/api/vex-hub.models.ts` | VEX status types |
| `BadgeComponent`, `StatCardComponent` | `shared/components/` | Statistics display |
## Documentation Prerequisites
- CLAUDE.md (Section 8: Code Quality rules)
- src/Web/StellaOps.Web/README.md
- docs/modules/vexlens/architecture.md
## Delivery Tracker
| # | Task ID | Status | Key dependency | Task Definition |
|---|---------|--------|----------------|-----------------|
| 1 | NG-FE-001 | DONE | None | Add delta report API endpoints to VexLens.WebService |
| 2 | NG-FE-002 | DONE | None | Create TypeScript models for noise-gating delta (noise-gating.models.ts) |
| 3 | NG-FE-003 | DONE | NG-FE-002 | Create NoiseGatingApiClient service |
| 4 | NG-FE-004 | DONE | NG-FE-003 | Create NoiseGatingSummaryStripComponent (extends DeltaSummaryStrip) |
| 5 | NG-FE-005 | DONE | NG-FE-003 | Create DeltaEntryCardComponent for individual entries |
| 6 | NG-FE-006 | DONE | NG-FE-004,005 | Create NoiseGatingDeltaReportComponent (container with tabs) |
| 7 | NG-FE-007 | DONE | NG-FE-006 | Create GatingStatisticsCardComponent |
| 8 | NG-FE-008 | DONE | NG-FE-006 | Integrate into vuln-explorer/triage workspace |
| 9 | NG-FE-009 | DONE | All | Update feature module exports and routing |
## Task Details
### NG-FE-001: Backend API Endpoints
Add endpoints to `VexLensEndpointExtensions.cs`:
```csharp
// Delta computation
POST /api/v1/vexlens/deltas/compute
Body: { fromSnapshotId, toSnapshotId, options }
Returns: DeltaReportResponse
// Get gated snapshot
GET /api/v1/vexlens/snapshots/{snapshotId}/gated
Returns: GatedGraphSnapshotResponse
// Get gating statistics
GET /api/v1/vexlens/gating/statistics
Query: tenantId, fromDate, toDate
Returns: GatingStatisticsResponse
```
### NG-FE-002: TypeScript Models
Create `src/app/core/api/noise-gating.models.ts`:
```typescript
// Match backend DeltaSection enum
export type NoiseGatingDeltaSection =
| 'new' | 'resolved' | 'confidence_up' | 'confidence_down'
| 'policy_impact' | 'damped' | 'evidence_changed';
// Match backend DeltaEntry
export interface NoiseGatingDeltaEntry {
section: NoiseGatingDeltaSection;
vulnerabilityId: string;
productKey: string;
fromStatus?: VexStatementStatus;
toStatus?: VexStatementStatus;
fromConfidence?: number;
toConfidence?: number;
justification?: string;
rationaleClass?: string;
summary?: string;
contributingSources?: string[];
createdAt: string;
}
// Match backend DeltaReport
export interface NoiseGatingDeltaReport {
reportId: string;
fromSnapshotDigest: string;
toSnapshotDigest: string;
generatedAt: string;
entries: NoiseGatingDeltaEntry[];
summary: NoiseGatingDeltaSummary;
hasActionableChanges: boolean;
}
// Summary counts
export interface NoiseGatingDeltaSummary {
totalCount: number;
newCount: number;
resolvedCount: number;
confidenceUpCount: number;
confidenceDownCount: number;
policyImpactCount: number;
dampedCount: number;
evidenceChangedCount: number;
}
// Gating statistics
export interface GatingStatistics {
originalEdgeCount: number;
deduplicatedEdgeCount: number;
edgeReductionPercent: number;
totalVerdictCount: number;
surfacedVerdictCount: number;
dampedVerdictCount: number;
duration: string;
}
```
### NG-FE-003: API Client
Create `src/app/core/api/noise-gating.client.ts`:
```typescript
@Injectable({ providedIn: 'root' })
export class NoiseGatingApiClient {
// Follow VexHubApiHttpClient patterns
// Signal-based state management
// Caching with Map<string, Observable>
}
```
### NG-FE-004: Summary Strip Component
Extend `DeltaSummaryStripComponent` pattern for noise-gating sections:
- New (green), Resolved (blue), ConfidenceUp (teal), ConfidenceDown (orange)
- PolicyImpact (red), Damped (gray), EvidenceChanged (purple)
### NG-FE-005: Delta Entry Card
Create `delta-entry-card.component.ts`:
- Display CVE ID, package, status transition
- Confidence change visualization (before -> after with delta %)
- Section-specific styling
- Link to GatingExplainerComponent for details
### NG-FE-006: Container Component
Create `noise-gating-delta-report.component.ts`:
- Uses `TabsComponent` with section tabs (badge counts)
- Uses `NoiseGatingSummaryStripComponent` for overview
- Filterable entry list within each tab
- Follows three-pane pattern from compare feature
### NG-FE-007: Statistics Card
Create `gating-statistics-card.component.ts`:
- Edge reduction percentage visualization
- Verdict surfacing/damping ratios
- Processing duration display
- Follows `StatCardComponent` patterns
### NG-FE-008: Triage Integration
Add to vuln-explorer:
- "Delta Report" tab or drawer
- Trigger from snapshot comparison
- Link from finding detail to delta context
### NG-FE-009: Module Exports
Update feature module:
- Export new components
- Add to routing if needed
- Register API client
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| Compose existing components | ~70% code reuse, consistent UX |
| Signal-based state | Matches existing Angular 17 patterns |
| Section tabs vs flat list | Better UX for categorized changes |
| Lazy-load delta data | Large reports should not block initial render |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Based on backend noise-gating completion |
| 2026-01-04 | NG-FE-001 | Added endpoints to VexLensEndpointExtensions.cs, created NoiseGatingApiModels.cs |
| 2026-01-04 | NG-FE-001 | Created ISnapshotStore, IGatingStatisticsStore with in-memory implementations |
| 2026-01-04 | NG-FE-001 | Updated INoiseGate.DiffAsync to accept DeltaReportOptions |
| 2026-01-04 | NG-FE-001 | Registered storage services in VexLensServiceCollectionExtensions |
| 2026-01-04 | NG-FE-002 | Created noise-gating.models.ts with all API types and helper functions |
| 2026-01-04 | NG-FE-003 | Created noise-gating.client.ts with signal-based state and caching |
| 2026-01-04 | NG-FE-004 | Created NoiseGatingSummaryStripComponent with section badges |
| 2026-01-04 | NG-FE-005 | Created DeltaEntryCardComponent for individual entries |
| 2026-01-04 | NG-FE-006 | Created NoiseGatingDeltaReportComponent container with tabs |
| 2026-01-04 | NG-FE-007 | Created GatingStatisticsCardComponent with progress bars |
| 2026-01-04 | NG-FE-009 | Created index.ts barrel export for noise-gating components |
| 2026-01-04 | NG-FE-008 | Integrated noise-gating into TriageCanvasComponent with Delta tab |
| 2026-01-04 | NG-FE-008 | Added keyboard shortcut 'd' for delta tab |
| 2026-01-04 | NG-FE-008 | Updated triage components index.ts to export noise-gating components |
| 2026-01-04 | Sprint complete | All 9 tasks completed |