Merge remote changes (theirs)

This commit is contained in:
Codex Assistant
2026-01-08 09:01:53 +02:00
4195 changed files with 249446 additions and 83444 deletions

View File

@@ -0,0 +1,195 @@
# Sprint 20260104_001_BE - Adaptive Noise-Gating for Vulnerability Graphs
## Topic & Scope
Implement adaptive noise-gating for vulnerability graphs to reduce alert fatigue and improve triage UX. The feature enables:
1. **Semantic Edge Deduplication**: Collapse redundant edges from multiple sources into single edges with provenance sets
2. **Proof Strength Hierarchy**: Formalize evidence authority ordering (Authority > Binary > Static > Heuristic)
3. **Stability Damping**: Prevent flip-flopping verdicts through hysteresis-based state transitions
4. **Delta Reports**: Surface only meaningful changes with typed sections (New, Resolved, ConfidenceUp, ConfidenceDown, PolicyImpact)
**Working directory:** `src/__Libraries/`, `src/VexLens/`, `src/Policy/`
## Dependencies & Concurrency
- Builds on existing `VexConsensusEngine`, `PolicyGateEvaluator`, and `NoisePriorService`
- No external dependencies; integrates with existing modules
- Tasks can be executed in parallel across modules
## Documentation Prerequisites
- docs/README.md
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
- docs/modules/platform/architecture-overview.md
- CLAUDE.md (especially Section 8: Code Quality & Determinism Rules)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | NG-001 | DONE | None | Guild | Add ProofStrength enum to StellaOps.Evidence.Core |
| 2 | NG-002 | DONE | NG-001 | Guild | Add ProofStrength field to EvidenceRecord |
| 3 | NG-003 | DONE | None | Guild | Create EdgeSemanticKey and deduplication logic in ReachGraph |
| 4 | NG-004 | DONE | None | Guild | Add StabilityDampingGate to Policy.Engine.Gates |
| 5 | NG-005 | DONE | NG-004 | Guild | Add StabilityDampingOptions with configurable thresholds |
| 6 | NG-006 | DONE | None | Guild | Create DeltaSection enum in VexLens |
| 7 | NG-007 | DONE | NG-006 | Guild | Extend VexDelta with section categorization |
| 8 | NG-008 | DONE | NG-001,NG-003,NG-004,NG-006 | Guild | Create INoiseGate interface and NoiseGateService |
| 9 | NG-009 | DONE | NG-008 | Guild | Add DI registration in VexLensServiceCollectionExtensions |
| 10 | NG-010 | DONE | All | Guild | Add unit tests for all new components |
| 11 | NG-011 | DONE | NG-010 | Guild | Update module AGENTS.md files |
## Task Details
### NG-001: ProofStrength Enum
Add `ProofStrength` enum to formalize evidence authority hierarchy:
```csharp
public enum ProofStrength
{
Authoritative = 100, // Vendor VEX, CSAF publisher
BinaryProof = 80, // Patch signature, binary analysis
StaticAnalysis = 60, // Reachability, call graph
Heuristic = 40 // Version matching, advisory correlation
}
```
Location: `src/__Libraries/StellaOps.Evidence/ProofStrength.cs`
### NG-002: EvidenceRecord Extension
Add optional `ProofStrength` field to existing evidence models for backward compatibility.
### NG-003: Edge Semantic Key
Create semantic key for edge deduplication:
```csharp
public readonly record struct EdgeSemanticKey(
string EntryPointId,
string SinkId,
string VulnerabilityId,
string? GateApplied)
{
public string ComputeKey() =>
$"{EntryPointId}|{SinkId}|{VulnerabilityId}|{GateApplied ?? "none"}";
}
```
Location: `src/__Libraries/StellaOps.ReachGraph/Deduplication/`
### NG-004: StabilityDampingGate
Implement hysteresis-based gate that:
- Tracks last verdict state per (artifact, CVE) tuple
- Requires score to persist for N hours OR change by X% before state transition
- Prevents flip-flopping notifications
Location: `src/Policy/StellaOps.Policy.Engine/Gates/StabilityDampingGate.cs`
### NG-005: StabilityDampingOptions
Configuration options:
- `MinDurationBeforeChange`: TimeSpan (default: 4 hours)
- `MinConfidenceDeltaPercent`: double (default: 15%)
- `EnabledStatuses`: List of VexStatus to apply damping
### NG-006: DeltaSection Enum
Categorize delta entries for UX:
```csharp
public enum DeltaSection
{
New, // First-time finding
Resolved, // Status changed to not_affected/fixed
ConfidenceUp, // Confidence increased significantly
ConfidenceDown, // Confidence decreased significantly
PolicyImpact // Gate decision changed
}
```
### NG-007: VexDelta Extension
Extend existing VexDelta with section categorization and aggregate summary.
### NG-008: INoiseGate Interface
Central interface for noise-gating operations:
```csharp
public interface INoiseGate
{
Task<IReadOnlyList<Edge>> DedupeEdgesAsync(
IReadOnlyList<Edge> edges,
CancellationToken ct = default);
Task<Verdict> ResolveNodeAsync(
string nodeId,
IReadOnlyList<Evidence> evidences,
CancellationToken ct = default);
Task<GraphSnapshot> GateAsync(
GraphSnapshot raw,
CancellationToken ct = default);
Task<DeltaReport> DiffAsync(
GraphSnapshot previous,
GraphSnapshot current,
CancellationToken ct = default);
}
```
### NG-009: DI Registration
Register services in `VexLensServiceCollectionExtensions`:
```csharp
services.AddSingleton<INoiseGate, NoiseGateService>();
services.AddOptions<StabilityDampingOptions>()
.Bind(config.GetSection("NoiseGate:StabilityDamping"))
.ValidateDataAnnotations()
.ValidateOnStart();
```
### NG-010: Unit Tests
Required test coverage:
- Edge deduplication with multi-source inputs
- Proof strength ordering in verdict resolution
- Hysteresis behavior (flip-flop prevention)
- Delta section categorization
- Determinism (same inputs = same outputs)
### NG-011: AGENTS.md Updates
Update module documentation:
- `src/VexLens/AGENTS.md`
- `src/Policy/AGENTS.md`
- `src/__Libraries/StellaOps.Evidence/AGENTS.md`
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| Use ProofStrength instead of EvidenceClass | Avoids naming collision with existing EvidenceType enum |
| Integrate with existing VexConsensusEngine | Leverages proven consensus logic rather than creating parallel infrastructure |
| Make damping optional per-status | Production environments can enable for affected/not_affected but skip for under_investigation |
| Store dedup metadata for audit | Provenance tracking required for transparency |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Based on product advisory review |
| 2026-01-04 | NG-001,NG-002 | Created ProofStrength enum, ProofStrengthExtensions, ProofRecord in StellaOps.Evidence.Models |
| 2026-01-04 | NG-003 | Created EdgeSemanticKey, DeduplicatedEdge, EdgeDeduplicator in StellaOps.ReachGraph.Deduplication |
| 2026-01-04 | NG-004,NG-005 | Created StabilityDampingGate, StabilityDampingOptions in StellaOps.Policy.Engine.Gates |
| 2026-01-04 | NG-006,NG-007 | Created DeltaSection, DeltaEntry, DeltaReport, DeltaReportBuilder in StellaOps.VexLens.Delta |
| 2026-01-04 | NG-008,NG-009 | Created INoiseGate, NoiseGateService, NoiseGateOptions; registered DI in VexLensServiceCollectionExtensions |
| 2026-01-04 | NG-010 | Added StabilityDampingGateTests, NoiseGateServiceTests, DeltaReportBuilderTests |
| 2026-01-04 | NG-011 | Updated VexLens and Policy.Engine AGENTS.md files |
| 2026-01-04 | Sprint complete | All 11 tasks DONE |

View File

@@ -0,0 +1,549 @@
# Sprint 20260104_002_SCANNER - Secret Leak Detection Core Analyzer
## Topic & Scope
Implement the core `StellaOps.Scanner.Analyzers.Secrets` plugin that detects accidentally committed secrets in container layers during scans. This is the foundational sprint for secret leak detection capability.
**Key deliverables:**
1. **Secrets Analyzer Plugin**: Core analyzer that executes regex/entropy-based detection rules
2. **Rule Engine**: Rule definition models, matching logic, and deterministic execution
3. **Masking Engine**: Payload masking to ensure secrets never leak in outputs
4. **Evidence Emission**: `secret.leak` evidence type integration with ScanAnalysisStore
5. **Feature Flag**: Experimental toggle for gradual rollout
**Working directory:** `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/`
## Dependencies & Concurrency
- **Depends on**: Surface.Secrets, Surface.Validation, Surface.Env (already implemented)
- **Required by**: Sprint 20260104_003 (Rule Bundle Infrastructure), Sprint 20260104_004 (Policy DSL)
- **Parallel work**: Tasks SLD-001 through SLD-008 can be developed concurrently
- **Integration tasks** (SLD-009+) require prior tasks complete
## Documentation Prerequisites
- docs/README.md
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
- docs/modules/scanner/architecture.md
- docs/modules/scanner/design/surface-secrets.md
- docs/modules/scanner/operations/secret-leak-detection.md (target spec)
- CLAUDE.md (especially Section 8: Code Quality & Determinism Rules)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | SLD-001 | DONE | None | Scanner Guild | Create project structure and csproj |
| 2 | SLD-002 | DONE | None | Scanner Guild | Define SecretRule and SecretRuleset models |
| 3 | SLD-003 | DONE | None | Scanner Guild | Implement ISecretDetector interface and RegexDetector |
| 4 | SLD-004 | DONE | None | Scanner Guild | Implement EntropyDetector for high-entropy string detection |
| 5 | SLD-005 | DONE | None | Scanner Guild | Implement PayloadMasker with configurable masking strategies |
| 6 | SLD-006 | DONE | None | Scanner Guild | Define SecretLeakEvidence record and finding model |
| 7 | SLD-007 | DONE | SLD-002 | Scanner Guild | Implement RulesetLoader with JSON parsing |
| 8 | SLD-008 | DONE | None | Scanner Guild | Add SecretsAnalyzerOptions with feature flag support |
| 9 | SLD-009 | DONE | SLD-003,SLD-004 | Scanner Guild | Implement CompositeSecretDetector combining regex and entropy |
| 10 | SLD-010 | DONE | SLD-006,SLD-009 | Scanner Guild | Implement SecretsAnalyzer (ILanguageAnalyzer) |
| 11 | SLD-011 | DONE | SLD-010 | Scanner Guild | Add SecretsAnalyzerHost for plugin lifecycle |
| 12 | SLD-012 | DONE | SLD-011 | Scanner Guild | Integrate with Scanner Worker pipeline |
| 13 | SLD-013 | DONE | SLD-010 | Scanner Guild | Add DI registration in ServiceCollectionExtensions |
| 14 | SLD-014 | DONE | All | Scanner Guild | Add comprehensive unit tests |
| 15 | SLD-015 | DONE | SLD-014 | Scanner Guild | Add integration tests with test fixtures |
| 16 | SLD-016 | DONE | All | Scanner Guild | Create AGENTS.md for module |
## Task Details
### SLD-001: Project Structure
Create the project skeleton following Scanner conventions:
```
src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/
├── StellaOps.Scanner.Analyzers.Secrets.csproj
├── AGENTS.md
├── AssemblyInfo.cs
├── Detectors/
│ ├── ISecretDetector.cs
│ ├── RegexDetector.cs
│ ├── EntropyDetector.cs
│ └── CompositeSecretDetector.cs
├── Rules/
│ ├── SecretRule.cs
│ ├── SecretRuleset.cs
│ └── RulesetLoader.cs
├── Masking/
│ ├── IPayloadMasker.cs
│ └── PayloadMasker.cs
├── Evidence/
│ ├── SecretLeakEvidence.cs
│ └── SecretFinding.cs
├── SecretsAnalyzer.cs
├── SecretsAnalyzerHost.cs
├── SecretsAnalyzerOptions.cs
└── ServiceCollectionExtensions.cs
```
csproj should reference:
- StellaOps.Scanner.Core
- StellaOps.Scanner.Surface
- StellaOps.Evidence.Core
### SLD-002: Rule Models
Define the rule structure for secret detection:
```csharp
/// <summary>
/// A single secret detection rule.
/// </summary>
public sealed record SecretRule
{
public required string Id { get; init; } // e.g., "stellaops.secrets.aws-access-key"
public required string Version { get; init; } // e.g., "2025.11.0"
public required string Name { get; init; } // Human-readable name
public required string Description { get; init; }
public required SecretRuleType Type { get; init; } // Regex, Entropy, Composite
public required string Pattern { get; init; } // Regex pattern or entropy config
public required SecretSeverity Severity { get; init; }
public required SecretConfidence Confidence { get; init; }
public string? MaskingHint { get; init; } // e.g., "prefix:4,suffix:2"
public ImmutableArray<string> Keywords { get; init; } // Pre-filter keywords
public ImmutableArray<string> FilePatterns { get; init; } // Glob patterns for file filtering
public bool Enabled { get; init; } = true;
}
public enum SecretRuleType { Regex, Entropy, Composite }
public enum SecretSeverity { Low, Medium, High, Critical }
public enum SecretConfidence { Low, Medium, High }
/// <summary>
/// A versioned collection of secret detection rules.
/// </summary>
public sealed record SecretRuleset
{
public required string Id { get; init; } // e.g., "secrets.ruleset"
public required string Version { get; init; } // e.g., "2025.11"
public required DateTimeOffset CreatedAt { get; init; }
public required ImmutableArray<SecretRule> Rules { get; init; }
public string? Sha256Digest { get; init; } // Integrity hash
}
```
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Rules/`
### SLD-003: Regex Detector
Implement regex-based secret detection:
```csharp
public interface ISecretDetector
{
string DetectorId { get; }
ValueTask<IReadOnlyList<SecretMatch>> DetectAsync(
ReadOnlyMemory<byte> content,
string filePath,
SecretRule rule,
CancellationToken ct = default);
}
public sealed record SecretMatch(
SecretRule Rule,
string FilePath,
int LineNumber,
int ColumnStart,
int ColumnEnd,
ReadOnlyMemory<byte> RawMatch, // For masking
double ConfidenceScore);
public sealed class RegexDetector : ISecretDetector
{
public string DetectorId => "regex";
// Implementation notes:
// - Use compiled regex for performance
// - Apply keyword pre-filter before regex matching
// - Respect file pattern filters
// - Track line/column for precise location
// - Never log raw match content
}
```
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Detectors/`
### SLD-004: Entropy Detector
Implement Shannon entropy-based detection for high-entropy strings:
```csharp
public sealed class EntropyDetector : ISecretDetector
{
public string DetectorId => "entropy";
// Implementation notes:
// - Calculate Shannon entropy for candidate strings
// - Default threshold: 4.5 bits per character
// - Minimum length: 16 characters
// - Skip common high-entropy non-secrets (UUIDs, hashes in comments)
// - Apply charset detection (base64, hex, alphanumeric)
}
public static class EntropyCalculator
{
/// <summary>
/// Calculates Shannon entropy in bits per character.
/// </summary>
public static double Calculate(ReadOnlySpan<byte> data)
{
// Use CultureInfo.InvariantCulture for all formatting
// Return 0.0 for empty input
}
}
```
### SLD-005: Payload Masker
Implement secure payload masking:
```csharp
public interface IPayloadMasker
{
/// <summary>
/// Masks a secret payload preserving prefix/suffix for identification.
/// </summary>
/// <param name="payload">The raw secret bytes</param>
/// <param name="hint">Optional masking hint from rule (e.g., "prefix:4,suffix:2")</param>
/// <returns>Masked string (e.g., "AKIA****B7")</returns>
string Mask(ReadOnlySpan<byte> payload, string? hint = null);
}
public sealed class PayloadMasker : IPayloadMasker
{
// Default: preserve first 4 and last 2 characters
// Replace middle with asterisks (max 8 asterisks)
// Minimum output length: 8 characters
// Never expose more than 6 characters total
public const int DefaultPrefixLength = 4;
public const int DefaultSuffixLength = 2;
public const int MaxMaskLength = 8;
public const char MaskChar = '*';
}
```
### SLD-006: Evidence Models
Define the evidence structure for policy integration:
```csharp
/// <summary>
/// Evidence record for a detected secret leak.
/// </summary>
public sealed record SecretLeakEvidence
{
public required string EvidenceType => "secret.leak";
public required string RuleId { get; init; }
public required string RuleVersion { get; init; }
public required SecretSeverity Severity { get; init; }
public required SecretConfidence Confidence { get; init; }
public required string FilePath { get; init; }
public required int LineNumber { get; init; }
public required string Mask { get; init; } // Masked payload
public required string BundleId { get; init; }
public required string BundleVersion { get; init; }
public required DateTimeOffset DetectedAt { get; init; }
public ImmutableDictionary<string, string>? Metadata { get; init; }
}
/// <summary>
/// Aggregated finding for a single secret match.
/// </summary>
public sealed record SecretFinding
{
public required Guid Id { get; init; }
public required SecretLeakEvidence Evidence { get; init; }
public required string ScanId { get; init; }
public required string TenantId { get; init; }
public required string ArtifactDigest { get; init; }
}
```
### SLD-007: Ruleset Loader
Implement deterministic ruleset loading:
```csharp
public interface IRulesetLoader
{
ValueTask<SecretRuleset> LoadAsync(
string rulesetPath,
CancellationToken ct = default);
ValueTask<SecretRuleset> LoadFromJsonlAsync(
Stream rulesStream,
string bundleId,
string bundleVersion,
CancellationToken ct = default);
}
public sealed class RulesetLoader : IRulesetLoader
{
// Implementation notes:
// - Parse secrets.ruleset.rules.jsonl (NDJSON format)
// - Validate rule schema on load
// - Sort rules by ID for deterministic ordering
// - Calculate and verify SHA-256 digest
// - Use CultureInfo.InvariantCulture for all parsing
// - Log bundle version on successful load
}
```
### SLD-008: Analyzer Options
Configuration options with feature flag:
```csharp
public sealed class SecretsAnalyzerOptions
{
/// <summary>
/// Enable secret leak detection (experimental feature).
/// </summary>
public bool Enabled { get; set; } = false;
/// <summary>
/// Path to the ruleset bundle directory.
/// </summary>
public string RulesetPath { get; set; } = "/opt/stellaops/plugins/scanner/analyzers/secrets";
/// <summary>
/// Minimum confidence level to report findings.
/// </summary>
public SecretConfidence MinConfidence { get; set; } = SecretConfidence.Medium;
/// <summary>
/// Maximum findings per scan (circuit breaker).
/// </summary>
public int MaxFindingsPerScan { get; set; } = 1000;
/// <summary>
/// File size limit for scanning (bytes).
/// </summary>
public long MaxFileSizeBytes { get; set; } = 10 * 1024 * 1024; // 10MB
/// <summary>
/// Enable entropy-based detection.
/// </summary>
public bool EnableEntropyDetection { get; set; } = true;
/// <summary>
/// Entropy threshold (bits per character).
/// </summary>
public double EntropyThreshold { get; set; } = 4.5;
}
```
### SLD-009: Composite Detector
Combine multiple detection strategies:
```csharp
public sealed class CompositeSecretDetector : ISecretDetector
{
private readonly IReadOnlyList<ISecretDetector> _detectors;
private readonly ILogger<CompositeSecretDetector> _logger;
public string DetectorId => "composite";
// Implementation notes:
// - Execute detectors in parallel where possible
// - Deduplicate overlapping matches
// - Merge confidence scores for overlapping detections
// - Respect per-rule detector type preference
}
```
### SLD-010: Secrets Analyzer
Main analyzer implementation:
```csharp
public sealed class SecretsAnalyzer : ILayerAnalyzer
{
public string AnalyzerId => "secrets";
public string DisplayName => "Secret Leak Detector";
// Implementation notes:
// - Check feature flag before processing
// - Load ruleset once at startup (cached)
// - Apply file pattern filters efficiently
// - Execute detection on text files only
// - Emit SecretLeakEvidence for each finding
// - Apply masking before any output
// - Track metrics: scanner.secret.finding_total
// - Add tracing span: scanner.secrets.scan
}
```
### SLD-011: Analyzer Host
Lifecycle management for the analyzer:
```csharp
public sealed class SecretsAnalyzerHost : IHostedService
{
// Implementation notes:
// - Load and validate ruleset on startup
// - Log bundle version and rule count
// - Verify DSSE signature if available
// - Graceful shutdown with finding flush
// - Emit startup log: "SecretsAnalyzerHost: Loaded bundle {version} with {count} rules"
}
```
### SLD-012: Worker Integration
Integrate with Scanner Worker pipeline:
```csharp
// In Scanner.Worker processing pipeline:
// 1. Add SecretsAnalyzer to analyzer chain (after language analyzers)
// 2. Gate execution on feature flag
// 3. Store findings in ScanAnalysisStore
// 4. Include in scan completion event
```
### SLD-013: DI Registration
```csharp
public static class ServiceCollectionExtensions
{
public static IServiceCollection AddSecretsAnalyzer(
this IServiceCollection services,
IConfiguration configuration)
{
services.AddOptions<SecretsAnalyzerOptions>()
.Bind(configuration.GetSection("Scanner:Analyzers:Secrets"))
.ValidateDataAnnotations()
.ValidateOnStart();
services.AddSingleton<IPayloadMasker, PayloadMasker>();
services.AddSingleton<IRulesetLoader, RulesetLoader>();
services.AddSingleton<ISecretDetector, RegexDetector>();
services.AddSingleton<ISecretDetector, EntropyDetector>();
services.AddSingleton<ISecretDetector, CompositeSecretDetector>();
services.AddSingleton<SecretsAnalyzer>();
services.AddHostedService<SecretsAnalyzerHost>();
return services;
}
}
```
### SLD-014: Unit Tests
Required test coverage in `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Secrets.Tests/`:
```
├── Detectors/
│ ├── RegexDetectorTests.cs
│ ├── EntropyDetectorTests.cs
│ ├── EntropyCalculatorTests.cs
│ └── CompositeSecretDetectorTests.cs
├── Rules/
│ ├── SecretRuleTests.cs
│ └── RulesetLoaderTests.cs
├── Masking/
│ └── PayloadMaskerTests.cs
├── Evidence/
│ └── SecretLeakEvidenceTests.cs
├── SecretsAnalyzerTests.cs
└── Fixtures/
├── aws-access-key.txt
├── github-token.txt
├── private-key.pem
└── test-ruleset.jsonl
```
Test requirements:
- All tests must be deterministic
- Use `[Trait("Category", "Unit")]` for unit tests
- Test masking never exposes full secrets
- Test entropy calculation with known inputs
- Test regex patterns match expected secrets
### SLD-015: Integration Tests
Integration tests with Scanner Worker:
```
├── SecretsAnalyzerIntegrationTests.cs
│ - Test full scan with secrets embedded
│ - Verify findings in ScanAnalysisStore
│ - Verify masking in output
│ - Test feature flag disables analyzer
├── RulesetLoadingTests.cs
│ - Test loading from file system
│ - Test invalid ruleset handling
│ - Test missing bundle handling
```
### SLD-016: Module AGENTS.md
Create `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/AGENTS.md` with:
- Mission statement
- Scope definition
- Required reading list
- Working agreements
- Security considerations
## Built-in Rule Examples
Initial rules to include in default bundle:
| Rule ID | Pattern Type | Description |
|---------|--------------|-------------|
| `stellaops.secrets.aws-access-key` | Regex | AWS Access Key ID (AKIA...) |
| `stellaops.secrets.aws-secret-key` | Regex + Entropy | AWS Secret Access Key |
| `stellaops.secrets.github-pat` | Regex | GitHub Personal Access Token |
| `stellaops.secrets.github-app` | Regex | GitHub App Token (ghs_, ghp_) |
| `stellaops.secrets.gitlab-pat` | Regex | GitLab Personal Access Token |
| `stellaops.secrets.private-key-rsa` | Regex | RSA Private Key (PEM) |
| `stellaops.secrets.private-key-ec` | Regex | EC Private Key (PEM) |
| `stellaops.secrets.jwt` | Regex + Entropy | JSON Web Token |
| `stellaops.secrets.basic-auth` | Regex | Basic Auth credentials in URLs |
| `stellaops.secrets.generic-api-key` | Entropy | High-entropy API key patterns |
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| Use NDJSON for rule format | Line-based parsing, easy streaming, git-friendly diffs |
| Mask before any persistence | Defense in depth - secrets never stored |
| Feature flag default off | Safe rollout, tenant opt-in required |
| Entropy threshold 4.5 bits | Balance between false positives and detection rate |
| Max 1000 findings per scan | Circuit breaker prevents DoS on noisy images |
| Text files only | Binary secret detection deferred to future sprint |
## Metrics & Observability
| Metric | Type | Labels |
|--------|------|--------|
| `scanner.secret.finding_total` | Counter | tenant, ruleId, severity, confidence |
| `scanner.secret.scan_duration_seconds` | Histogram | tenant |
| `scanner.secret.rules_loaded` | Gauge | bundleVersion |
| `scanner.secret.files_scanned` | Counter | tenant |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Based on gap analysis of secrets scanning support |
<<<<<<<< HEAD:docs-archived/implplan/2026-01-04-completed-sprints/SPRINT_20260104_002_SCANNER_secret_leak_detection_core.md
| 2026-01-07 | All tasks marked DONE | Implementation verified: project structure, detectors (Regex, Entropy, Composite), models (SecretRule, SecretRuleset, SecretLeakEvidence), RulesetLoader, SecretsAnalyzer, SecretsAnalyzerHost, ServiceCollectionExtensions, unit tests all exist |
| 2026-01-07 | Gap analysis | Found missing tests: SecretsAnalyzerTests.cs, SecretsAnalyzerHostTests.cs, Fixtures/, integration tests |
| 2026-01-07 | Tests completed | Added SecretsAnalyzerTests.cs (20 tests), SecretsAnalyzerHostTests.cs (9 tests), SecretsAnalyzerIntegrationTests.cs (11 tests), Fixtures/ with aws-access-key.txt, github-token.txt, private-key.pem, test-ruleset.jsonl |
| 2026-01-07 | Sprint complete | All tasks verified and ready for archive |
========
| 2026-01-04 | SLD-001 to SLD-014, SLD-016 completed | Full implementation: project structure, rule models, RegexDetector, EntropyDetector, PayloadMasker, SecretLeakEvidence, RulesetLoader, SecretsAnalyzerOptions, CompositeSecretDetector, SecretsAnalyzer, SecretsAnalyzerHost, ServiceCollectionExtensions, unit tests (EntropyCalculatorTests, PayloadMaskerTests, RegexDetectorTests, RulesetLoaderTests, SecretRuleTests, SecretRulesetTests), AGENTS.md. All builds verified. |
| 2026-01-04 | SLD-015 completed | Created integration test project with test fixtures (aws-access-key.txt, github-token.txt, private-key.pem, test-ruleset.jsonl) and SecretsAnalyzerIntegrationTests.cs covering full scan detection, feature flags, circuit breaker, masking, evidence fields, and determinism. All builds verified. **Sprint complete.** |
>>>>>>>> 2096cf49a629f8e20e0775958cf82718dab85884:docs-archived/implplan/2026-01-04-secret-detection/SPRINT_20260104_002_SCANNER_secret_leak_detection_core.md