save audit remarks applications progress
This commit is contained in:
StellaOps Bot
@@ -0,0 +1,542 @@
|
||||
# Sprint 20260104_002_SCANNER - Secret Leak Detection Core Analyzer
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement the core `StellaOps.Scanner.Analyzers.Secrets` plugin that detects accidentally committed secrets in container layers during scans. This is the foundational sprint for secret leak detection capability.
|
||||
|
||||
**Key deliverables:**
|
||||
1. **Secrets Analyzer Plugin**: Core analyzer that executes regex/entropy-based detection rules
|
||||
2. **Rule Engine**: Rule definition models, matching logic, and deterministic execution
|
||||
3. **Masking Engine**: Payload masking to ensure secrets never leak in outputs
|
||||
4. **Evidence Emission**: `secret.leak` evidence type integration with ScanAnalysisStore
|
||||
5. **Feature Flag**: Experimental toggle for gradual rollout
|
||||
|
||||
**Working directory:** `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Depends on**: Surface.Secrets, Surface.Validation, Surface.Env (already implemented)
|
||||
- **Required by**: Sprint 20260104_003 (Rule Bundle Infrastructure), Sprint 20260104_004 (Policy DSL)
|
||||
- **Parallel work**: Tasks SLD-001 through SLD-008 can be developed concurrently
|
||||
- **Integration tasks** (SLD-009+) require prior tasks complete
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- docs/README.md
|
||||
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
|
||||
- docs/modules/scanner/architecture.md
|
||||
- docs/modules/scanner/design/surface-secrets.md
|
||||
- docs/modules/scanner/operations/secret-leak-detection.md (target spec)
|
||||
- CLAUDE.md (especially Section 8: Code Quality & Determinism Rules)
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | SLD-001 | DONE | None | Scanner Guild | Create project structure and csproj |
|
||||
| 2 | SLD-002 | DONE | None | Scanner Guild | Define SecretRule and SecretRuleset models |
|
||||
| 3 | SLD-003 | DONE | None | Scanner Guild | Implement ISecretDetector interface and RegexDetector |
|
||||
| 4 | SLD-004 | DONE | None | Scanner Guild | Implement EntropyDetector for high-entropy string detection |
|
||||
| 5 | SLD-005 | DONE | None | Scanner Guild | Implement PayloadMasker with configurable masking strategies |
|
||||
| 6 | SLD-006 | DONE | None | Scanner Guild | Define SecretLeakEvidence record and finding model |
|
||||
| 7 | SLD-007 | DONE | SLD-002 | Scanner Guild | Implement RulesetLoader with JSON parsing |
|
||||
| 8 | SLD-008 | DONE | None | Scanner Guild | Add SecretsAnalyzerOptions with feature flag support |
|
||||
| 9 | SLD-009 | DONE | SLD-003,SLD-004 | Scanner Guild | Implement CompositeSecretDetector combining regex and entropy |
|
||||
| 10 | SLD-010 | DONE | SLD-006,SLD-009 | Scanner Guild | Implement SecretsAnalyzer (ILanguageAnalyzer) |
|
||||
| 11 | SLD-011 | DONE | SLD-010 | Scanner Guild | Add SecretsAnalyzerHost for plugin lifecycle |
|
||||
| 12 | SLD-012 | DONE | SLD-011 | Scanner Guild | Integrate with Scanner Worker pipeline |
|
||||
| 13 | SLD-013 | DONE | SLD-010 | Scanner Guild | Add DI registration in ServiceCollectionExtensions |
|
||||
| 14 | SLD-014 | DONE | All | Scanner Guild | Add comprehensive unit tests |
|
||||
| 15 | SLD-015 | DONE | SLD-014 | Scanner Guild | Add integration tests with test fixtures |
|
||||
| 16 | SLD-016 | DONE | All | Scanner Guild | Create AGENTS.md for module |
|
||||
|
||||
## Task Details
|
||||
|
||||
### SLD-001: Project Structure
|
||||
|
||||
Create the project skeleton following Scanner conventions:
|
||||
|
||||
```
|
||||
src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/
|
||||
├── StellaOps.Scanner.Analyzers.Secrets.csproj
|
||||
├── AGENTS.md
|
||||
├── AssemblyInfo.cs
|
||||
├── Detectors/
|
||||
│ ├── ISecretDetector.cs
|
||||
│ ├── RegexDetector.cs
|
||||
│ ├── EntropyDetector.cs
|
||||
│ └── CompositeSecretDetector.cs
|
||||
├── Rules/
|
||||
│ ├── SecretRule.cs
|
||||
│ ├── SecretRuleset.cs
|
||||
│ └── RulesetLoader.cs
|
||||
├── Masking/
|
||||
│ ├── IPayloadMasker.cs
|
||||
│ └── PayloadMasker.cs
|
||||
├── Evidence/
|
||||
│ ├── SecretLeakEvidence.cs
|
||||
│ └── SecretFinding.cs
|
||||
├── SecretsAnalyzer.cs
|
||||
├── SecretsAnalyzerHost.cs
|
||||
├── SecretsAnalyzerOptions.cs
|
||||
└── ServiceCollectionExtensions.cs
|
||||
```
|
||||
|
||||
csproj should reference:
|
||||
- StellaOps.Scanner.Core
|
||||
- StellaOps.Scanner.Surface
|
||||
- StellaOps.Evidence.Core
|
||||
|
||||
### SLD-002: Rule Models
|
||||
|
||||
Define the rule structure for secret detection:
|
||||
|
||||
```csharp
|
||||
/// <summary>
|
||||
/// A single secret detection rule.
|
||||
/// </summary>
|
||||
public sealed record SecretRule
|
||||
{
|
||||
public required string Id { get; init; } // e.g., "stellaops.secrets.aws-access-key"
|
||||
public required string Version { get; init; } // e.g., "2025.11.0"
|
||||
public required string Name { get; init; } // Human-readable name
|
||||
public required string Description { get; init; }
|
||||
public required SecretRuleType Type { get; init; } // Regex, Entropy, Composite
|
||||
public required string Pattern { get; init; } // Regex pattern or entropy config
|
||||
public required SecretSeverity Severity { get; init; }
|
||||
public required SecretConfidence Confidence { get; init; }
|
||||
public string? MaskingHint { get; init; } // e.g., "prefix:4,suffix:2"
|
||||
public ImmutableArray<string> Keywords { get; init; } // Pre-filter keywords
|
||||
public ImmutableArray<string> FilePatterns { get; init; } // Glob patterns for file filtering
|
||||
public bool Enabled { get; init; } = true;
|
||||
}
|
||||
|
||||
public enum SecretRuleType { Regex, Entropy, Composite }
|
||||
public enum SecretSeverity { Low, Medium, High, Critical }
|
||||
public enum SecretConfidence { Low, Medium, High }
|
||||
|
||||
/// <summary>
|
||||
/// A versioned collection of secret detection rules.
|
||||
/// </summary>
|
||||
public sealed record SecretRuleset
|
||||
{
|
||||
public required string Id { get; init; } // e.g., "secrets.ruleset"
|
||||
public required string Version { get; init; } // e.g., "2025.11"
|
||||
public required DateTimeOffset CreatedAt { get; init; }
|
||||
public required ImmutableArray<SecretRule> Rules { get; init; }
|
||||
public string? Sha256Digest { get; init; } // Integrity hash
|
||||
}
|
||||
```
|
||||
|
||||
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Rules/`
|
||||
|
||||
### SLD-003: Regex Detector
|
||||
|
||||
Implement regex-based secret detection:
|
||||
|
||||
```csharp
|
||||
public interface ISecretDetector
|
||||
{
|
||||
string DetectorId { get; }
|
||||
|
||||
ValueTask<IReadOnlyList<SecretMatch>> DetectAsync(
|
||||
ReadOnlyMemory<byte> content,
|
||||
string filePath,
|
||||
SecretRule rule,
|
||||
CancellationToken ct = default);
|
||||
}
|
||||
|
||||
public sealed record SecretMatch(
|
||||
SecretRule Rule,
|
||||
string FilePath,
|
||||
int LineNumber,
|
||||
int ColumnStart,
|
||||
int ColumnEnd,
|
||||
ReadOnlyMemory<byte> RawMatch, // For masking
|
||||
double ConfidenceScore);
|
||||
|
||||
public sealed class RegexDetector : ISecretDetector
|
||||
{
|
||||
public string DetectorId => "regex";
|
||||
|
||||
// Implementation notes:
|
||||
// - Use compiled regex for performance
|
||||
// - Apply keyword pre-filter before regex matching
|
||||
// - Respect file pattern filters
|
||||
// - Track line/column for precise location
|
||||
// - Never log raw match content
|
||||
}
|
||||
```
|
||||
|
||||
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Detectors/`
|
||||
|
||||
### SLD-004: Entropy Detector
|
||||
|
||||
Implement Shannon entropy-based detection for high-entropy strings:
|
||||
|
||||
```csharp
|
||||
public sealed class EntropyDetector : ISecretDetector
|
||||
{
|
||||
public string DetectorId => "entropy";
|
||||
|
||||
// Implementation notes:
|
||||
// - Calculate Shannon entropy for candidate strings
|
||||
// - Default threshold: 4.5 bits per character
|
||||
// - Minimum length: 16 characters
|
||||
// - Skip common high-entropy non-secrets (UUIDs, hashes in comments)
|
||||
// - Apply charset detection (base64, hex, alphanumeric)
|
||||
}
|
||||
|
||||
public static class EntropyCalculator
|
||||
{
|
||||
/// <summary>
|
||||
/// Calculates Shannon entropy in bits per character.
|
||||
/// </summary>
|
||||
public static double Calculate(ReadOnlySpan<byte> data)
|
||||
{
|
||||
// Use CultureInfo.InvariantCulture for all formatting
|
||||
// Return 0.0 for empty input
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### SLD-005: Payload Masker
|
||||
|
||||
Implement secure payload masking:
|
||||
|
||||
```csharp
|
||||
public interface IPayloadMasker
|
||||
{
|
||||
/// <summary>
|
||||
/// Masks a secret payload preserving prefix/suffix for identification.
|
||||
/// </summary>
|
||||
/// <param name="payload">The raw secret bytes</param>
|
||||
/// <param name="hint">Optional masking hint from rule (e.g., "prefix:4,suffix:2")</param>
|
||||
/// <returns>Masked string (e.g., "AKIA****B7")</returns>
|
||||
string Mask(ReadOnlySpan<byte> payload, string? hint = null);
|
||||
}
|
||||
|
||||
public sealed class PayloadMasker : IPayloadMasker
|
||||
{
|
||||
// Default: preserve first 4 and last 2 characters
|
||||
// Replace middle with asterisks (max 8 asterisks)
|
||||
// Minimum output length: 8 characters
|
||||
// Never expose more than 6 characters total
|
||||
|
||||
public const int DefaultPrefixLength = 4;
|
||||
public const int DefaultSuffixLength = 2;
|
||||
public const int MaxMaskLength = 8;
|
||||
public const char MaskChar = '*';
|
||||
}
|
||||
```
|
||||
|
||||
### SLD-006: Evidence Models
|
||||
|
||||
Define the evidence structure for policy integration:
|
||||
|
||||
```csharp
|
||||
/// <summary>
|
||||
/// Evidence record for a detected secret leak.
|
||||
/// </summary>
|
||||
public sealed record SecretLeakEvidence
|
||||
{
|
||||
public required string EvidenceType => "secret.leak";
|
||||
public required string RuleId { get; init; }
|
||||
public required string RuleVersion { get; init; }
|
||||
public required SecretSeverity Severity { get; init; }
|
||||
public required SecretConfidence Confidence { get; init; }
|
||||
public required string FilePath { get; init; }
|
||||
public required int LineNumber { get; init; }
|
||||
public required string Mask { get; init; } // Masked payload
|
||||
public required string BundleId { get; init; }
|
||||
public required string BundleVersion { get; init; }
|
||||
public required DateTimeOffset DetectedAt { get; init; }
|
||||
public ImmutableDictionary<string, string>? Metadata { get; init; }
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Aggregated finding for a single secret match.
|
||||
/// </summary>
|
||||
public sealed record SecretFinding
|
||||
{
|
||||
public required Guid Id { get; init; }
|
||||
public required SecretLeakEvidence Evidence { get; init; }
|
||||
public required string ScanId { get; init; }
|
||||
public required string TenantId { get; init; }
|
||||
public required string ArtifactDigest { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### SLD-007: Ruleset Loader
|
||||
|
||||
Implement deterministic ruleset loading:
|
||||
|
||||
```csharp
|
||||
public interface IRulesetLoader
|
||||
{
|
||||
ValueTask<SecretRuleset> LoadAsync(
|
||||
string rulesetPath,
|
||||
CancellationToken ct = default);
|
||||
|
||||
ValueTask<SecretRuleset> LoadFromJsonlAsync(
|
||||
Stream rulesStream,
|
||||
string bundleId,
|
||||
string bundleVersion,
|
||||
CancellationToken ct = default);
|
||||
}
|
||||
|
||||
public sealed class RulesetLoader : IRulesetLoader
|
||||
{
|
||||
// Implementation notes:
|
||||
// - Parse secrets.ruleset.rules.jsonl (NDJSON format)
|
||||
// - Validate rule schema on load
|
||||
// - Sort rules by ID for deterministic ordering
|
||||
// - Calculate and verify SHA-256 digest
|
||||
// - Use CultureInfo.InvariantCulture for all parsing
|
||||
// - Log bundle version on successful load
|
||||
}
|
||||
```
|
||||
|
||||
### SLD-008: Analyzer Options
|
||||
|
||||
Configuration options with feature flag:
|
||||
|
||||
```csharp
|
||||
public sealed class SecretsAnalyzerOptions
|
||||
{
|
||||
/// <summary>
|
||||
/// Enable secret leak detection (experimental feature).
|
||||
/// </summary>
|
||||
public bool Enabled { get; set; } = false;
|
||||
|
||||
/// <summary>
|
||||
/// Path to the ruleset bundle directory.
|
||||
/// </summary>
|
||||
public string RulesetPath { get; set; } = "/opt/stellaops/plugins/scanner/analyzers/secrets";
|
||||
|
||||
/// <summary>
|
||||
/// Minimum confidence level to report findings.
|
||||
/// </summary>
|
||||
public SecretConfidence MinConfidence { get; set; } = SecretConfidence.Medium;
|
||||
|
||||
/// <summary>
|
||||
/// Maximum findings per scan (circuit breaker).
|
||||
/// </summary>
|
||||
public int MaxFindingsPerScan { get; set; } = 1000;
|
||||
|
||||
/// <summary>
|
||||
/// File size limit for scanning (bytes).
|
||||
/// </summary>
|
||||
public long MaxFileSizeBytes { get; set; } = 10 * 1024 * 1024; // 10MB
|
||||
|
||||
/// <summary>
|
||||
/// Enable entropy-based detection.
|
||||
/// </summary>
|
||||
public bool EnableEntropyDetection { get; set; } = true;
|
||||
|
||||
/// <summary>
|
||||
/// Entropy threshold (bits per character).
|
||||
/// </summary>
|
||||
public double EntropyThreshold { get; set; } = 4.5;
|
||||
}
|
||||
```
|
||||
|
||||
### SLD-009: Composite Detector
|
||||
|
||||
Combine multiple detection strategies:
|
||||
|
||||
```csharp
|
||||
public sealed class CompositeSecretDetector : ISecretDetector
|
||||
{
|
||||
private readonly IReadOnlyList<ISecretDetector> _detectors;
|
||||
private readonly ILogger<CompositeSecretDetector> _logger;
|
||||
|
||||
public string DetectorId => "composite";
|
||||
|
||||
// Implementation notes:
|
||||
// - Execute detectors in parallel where possible
|
||||
// - Deduplicate overlapping matches
|
||||
// - Merge confidence scores for overlapping detections
|
||||
// - Respect per-rule detector type preference
|
||||
}
|
||||
```
|
||||
|
||||
### SLD-010: Secrets Analyzer
|
||||
|
||||
Main analyzer implementation:
|
||||
|
||||
```csharp
|
||||
public sealed class SecretsAnalyzer : ILayerAnalyzer
|
||||
{
|
||||
public string AnalyzerId => "secrets";
|
||||
public string DisplayName => "Secret Leak Detector";
|
||||
|
||||
// Implementation notes:
|
||||
// - Check feature flag before processing
|
||||
// - Load ruleset once at startup (cached)
|
||||
// - Apply file pattern filters efficiently
|
||||
// - Execute detection on text files only
|
||||
// - Emit SecretLeakEvidence for each finding
|
||||
// - Apply masking before any output
|
||||
// - Track metrics: scanner.secret.finding_total
|
||||
// - Add tracing span: scanner.secrets.scan
|
||||
}
|
||||
```
|
||||
|
||||
### SLD-011: Analyzer Host
|
||||
|
||||
Lifecycle management for the analyzer:
|
||||
|
||||
```csharp
|
||||
public sealed class SecretsAnalyzerHost : IHostedService
|
||||
{
|
||||
// Implementation notes:
|
||||
// - Load and validate ruleset on startup
|
||||
// - Log bundle version and rule count
|
||||
// - Verify DSSE signature if available
|
||||
// - Graceful shutdown with finding flush
|
||||
// - Emit startup log: "SecretsAnalyzerHost: Loaded bundle {version} with {count} rules"
|
||||
}
|
||||
```
|
||||
|
||||
### SLD-012: Worker Integration
|
||||
|
||||
Integrate with Scanner Worker pipeline:
|
||||
|
||||
```csharp
|
||||
// In Scanner.Worker processing pipeline:
|
||||
// 1. Add SecretsAnalyzer to analyzer chain (after language analyzers)
|
||||
// 2. Gate execution on feature flag
|
||||
// 3. Store findings in ScanAnalysisStore
|
||||
// 4. Include in scan completion event
|
||||
```
|
||||
|
||||
### SLD-013: DI Registration
|
||||
|
||||
```csharp
|
||||
public static class ServiceCollectionExtensions
|
||||
{
|
||||
public static IServiceCollection AddSecretsAnalyzer(
|
||||
this IServiceCollection services,
|
||||
IConfiguration configuration)
|
||||
{
|
||||
services.AddOptions<SecretsAnalyzerOptions>()
|
||||
.Bind(configuration.GetSection("Scanner:Analyzers:Secrets"))
|
||||
.ValidateDataAnnotations()
|
||||
.ValidateOnStart();
|
||||
|
||||
services.AddSingleton<IPayloadMasker, PayloadMasker>();
|
||||
services.AddSingleton<IRulesetLoader, RulesetLoader>();
|
||||
services.AddSingleton<ISecretDetector, RegexDetector>();
|
||||
services.AddSingleton<ISecretDetector, EntropyDetector>();
|
||||
services.AddSingleton<ISecretDetector, CompositeSecretDetector>();
|
||||
services.AddSingleton<SecretsAnalyzer>();
|
||||
services.AddHostedService<SecretsAnalyzerHost>();
|
||||
|
||||
return services;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### SLD-014: Unit Tests
|
||||
|
||||
Required test coverage in `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Secrets.Tests/`:
|
||||
|
||||
```
|
||||
├── Detectors/
|
||||
│ ├── RegexDetectorTests.cs
|
||||
│ ├── EntropyDetectorTests.cs
|
||||
│ ├── EntropyCalculatorTests.cs
|
||||
│ └── CompositeSecretDetectorTests.cs
|
||||
├── Rules/
|
||||
│ ├── SecretRuleTests.cs
|
||||
│ └── RulesetLoaderTests.cs
|
||||
├── Masking/
|
||||
│ └── PayloadMaskerTests.cs
|
||||
├── Evidence/
|
||||
│ └── SecretLeakEvidenceTests.cs
|
||||
├── SecretsAnalyzerTests.cs
|
||||
└── Fixtures/
|
||||
├── aws-access-key.txt
|
||||
├── github-token.txt
|
||||
├── private-key.pem
|
||||
└── test-ruleset.jsonl
|
||||
```
|
||||
|
||||
Test requirements:
|
||||
- All tests must be deterministic
|
||||
- Use `[Trait("Category", "Unit")]` for unit tests
|
||||
- Test masking never exposes full secrets
|
||||
- Test entropy calculation with known inputs
|
||||
- Test regex patterns match expected secrets
|
||||
|
||||
### SLD-015: Integration Tests
|
||||
|
||||
Integration tests with Scanner Worker:
|
||||
|
||||
```
|
||||
├── SecretsAnalyzerIntegrationTests.cs
|
||||
│ - Test full scan with secrets embedded
|
||||
│ - Verify findings in ScanAnalysisStore
|
||||
│ - Verify masking in output
|
||||
│ - Test feature flag disables analyzer
|
||||
├── RulesetLoadingTests.cs
|
||||
│ - Test loading from file system
|
||||
│ - Test invalid ruleset handling
|
||||
│ - Test missing bundle handling
|
||||
```
|
||||
|
||||
### SLD-016: Module AGENTS.md
|
||||
|
||||
Create `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/AGENTS.md` with:
|
||||
- Mission statement
|
||||
- Scope definition
|
||||
- Required reading list
|
||||
- Working agreements
|
||||
- Security considerations
|
||||
|
||||
## Built-in Rule Examples
|
||||
|
||||
Initial rules to include in default bundle:
|
||||
|
||||
| Rule ID | Pattern Type | Description |
|
||||
|---------|--------------|-------------|
|
||||
| `stellaops.secrets.aws-access-key` | Regex | AWS Access Key ID (AKIA...) |
|
||||
| `stellaops.secrets.aws-secret-key` | Regex + Entropy | AWS Secret Access Key |
|
||||
| `stellaops.secrets.github-pat` | Regex | GitHub Personal Access Token |
|
||||
| `stellaops.secrets.github-app` | Regex | GitHub App Token (ghs_, ghp_) |
|
||||
| `stellaops.secrets.gitlab-pat` | Regex | GitLab Personal Access Token |
|
||||
| `stellaops.secrets.private-key-rsa` | Regex | RSA Private Key (PEM) |
|
||||
| `stellaops.secrets.private-key-ec` | Regex | EC Private Key (PEM) |
|
||||
| `stellaops.secrets.jwt` | Regex + Entropy | JSON Web Token |
|
||||
| `stellaops.secrets.basic-auth` | Regex | Basic Auth credentials in URLs |
|
||||
| `stellaops.secrets.generic-api-key` | Entropy | High-entropy API key patterns |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| Use NDJSON for rule format | Line-based parsing, easy streaming, git-friendly diffs |
|
||||
| Mask before any persistence | Defense in depth - secrets never stored |
|
||||
| Feature flag default off | Safe rollout, tenant opt-in required |
|
||||
| Entropy threshold 4.5 bits | Balance between false positives and detection rate |
|
||||
| Max 1000 findings per scan | Circuit breaker prevents DoS on noisy images |
|
||||
| Text files only | Binary secret detection deferred to future sprint |
|
||||
|
||||
## Metrics & Observability
|
||||
|
||||
| Metric | Type | Labels |
|
||||
|--------|------|--------|
|
||||
| `scanner.secret.finding_total` | Counter | tenant, ruleId, severity, confidence |
|
||||
| `scanner.secret.scan_duration_seconds` | Histogram | tenant |
|
||||
| `scanner.secret.rules_loaded` | Gauge | bundleVersion |
|
||||
| `scanner.secret.files_scanned` | Counter | tenant |
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date | Action | Notes |
|
||||
|------|--------|-------|
|
||||
| 2026-01-04 | Sprint created | Based on gap analysis of secrets scanning support |
|
||||
| 2026-01-04 | SLD-001 to SLD-014, SLD-016 completed | Full implementation: project structure, rule models, RegexDetector, EntropyDetector, PayloadMasker, SecretLeakEvidence, RulesetLoader, SecretsAnalyzerOptions, CompositeSecretDetector, SecretsAnalyzer, SecretsAnalyzerHost, ServiceCollectionExtensions, unit tests (EntropyCalculatorTests, PayloadMaskerTests, RegexDetectorTests, RulesetLoaderTests, SecretRuleTests, SecretRulesetTests), AGENTS.md. All builds verified. |
|
||||
| 2026-01-04 | SLD-015 completed | Created integration test project with test fixtures (aws-access-key.txt, github-token.txt, private-key.pem, test-ruleset.jsonl) and SecretsAnalyzerIntegrationTests.cs covering full scan detection, feature flags, circuit breaker, masking, evidence fields, and determinism. All builds verified. **Sprint complete.** |
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
# Sprint 20260104_006_BE - Secret Detection Configuration API
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Backend APIs and data models for configuring secret detection behavior per tenant. This sprint provides the foundation for UI configuration of secret leak detection.
|
||||
|
||||
**Key deliverables:**
|
||||
1. **Tenant Settings Model**: Per-tenant secret detection configuration
|
||||
2. **Revelation Policy**: Control how detected secrets are displayed/masked
|
||||
3. **Exception Management**: Allowlist patterns for false positives
|
||||
4. **Configuration API**: CRUD endpoints for settings
|
||||
|
||||
**Working directory:** `src/Scanner/`, `src/Platform/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Depends on**: Sprint 20260104_001 (Core Analyzer), Sprint 20260104_002 (Rule Bundles)
|
||||
- **Parallel with**: Sprint 20260104_007 (Alert Integration)
|
||||
- **Blocks**: Sprint 20260104_008 (UI)
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- docs/modules/scanner/operations/secret-leak-detection.md
|
||||
- CLAUDE.md Section 8 (Determinism)
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | SDC-001 | DONE | None | Scanner Guild | Define SecretDetectionSettings domain model |
|
||||
| 2 | SDC-002 | DONE | SDC-001 | Scanner Guild | Create SecretRevelationPolicy enum and config |
|
||||
| 3 | SDC-003 | DONE | SDC-001 | Scanner Guild | Create SecretExceptionPattern model for allowlists |
|
||||
| 4 | SDC-004 | DONE | SDC-001 | Platform Guild | Add persistence (Dapper migrations) |
|
||||
| 5 | SDC-005 | DONE | SDC-004 | Platform Guild | Create Settings CRUD API endpoints |
|
||||
| 6 | SDC-006 | DONE | SDC-005 | Platform Guild | Add OpenAPI spec for settings endpoints |
|
||||
| 7 | SDC-007 | DONE | SDC-003 | Scanner Guild | Integrate exception patterns into SecretsAnalyzerHost |
|
||||
| 8 | SDC-008 | DONE | SDC-002 | Scanner Guild | Implement revelation policy in findings output |
|
||||
| 9 | SDC-009 | DONE | All | Scanner Guild | Add unit and integration tests |
|
||||
|
||||
## Task Details
|
||||
|
||||
### SDC-001: SecretDetectionSettings Domain Model
|
||||
|
||||
```csharp
|
||||
public sealed record SecretDetectionSettings
|
||||
{
|
||||
public required Guid TenantId { get; init; }
|
||||
public required bool Enabled { get; init; }
|
||||
public required SecretRevelationPolicy RevelationPolicy { get; init; }
|
||||
public required IReadOnlyList<string> EnabledRuleCategories { get; init; }
|
||||
public required IReadOnlyList<SecretExceptionPattern> Exceptions { get; init; }
|
||||
public required SecretAlertSettings AlertSettings { get; init; }
|
||||
public required DateTimeOffset UpdatedAt { get; init; }
|
||||
public required string UpdatedBy { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Core/Secrets/Configuration/`
|
||||
|
||||
### SDC-002: SecretRevelationPolicy
|
||||
|
||||
Control how detected secrets appear in different contexts:
|
||||
|
||||
```csharp
|
||||
public enum SecretRevelationPolicy
|
||||
{
|
||||
/// <summary>
|
||||
/// Show only that a secret was detected, no value shown.
|
||||
/// Example: [SECRET_DETECTED: aws_access_key_id]
|
||||
/// </summary>
|
||||
FullMask = 0,
|
||||
|
||||
/// <summary>
|
||||
/// Show first and last 4 characters.
|
||||
/// Example: AKIA****WXYZ
|
||||
/// </summary>
|
||||
PartialReveal = 1,
|
||||
|
||||
/// <summary>
|
||||
/// Show full value (requires elevated permissions).
|
||||
/// Use only for debugging/incident response.
|
||||
/// </summary>
|
||||
FullReveal = 2
|
||||
}
|
||||
|
||||
public sealed record RevelationPolicyConfig
|
||||
{
|
||||
/// <summary>Default policy for UI/API responses.</summary>
|
||||
public SecretRevelationPolicy DefaultPolicy { get; init; } = SecretRevelationPolicy.PartialReveal;
|
||||
|
||||
/// <summary>Policy for exported reports (PDF, JSON).</summary>
|
||||
public SecretRevelationPolicy ExportPolicy { get; init; } = SecretRevelationPolicy.FullMask;
|
||||
|
||||
/// <summary>Policy for logs and telemetry.</summary>
|
||||
public SecretRevelationPolicy LogPolicy { get; init; } = SecretRevelationPolicy.FullMask;
|
||||
|
||||
/// <summary>Roles allowed to use FullReveal.</summary>
|
||||
public IReadOnlyList<string> FullRevealRoles { get; init; } = ["security-admin", "incident-responder"];
|
||||
|
||||
/// <summary>Number of characters to show at start/end for PartialReveal.</summary>
|
||||
public int PartialRevealChars { get; init; } = 4;
|
||||
}
|
||||
```
|
||||
|
||||
### SDC-003: SecretExceptionPattern (Allowlist)
|
||||
|
||||
```csharp
|
||||
public sealed record SecretExceptionPattern
|
||||
{
|
||||
public required Guid Id { get; init; }
|
||||
public required string Name { get; init; }
|
||||
public required string Description { get; init; }
|
||||
|
||||
/// <summary>Regex pattern to match against detected secret value.</summary>
|
||||
public required string Pattern { get; init; }
|
||||
|
||||
/// <summary>Optional: Only apply to specific rule IDs.</summary>
|
||||
public IReadOnlyList<string>? ApplicableRuleIds { get; init; }
|
||||
|
||||
/// <summary>Optional: Only apply to specific file paths.</summary>
|
||||
public string? FilePathGlob { get; init; }
|
||||
|
||||
/// <summary>Reason for exception (audit trail).</summary>
|
||||
public required string Justification { get; init; }
|
||||
|
||||
/// <summary>Expiration date (null = permanent).</summary>
|
||||
public DateTimeOffset? ExpiresAt { get; init; }
|
||||
|
||||
public required DateTimeOffset CreatedAt { get; init; }
|
||||
public required string CreatedBy { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### SDC-005: Settings API Endpoints
|
||||
|
||||
```
|
||||
GET /api/v1/tenants/{tenantId}/settings/secret-detection
|
||||
PUT /api/v1/tenants/{tenantId}/settings/secret-detection
|
||||
PATCH /api/v1/tenants/{tenantId}/settings/secret-detection
|
||||
|
||||
GET /api/v1/tenants/{tenantId}/settings/secret-detection/exceptions
|
||||
POST /api/v1/tenants/{tenantId}/settings/secret-detection/exceptions
|
||||
DELETE /api/v1/tenants/{tenantId}/settings/secret-detection/exceptions/{exceptionId}
|
||||
|
||||
GET /api/v1/tenants/{tenantId}/settings/secret-detection/rule-categories
|
||||
```
|
||||
|
||||
### SDC-008: Revelation Policy Implementation
|
||||
|
||||
```csharp
|
||||
public static class SecretMasker
|
||||
{
|
||||
public static string Mask(string secretValue, SecretRevelationPolicy policy, int partialChars = 4)
|
||||
{
|
||||
return policy switch
|
||||
{
|
||||
SecretRevelationPolicy.FullMask => "[REDACTED]",
|
||||
SecretRevelationPolicy.PartialReveal => MaskPartial(secretValue, partialChars),
|
||||
SecretRevelationPolicy.FullReveal => secretValue,
|
||||
_ => "[REDACTED]"
|
||||
};
|
||||
}
|
||||
|
||||
private static string MaskPartial(string value, int chars)
|
||||
{
|
||||
if (value.Length <= chars * 2)
|
||||
return new string('*', value.Length);
|
||||
|
||||
var prefix = value[..chars];
|
||||
var suffix = value[^chars..];
|
||||
var masked = new string('*', Math.Min(value.Length - chars * 2, 8));
|
||||
return $"{prefix}{masked}{suffix}";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
src/Scanner/__Libraries/StellaOps.Scanner.Core/
|
||||
├── Secrets/
|
||||
│ ├── Configuration/
|
||||
│ │ ├── SecretDetectionSettings.cs
|
||||
│ │ ├── SecretRevelationPolicy.cs
|
||||
│ │ ├── RevelationPolicyConfig.cs
|
||||
│ │ ├── SecretExceptionPattern.cs
|
||||
│ │ └── SecretAlertSettings.cs
|
||||
│ └── Masking/
|
||||
│ └── SecretMasker.cs
|
||||
|
||||
src/Platform/StellaOps.Platform.WebService/
|
||||
├── Endpoints/
|
||||
│ └── SecretDetectionSettingsEndpoints.cs
|
||||
└── Persistence/
|
||||
└── Migrations/
|
||||
└── AddSecretDetectionSettings.cs
|
||||
```
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| Per-tenant settings | Multi-tenant isolation requirement |
|
||||
| Role-based full reveal | Security: prevent accidental exposure |
|
||||
| Exception expiration | Force periodic review of allowlists |
|
||||
| Separate export/log policies | Defense in depth for sensitive data |
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date | Action | Notes |
|
||||
|------|--------|-------|
|
||||
| 2026-01-04 | Sprint created | Gap identified in secret detection feature |
|
||||
| 2026-01-04 | SDC-001 to SDC-008 DONE | Domain models, persistence, API endpoints, exception matcher, masker implemented |
|
||||
| 2026-01-04 | Files created | SecretDetectionSettings.cs, SecretRevelationPolicy.cs, SecretExceptionPattern.cs, SecretAlertSettings.cs, SecretMasker.cs, SecretExceptionMatcher.cs, migration 021_secret_detection_settings.sql, SecretDetectionSettingsRow.cs, ISecretDetectionSettingsRepository.cs, PostgresSecretDetectionSettingsRepository.cs, SecretDetectionConfigContracts.cs, SecretDetectionSettingsService.cs, SecretDetectionSettingsEndpoints.cs |
|
||||
| 2026-01-04 | SDC-009 DONE | Unit tests created: SecretDetectionSettingsTests.cs, SecretMaskerTests.cs, SecretExceptionPatternTests.cs, SecretExceptionMatcherTests.cs - build verified |
|
||||
|
||||
@@ -0,0 +1,298 @@
|
||||
# Sprint 20260104_007_BE - Secret Detection Alert Integration
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Integration between secret detection findings and the Notify service for real-time alerting when secrets are discovered in scans.
|
||||
|
||||
**Key deliverables:**
|
||||
1. **Alert Routing**: Route secret findings to configured channels
|
||||
2. **Alert Templates**: Formatted notifications for different channels
|
||||
3. **Rate Limiting**: Prevent alert fatigue from mass findings
|
||||
4. **Severity Mapping**: Map rule severity to alert priority
|
||||
|
||||
**Working directory:** `src/Scanner/`, `src/Notify/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Depends on**: Sprint 20260104_001 (Core Analyzer), Sprint 20260104_006 (Config API)
|
||||
- **Parallel with**: Sprint 20260104_008 (UI)
|
||||
- **Blocks**: Production deployment with alerting
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- docs/modules/notify/architecture.md
|
||||
- docs/modules/scanner/operations/secret-leak-detection.md
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | SDA-001 | DONE | None | Scanner Guild | Define SecretAlertSettings model |
|
||||
| 2 | SDA-002 | DONE | SDA-001 | Scanner Guild | Create SecretFindingAlertEvent |
|
||||
| 3 | SDA-003 | DONE | SDA-002 | Notify Guild | Add secret-finding alert template |
|
||||
| 4 | SDA-004 | DONE | SDA-003 | Notify Guild | Implement Slack/Teams formatters |
|
||||
| 5 | SDA-005 | DONE | SDA-002 | Scanner Guild | Add alert emission to SecretsAnalyzerHost |
|
||||
| 6 | SDA-006 | DONE | SDA-005 | Scanner Guild | Implement rate limiting / deduplication |
|
||||
| 7 | SDA-007 | DONE | SDA-006 | Scanner Guild | Add severity-based routing |
|
||||
| 8 | SDA-008 | DONE | SDA-001 | Platform Guild | Add alert settings to config API |
|
||||
| 9 | SDA-009 | DONE | All | Scanner Guild | Add integration tests |
|
||||
|
||||
## Task Details
|
||||
|
||||
### SDA-001: SecretAlertSettings Model
|
||||
|
||||
```csharp
|
||||
public sealed record SecretAlertSettings
|
||||
{
|
||||
/// <summary>Enable/disable alerting for this tenant.</summary>
|
||||
public bool Enabled { get; init; } = true;
|
||||
|
||||
/// <summary>Minimum severity to trigger alert (Critical, High, Medium, Low).</summary>
|
||||
public SecretSeverity MinimumAlertSeverity { get; init; } = SecretSeverity.High;
|
||||
|
||||
/// <summary>Alert destinations by channel type.</summary>
|
||||
public IReadOnlyList<SecretAlertDestination> Destinations { get; init; } = [];
|
||||
|
||||
/// <summary>Rate limit: max alerts per scan.</summary>
|
||||
public int MaxAlertsPerScan { get; init; } = 10;
|
||||
|
||||
/// <summary>Deduplication window: don't re-alert same secret within this period.</summary>
|
||||
public TimeSpan DeduplicationWindow { get; init; } = TimeSpan.FromHours(24);
|
||||
|
||||
/// <summary>Include file path in alert (may reveal repo structure).</summary>
|
||||
public bool IncludeFilePath { get; init; } = true;
|
||||
|
||||
/// <summary>Include masked secret value in alert.</summary>
|
||||
public bool IncludeMaskedValue { get; init; } = true;
|
||||
}
|
||||
|
||||
public sealed record SecretAlertDestination
|
||||
{
|
||||
public required Guid Id { get; init; }
|
||||
public required AlertChannelType ChannelType { get; init; }
|
||||
public required string ChannelId { get; init; } // Slack channel ID, email, webhook URL
|
||||
public IReadOnlyList<SecretSeverity>? SeverityFilter { get; init; }
|
||||
public IReadOnlyList<string>? RuleCategoryFilter { get; init; }
|
||||
}
|
||||
|
||||
public enum AlertChannelType
|
||||
{
|
||||
Slack,
|
||||
Teams,
|
||||
Email,
|
||||
Webhook,
|
||||
PagerDuty
|
||||
}
|
||||
```
|
||||
|
||||
### SDA-002: SecretFindingAlertEvent
|
||||
|
||||
```csharp
|
||||
public sealed record SecretFindingAlertEvent
|
||||
{
|
||||
public required Guid EventId { get; init; }
|
||||
public required Guid TenantId { get; init; }
|
||||
public required Guid ScanId { get; init; }
|
||||
public required string ImageRef { get; init; }
|
||||
|
||||
public required SecretSeverity Severity { get; init; }
|
||||
public required string RuleId { get; init; }
|
||||
public required string RuleName { get; init; }
|
||||
public required string RuleCategory { get; init; }
|
||||
|
||||
public required string FilePath { get; init; }
|
||||
public required int LineNumber { get; init; }
|
||||
public required string MaskedValue { get; init; }
|
||||
|
||||
public required DateTimeOffset DetectedAt { get; init; }
|
||||
public required string ScanTriggeredBy { get; init; }
|
||||
|
||||
/// <summary>Deduplication key for rate limiting.</summary>
|
||||
public string DeduplicationKey => $"{TenantId}:{RuleId}:{FilePath}:{LineNumber}";
|
||||
}
|
||||
```
|
||||
|
||||
### SDA-003: Alert Templates
|
||||
|
||||
**Slack Template:**
|
||||
```json
|
||||
{
|
||||
"blocks": [
|
||||
{
|
||||
"type": "header",
|
||||
"text": {
|
||||
"type": "plain_text",
|
||||
"text": "🚨 Secret Detected in Container Scan"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "section",
|
||||
"fields": [
|
||||
{ "type": "mrkdwn", "text": "*Severity:*\n{{severity}}" },
|
||||
{ "type": "mrkdwn", "text": "*Rule:*\n{{ruleName}}" },
|
||||
{ "type": "mrkdwn", "text": "*Image:*\n`{{imageRef}}`" },
|
||||
{ "type": "mrkdwn", "text": "*File:*\n`{{filePath}}:{{lineNumber}}`" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "section",
|
||||
"text": {
|
||||
"type": "mrkdwn",
|
||||
"text": "*Detected Value:*\n```{{maskedValue}}```"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "actions",
|
||||
"elements": [
|
||||
{
|
||||
"type": "button",
|
||||
"text": { "type": "plain_text", "text": "View in StellaOps" },
|
||||
"url": "{{findingUrl}}"
|
||||
},
|
||||
{
|
||||
"type": "button",
|
||||
"text": { "type": "plain_text", "text": "Add Exception" },
|
||||
"url": "{{exceptionUrl}}"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### SDA-005: Alert Emission in SecretsAnalyzerHost
|
||||
|
||||
```csharp
|
||||
public sealed class SecretsAnalyzerHost
|
||||
{
|
||||
private readonly ISecretAlertEmitter _alertEmitter;
|
||||
private readonly ISecretAlertDeduplicator _deduplicator;
|
||||
|
||||
public async Task OnSecretFoundAsync(
|
||||
SecretFinding finding,
|
||||
ScanContext context,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var settings = await _settingsProvider.GetAlertSettingsAsync(context.TenantId, ct);
|
||||
|
||||
if (!settings.Enabled)
|
||||
return;
|
||||
|
||||
if (finding.Severity < settings.MinimumAlertSeverity)
|
||||
return;
|
||||
|
||||
var alertEvent = MapToAlertEvent(finding, context);
|
||||
|
||||
// Check deduplication
|
||||
if (await _deduplicator.IsDuplicateAsync(alertEvent, settings.DeduplicationWindow, ct))
|
||||
{
|
||||
_logger.LogDebug("secret.alert.deduplicated key={key}", alertEvent.DeduplicationKey);
|
||||
return;
|
||||
}
|
||||
|
||||
// Rate limiting
|
||||
var alertCount = await _alertEmitter.GetAlertCountForScanAsync(context.ScanId, ct);
|
||||
if (alertCount >= settings.MaxAlertsPerScan)
|
||||
{
|
||||
_logger.LogWarning("secret.alert.rate_limited scan_id={scan_id} count={count}",
|
||||
context.ScanId, alertCount);
|
||||
return;
|
||||
}
|
||||
|
||||
// Emit to configured destinations
|
||||
await _alertEmitter.EmitAsync(alertEvent, settings.Destinations, ct);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### SDA-006: Rate Limiting & Deduplication
|
||||
|
||||
```csharp
|
||||
public interface ISecretAlertDeduplicator
|
||||
{
|
||||
Task<bool> IsDuplicateAsync(
|
||||
SecretFindingAlertEvent alert,
|
||||
TimeSpan window,
|
||||
CancellationToken ct);
|
||||
|
||||
Task RecordAlertAsync(
|
||||
SecretFindingAlertEvent alert,
|
||||
CancellationToken ct);
|
||||
}
|
||||
|
||||
public sealed class ValkeySecretAlertDeduplicator : ISecretAlertDeduplicator
|
||||
{
|
||||
private readonly IValkeyConnection _valkey;
|
||||
|
||||
public async Task<bool> IsDuplicateAsync(
|
||||
SecretFindingAlertEvent alert,
|
||||
TimeSpan window,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var key = $"secret:alert:dedup:{alert.DeduplicationKey}";
|
||||
var exists = await _valkey.ExistsAsync(key);
|
||||
return exists;
|
||||
}
|
||||
|
||||
public async Task RecordAlertAsync(
|
||||
SecretFindingAlertEvent alert,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var key = $"secret:alert:dedup:{alert.DeduplicationKey}";
|
||||
await _valkey.SetAsync(key, alert.EventId.ToString(), expiry: TimeSpan.FromHours(24));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Severity Mapping
|
||||
|
||||
| Rule Severity | Alert Priority | Default Behavior |
|
||||
|---------------|----------------|------------------|
|
||||
| Critical | P1 / Immediate | Always alert, page on-call |
|
||||
| High | P2 / Urgent | Alert to security channel |
|
||||
| Medium | P3 / Normal | Alert if configured |
|
||||
| Low | P4 / Info | No alert by default |
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
src/Scanner/__Libraries/StellaOps.Scanner.Core/
|
||||
├── Secrets/
|
||||
│ ├── Alerts/
|
||||
│ │ ├── SecretAlertSettings.cs
|
||||
│ │ ├── SecretFindingAlertEvent.cs
|
||||
│ │ ├── ISecretAlertEmitter.cs
|
||||
│ │ ├── ISecretAlertDeduplicator.cs
|
||||
│ │ └── ValkeySecretAlertDeduplicator.cs
|
||||
|
||||
src/Notify/__Libraries/StellaOps.Notify.Engine/
|
||||
├── Templates/
|
||||
│ └── SecretFindingAlertTemplate.cs
|
||||
├── Formatters/
|
||||
│ ├── SlackSecretAlertFormatter.cs
|
||||
│ └── TeamsSecretAlertFormatter.cs
|
||||
```
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| Valkey for deduplication | Fast, distributed, TTL support |
|
||||
| Per-scan rate limit | Prevent alert storms on large findings |
|
||||
| Masked values in alerts | Balance security awareness vs exposure |
|
||||
| Severity-based routing | Different channels for different priorities |
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date | Action | Notes |
|
||||
|------|--------|-------|
|
||||
| 2026-01-04 | Sprint created | Alert integration for secret detection |
|
||||
| 2026-01-04 | SDA-001 DONE | SecretAlertSettings already implemented in Sprint 006 (SecretAlertSettings.cs) |
|
||||
| 2026-01-04 | SDA-008 DONE | Alert settings already included in SecretDetectionSettings config API |
|
||||
| 2026-01-04 | SDA-002 DONE | Created SecretFindingAlertEvent.cs and SecretFindingInfo.cs |
|
||||
| 2026-01-04 | SDA-005 DONE | Created ISecretAlertEmitter.cs and SecretAlertEmitter.cs |
|
||||
| 2026-01-04 | SDA-006 DONE | Created ISecretAlertDeduplicator.cs interface |
|
||||
| 2026-01-04 | SDA-007 DONE | Created ISecretAlertRouter.cs and SecretAlertRouter.cs |
|
||||
| 2026-01-04 | SDA-003/004 DONE | Created SecretFindingAlertTemplates.cs with Slack, Teams, Email, Webhook, PagerDuty templates |
|
||||
| 2026-01-04 | SDA-009 DONE | Unit tests: SecretFindingAlertEventTests, SecretAlertRouterTests, SecretAlertEmitterTests |
|
||||
|
||||
@@ -0,0 +1,503 @@
|
||||
# Sprint 20260104_008_FE - Secret Detection UI
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Frontend components for configuring and viewing secret detection findings. Provides tenant administrators with tools to manage detection settings, view findings, and configure alerts.
|
||||
|
||||
**Key deliverables:**
|
||||
1. **Settings Page**: Configure secret detection for tenant
|
||||
2. **Findings Viewer**: View detected secrets with proper masking
|
||||
3. **Exception Manager**: Add/remove allowlist patterns
|
||||
4. **Alert Configuration**: Set up notification channels
|
||||
|
||||
**Working directory:** `src/Web/StellaOps.Web/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Depends on**: Sprint 20260104_006 (Config API), Sprint 20260104_007 (Alerts)
|
||||
- **Parallel with**: None (final UI sprint)
|
||||
- **Blocks**: Feature release
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- docs/modules/web/architecture.md
|
||||
- Angular v17 component patterns
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | SDU-001 | DONE | None | Frontend Guild | Create secret-detection feature module |
|
||||
| 2 | SDU-002 | DONE | SDU-001 | Frontend Guild | Build settings page component |
|
||||
| 3 | SDU-003 | DONE | SDU-002 | Frontend Guild | Add revelation policy selector |
|
||||
| 4 | SDU-004 | DONE | SDU-002 | Frontend Guild | Build rule category toggles |
|
||||
| 5 | SDU-005 | DONE | SDU-001 | Frontend Guild | Create findings list component |
|
||||
| 6 | SDU-006 | DONE | SDU-005 | Frontend Guild | Implement masked value display |
|
||||
| 7 | SDU-007 | DONE | SDU-005 | Frontend Guild | Add finding detail drawer (via exception-manager) |
|
||||
| 8 | SDU-008 | DONE | SDU-001 | Frontend Guild | Build exception manager component |
|
||||
| 9 | SDU-009 | DONE | SDU-008 | Frontend Guild | Create exception form with validation |
|
||||
| 10 | SDU-010 | DONE | SDU-001 | Frontend Guild | Build alert destination config |
|
||||
| 11 | SDU-011 | DONE | SDU-010 | Frontend Guild | Add channel test functionality |
|
||||
| 12 | SDU-012 | DONE | All | Frontend Guild | Add E2E tests |
|
||||
|
||||
## Task Details
|
||||
|
||||
### SDU-002: Settings Page Component
|
||||
|
||||
```typescript
|
||||
// secret-detection-settings.component.ts
|
||||
@Component({
|
||||
selector: 'app-secret-detection-settings',
|
||||
template: `
|
||||
<div class="settings-container">
|
||||
<header class="settings-header">
|
||||
<h1>Secret Detection</h1>
|
||||
<mat-slide-toggle
|
||||
[checked]="settings()?.enabled"
|
||||
(change)="onEnabledChange($event)"
|
||||
color="primary">
|
||||
{{ settings()?.enabled ? 'Enabled' : 'Disabled' }}
|
||||
</mat-slide-toggle>
|
||||
</header>
|
||||
|
||||
<mat-tab-group>
|
||||
<mat-tab label="General">
|
||||
<app-revelation-policy-config
|
||||
[policy]="settings()?.revelationPolicy"
|
||||
(policyChange)="onPolicyChange($event)" />
|
||||
|
||||
<app-rule-category-selector
|
||||
[categories]="availableCategories()"
|
||||
[selected]="settings()?.enabledRuleCategories"
|
||||
(selectionChange)="onCategoriesChange($event)" />
|
||||
</mat-tab>
|
||||
|
||||
<mat-tab label="Exceptions">
|
||||
<app-exception-manager
|
||||
[exceptions]="settings()?.exceptions"
|
||||
(add)="onAddException($event)"
|
||||
(remove)="onRemoveException($event)" />
|
||||
</mat-tab>
|
||||
|
||||
<mat-tab label="Alerts">
|
||||
<app-alert-destination-config
|
||||
[settings]="settings()?.alertSettings"
|
||||
(settingsChange)="onAlertSettingsChange($event)" />
|
||||
</mat-tab>
|
||||
</mat-tab-group>
|
||||
</div>
|
||||
`
|
||||
})
|
||||
export class SecretDetectionSettingsComponent {
|
||||
private settingsService = inject(SecretDetectionSettingsService);
|
||||
|
||||
settings = this.settingsService.settings;
|
||||
availableCategories = this.settingsService.availableCategories;
|
||||
|
||||
// ... handlers
|
||||
}
|
||||
```
|
||||
|
||||
### SDU-003: Revelation Policy Selector
|
||||
|
||||
```typescript
|
||||
// revelation-policy-config.component.ts
|
||||
@Component({
|
||||
selector: 'app-revelation-policy-config',
|
||||
template: `
|
||||
<mat-card>
|
||||
<mat-card-header>
|
||||
<mat-card-title>Secret Revelation Policy</mat-card-title>
|
||||
<mat-card-subtitle>
|
||||
Control how detected secrets are displayed
|
||||
</mat-card-subtitle>
|
||||
</mat-card-header>
|
||||
|
||||
<mat-card-content>
|
||||
<mat-radio-group
|
||||
[value]="policy()?.defaultPolicy"
|
||||
(change)="onDefaultPolicyChange($event)">
|
||||
|
||||
<mat-radio-button value="FullMask">
|
||||
<div class="policy-option">
|
||||
<strong>Full Mask</strong>
|
||||
<span class="example">[REDACTED]</span>
|
||||
<p>No secret value shown. Safest option.</p>
|
||||
</div>
|
||||
</mat-radio-button>
|
||||
|
||||
<mat-radio-button value="PartialReveal">
|
||||
<div class="policy-option">
|
||||
<strong>Partial Reveal</strong>
|
||||
<span class="example">AKIA****WXYZ</span>
|
||||
<p>Show first/last 4 characters. Helps identify specific secrets.</p>
|
||||
</div>
|
||||
</mat-radio-button>
|
||||
|
||||
<mat-radio-button value="FullReveal" [disabled]="!canFullReveal()">
|
||||
<div class="policy-option">
|
||||
<strong>Full Reveal</strong>
|
||||
<span class="example">AKIAIOSFODNN7EXAMPLE</span>
|
||||
<p>Show complete value. Requires security-admin role.</p>
|
||||
</div>
|
||||
</mat-radio-button>
|
||||
</mat-radio-group>
|
||||
|
||||
<mat-divider />
|
||||
|
||||
<h4>Context-Specific Policies</h4>
|
||||
<div class="context-policies">
|
||||
<mat-form-field>
|
||||
<mat-label>Export Reports</mat-label>
|
||||
<mat-select [value]="policy()?.exportPolicy">
|
||||
<mat-option value="FullMask">Full Mask</mat-option>
|
||||
<mat-option value="PartialReveal">Partial Reveal</mat-option>
|
||||
</mat-select>
|
||||
</mat-form-field>
|
||||
|
||||
<mat-form-field>
|
||||
<mat-label>Logs & Telemetry</mat-label>
|
||||
<mat-select [value]="policy()?.logPolicy" disabled>
|
||||
<mat-option value="FullMask">Full Mask (Enforced)</mat-option>
|
||||
</mat-select>
|
||||
<mat-hint>Secrets are never logged in full</mat-hint>
|
||||
</mat-form-field>
|
||||
</div>
|
||||
</mat-card-content>
|
||||
</mat-card>
|
||||
`
|
||||
})
|
||||
```
|
||||
|
||||
### SDU-005: Findings List Component
|
||||
|
||||
```typescript
|
||||
// secret-findings-list.component.ts
|
||||
@Component({
|
||||
selector: 'app-secret-findings-list',
|
||||
template: `
|
||||
<div class="findings-container">
|
||||
<header class="findings-header">
|
||||
<h2>Secret Findings</h2>
|
||||
<div class="filters">
|
||||
<mat-form-field>
|
||||
<mat-label>Severity</mat-label>
|
||||
<mat-select multiple [(value)]="severityFilter">
|
||||
<mat-option value="Critical">Critical</mat-option>
|
||||
<mat-option value="High">High</mat-option>
|
||||
<mat-option value="Medium">Medium</mat-option>
|
||||
<mat-option value="Low">Low</mat-option>
|
||||
</mat-select>
|
||||
</mat-form-field>
|
||||
|
||||
<mat-form-field>
|
||||
<mat-label>Status</mat-label>
|
||||
<mat-select [(value)]="statusFilter">
|
||||
<mat-option value="Open">Open</mat-option>
|
||||
<mat-option value="Dismissed">Dismissed</mat-option>
|
||||
<mat-option value="Excepted">Excepted</mat-option>
|
||||
</mat-select>
|
||||
</mat-form-field>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<table mat-table [dataSource]="findings()">
|
||||
<ng-container matColumnDef="severity">
|
||||
<th mat-header-cell *matHeaderCellDef>Severity</th>
|
||||
<td mat-cell *matCellDef="let finding">
|
||||
<app-severity-badge [severity]="finding.severity" />
|
||||
</td>
|
||||
</ng-container>
|
||||
|
||||
<ng-container matColumnDef="rule">
|
||||
<th mat-header-cell *matHeaderCellDef>Rule</th>
|
||||
<td mat-cell *matCellDef="let finding">
|
||||
<div class="rule-info">
|
||||
<span class="rule-name">{{ finding.ruleName }}</span>
|
||||
<span class="rule-category">{{ finding.ruleCategory }}</span>
|
||||
</div>
|
||||
</td>
|
||||
</ng-container>
|
||||
|
||||
<ng-container matColumnDef="location">
|
||||
<th mat-header-cell *matHeaderCellDef>Location</th>
|
||||
<td mat-cell *matCellDef="let finding">
|
||||
<code class="file-path">{{ finding.filePath }}:{{ finding.lineNumber }}</code>
|
||||
</td>
|
||||
</ng-container>
|
||||
|
||||
<ng-container matColumnDef="value">
|
||||
<th mat-header-cell *matHeaderCellDef>Detected Value</th>
|
||||
<td mat-cell *matCellDef="let finding">
|
||||
<app-masked-secret-value
|
||||
[value]="finding.value"
|
||||
[policy]="revelationPolicy()"
|
||||
[canReveal]="canRevealSecrets()" />
|
||||
</td>
|
||||
</ng-container>
|
||||
|
||||
<ng-container matColumnDef="actions">
|
||||
<th mat-header-cell *matHeaderCellDef></th>
|
||||
<td mat-cell *matCellDef="let finding">
|
||||
<button mat-icon-button [matMenuTriggerFor]="actionMenu">
|
||||
<mat-icon>more_vert</mat-icon>
|
||||
</button>
|
||||
<mat-menu #actionMenu>
|
||||
<button mat-menu-item (click)="viewDetails(finding)">
|
||||
<mat-icon>visibility</mat-icon>
|
||||
View Details
|
||||
</button>
|
||||
<button mat-menu-item (click)="dismiss(finding)">
|
||||
<mat-icon>cancel</mat-icon>
|
||||
Dismiss
|
||||
</button>
|
||||
<button mat-menu-item (click)="addException(finding)">
|
||||
<mat-icon>playlist_add</mat-icon>
|
||||
Add Exception
|
||||
</button>
|
||||
</mat-menu>
|
||||
</td>
|
||||
</ng-container>
|
||||
|
||||
<tr mat-header-row *matHeaderRowDef="displayedColumns"></tr>
|
||||
<tr mat-row *matRowDef="let row; columns: displayedColumns"
|
||||
(click)="viewDetails(row)"></tr>
|
||||
</table>
|
||||
</div>
|
||||
`
|
||||
})
|
||||
```
|
||||
|
||||
### SDU-006: Masked Value Display
|
||||
|
||||
```typescript
|
||||
// masked-secret-value.component.ts
|
||||
@Component({
|
||||
selector: 'app-masked-secret-value',
|
||||
template: `
|
||||
<div class="masked-value" [class.revealed]="isRevealed()">
|
||||
<code>{{ displayValue() }}</code>
|
||||
|
||||
@if (canReveal() && !isRevealed()) {
|
||||
<button
|
||||
mat-icon-button
|
||||
matTooltip="Reveal value (logged)"
|
||||
(click)="reveal()">
|
||||
<mat-icon>visibility</mat-icon>
|
||||
</button>
|
||||
}
|
||||
|
||||
@if (isRevealed()) {
|
||||
<button
|
||||
mat-icon-button
|
||||
matTooltip="Hide value"
|
||||
(click)="hide()">
|
||||
<mat-icon>visibility_off</mat-icon>
|
||||
</button>
|
||||
<button
|
||||
mat-icon-button
|
||||
matTooltip="Copy to clipboard"
|
||||
(click)="copy()">
|
||||
<mat-icon>content_copy</mat-icon>
|
||||
</button>
|
||||
}
|
||||
</div>
|
||||
`,
|
||||
styles: [`
|
||||
.masked-value {
|
||||
font-family: monospace;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
}
|
||||
.revealed code {
|
||||
background: #fff3cd;
|
||||
padding: 4px 8px;
|
||||
border-radius: 4px;
|
||||
}
|
||||
`]
|
||||
})
|
||||
export class MaskedSecretValueComponent {
|
||||
value = input.required<string>();
|
||||
policy = input.required<SecretRevelationPolicy>();
|
||||
canReveal = input<boolean>(false);
|
||||
|
||||
private revealed = signal(false);
|
||||
isRevealed = computed(() => this.revealed() && this.canReveal());
|
||||
|
||||
displayValue = computed(() => {
|
||||
if (this.isRevealed()) {
|
||||
return this.value();
|
||||
}
|
||||
return this.maskValue(this.value(), this.policy());
|
||||
});
|
||||
|
||||
reveal() {
|
||||
// Log reveal action for audit
|
||||
this.auditService.logSecretReveal(this.value());
|
||||
this.revealed.set(true);
|
||||
}
|
||||
|
||||
hide() {
|
||||
this.revealed.set(false);
|
||||
}
|
||||
|
||||
private maskValue(value: string, policy: SecretRevelationPolicy): string {
|
||||
switch (policy) {
|
||||
case 'FullMask':
|
||||
return '[REDACTED]';
|
||||
case 'PartialReveal':
|
||||
if (value.length <= 8) return '*'.repeat(value.length);
|
||||
return `${value.slice(0, 4)}${'*'.repeat(Math.min(8, value.length - 8))}${value.slice(-4)}`;
|
||||
default:
|
||||
return '[REDACTED]';
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### SDU-010: Alert Destination Configuration
|
||||
|
||||
```typescript
|
||||
// alert-destination-config.component.ts
|
||||
@Component({
|
||||
selector: 'app-alert-destination-config',
|
||||
template: `
|
||||
<mat-card>
|
||||
<mat-card-header>
|
||||
<mat-card-title>Alert Destinations</mat-card-title>
|
||||
</mat-card-header>
|
||||
|
||||
<mat-card-content>
|
||||
<div class="alert-settings">
|
||||
<mat-slide-toggle [(ngModel)]="settings().enabled">
|
||||
Enable Alerts
|
||||
</mat-slide-toggle>
|
||||
|
||||
<mat-form-field>
|
||||
<mat-label>Minimum Severity</mat-label>
|
||||
<mat-select [(value)]="settings().minimumAlertSeverity">
|
||||
<mat-option value="Critical">Critical only</mat-option>
|
||||
<mat-option value="High">High and above</mat-option>
|
||||
<mat-option value="Medium">Medium and above</mat-option>
|
||||
<mat-option value="Low">All findings</mat-option>
|
||||
</mat-select>
|
||||
</mat-form-field>
|
||||
</div>
|
||||
|
||||
<mat-divider />
|
||||
|
||||
<h4>Configured Channels</h4>
|
||||
<div class="destinations-list">
|
||||
@for (dest of settings().destinations; track dest.id) {
|
||||
<mat-card class="destination-card">
|
||||
<div class="destination-info">
|
||||
<mat-icon>{{ getChannelIcon(dest.channelType) }}</mat-icon>
|
||||
<span>{{ dest.channelType }}</span>
|
||||
<code>{{ dest.channelId }}</code>
|
||||
</div>
|
||||
<div class="destination-actions">
|
||||
<button mat-icon-button (click)="testChannel(dest)">
|
||||
<mat-icon>send</mat-icon>
|
||||
</button>
|
||||
<button mat-icon-button color="warn" (click)="removeDestination(dest)">
|
||||
<mat-icon>delete</mat-icon>
|
||||
</button>
|
||||
</div>
|
||||
</mat-card>
|
||||
}
|
||||
</div>
|
||||
|
||||
<button mat-stroked-button (click)="addDestination()">
|
||||
<mat-icon>add</mat-icon>
|
||||
Add Destination
|
||||
</button>
|
||||
</mat-card-content>
|
||||
</mat-card>
|
||||
`
|
||||
})
|
||||
```
|
||||
|
||||
## UI Mockups
|
||||
|
||||
### Settings Page Layout
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Secret Detection [Enabled ●] │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ [General] [Exceptions] [Alerts] │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─ Revelation Policy ─────────────────────────────────┐ │
|
||||
│ │ ○ Full Mask [REDACTED] │ │
|
||||
│ │ ● Partial Reveal AKIA****WXYZ │ │
|
||||
│ │ ○ Full Reveal (requires security-admin) │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─ Rule Categories ───────────────────────────────────┐ │
|
||||
│ │ ☑ AWS Credentials ☑ GCP Service Accounts │ │
|
||||
│ │ ☑ Generic API Keys ☑ Private Keys │ │
|
||||
│ │ ☐ Internal Tokens ☑ Database Credentials │ │
|
||||
│ └─────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Findings List
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Secret Findings │
|
||||
│ Severity: [All ▼] Status: [Open ▼] Image: [All ▼] │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ SEV │ RULE │ LOCATION │ VALUE │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ 🔴 │ AWS Access Key │ config.yaml:42 │ AKIA****XYZ │
|
||||
│ 🟠 │ Generic API Key │ .env:15 │ sk_l****abc │
|
||||
│ 🟡 │ Private Key │ certs/server.key │ [REDACTED] │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
src/Web/StellaOps.Web/src/app/
|
||||
├── features/
|
||||
│ └── secret-detection/
|
||||
│ ├── secret-detection.module.ts
|
||||
│ ├── secret-detection.routes.ts
|
||||
│ ├── pages/
|
||||
│ │ ├── settings/
|
||||
│ │ │ └── secret-detection-settings.component.ts
|
||||
│ │ └── findings/
|
||||
│ │ └── secret-findings-list.component.ts
|
||||
│ ├── components/
|
||||
│ │ ├── revelation-policy-config/
|
||||
│ │ ├── rule-category-selector/
|
||||
│ │ ├── exception-manager/
|
||||
│ │ ├── alert-destination-config/
|
||||
│ │ ├── masked-secret-value/
|
||||
│ │ └── finding-detail-drawer/
|
||||
│ └── services/
|
||||
│ ├── secret-detection-settings.service.ts
|
||||
│ └── secret-findings.service.ts
|
||||
```
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| Angular Material | Consistent with existing UI |
|
||||
| Signal-based state | Modern Angular patterns |
|
||||
| Audit logging on reveal | Compliance requirement |
|
||||
| Lazy-loaded module | Performance optimization |
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date | Action | Notes |
|
||||
|------|--------|-------|
|
||||
| 2026-01-04 | Sprint created | UI components for secret detection |
|
||||
| 2026-01-05 | SDU-001 to SDU-010 completed | Feature module, settings page, revelation policy, rule toggles, findings list, masked display, exception manager, alert config all implemented |
|
||||
| 2026-01-05 | SDU-011 completed | Channel test functionality added to alert config |
|
||||
| 2026-01-05 | SDU-012 completed | E2E tests created in e2e/secret-detection.e2e.spec.ts |
|
||||
| 2026-01-05 | Sprint COMPLETE | All 12 tasks done |
|
||||
|
||||
Reference in New Issue
Block a user