save progress

This commit is contained in:
StellaOps Bot
2026-01-04 14:54:52 +02:00
parent c49b03a254
commit 3098e84de4
132 changed files with 19783 additions and 31 deletions

View File

@@ -1,6 +1,10 @@
# Secret Handling & Leak Detection
## StellaOps approach (2025.11 release)
> **Implementation Status:** Secret leak detection (`StellaOps.Scanner.Analyzers.Secrets`) is **NOT YET IMPLEMENTED**.
> Only Surface.Secrets (operational credential management) is currently available.
> See implementation sprints: SPRINT_20260104_002 through SPRINT_20260104_005.
## StellaOps approach (TARGET - 2026.01+ release)
- Read the Policy/Security briefing: `../../modules/policy/secret-leak-detection-readiness.md`.
- Operational runbook: `../../modules/scanner/operations/secret-leak-detection.md`.
- Surface.Secrets continues to deliver operational credentials through secure handles (`docs/modules/scanner/design/surface-secrets.md`), with providers supporting Kubernetes Secrets, file bundles, and inline definitions validated by Surface.Validation.
@@ -25,15 +29,15 @@
- Operators must rely on external tooling for leak detection while Grype focuses exclusively on vulnerability matching.[g1]
## Key differences
- **Purpose**: StellaOps now covers both operational secret retrieval *and* deterministic leak detection; Trivy and Snyk focus exclusively on leak detection while Grype omits it.
- **Workflow**: StellaOps performs leak detection in-line during scans with offline rule bundles and policy-aware outcomes; Trivy/Snyk rely on mutable rule packs or SaaS classifiers; Grype delegates to external tools.
- **Determinism**: StellaOps signs every bundle and records bundle IDs in explain traces; Trivy and Snyk update rules continuously (risking drift); Grype remains deterministic by not scanning.
- **Purpose**: StellaOps currently provides operational secret retrieval only; leak detection is **PLANNED** (see implementation sprints). Trivy and Snyk focus exclusively on leak detection while Grype omits it.
- **Workflow**: StellaOps will perform leak detection in-line during scans with offline rule bundles and policy-aware outcomes (when implemented); Trivy/Snyk rely on mutable rule packs or SaaS classifiers; Grype delegates to external tools.
- **Determinism**: StellaOps will sign every bundle and record bundle IDs in explain traces (when implemented); Trivy and Snyk update rules continuously (risking drift); Grype remains deterministic by not scanning.
### Detection technique comparison
| Tool | Detection technique(s) | Merge / result handling | Notes |
| --- | --- | --- | --- |
| **StellaOps (≤2025.10)** | `Surface.Secrets` providers fetch credentials at runtime; no leak scanning. | Secrets resolve to opaque handles stored in scan metadata; no SBOM entries emitted. | Deterministic secret retrieval only (legacy behaviour). |
| **StellaOps (2025.11+)** | `StellaOps.Scanner.Analyzers.Secrets` plug-in executes DSSE-signed rule bundles. | Findings inserted into `ScanAnalysisStore` as `secret.leak` evidence; Policy Engine merges with component context and lattice scores; CLI/export mask payloads. | Rule bundles ship offline, signatures verified locally; see operations runbook for rollout. |
| **StellaOps (≤2025.10)** | `Surface.Secrets` providers fetch credentials at runtime; no leak scanning. | Secrets resolve to opaque handles stored in scan metadata; no SBOM entries emitted. | Deterministic secret retrieval only. **This is the current implementation.** |
| **StellaOps (PLANNED)** | `StellaOps.Scanner.Analyzers.Secrets` plug-in executes DSSE-signed rule bundles. | Findings inserted into `ScanAnalysisStore` as `secret.leak` evidence; Policy Engine merges with component context and lattice scores; CLI/export mask payloads. | **NOT YET IMPLEMENTED** - See SPRINT_20260104_002 through SPRINT_20260104_005. |
| **Trivy** | Regex + entropy detectors under `pkg/fanal/secret` (configurable via `trivy-secret.yaml`). | Detectors aggregate per file; results exported alongside vulnerability findings without provenance binding. | Ships built-in rule sets; users can add allow/block lists. |
| **Snyk** | Snyk Code SaaS classifiers invoked by CLI plugin (`src/lib/plugins/sast`). | Source uploaded to SaaS; issues returned with severity + remediation; no offline merge with SBOM data. | Requires authenticated cloud access; rules evolve server-side. |
| **Grype** | None (focuses on vulnerability matching). | — | Operators must integrate separate tooling for leak detection. |

1641
docs/full-features-list.md Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,540 @@
# Sprint 20260104_002_SCANNER - Secret Leak Detection Core Analyzer
## Topic & Scope
Implement the core `StellaOps.Scanner.Analyzers.Secrets` plugin that detects accidentally committed secrets in container layers during scans. This is the foundational sprint for secret leak detection capability.
**Key deliverables:**
1. **Secrets Analyzer Plugin**: Core analyzer that executes regex/entropy-based detection rules
2. **Rule Engine**: Rule definition models, matching logic, and deterministic execution
3. **Masking Engine**: Payload masking to ensure secrets never leak in outputs
4. **Evidence Emission**: `secret.leak` evidence type integration with ScanAnalysisStore
5. **Feature Flag**: Experimental toggle for gradual rollout
**Working directory:** `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/`
## Dependencies & Concurrency
- **Depends on**: Surface.Secrets, Surface.Validation, Surface.Env (already implemented)
- **Required by**: Sprint 20260104_003 (Rule Bundle Infrastructure), Sprint 20260104_004 (Policy DSL)
- **Parallel work**: Tasks SLD-001 through SLD-008 can be developed concurrently
- **Integration tasks** (SLD-009+) require prior tasks complete
## Documentation Prerequisites
- docs/README.md
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
- docs/modules/scanner/architecture.md
- docs/modules/scanner/design/surface-secrets.md
- docs/modules/scanner/operations/secret-leak-detection.md (target spec)
- CLAUDE.md (especially Section 8: Code Quality & Determinism Rules)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | SLD-001 | TODO | None | Scanner Guild | Create project structure and csproj |
| 2 | SLD-002 | TODO | None | Scanner Guild | Define SecretRule and SecretRuleset models |
| 3 | SLD-003 | TODO | None | Scanner Guild | Implement ISecretDetector interface and RegexDetector |
| 4 | SLD-004 | TODO | None | Scanner Guild | Implement EntropyDetector for high-entropy string detection |
| 5 | SLD-005 | TODO | None | Scanner Guild | Implement PayloadMasker with configurable masking strategies |
| 6 | SLD-006 | TODO | None | Scanner Guild | Define SecretLeakEvidence record and finding model |
| 7 | SLD-007 | TODO | SLD-002 | Scanner Guild | Implement RulesetLoader with JSON parsing |
| 8 | SLD-008 | TODO | None | Scanner Guild | Add SecretsAnalyzerOptions with feature flag support |
| 9 | SLD-009 | TODO | SLD-003,SLD-004 | Scanner Guild | Implement CompositeSecretDetector combining regex and entropy |
| 10 | SLD-010 | TODO | SLD-006,SLD-009 | Scanner Guild | Implement SecretsAnalyzer (ILanguageAnalyzer) |
| 11 | SLD-011 | TODO | SLD-010 | Scanner Guild | Add SecretsAnalyzerHost for plugin lifecycle |
| 12 | SLD-012 | TODO | SLD-011 | Scanner Guild | Integrate with Scanner Worker pipeline |
| 13 | SLD-013 | TODO | SLD-010 | Scanner Guild | Add DI registration in ServiceCollectionExtensions |
| 14 | SLD-014 | TODO | All | Scanner Guild | Add comprehensive unit tests |
| 15 | SLD-015 | TODO | SLD-014 | Scanner Guild | Add integration tests with test fixtures |
| 16 | SLD-016 | TODO | All | Scanner Guild | Create AGENTS.md for module |
## Task Details
### SLD-001: Project Structure
Create the project skeleton following Scanner conventions:
```
src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/
├── StellaOps.Scanner.Analyzers.Secrets.csproj
├── AGENTS.md
├── AssemblyInfo.cs
├── Detectors/
│ ├── ISecretDetector.cs
│ ├── RegexDetector.cs
│ ├── EntropyDetector.cs
│ └── CompositeSecretDetector.cs
├── Rules/
│ ├── SecretRule.cs
│ ├── SecretRuleset.cs
│ └── RulesetLoader.cs
├── Masking/
│ ├── IPayloadMasker.cs
│ └── PayloadMasker.cs
├── Evidence/
│ ├── SecretLeakEvidence.cs
│ └── SecretFinding.cs
├── SecretsAnalyzer.cs
├── SecretsAnalyzerHost.cs
├── SecretsAnalyzerOptions.cs
└── ServiceCollectionExtensions.cs
```
csproj should reference:
- StellaOps.Scanner.Core
- StellaOps.Scanner.Surface
- StellaOps.Evidence.Core
### SLD-002: Rule Models
Define the rule structure for secret detection:
```csharp
/// <summary>
/// A single secret detection rule.
/// </summary>
public sealed record SecretRule
{
public required string Id { get; init; } // e.g., "stellaops.secrets.aws-access-key"
public required string Version { get; init; } // e.g., "2025.11.0"
public required string Name { get; init; } // Human-readable name
public required string Description { get; init; }
public required SecretRuleType Type { get; init; } // Regex, Entropy, Composite
public required string Pattern { get; init; } // Regex pattern or entropy config
public required SecretSeverity Severity { get; init; }
public required SecretConfidence Confidence { get; init; }
public string? MaskingHint { get; init; } // e.g., "prefix:4,suffix:2"
public ImmutableArray<string> Keywords { get; init; } // Pre-filter keywords
public ImmutableArray<string> FilePatterns { get; init; } // Glob patterns for file filtering
public bool Enabled { get; init; } = true;
}
public enum SecretRuleType { Regex, Entropy, Composite }
public enum SecretSeverity { Low, Medium, High, Critical }
public enum SecretConfidence { Low, Medium, High }
/// <summary>
/// A versioned collection of secret detection rules.
/// </summary>
public sealed record SecretRuleset
{
public required string Id { get; init; } // e.g., "secrets.ruleset"
public required string Version { get; init; } // e.g., "2025.11"
public required DateTimeOffset CreatedAt { get; init; }
public required ImmutableArray<SecretRule> Rules { get; init; }
public string? Sha256Digest { get; init; } // Integrity hash
}
```
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Rules/`
### SLD-003: Regex Detector
Implement regex-based secret detection:
```csharp
public interface ISecretDetector
{
string DetectorId { get; }
ValueTask<IReadOnlyList<SecretMatch>> DetectAsync(
ReadOnlyMemory<byte> content,
string filePath,
SecretRule rule,
CancellationToken ct = default);
}
public sealed record SecretMatch(
SecretRule Rule,
string FilePath,
int LineNumber,
int ColumnStart,
int ColumnEnd,
ReadOnlyMemory<byte> RawMatch, // For masking
double ConfidenceScore);
public sealed class RegexDetector : ISecretDetector
{
public string DetectorId => "regex";
// Implementation notes:
// - Use compiled regex for performance
// - Apply keyword pre-filter before regex matching
// - Respect file pattern filters
// - Track line/column for precise location
// - Never log raw match content
}
```
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Detectors/`
### SLD-004: Entropy Detector
Implement Shannon entropy-based detection for high-entropy strings:
```csharp
public sealed class EntropyDetector : ISecretDetector
{
public string DetectorId => "entropy";
// Implementation notes:
// - Calculate Shannon entropy for candidate strings
// - Default threshold: 4.5 bits per character
// - Minimum length: 16 characters
// - Skip common high-entropy non-secrets (UUIDs, hashes in comments)
// - Apply charset detection (base64, hex, alphanumeric)
}
public static class EntropyCalculator
{
/// <summary>
/// Calculates Shannon entropy in bits per character.
/// </summary>
public static double Calculate(ReadOnlySpan<byte> data)
{
// Use CultureInfo.InvariantCulture for all formatting
// Return 0.0 for empty input
}
}
```
### SLD-005: Payload Masker
Implement secure payload masking:
```csharp
public interface IPayloadMasker
{
/// <summary>
/// Masks a secret payload preserving prefix/suffix for identification.
/// </summary>
/// <param name="payload">The raw secret bytes</param>
/// <param name="hint">Optional masking hint from rule (e.g., "prefix:4,suffix:2")</param>
/// <returns>Masked string (e.g., "AKIA****B7")</returns>
string Mask(ReadOnlySpan<byte> payload, string? hint = null);
}
public sealed class PayloadMasker : IPayloadMasker
{
// Default: preserve first 4 and last 2 characters
// Replace middle with asterisks (max 8 asterisks)
// Minimum output length: 8 characters
// Never expose more than 6 characters total
public const int DefaultPrefixLength = 4;
public const int DefaultSuffixLength = 2;
public const int MaxMaskLength = 8;
public const char MaskChar = '*';
}
```
### SLD-006: Evidence Models
Define the evidence structure for policy integration:
```csharp
/// <summary>
/// Evidence record for a detected secret leak.
/// </summary>
public sealed record SecretLeakEvidence
{
public required string EvidenceType => "secret.leak";
public required string RuleId { get; init; }
public required string RuleVersion { get; init; }
public required SecretSeverity Severity { get; init; }
public required SecretConfidence Confidence { get; init; }
public required string FilePath { get; init; }
public required int LineNumber { get; init; }
public required string Mask { get; init; } // Masked payload
public required string BundleId { get; init; }
public required string BundleVersion { get; init; }
public required DateTimeOffset DetectedAt { get; init; }
public ImmutableDictionary<string, string>? Metadata { get; init; }
}
/// <summary>
/// Aggregated finding for a single secret match.
/// </summary>
public sealed record SecretFinding
{
public required Guid Id { get; init; }
public required SecretLeakEvidence Evidence { get; init; }
public required string ScanId { get; init; }
public required string TenantId { get; init; }
public required string ArtifactDigest { get; init; }
}
```
### SLD-007: Ruleset Loader
Implement deterministic ruleset loading:
```csharp
public interface IRulesetLoader
{
ValueTask<SecretRuleset> LoadAsync(
string rulesetPath,
CancellationToken ct = default);
ValueTask<SecretRuleset> LoadFromJsonlAsync(
Stream rulesStream,
string bundleId,
string bundleVersion,
CancellationToken ct = default);
}
public sealed class RulesetLoader : IRulesetLoader
{
// Implementation notes:
// - Parse secrets.ruleset.rules.jsonl (NDJSON format)
// - Validate rule schema on load
// - Sort rules by ID for deterministic ordering
// - Calculate and verify SHA-256 digest
// - Use CultureInfo.InvariantCulture for all parsing
// - Log bundle version on successful load
}
```
### SLD-008: Analyzer Options
Configuration options with feature flag:
```csharp
public sealed class SecretsAnalyzerOptions
{
/// <summary>
/// Enable secret leak detection (experimental feature).
/// </summary>
public bool Enabled { get; set; } = false;
/// <summary>
/// Path to the ruleset bundle directory.
/// </summary>
public string RulesetPath { get; set; } = "/opt/stellaops/plugins/scanner/analyzers/secrets";
/// <summary>
/// Minimum confidence level to report findings.
/// </summary>
public SecretConfidence MinConfidence { get; set; } = SecretConfidence.Medium;
/// <summary>
/// Maximum findings per scan (circuit breaker).
/// </summary>
public int MaxFindingsPerScan { get; set; } = 1000;
/// <summary>
/// File size limit for scanning (bytes).
/// </summary>
public long MaxFileSizeBytes { get; set; } = 10 * 1024 * 1024; // 10MB
/// <summary>
/// Enable entropy-based detection.
/// </summary>
public bool EnableEntropyDetection { get; set; } = true;
/// <summary>
/// Entropy threshold (bits per character).
/// </summary>
public double EntropyThreshold { get; set; } = 4.5;
}
```
### SLD-009: Composite Detector
Combine multiple detection strategies:
```csharp
public sealed class CompositeSecretDetector : ISecretDetector
{
private readonly IReadOnlyList<ISecretDetector> _detectors;
private readonly ILogger<CompositeSecretDetector> _logger;
public string DetectorId => "composite";
// Implementation notes:
// - Execute detectors in parallel where possible
// - Deduplicate overlapping matches
// - Merge confidence scores for overlapping detections
// - Respect per-rule detector type preference
}
```
### SLD-010: Secrets Analyzer
Main analyzer implementation:
```csharp
public sealed class SecretsAnalyzer : ILayerAnalyzer
{
public string AnalyzerId => "secrets";
public string DisplayName => "Secret Leak Detector";
// Implementation notes:
// - Check feature flag before processing
// - Load ruleset once at startup (cached)
// - Apply file pattern filters efficiently
// - Execute detection on text files only
// - Emit SecretLeakEvidence for each finding
// - Apply masking before any output
// - Track metrics: scanner.secret.finding_total
// - Add tracing span: scanner.secrets.scan
}
```
### SLD-011: Analyzer Host
Lifecycle management for the analyzer:
```csharp
public sealed class SecretsAnalyzerHost : IHostedService
{
// Implementation notes:
// - Load and validate ruleset on startup
// - Log bundle version and rule count
// - Verify DSSE signature if available
// - Graceful shutdown with finding flush
// - Emit startup log: "SecretsAnalyzerHost: Loaded bundle {version} with {count} rules"
}
```
### SLD-012: Worker Integration
Integrate with Scanner Worker pipeline:
```csharp
// In Scanner.Worker processing pipeline:
// 1. Add SecretsAnalyzer to analyzer chain (after language analyzers)
// 2. Gate execution on feature flag
// 3. Store findings in ScanAnalysisStore
// 4. Include in scan completion event
```
### SLD-013: DI Registration
```csharp
public static class ServiceCollectionExtensions
{
public static IServiceCollection AddSecretsAnalyzer(
this IServiceCollection services,
IConfiguration configuration)
{
services.AddOptions<SecretsAnalyzerOptions>()
.Bind(configuration.GetSection("Scanner:Analyzers:Secrets"))
.ValidateDataAnnotations()
.ValidateOnStart();
services.AddSingleton<IPayloadMasker, PayloadMasker>();
services.AddSingleton<IRulesetLoader, RulesetLoader>();
services.AddSingleton<ISecretDetector, RegexDetector>();
services.AddSingleton<ISecretDetector, EntropyDetector>();
services.AddSingleton<ISecretDetector, CompositeSecretDetector>();
services.AddSingleton<SecretsAnalyzer>();
services.AddHostedService<SecretsAnalyzerHost>();
return services;
}
}
```
### SLD-014: Unit Tests
Required test coverage in `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Secrets.Tests/`:
```
├── Detectors/
│ ├── RegexDetectorTests.cs
│ ├── EntropyDetectorTests.cs
│ ├── EntropyCalculatorTests.cs
│ └── CompositeSecretDetectorTests.cs
├── Rules/
│ ├── SecretRuleTests.cs
│ └── RulesetLoaderTests.cs
├── Masking/
│ └── PayloadMaskerTests.cs
├── Evidence/
│ └── SecretLeakEvidenceTests.cs
├── SecretsAnalyzerTests.cs
└── Fixtures/
├── aws-access-key.txt
├── github-token.txt
├── private-key.pem
└── test-ruleset.jsonl
```
Test requirements:
- All tests must be deterministic
- Use `[Trait("Category", "Unit")]` for unit tests
- Test masking never exposes full secrets
- Test entropy calculation with known inputs
- Test regex patterns match expected secrets
### SLD-015: Integration Tests
Integration tests with Scanner Worker:
```
├── SecretsAnalyzerIntegrationTests.cs
│ - Test full scan with secrets embedded
│ - Verify findings in ScanAnalysisStore
│ - Verify masking in output
│ - Test feature flag disables analyzer
├── RulesetLoadingTests.cs
│ - Test loading from file system
│ - Test invalid ruleset handling
│ - Test missing bundle handling
```
### SLD-016: Module AGENTS.md
Create `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/AGENTS.md` with:
- Mission statement
- Scope definition
- Required reading list
- Working agreements
- Security considerations
## Built-in Rule Examples
Initial rules to include in default bundle:
| Rule ID | Pattern Type | Description |
|---------|--------------|-------------|
| `stellaops.secrets.aws-access-key` | Regex | AWS Access Key ID (AKIA...) |
| `stellaops.secrets.aws-secret-key` | Regex + Entropy | AWS Secret Access Key |
| `stellaops.secrets.github-pat` | Regex | GitHub Personal Access Token |
| `stellaops.secrets.github-app` | Regex | GitHub App Token (ghs_, ghp_) |
| `stellaops.secrets.gitlab-pat` | Regex | GitLab Personal Access Token |
| `stellaops.secrets.private-key-rsa` | Regex | RSA Private Key (PEM) |
| `stellaops.secrets.private-key-ec` | Regex | EC Private Key (PEM) |
| `stellaops.secrets.jwt` | Regex + Entropy | JSON Web Token |
| `stellaops.secrets.basic-auth` | Regex | Basic Auth credentials in URLs |
| `stellaops.secrets.generic-api-key` | Entropy | High-entropy API key patterns |
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| Use NDJSON for rule format | Line-based parsing, easy streaming, git-friendly diffs |
| Mask before any persistence | Defense in depth - secrets never stored |
| Feature flag default off | Safe rollout, tenant opt-in required |
| Entropy threshold 4.5 bits | Balance between false positives and detection rate |
| Max 1000 findings per scan | Circuit breaker prevents DoS on noisy images |
| Text files only | Binary secret detection deferred to future sprint |
## Metrics & Observability
| Metric | Type | Labels |
|--------|------|--------|
| `scanner.secret.finding_total` | Counter | tenant, ruleId, severity, confidence |
| `scanner.secret.scan_duration_seconds` | Histogram | tenant |
| `scanner.secret.rules_loaded` | Gauge | bundleVersion |
| `scanner.secret.files_scanned` | Counter | tenant |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Based on gap analysis of secrets scanning support |

View File

@@ -0,0 +1,451 @@
# Sprint 20260104_003_SCANNER - Secret Detection Rule Bundle Infrastructure
## Topic & Scope
Implement the DSSE-signed rule bundle infrastructure for secret leak detection. This sprint delivers the signing, verification, and distribution pipeline for deterministic rule bundles.
**Key deliverables:**
1. **Bundle Schema**: Formal JSON schema for rule bundles
2. **Bundle Builder**: CLI tool to create and sign rule bundles
3. **DSSE Integration**: Signing and verification using existing Signer/Attestor modules
4. **Bundle Verification**: Runtime verification of bundle integrity and provenance
5. **Default Bundle**: Initial set of secret detection rules
**Working directory:** `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/`, `src/Cli/`, `offline/rules/secrets/`
## Dependencies & Concurrency
- **Depends on**: Sprint 20260104_002 (Core Analyzer), StellaOps.Attestor, StellaOps.Signer
- **Required by**: Sprint 20260104_005 (Offline Kit Integration)
- **Parallel work**: Tasks RB-001 through RB-005 can be developed concurrently
## Documentation Prerequisites
- docs/modules/scanner/operations/secret-leak-detection.md
- docs/modules/attestor/architecture.md
- docs/modules/signer/architecture.md
- docs/ci/dsse-build-flow.md
- CLAUDE.md Section 8.6 (DSSE PAE Consistency)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | RB-001 | DONE | None | Scanner Guild | Define bundle manifest JSON schema |
| 2 | RB-002 | DONE | None | Scanner Guild | Define rules JSONL schema and validation |
| 3 | RB-003 | DONE | RB-001,RB-002 | Scanner Guild | Create BundleBuilder class for bundle creation |
| 4 | RB-004 | DONE | RB-003 | Scanner Guild | Add DSSE signing integration |
| 5 | RB-005 | DONE | RB-004 | Scanner Guild | Implement BundleVerifier with Attestor integration |
| 6 | RB-006 | DONE | RB-005 | CLI Guild | Add `stella secrets bundle create` CLI command |
| 7 | RB-007 | DONE | RB-005 | CLI Guild | Add `stella secrets bundle verify` CLI command |
| 8 | RB-008 | DONE | RB-005 | Scanner Guild | Integrate verification into SecretsAnalyzerHost |
| 9 | RB-009 | DONE | RB-002 | Scanner Guild | Create default rule definitions |
| 10 | RB-010 | DONE | RB-009 | Scanner Guild | Build and sign initial bundle (2026.01) |
| 11 | RB-011 | DONE | All | Scanner Guild | Add unit and integration tests |
| 12 | RB-012 | DONE | RB-010 | Docs Guild | Document bundle lifecycle and rotation |
## Task Details
### RB-001: Bundle Manifest Schema
Define the manifest schema (`secrets.ruleset.manifest.json`):
```json
{
"$schema": "https://stellaops.io/schemas/secrets-ruleset-manifest-v1.json",
"schemaVersion": "1.0",
"id": "secrets.ruleset",
"version": "2026.01",
"createdAt": "2026-01-04T00:00:00Z",
"description": "StellaOps Secret Detection Rules",
"rules": [
{
"id": "stellaops.secrets.aws-access-key",
"version": "1.0.0",
"severity": "high",
"enabled": true
}
],
"integrity": {
"rulesFile": "secrets.ruleset.rules.jsonl",
"rulesSha256": "abc123...",
"totalRules": 15,
"enabledRules": 15
},
"signatures": {
"dsseEnvelope": "secrets.ruleset.dsse.json"
}
}
```
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Bundles/Schemas/`
### RB-002: Rules JSONL Schema
Define the rule entry schema for NDJSON format:
```json
{
"id": "stellaops.secrets.aws-access-key",
"version": "1.0.0",
"name": "AWS Access Key ID",
"description": "Detects AWS Access Key IDs in source code and configuration files",
"type": "regex",
"pattern": "(?:A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}",
"severity": "high",
"confidence": "high",
"keywords": ["AKIA", "ASIA", "AIDA", "aws"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.conf", "*.config"],
"maskingHint": "prefix:4,suffix:2",
"metadata": {
"category": "cloud-credentials",
"provider": "aws",
"references": ["https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html"]
}
}
```
Validation rules:
- `id` must be namespaced (e.g., `stellaops.secrets.*`)
- `version` must be valid SemVer
- `pattern` must be valid regex (compile-time validation)
- `severity` must be one of: low, medium, high, critical
- `confidence` must be one of: low, medium, high
### RB-003: Bundle Builder
Implement the bundle creation logic:
```csharp
public interface IBundleBuilder
{
/// <summary>
/// Creates a bundle from individual rule files.
/// </summary>
Task<BundleArtifact> BuildAsync(
BundleBuildOptions options,
CancellationToken ct = default);
}
public sealed record BundleBuildOptions
{
public required string OutputDirectory { get; init; }
public required string BundleId { get; init; }
public required string Version { get; init; }
public required IReadOnlyList<string> RuleFiles { get; init; }
public TimeProvider? TimeProvider { get; init; }
}
public sealed record BundleArtifact
{
public required string ManifestPath { get; init; }
public required string RulesPath { get; init; }
public required string RulesSha256 { get; init; }
public required int TotalRules { get; init; }
}
public sealed class BundleBuilder : IBundleBuilder
{
// Implementation notes:
// - Validate each rule on load
// - Sort rules by ID for deterministic output
// - Compute SHA-256 of rules file
// - Generate manifest with integrity info
// - Use TimeProvider for timestamps (determinism)
}
```
Location: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/Bundles/`
### RB-004: DSSE Signing Integration
Integrate with Signer module for bundle signing:
```csharp
public interface IBundleSigner
{
/// <summary>
/// Signs a bundle artifact producing a DSSE envelope.
/// </summary>
Task<DsseEnvelope> SignAsync(
BundleArtifact artifact,
SigningOptions options,
CancellationToken ct = default);
}
public sealed record SigningOptions
{
public required string KeyId { get; init; }
public string PayloadType { get; init; } = "application/vnd.stellaops.secrets-ruleset+json";
public bool IncludeCertificateChain { get; init; } = true;
}
public sealed class BundleSigner : IBundleSigner
{
private readonly ISigner _signer;
// Implementation notes:
// - Use existing Signer infrastructure
// - Payload is the manifest JSON (not rules file)
// - Include rules file digest in signed payload
// - Support multiple signature algorithms
}
```
### RB-005: Bundle Verifier
Implement verification with Attestor integration:
```csharp
public interface IBundleVerifier
{
/// <summary>
/// Verifies a bundle's DSSE signature and integrity.
/// </summary>
Task<BundleVerificationResult> VerifyAsync(
string bundleDirectory,
VerificationOptions options,
CancellationToken ct = default);
}
public sealed record VerificationOptions
{
public string? AttestorUrl { get; init; }
public bool RequireRekorProof { get; init; } = false;
public IReadOnlyList<string>? TrustedKeyIds { get; init; }
}
public sealed record BundleVerificationResult
{
public required bool IsValid { get; init; }
public required string BundleVersion { get; init; }
public required DateTimeOffset SignedAt { get; init; }
public required string SignerKeyId { get; init; }
public string? RekorLogId { get; init; }
public IReadOnlyList<string>? ValidationErrors { get; init; }
}
public sealed class BundleVerifier : IBundleVerifier
{
private readonly IAttestorClient _attestorClient;
// Implementation notes:
// - Verify DSSE envelope signature
// - Verify rules file SHA-256 matches manifest
// - Optionally verify Rekor transparency log entry
// - Support offline verification (no network calls)
}
```
### RB-006: CLI Bundle Create Command
Add CLI command for bundle creation:
```bash
stella secrets bundle create \
--output ./bundles/2026.01 \
--bundle-id secrets.ruleset \
--version 2026.01 \
--rules ./rules/*.json \
--sign \
--key-id stellaops-secrets-signer
```
Implementation in `src/Cli/StellaOps.Cli/Commands/Secrets/`:
```csharp
[Command("secrets bundle create")]
public class BundleCreateCommand
{
[Option("--output", Required = true)]
public string OutputDirectory { get; set; }
[Option("--bundle-id", Required = true)]
public string BundleId { get; set; }
[Option("--version", Required = true)]
public string Version { get; set; }
[Option("--rules", Required = true)]
public IReadOnlyList<string> RuleFiles { get; set; }
[Option("--sign")]
public bool Sign { get; set; }
[Option("--key-id")]
public string? KeyId { get; set; }
}
```
### RB-007: CLI Bundle Verify Command
Add CLI command for bundle verification:
```bash
stella secrets bundle verify \
--bundle ./bundles/2026.01 \
--attestor-url http://attestor.local \
--require-rekor
```
```csharp
[Command("secrets bundle verify")]
public class BundleVerifyCommand
{
[Option("--bundle", Required = true)]
public string BundleDirectory { get; set; }
[Option("--attestor-url")]
public string? AttestorUrl { get; set; }
[Option("--require-rekor")]
public bool RequireRekor { get; set; }
}
```
### RB-008: Analyzer Host Integration
Update SecretsAnalyzerHost to verify bundles on startup:
```csharp
public sealed class SecretsAnalyzerHost : IHostedService
{
public async Task StartAsync(CancellationToken ct)
{
var bundlePath = _options.RulesetPath;
// Verify bundle integrity
var verification = await _verifier.VerifyAsync(bundlePath, new VerificationOptions
{
RequireRekorProof = _options.RequireSignatureVerification
}, ct);
if (!verification.IsValid)
{
_logger.LogError("Bundle verification failed: {Errors}",
string.Join(", ", verification.ValidationErrors ?? []));
if (_options.FailOnInvalidBundle)
throw new InvalidOperationException("Secret detection bundle verification failed");
return; // Analyzer disabled
}
_logger.LogInformation(
"SecretsAnalyzerHost: Loaded bundle {Version} signed by {KeyId} with {Count} rules",
verification.BundleVersion,
verification.SignerKeyId,
_ruleset.Rules.Length);
}
}
```
### RB-009: Default Rule Definitions
Create initial rule set in `offline/rules/secrets/sources/`:
| File | Rule ID | Description |
|------|---------|-------------|
| `aws-access-key.json` | `stellaops.secrets.aws-access-key` | AWS Access Key ID |
| `aws-secret-key.json` | `stellaops.secrets.aws-secret-key` | AWS Secret Access Key |
| `github-pat.json` | `stellaops.secrets.github-pat` | GitHub Personal Access Token |
| `github-app.json` | `stellaops.secrets.github-app` | GitHub App Token (ghs_, ghp_) |
| `gitlab-pat.json` | `stellaops.secrets.gitlab-pat` | GitLab Personal Access Token |
| `azure-storage.json` | `stellaops.secrets.azure-storage-key` | Azure Storage Account Key |
| `gcp-service-account.json` | `stellaops.secrets.gcp-service-account` | GCP Service Account JSON |
| `private-key-rsa.json` | `stellaops.secrets.private-key-rsa` | RSA Private Key (PEM) |
| `private-key-ec.json` | `stellaops.secrets.private-key-ec` | EC Private Key (PEM) |
| `private-key-openssh.json` | `stellaops.secrets.private-key-openssh` | OpenSSH Private Key |
| `jwt.json` | `stellaops.secrets.jwt` | JSON Web Token |
| `slack-token.json` | `stellaops.secrets.slack-token` | Slack API Token |
| `stripe-key.json` | `stellaops.secrets.stripe-key` | Stripe API Key |
| `sendgrid-key.json` | `stellaops.secrets.sendgrid-key` | SendGrid API Key |
| `generic-api-key.json` | `stellaops.secrets.generic-api-key` | Generic high-entropy API key |
### RB-010: Initial Bundle Build
Create the signed 2026.01 bundle:
```
offline/rules/secrets/2026.01/
├── secrets.ruleset.manifest.json
├── secrets.ruleset.rules.jsonl
└── secrets.ruleset.dsse.json
```
Build script in `.gitea/scripts/build/build-secrets-bundle.sh`:
```bash
#!/bin/bash
set -euo pipefail
VERSION="${1:-2026.01}"
OUTPUT_DIR="offline/rules/secrets/${VERSION}"
stella secrets bundle create \
--output "${OUTPUT_DIR}" \
--bundle-id secrets.ruleset \
--version "${VERSION}" \
--rules offline/rules/secrets/sources/*.json \
--sign \
--key-id stellaops-secrets-signer
```
### RB-011: Tests
Unit tests:
- `BundleBuilderTests.cs` - Bundle creation and validation
- `BundleVerifierTests.cs` - Signature verification
- `RuleValidatorTests.cs` - Rule schema validation
Integration tests:
- `BundleRoundtripTests.cs` - Create, sign, verify cycle
- `CliCommandTests.cs` - CLI command execution
### RB-012: Documentation
Update `docs/modules/scanner/operations/secret-leak-detection.md`:
- Add bundle creation workflow
- Document verification process
- Add troubleshooting for signature failures
Create `docs/modules/scanner/operations/secrets-bundle-rotation.md`:
- Bundle versioning strategy
- Rotation procedures
- Rollback instructions
## Bundle Directory Structure
```
offline/rules/secrets/
├── sources/ # Source rule definitions (not distributed)
│ ├── aws-access-key.json
│ ├── aws-secret-key.json
│ └── ...
├── 2026.01/ # Signed release bundle
│ ├── secrets.ruleset.manifest.json
│ ├── secrets.ruleset.rules.jsonl
│ └── secrets.ruleset.dsse.json
└── latest -> 2026.01 # Symlink to latest stable
```
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| NDJSON for rules | Streaming parse, git-friendly, easy validation |
| Sign manifest (not rules) | Manifest includes rules digest; smaller signature payload |
| Optional Rekor verification | Supports air-gapped deployments |
| Symlink for latest | Simple upgrade path, atomic switch |
| Source rules in repo | Version control, review process for rule changes |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Part of secret leak detection implementation |
| 2026-01-04 | RB-001 to RB-010 | Implemented bundle infrastructure, signing, verification, CLI commands |
| 2026-01-04 | RB-011 | Fixed and validated 37 unit tests for bundle system |
| 2026-01-04 | RB-012 | Updated secret-leak-detection.md, created secrets-bundle-rotation.md |
| 2026-01-04 | Sprint completed | All 12 tasks DONE |

View File

@@ -0,0 +1,543 @@
# Sprint 20260104_004_POLICY - Secret Leak Detection Policy DSL Integration
## Topic & Scope
Extend the Policy Engine and stella-dsl with `secret.*` predicates to enable policy-driven decisions on secret leak findings. This sprint delivers the policy integration layer.
**Key deliverables:**
1. **Policy Predicates**: `secret.hasFinding()`, `secret.bundle.version()`, `secret.match.count()`, `secret.mask.applied`
2. **Evidence Binding**: Connect SecretLeakEvidence to policy evaluation context
3. **Example Policies**: Sample policies for common secret blocking/warning scenarios
4. **Policy Validation**: Schema updates for secret-related predicates
**Working directory:** `src/Policy/`, `src/PolicyDsl/`
## Dependencies & Concurrency
- **Depends on**: Sprint 20260104_002 (Core Analyzer - SecretLeakEvidence model)
- **Parallel with**: Sprint 20260104_003 (Rule Bundles)
- **Required by**: Sprint 20260104_005 (Offline Kit Integration)
## Documentation Prerequisites
- docs/modules/policy/architecture.md
- docs/policy/dsl.md
- docs/modules/policy/secret-leak-detection-readiness.md
- CLAUDE.md Section 8 (Code Quality)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PSD-001 | TODO | None | Policy Guild | Define ISecretEvidenceProvider interface |
| 2 | PSD-002 | TODO | PSD-001 | Policy Guild | Implement SecretEvidenceContext binding |
| 3 | PSD-003 | TODO | None | Policy Guild | Add secret.hasFinding() predicate |
| 4 | PSD-004 | TODO | None | Policy Guild | Add secret.bundle.version() predicate |
| 5 | PSD-005 | TODO | None | Policy Guild | Add secret.match.count() predicate |
| 6 | PSD-006 | TODO | None | Policy Guild | Add secret.mask.applied predicate |
| 7 | PSD-007 | TODO | None | Policy Guild | Add secret.path.allowlist() predicate |
| 8 | PSD-008 | TODO | PSD-003-007 | Policy Guild | Register predicates in PolicyDslRegistry |
| 9 | PSD-009 | TODO | PSD-008 | Policy Guild | Update DSL schema validation |
| 10 | PSD-010 | TODO | PSD-008 | Policy Guild | Create example policy templates |
| 11 | PSD-011 | TODO | All | Policy Guild | Add unit and integration tests |
| 12 | PSD-012 | TODO | All | Docs Guild | Update policy/dsl.md documentation |
## Task Details
### PSD-001: Secret Evidence Provider Interface
Define the interface for secret evidence access:
```csharp
/// <summary>
/// Provides secret leak evidence for policy evaluation.
/// </summary>
public interface ISecretEvidenceProvider
{
/// <summary>
/// Gets all secret findings for the current evaluation context.
/// </summary>
IReadOnlyList<SecretLeakEvidence> GetFindings();
/// <summary>
/// Gets the active rule bundle metadata.
/// </summary>
SecretBundleMetadata? GetBundleMetadata();
/// <summary>
/// Checks if masking was successfully applied to all findings.
/// </summary>
bool IsMaskingApplied();
}
public sealed record SecretBundleMetadata(
string BundleId,
string Version,
DateTimeOffset SignedAt,
int RuleCount);
```
Location: `src/Policy/__Libraries/StellaOps.Policy/Secrets/`
### PSD-002: Evidence Context Binding
Bind secret evidence to the policy evaluation context:
```csharp
public sealed class SecretEvidenceContext
{
private readonly ISecretEvidenceProvider _provider;
public SecretEvidenceContext(ISecretEvidenceProvider provider)
{
_provider = provider;
}
public IReadOnlyList<SecretLeakEvidence> Findings => _provider.GetFindings();
public SecretBundleMetadata? Bundle => _provider.GetBundleMetadata();
public bool MaskingApplied => _provider.IsMaskingApplied();
}
// Integration with PolicyEvaluationContext
public sealed class PolicyEvaluationContext
{
// ... existing properties ...
public SecretEvidenceContext? Secrets { get; init; }
}
```
### PSD-003: hasFinding Predicate
Implement the `secret.hasFinding()` predicate:
```csharp
/// <summary>
/// Returns true if any secret finding matches the filter criteria.
/// </summary>
/// <example>
/// secret.hasFinding() // Any finding
/// secret.hasFinding(severity: "high") // High severity
/// secret.hasFinding(ruleId: "stellaops.secrets.aws-*") // AWS rules (glob)
/// secret.hasFinding(severity: "high", confidence: "high") // Both filters
/// </example>
[DslPredicate("secret.hasFinding")]
public sealed class SecretHasFindingPredicate : IPolicyPredicate
{
public bool Evaluate(
PolicyEvaluationContext context,
IReadOnlyDictionary<string, object?>? args)
{
var findings = context.Secrets?.Findings ?? [];
if (findings.Count == 0) return false;
var ruleIdPattern = args?.GetValueOrDefault("ruleId") as string;
var severity = args?.GetValueOrDefault("severity") as string;
var confidence = args?.GetValueOrDefault("confidence") as string;
return findings.Any(f =>
MatchesRuleId(f.RuleId, ruleIdPattern) &&
MatchesSeverity(f.Severity, severity) &&
MatchesConfidence(f.Confidence, confidence));
}
private static bool MatchesRuleId(string ruleId, string? pattern)
{
if (string.IsNullOrEmpty(pattern)) return true;
if (pattern.EndsWith("*"))
return ruleId.StartsWith(pattern[..^1], StringComparison.Ordinal);
return ruleId.Equals(pattern, StringComparison.Ordinal);
}
}
```
Location: `src/Policy/__Libraries/StellaOps.Policy/Predicates/Secret/`
### PSD-004: bundle.version Predicate
Implement the `secret.bundle.version()` predicate:
```csharp
/// <summary>
/// Returns true if the active bundle meets or exceeds the required version.
/// </summary>
/// <example>
/// secret.bundle.version("2026.01") // At least 2026.01
/// </example>
[DslPredicate("secret.bundle.version")]
public sealed class SecretBundleVersionPredicate : IPolicyPredicate
{
public bool Evaluate(
PolicyEvaluationContext context,
IReadOnlyDictionary<string, object?>? args)
{
var requiredVersion = args?.GetValueOrDefault("requiredVersion") as string
?? throw new PolicyEvaluationException("secret.bundle.version requires requiredVersion argument");
var bundle = context.Secrets?.Bundle;
if (bundle == null) return false;
return CompareVersions(bundle.Version, requiredVersion) >= 0;
}
private static int CompareVersions(string current, string required)
{
// Simple calendar version comparison (YYYY.MM format)
return string.Compare(current, required, StringComparison.Ordinal);
}
}
```
### PSD-005: match.count Predicate
Implement the `secret.match.count()` predicate:
```csharp
/// <summary>
/// Returns the count of findings matching the filter criteria.
/// </summary>
/// <example>
/// secret.match.count() > 0 // Any findings
/// secret.match.count(ruleId: "stellaops.secrets.aws-*") >= 5 // Many AWS findings
/// </example>
[DslPredicate("secret.match.count")]
public sealed class SecretMatchCountPredicate : IPolicyPredicate<int>
{
public int Evaluate(
PolicyEvaluationContext context,
IReadOnlyDictionary<string, object?>? args)
{
var findings = context.Secrets?.Findings ?? [];
var ruleIdPattern = args?.GetValueOrDefault("ruleId") as string;
return findings.Count(f => MatchesRuleId(f.RuleId, ruleIdPattern));
}
}
```
### PSD-006: mask.applied Predicate
Implement the `secret.mask.applied` predicate:
```csharp
/// <summary>
/// Returns true if masking was successfully applied to all findings.
/// </summary>
/// <example>
/// secret.mask.applied // Verify masking succeeded
/// </example>
[DslPredicate("secret.mask.applied")]
public sealed class SecretMaskAppliedPredicate : IPolicyPredicate
{
public bool Evaluate(
PolicyEvaluationContext context,
IReadOnlyDictionary<string, object?>? args)
{
return context.Secrets?.MaskingApplied ?? true; // Default true if no findings
}
}
```
### PSD-007: path.allowlist Predicate
Implement the `secret.path.allowlist()` predicate for false positive suppression:
```csharp
/// <summary>
/// Returns true if all findings are in paths matching the allowlist patterns.
/// </summary>
/// <example>
/// secret.path.allowlist(["**/test/**", "**/fixtures/**"]) // Ignore test files
/// </example>
[DslPredicate("secret.path.allowlist")]
public sealed class SecretPathAllowlistPredicate : IPolicyPredicate
{
public bool Evaluate(
PolicyEvaluationContext context,
IReadOnlyDictionary<string, object?>? args)
{
var patterns = args?.GetValueOrDefault("patterns") as IReadOnlyList<string>;
if (patterns == null || patterns.Count == 0)
throw new PolicyEvaluationException("secret.path.allowlist requires patterns argument");
var findings = context.Secrets?.Findings ?? [];
if (findings.Count == 0) return true;
return findings.All(f => patterns.Any(p => GlobMatcher.IsMatch(f.FilePath, p)));
}
}
```
### PSD-008: Predicate Registration
Register predicates in the DSL registry:
```csharp
public static class SecretPredicateRegistration
{
public static void RegisterSecretPredicates(this PolicyDslRegistry registry)
{
registry.RegisterPredicate<SecretHasFindingPredicate>("secret.hasFinding");
registry.RegisterPredicate<SecretBundleVersionPredicate>("secret.bundle.version");
registry.RegisterPredicate<SecretMatchCountPredicate>("secret.match.count");
registry.RegisterPredicate<SecretMaskAppliedPredicate>("secret.mask.applied");
registry.RegisterPredicate<SecretPathAllowlistPredicate>("secret.path.allowlist");
}
}
```
### PSD-009: DSL Schema Validation
Update the DSL schema to include secret predicates:
```json
{
"predicates": {
"secret.hasFinding": {
"description": "Returns true if any secret finding matches the filter",
"arguments": {
"ruleId": { "type": "string", "optional": true, "description": "Rule ID pattern (supports * glob)" },
"severity": { "type": "string", "optional": true, "enum": ["low", "medium", "high", "critical"] },
"confidence": { "type": "string", "optional": true, "enum": ["low", "medium", "high"] }
},
"returns": "boolean"
},
"secret.bundle.version": {
"description": "Returns true if the active bundle meets or exceeds the required version",
"arguments": {
"requiredVersion": { "type": "string", "required": true, "description": "Minimum required version (YYYY.MM format)" }
},
"returns": "boolean"
},
"secret.match.count": {
"description": "Returns the count of findings matching the filter",
"arguments": {
"ruleId": { "type": "string", "optional": true, "description": "Rule ID pattern (supports * glob)" }
},
"returns": "integer"
},
"secret.mask.applied": {
"description": "Returns true if masking was successfully applied to all findings",
"arguments": {},
"returns": "boolean"
},
"secret.path.allowlist": {
"description": "Returns true if all findings are in paths matching the allowlist",
"arguments": {
"patterns": { "type": "array", "items": { "type": "string" }, "required": true }
},
"returns": "boolean"
}
}
}
```
### PSD-010: Example Policy Templates
Create example policies in `docs/modules/policy/examples/`:
**secret-blocker.stella** - Block high-severity secrets:
```dsl
policy "Secret Leak Guard" syntax "stella-dsl@1" {
metadata {
description = "Block high-confidence secret leaks in production scans"
tags = ["secrets", "compliance", "security"]
}
rule block_critical priority 100 {
when secret.hasFinding(severity: "critical")
then escalate to "block";
because "Critical secret leak detected - deployment blocked";
}
rule block_high_confidence priority 90 {
when secret.hasFinding(severity: "high", confidence: "high")
then escalate to "block";
because "High severity secret leak with high confidence detected";
}
rule warn_medium priority 50 {
when secret.hasFinding(severity: "medium")
then warn message "Medium severity secret detected - review required";
}
rule require_current_bundle priority 10 {
when not secret.bundle.version("2026.01")
then warn message "Secret detection bundle is out of date";
}
}
```
**secret-allowlist.stella** - Suppress test file findings:
```dsl
policy "Secret Allowlist" syntax "stella-dsl@1" {
metadata {
description = "Suppress false positives in test fixtures"
tags = ["secrets", "testing"]
}
rule allow_test_fixtures priority 200 {
when secret.path.allowlist([
"**/test/**",
"**/tests/**",
"**/fixtures/**",
"**/__fixtures__/**",
"**/testdata/**"
])
then annotate decision.notes := "Findings in test paths - suppressed";
else continue;
}
rule allow_examples priority 190 {
when secret.path.allowlist([
"**/examples/**",
"**/samples/**",
"**/docs/**"
])
and secret.hasFinding(confidence: "low")
then annotate decision.notes := "Low confidence findings in example paths";
else continue;
}
}
```
**secret-threshold.stella** - Threshold-based blocking:
```dsl
policy "Secret Threshold Guard" syntax "stella-dsl@1" {
metadata {
description = "Block scans exceeding secret finding thresholds"
tags = ["secrets", "thresholds"]
}
rule excessive_secrets priority 80 {
when secret.match.count() > 50
then escalate to "block";
because "Excessive number of secret findings (>50) - likely misconfigured scan";
}
rule many_aws_secrets priority 70 {
when secret.match.count(ruleId: "stellaops.secrets.aws-*") > 10
then escalate to "review";
because "Multiple AWS credentials detected - security review required";
}
}
```
### PSD-011: Tests
Unit tests in `src/Policy/__Tests/StellaOps.Policy.Tests/Predicates/Secret/`:
```
├── SecretHasFindingPredicateTests.cs
│ - Test empty findings returns false
│ - Test matching severity filter
│ - Test matching confidence filter
│ - Test ruleId glob matching
│ - Test combined filters
├── SecretBundleVersionPredicateTests.cs
│ - Test version comparison
│ - Test missing bundle returns false
│ - Test exact version match
├── SecretMatchCountPredicateTests.cs
│ - Test empty findings returns 0
│ - Test count with filter
│ - Test count without filter
├── SecretMaskAppliedPredicateTests.cs
│ - Test masking applied
│ - Test masking not applied
│ - Test default for no findings
├── SecretPathAllowlistPredicateTests.cs
│ - Test glob pattern matching
│ - Test multiple patterns
│ - Test no matching patterns
└── PolicyEvaluationIntegrationTests.cs
- Test full policy evaluation with secrets
- Test policy chaining
- Test decision propagation
```
### PSD-012: Documentation
Update `docs/policy/dsl.md` with new predicates section:
```markdown
## Secret Leak Detection Predicates
The following predicates are available for secret leak detection policy rules:
### secret.hasFinding(ruleId?, severity?, confidence?)
Returns `true` if any secret finding matches the specified filters.
**Arguments:**
- `ruleId` (string, optional): Rule ID pattern with optional `*` glob suffix
- `severity` (string, optional): One of `low`, `medium`, `high`, `critical`
- `confidence` (string, optional): One of `low`, `medium`, `high`
**Example:**
```dsl
when secret.hasFinding(severity: "high", confidence: "high")
```
### secret.bundle.version(requiredVersion)
Returns `true` if the active rule bundle version meets or exceeds the required version.
**Arguments:**
- `requiredVersion` (string, required): Minimum version in `YYYY.MM` format
**Example:**
```dsl
when not secret.bundle.version("2026.01")
then warn message "Bundle out of date";
```
### secret.match.count(ruleId?)
Returns the integer count of findings matching the optional rule ID filter.
**Example:**
```dsl
when secret.match.count() > 50
```
### secret.mask.applied
Returns `true` if payload masking was successfully applied to all findings.
**Example:**
```dsl
when not secret.mask.applied
then escalate to "block";
because "Masking failed - secrets may be exposed";
```
### secret.path.allowlist(patterns)
Returns `true` if all findings are in file paths matching at least one allowlist pattern.
**Arguments:**
- `patterns` (array of strings, required): Glob patterns for allowed paths
**Example:**
```dsl
when secret.path.allowlist(["**/test/**", "**/fixtures/**"])
```
```
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| Glob patterns for ruleId | Simple, familiar syntax for rule filtering |
| YYYY.MM version format | Matches bundle versioning convention |
| Default true for mask.applied with no findings | Conservative - don't fail on clean scans |
| Path allowlist as AND | All findings must be in allowed paths to pass |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Part of secret leak detection implementation |

View File

@@ -0,0 +1,591 @@
# Sprint 20260104_005_AIRGAP - Secret Detection Offline Kit Integration
## Topic & Scope
Integrate secret detection rule bundles with the Offline Kit infrastructure for air-gapped deployments. This sprint ensures complete offline parity for secret leak detection.
**Key deliverables:**
1. **Bundle Distribution**: Include signed bundles in Offline Kit exports
2. **Import Workflow**: Bundle import and verification scripts
3. **Attestor Mirror**: Local verification support without internet
4. **CI/CD Integration**: Automated bundle inclusion in releases
5. **Upgrade Path**: Bundle rotation procedures for offline environments
**Working directory:** `src/AirGap/`, `devops/offline/`, `offline/rules/secrets/`
## Dependencies & Concurrency
- **Depends on**: Sprint 20260104_002 (Core Analyzer), Sprint 20260104_003 (Rule Bundles)
- **Parallel with**: Sprint 20260104_004 (Policy DSL)
- **Blocks**: Production deployment of secret leak detection
## Documentation Prerequisites
- docs/24_OFFLINE_KIT.md
- docs/modules/airgap/airgap-mode.md
- docs/modules/scanner/operations/secret-leak-detection.md
- CLAUDE.md Section 8 (Determinism)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | OKS-001 | TODO | None | AirGap Guild | Update Offline Kit manifest schema for rules |
| 2 | OKS-002 | TODO | OKS-001 | AirGap Guild | Add secrets bundle to BundleBuilder |
| 3 | OKS-003 | TODO | OKS-002 | AirGap Guild | Create bundle verification in Importer |
| 4 | OKS-004 | TODO | None | AirGap Guild | Add Attestor mirror support for bundle verification |
| 5 | OKS-005 | TODO | OKS-003 | AirGap Guild | Create bundle installation script |
| 6 | OKS-006 | TODO | OKS-005 | AirGap Guild | Add bundle rotation/upgrade workflow |
| 7 | OKS-007 | TODO | None | CI/CD Guild | Add bundle to release workflow |
| 8 | OKS-008 | TODO | All | AirGap Guild | Add integration tests for offline flow |
| 9 | OKS-009 | TODO | All | Docs Guild | Update offline kit documentation |
| 10 | OKS-010 | TODO | All | DevOps Guild | Update Helm charts for bundle mounting |
## Task Details
### OKS-001: Manifest Schema Update
Update the Offline Kit manifest to include rule bundles:
```json
{
"version": "2.1",
"created": "2026-01-04T00:00:00Z",
"components": {
"advisory": { ... },
"policy": { ... },
"vex": { ... },
"rules": {
"secrets": {
"bundleId": "secrets.ruleset",
"version": "2026.01",
"path": "rules/secrets/2026.01",
"files": [
{
"name": "secrets.ruleset.manifest.json",
"sha256": "abc123..."
},
{
"name": "secrets.ruleset.rules.jsonl",
"sha256": "def456..."
},
{
"name": "secrets.ruleset.dsse.json",
"sha256": "ghi789..."
}
],
"signature": {
"keyId": "stellaops-secrets-signer",
"verifiedAt": "2026-01-04T00:00:00Z"
}
}
}
}
}
```
Location: `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Schemas/`
### OKS-002: Bundle Builder Extension
Extend BundleBuilder to include secrets rule bundles:
```csharp
public sealed class SnapshotBundleBuilder
{
// Add secrets bundle extraction
public async Task<BundleResult> BuildAsync(
BundleBuildContext context,
CancellationToken ct = default)
{
// ... existing extractors ...
// Add secrets rules extractor
await ExtractSecretsRulesAsync(context, ct);
return result;
}
private async Task ExtractSecretsRulesAsync(
BundleBuildContext context,
CancellationToken ct)
{
var sourcePath = _options.SecretsRulesBundlePath;
if (string.IsNullOrEmpty(sourcePath) || !Directory.Exists(sourcePath))
{
_logger.LogWarning("Secrets rules bundle not found at {Path}", sourcePath);
return;
}
var targetPath = Path.Combine(context.OutputPath, "rules", "secrets");
Directory.CreateDirectory(targetPath);
// Copy bundle files
foreach (var file in Directory.GetFiles(sourcePath, "secrets.ruleset.*"))
{
var targetFile = Path.Combine(targetPath, Path.GetFileName(file));
await CopyWithIntegrityAsync(file, targetFile, ct);
}
// Add to manifest
context.Manifest.Rules["secrets"] = new RuleBundleManifest
{
BundleId = "secrets.ruleset",
Version = await ReadBundleVersionAsync(sourcePath, ct),
Path = "rules/secrets"
};
}
}
```
Location: `src/AirGap/__Libraries/StellaOps.AirGap.Bundle/Services/`
### OKS-003: Importer Verification
Add bundle verification to the Offline Kit importer:
```csharp
public sealed class OfflineKitImporter
{
public async Task<ImportResult> ImportAsync(
string kitPath,
ImportOptions options,
CancellationToken ct = default)
{
// ... existing imports ...
// Import and verify secrets rules
if (manifest.Rules.TryGetValue("secrets", out var secretsBundle))
{
await ImportSecretsRulesAsync(kitPath, secretsBundle, options, ct);
}
return result;
}
private async Task ImportSecretsRulesAsync(
string kitPath,
RuleBundleManifest bundle,
ImportOptions options,
CancellationToken ct)
{
var sourcePath = Path.Combine(kitPath, bundle.Path);
var targetPath = _options.SecretsRulesInstallPath;
// Verify bundle integrity
var verifier = _serviceProvider.GetRequiredService<IBundleVerifier>();
var verification = await verifier.VerifyAsync(sourcePath, new VerificationOptions
{
AttestorUrl = options.AttestorMirrorUrl,
RequireRekorProof = options.RequireRekorProof
}, ct);
if (!verification.IsValid)
{
throw new ImportException($"Secrets bundle verification failed: {string.Join(", ", verification.ValidationErrors ?? [])}");
}
// Install bundle
await InstallBundleAsync(sourcePath, targetPath, ct);
_logger.LogInformation(
"Installed secrets bundle {Version} signed by {KeyId}",
verification.BundleVersion,
verification.SignerKeyId);
}
}
```
### OKS-004: Attestor Mirror Support
Configure Attestor mirror for offline bundle verification:
```csharp
public sealed class OfflineAttestorClient : IAttestorClient
{
private readonly string _mirrorPath;
public OfflineAttestorClient(string mirrorPath)
{
_mirrorPath = mirrorPath;
}
public async Task<VerificationResult> VerifyDsseAsync(
DsseEnvelope envelope,
VerifyOptions options,
CancellationToken ct = default)
{
// Load mirrored certificate chain
var chainPath = Path.Combine(_mirrorPath, "certs", options.KeyId + ".pem");
if (!File.Exists(chainPath))
{
return VerificationResult.Failed($"Certificate not found in mirror: {options.KeyId}");
}
var chain = await LoadCertificateChainAsync(chainPath, ct);
// Verify signature locally
var result = await _dsseVerifier.VerifyAsync(envelope, chain, ct);
// Optionally verify against mirrored Rekor entries
if (options.RequireRekorProof)
{
var rekorPath = Path.Combine(_mirrorPath, "rekor", envelope.PayloadDigest + ".json");
if (!File.Exists(rekorPath))
{
return VerificationResult.Failed("Rekor entry not found in mirror");
}
result = result.WithRekorEntry(await LoadRekorEntryAsync(rekorPath, ct));
}
return result;
}
}
```
### OKS-005: Installation Script
Create bundle installation script for operators:
**`devops/offline/scripts/install-secrets-bundle.sh`:**
```bash
#!/bin/bash
set -euo pipefail
BUNDLE_PATH="${1:-/mnt/offline-kit/rules/secrets}"
INSTALL_PATH="${2:-/opt/stellaops/plugins/scanner/analyzers/secrets}"
ATTESTOR_MIRROR="${3:-/mnt/offline-kit/attestor-mirror}"
echo "Installing secrets bundle from ${BUNDLE_PATH}"
# Verify bundle before installation
export STELLA_ATTESTOR_URL="file://${ATTESTOR_MIRROR}"
if ! stella secrets bundle verify --bundle "${BUNDLE_PATH}" --require-rekor; then
echo "ERROR: Bundle verification failed" >&2
exit 1
fi
# Create installation directory
mkdir -p "${INSTALL_PATH}"
# Install bundle files
cp -v "${BUNDLE_PATH}"/secrets.ruleset.* "${INSTALL_PATH}/"
# Set permissions
chmod 640 "${INSTALL_PATH}"/secrets.ruleset.*
chown stellaops:stellaops "${INSTALL_PATH}"/secrets.ruleset.*
# Verify installation
INSTALLED_VERSION=$(jq -r '.version' "${INSTALL_PATH}/secrets.ruleset.manifest.json")
echo "Successfully installed secrets bundle version ${INSTALLED_VERSION}"
echo ""
echo "To activate, restart Scanner Worker:"
echo " systemctl restart stellaops-scanner-worker"
echo ""
echo "Or with Kubernetes:"
echo " kubectl rollout restart deployment/scanner-worker"
```
### OKS-006: Bundle Rotation Workflow
Document and implement bundle upgrade procedure:
**Upgrade Workflow:**
1. **Pre-upgrade Verification**
```bash
# Verify new bundle
stella secrets bundle verify --bundle /path/to/new-bundle
# Compare with current
CURRENT=$(jq -r '.version' /opt/stellaops/.../secrets.ruleset.manifest.json)
NEW=$(jq -r '.version' /path/to/new-bundle/secrets.ruleset.manifest.json)
echo "Upgrading from ${CURRENT} to ${NEW}"
```
2. **Backup Current Bundle**
```bash
BACKUP_DIR="/opt/stellaops/backups/secrets-bundles/$(date +%Y%m%d)"
mkdir -p "${BACKUP_DIR}"
cp -a /opt/stellaops/plugins/scanner/analyzers/secrets/* "${BACKUP_DIR}/"
```
3. **Install New Bundle**
```bash
./install-secrets-bundle.sh /path/to/new-bundle
```
4. **Rolling Restart**
```bash
# Kubernetes
kubectl rollout restart deployment/scanner-worker --namespace stellaops
# Systemd
systemctl restart stellaops-scanner-worker
```
5. **Verify Upgrade**
```bash
# Check logs for new version
kubectl logs -l app=scanner-worker --tail=100 | grep "SecretsAnalyzerHost"
```
**Rollback Procedure:**
```bash
# Restore backup
cp -a "${BACKUP_DIR}"/* /opt/stellaops/plugins/scanner/analyzers/secrets/
# Restart workers
kubectl rollout restart deployment/scanner-worker
```
### OKS-007: Release Workflow Integration
Add secrets bundle to CI/CD release pipeline:
**`.gitea/workflows/release-offline-kit.yml`:**
```yaml
jobs:
build-offline-kit:
steps:
- name: Build secrets bundle
run: |
stella secrets bundle create \
--output ./offline-kit/rules/secrets/${VERSION} \
--bundle-id secrets.ruleset \
--version ${VERSION} \
--rules ./offline/rules/secrets/sources/*.json \
--sign \
--key-id ${SECRETS_SIGNER_KEY_ID}
- name: Include in offline kit
run: |
# Bundle is automatically included via BundleBuilder
- name: Verify bundle in kit
run: |
stella secrets bundle verify \
--bundle ./offline-kit/rules/secrets/${VERSION}
```
**Add to `.gitea/scripts/build/build-offline-kit.sh`:**
```bash
# Build and sign secrets bundle
echo "Building secrets rule bundle..."
stella secrets bundle create \
--output "${OUTPUT_DIR}/rules/secrets/${BUNDLE_VERSION}" \
--bundle-id secrets.ruleset \
--version "${BUNDLE_VERSION}" \
--rules offline/rules/secrets/sources/*.json \
--sign \
--key-id "${SECRETS_SIGNER_KEY_ID}"
```
### OKS-008: Integration Tests
Create integration tests for offline flow:
```csharp
[Trait("Category", "Integration")]
public class OfflineSecretsIntegrationTests : IClassFixture<OfflineKitFixture>
{
[Fact]
public async Task OfflineKit_IncludesSecretsBundleWithValidSignature()
{
// Arrange
var kit = await _fixture.BuildOfflineKitAsync();
// Act
var bundlePath = Path.Combine(kit.Path, "rules", "secrets");
var verifier = new BundleVerifier(_attestorMirror);
var result = await verifier.VerifyAsync(bundlePath, new VerificationOptions());
// Assert
result.IsValid.Should().BeTrue();
result.BundleVersion.Should().NotBeEmpty();
}
[Fact]
public async Task Importer_InstallsAndVerifiesBundle()
{
// Arrange
var kit = await _fixture.BuildOfflineKitAsync();
var importer = new OfflineKitImporter(_options);
// Act
var result = await importer.ImportAsync(kit.Path, new ImportOptions
{
AttestorMirrorUrl = _attestorMirrorUrl
});
// Assert
result.Success.Should().BeTrue();
Directory.Exists(_installPath).Should().BeTrue();
File.Exists(Path.Combine(_installPath, "secrets.ruleset.manifest.json")).Should().BeTrue();
}
[Fact]
public async Task Scanner_LoadsBundleFromOfflineInstallation()
{
// Arrange
await ImportOfflineKitAsync();
// Act
var host = new SecretsAnalyzerHost(_options, _logger);
await host.StartAsync(CancellationToken.None);
// Assert
host.IsEnabled.Should().BeTrue();
host.BundleVersion.Should().Be(_expectedVersion);
}
}
```
### OKS-009: Documentation Updates
Update `docs/24_OFFLINE_KIT.md`:
```markdown
## Secret Detection Rules
The Offline Kit includes DSSE-signed rule bundles for secret leak detection.
### Bundle Contents
```
offline-kit/
├── rules/
│ └── secrets/
│ └── 2026.01/
│ ├── secrets.ruleset.manifest.json # Rule metadata
│ ├── secrets.ruleset.rules.jsonl # Rule definitions
│ └── secrets.ruleset.dsse.json # DSSE signature
└── attestor-mirror/
├── certs/
│ └── stellaops-secrets-signer.pem # Signing certificate
└── rekor/
└── <digest>.json # Transparency log entry
```
### Installation
1. **Verify Bundle**
```bash
export STELLA_ATTESTOR_URL="file:///mnt/offline-kit/attestor-mirror"
stella secrets bundle verify --bundle /mnt/offline-kit/rules/secrets/2026.01
```
2. **Install Bundle**
```bash
./devops/offline/scripts/install-secrets-bundle.sh \
/mnt/offline-kit/rules/secrets/2026.01 \
/opt/stellaops/plugins/scanner/analyzers/secrets
```
3. **Enable Feature**
```yaml
scanner:
features:
experimental:
secret-leak-detection: true
```
4. **Restart Workers**
```bash
kubectl rollout restart deployment/scanner-worker
```
### Verification
Check that the bundle is loaded:
```bash
kubectl logs -l app=scanner-worker --tail=100 | grep SecretsAnalyzerHost
# Expected: SecretsAnalyzerHost: Loaded bundle 2026.01 signed by stellaops-secrets-signer with N rules
```
### Bundle Rotation
See [Secret Bundle Rotation](./modules/scanner/operations/secrets-bundle-rotation.md) for upgrade procedures.
```
### OKS-010: Helm Chart Updates
Update Helm charts for bundle mounting:
**`devops/helm/stellaops/templates/scanner-worker-deployment.yaml`:**
```yaml
spec:
template:
spec:
volumes:
# Existing volumes...
- name: secrets-rules
{{- if .Values.scanner.secretsRules.persistentVolumeClaim }}
persistentVolumeClaim:
claimName: {{ .Values.scanner.secretsRules.persistentVolumeClaim }}
{{- else }}
configMap:
name: {{ include "stellaops.fullname" . }}-secrets-rules
{{- end }}
containers:
- name: scanner-worker
volumeMounts:
# Existing mounts...
- name: secrets-rules
mountPath: /opt/stellaops/plugins/scanner/analyzers/secrets
readOnly: true
```
**`devops/helm/stellaops/values.yaml`:**
```yaml
scanner:
features:
experimental:
secretLeakDetection: false # Enable via override
secretsRules:
# Use PVC for air-gapped installations
persistentVolumeClaim: ""
# Or use ConfigMap for simple deployments
bundleVersion: "2026.01"
```
## Directory Structure
```
offline/rules/secrets/
├── sources/ # Source rule JSON files (not in kit)
│ ├── aws-access-key.json
│ └── ...
├── 2026.01/ # Signed bundle (in kit)
│ ├── secrets.ruleset.manifest.json
│ ├── secrets.ruleset.rules.jsonl
│ └── secrets.ruleset.dsse.json
└── latest -> 2026.01 # Symlink
devops/offline/
├── scripts/
│ ├── install-secrets-bundle.sh # Installation script
│ └── rotate-secrets-bundle.sh # Rotation script
└── templates/
└── secrets-bundle-pvc.yaml # PVC template for air-gap
```
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| Include Attestor mirror in kit | Enables fully offline verification |
| File:// URL for offline Attestor | Simple, no network required |
| ConfigMap fallback | Simpler for non-air-gapped deployments |
| Symlink for latest | Atomic version switching |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Part of secret leak detection implementation |

View File

@@ -28,17 +28,17 @@ Implement adaptive noise-gating for vulnerability graphs to reduce alert fatigue
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | NG-001 | TODO | None | Guild | Add ProofStrength enum to StellaOps.Evidence.Core |
| 2 | NG-002 | TODO | NG-001 | Guild | Add ProofStrength field to EvidenceRecord |
| 3 | NG-003 | TODO | None | Guild | Create EdgeSemanticKey and deduplication logic in ReachGraph |
| 4 | NG-004 | TODO | None | Guild | Add StabilityDampingGate to Policy.Engine.Gates |
| 5 | NG-005 | TODO | NG-004 | Guild | Add StabilityDampingOptions with configurable thresholds |
| 6 | NG-006 | TODO | None | Guild | Create DeltaSection enum in VexLens |
| 7 | NG-007 | TODO | NG-006 | Guild | Extend VexDelta with section categorization |
| 8 | NG-008 | TODO | NG-001,NG-003,NG-004,NG-006 | Guild | Create INoiseGate interface and NoiseGateService |
| 9 | NG-009 | TODO | NG-008 | Guild | Add DI registration in VexLensServiceCollectionExtensions |
| 10 | NG-010 | TODO | All | Guild | Add unit tests for all new components |
| 11 | NG-011 | TODO | NG-010 | Guild | Update module AGENTS.md files |
| 1 | NG-001 | DONE | None | Guild | Add ProofStrength enum to StellaOps.Evidence.Core |
| 2 | NG-002 | DONE | NG-001 | Guild | Add ProofStrength field to EvidenceRecord |
| 3 | NG-003 | DONE | None | Guild | Create EdgeSemanticKey and deduplication logic in ReachGraph |
| 4 | NG-004 | DONE | None | Guild | Add StabilityDampingGate to Policy.Engine.Gates |
| 5 | NG-005 | DONE | NG-004 | Guild | Add StabilityDampingOptions with configurable thresholds |
| 6 | NG-006 | DONE | None | Guild | Create DeltaSection enum in VexLens |
| 7 | NG-007 | DONE | NG-006 | Guild | Extend VexDelta with section categorization |
| 8 | NG-008 | DONE | NG-001,NG-003,NG-004,NG-006 | Guild | Create INoiseGate interface and NoiseGateService |
| 9 | NG-009 | DONE | NG-008 | Guild | Add DI registration in VexLensServiceCollectionExtensions |
| 10 | NG-010 | DONE | All | Guild | Add unit tests for all new components |
| 11 | NG-011 | DONE | NG-010 | Guild | Update module AGENTS.md files |
## Task Details
@@ -184,4 +184,12 @@ Update module documentation:
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Based on product advisory review |
| 2026-01-04 | NG-001,NG-002 | Created ProofStrength enum, ProofStrengthExtensions, ProofRecord in StellaOps.Evidence.Models |
| 2026-01-04 | NG-003 | Created EdgeSemanticKey, DeduplicatedEdge, EdgeDeduplicator in StellaOps.ReachGraph.Deduplication |
| 2026-01-04 | NG-004,NG-005 | Created StabilityDampingGate, StabilityDampingOptions in StellaOps.Policy.Engine.Gates |
| 2026-01-04 | NG-006,NG-007 | Created DeltaSection, DeltaEntry, DeltaReport, DeltaReportBuilder in StellaOps.VexLens.Delta |
| 2026-01-04 | NG-008,NG-009 | Created INoiseGate, NoiseGateService, NoiseGateOptions; registered DI in VexLensServiceCollectionExtensions |
| 2026-01-04 | NG-010 | Added StabilityDampingGateTests, NoiseGateServiceTests, DeltaReportBuilderTests |
| 2026-01-04 | NG-011 | Updated VexLens and Policy.Engine AGENTS.md files |
| 2026-01-04 | Sprint complete | All 11 tasks DONE |

View File

@@ -0,0 +1,223 @@
# Sprint 20260104_002_FE - Noise-Gating Delta Report UI
## Topic & Scope
Implement frontend components to display noise-gating delta reports from the VexLens backend. This sprint composes existing Angular components to minimize new code while providing a complete UI for:
1. **Delta Report Display**: Show changes between vulnerability graph snapshots
2. **Section-Based Navigation**: Tabs for New, Resolved, ConfidenceUp/Down, PolicyImpact, Damped sections
3. **Gating Statistics**: Edge deduplication rates and verdict damping metrics
4. **Backend API Endpoints**: Expose DeltaReport via VexLens WebService
**Working directories:**
- `src/Web/StellaOps.Web/src/app/` (frontend)
- `src/VexLens/StellaOps.VexLens.WebService/` (backend API)
## Dependencies & Concurrency
- Builds on completed Sprint 20260104_001_BE (backend NoiseGate implementation)
- Reuses existing components: `DeltaSummaryStripComponent`, `TabsComponent`, `GatingExplainerComponent`
- Tasks NG-FE-001 through NG-FE-003 (backend + models) must complete before NG-FE-004+
## Existing Components to Reuse
| Component | Location | Usage |
|-----------|----------|-------|
| `DeltaSummaryStripComponent` | `features/compare/components/` | Overview stats display |
| `TabsComponent` / `TabPanelDirective` | `shared/components/tabs/` | Section navigation |
| `GatingExplainerComponent` | `features/triage/components/gating-explainer/` | Per-entry explanations |
| `DeltaComputeService` patterns | `features/compare/services/` | Signal-based state management |
| `GatingReason`, `DeltaSummary` | `features/triage/models/gating.model.ts` | Existing delta/gating types |
| `VexStatementStatus` | `core/api/vex-hub.models.ts` | VEX status types |
| `BadgeComponent`, `StatCardComponent` | `shared/components/` | Statistics display |
## Documentation Prerequisites
- CLAUDE.md (Section 8: Code Quality rules)
- src/Web/StellaOps.Web/README.md
- docs/modules/vexlens/architecture.md
## Delivery Tracker
| # | Task ID | Status | Key dependency | Task Definition |
|---|---------|--------|----------------|-----------------|
| 1 | NG-FE-001 | DONE | None | Add delta report API endpoints to VexLens.WebService |
| 2 | NG-FE-002 | DONE | None | Create TypeScript models for noise-gating delta (noise-gating.models.ts) |
| 3 | NG-FE-003 | DONE | NG-FE-002 | Create NoiseGatingApiClient service |
| 4 | NG-FE-004 | DONE | NG-FE-003 | Create NoiseGatingSummaryStripComponent (extends DeltaSummaryStrip) |
| 5 | NG-FE-005 | DONE | NG-FE-003 | Create DeltaEntryCardComponent for individual entries |
| 6 | NG-FE-006 | DONE | NG-FE-004,005 | Create NoiseGatingDeltaReportComponent (container with tabs) |
| 7 | NG-FE-007 | DONE | NG-FE-006 | Create GatingStatisticsCardComponent |
| 8 | NG-FE-008 | DONE | NG-FE-006 | Integrate into vuln-explorer/triage workspace |
| 9 | NG-FE-009 | DONE | All | Update feature module exports and routing |
## Task Details
### NG-FE-001: Backend API Endpoints
Add endpoints to `VexLensEndpointExtensions.cs`:
```csharp
// Delta computation
POST /api/v1/vexlens/deltas/compute
Body: { fromSnapshotId, toSnapshotId, options }
Returns: DeltaReportResponse
// Get gated snapshot
GET /api/v1/vexlens/snapshots/{snapshotId}/gated
Returns: GatedGraphSnapshotResponse
// Get gating statistics
GET /api/v1/vexlens/gating/statistics
Query: tenantId, fromDate, toDate
Returns: GatingStatisticsResponse
```
### NG-FE-002: TypeScript Models
Create `src/app/core/api/noise-gating.models.ts`:
```typescript
// Match backend DeltaSection enum
export type NoiseGatingDeltaSection =
| 'new' | 'resolved' | 'confidence_up' | 'confidence_down'
| 'policy_impact' | 'damped' | 'evidence_changed';
// Match backend DeltaEntry
export interface NoiseGatingDeltaEntry {
section: NoiseGatingDeltaSection;
vulnerabilityId: string;
productKey: string;
fromStatus?: VexStatementStatus;
toStatus?: VexStatementStatus;
fromConfidence?: number;
toConfidence?: number;
justification?: string;
rationaleClass?: string;
summary?: string;
contributingSources?: string[];
createdAt: string;
}
// Match backend DeltaReport
export interface NoiseGatingDeltaReport {
reportId: string;
fromSnapshotDigest: string;
toSnapshotDigest: string;
generatedAt: string;
entries: NoiseGatingDeltaEntry[];
summary: NoiseGatingDeltaSummary;
hasActionableChanges: boolean;
}
// Summary counts
export interface NoiseGatingDeltaSummary {
totalCount: number;
newCount: number;
resolvedCount: number;
confidenceUpCount: number;
confidenceDownCount: number;
policyImpactCount: number;
dampedCount: number;
evidenceChangedCount: number;
}
// Gating statistics
export interface GatingStatistics {
originalEdgeCount: number;
deduplicatedEdgeCount: number;
edgeReductionPercent: number;
totalVerdictCount: number;
surfacedVerdictCount: number;
dampedVerdictCount: number;
duration: string;
}
```
### NG-FE-003: API Client
Create `src/app/core/api/noise-gating.client.ts`:
```typescript
@Injectable({ providedIn: 'root' })
export class NoiseGatingApiClient {
// Follow VexHubApiHttpClient patterns
// Signal-based state management
// Caching with Map<string, Observable>
}
```
### NG-FE-004: Summary Strip Component
Extend `DeltaSummaryStripComponent` pattern for noise-gating sections:
- New (green), Resolved (blue), ConfidenceUp (teal), ConfidenceDown (orange)
- PolicyImpact (red), Damped (gray), EvidenceChanged (purple)
### NG-FE-005: Delta Entry Card
Create `delta-entry-card.component.ts`:
- Display CVE ID, package, status transition
- Confidence change visualization (before -> after with delta %)
- Section-specific styling
- Link to GatingExplainerComponent for details
### NG-FE-006: Container Component
Create `noise-gating-delta-report.component.ts`:
- Uses `TabsComponent` with section tabs (badge counts)
- Uses `NoiseGatingSummaryStripComponent` for overview
- Filterable entry list within each tab
- Follows three-pane pattern from compare feature
### NG-FE-007: Statistics Card
Create `gating-statistics-card.component.ts`:
- Edge reduction percentage visualization
- Verdict surfacing/damping ratios
- Processing duration display
- Follows `StatCardComponent` patterns
### NG-FE-008: Triage Integration
Add to vuln-explorer:
- "Delta Report" tab or drawer
- Trigger from snapshot comparison
- Link from finding detail to delta context
### NG-FE-009: Module Exports
Update feature module:
- Export new components
- Add to routing if needed
- Register API client
## Decisions & Risks
| Decision | Rationale |
|----------|-----------|
| Compose existing components | ~70% code reuse, consistent UX |
| Signal-based state | Matches existing Angular 17 patterns |
| Section tabs vs flat list | Better UX for categorized changes |
| Lazy-load delta data | Large reports should not block initial render |
## Execution Log
| Date | Action | Notes |
|------|--------|-------|
| 2026-01-04 | Sprint created | Based on backend noise-gating completion |
| 2026-01-04 | NG-FE-001 | Added endpoints to VexLensEndpointExtensions.cs, created NoiseGatingApiModels.cs |
| 2026-01-04 | NG-FE-001 | Created ISnapshotStore, IGatingStatisticsStore with in-memory implementations |
| 2026-01-04 | NG-FE-001 | Updated INoiseGate.DiffAsync to accept DeltaReportOptions |
| 2026-01-04 | NG-FE-001 | Registered storage services in VexLensServiceCollectionExtensions |
| 2026-01-04 | NG-FE-002 | Created noise-gating.models.ts with all API types and helper functions |
| 2026-01-04 | NG-FE-003 | Created noise-gating.client.ts with signal-based state and caching |
| 2026-01-04 | NG-FE-004 | Created NoiseGatingSummaryStripComponent with section badges |
| 2026-01-04 | NG-FE-005 | Created DeltaEntryCardComponent for individual entries |
| 2026-01-04 | NG-FE-006 | Created NoiseGatingDeltaReportComponent container with tabs |
| 2026-01-04 | NG-FE-007 | Created GatingStatisticsCardComponent with progress bars |
| 2026-01-04 | NG-FE-009 | Created index.ts barrel export for noise-gating components |
| 2026-01-04 | NG-FE-008 | Integrated noise-gating into TriageCanvasComponent with Delta tab |
| 2026-01-04 | NG-FE-008 | Added keyboard shortcut 'd' for delta tab |
| 2026-01-04 | NG-FE-008 | Updated triage components index.ts to export noise-gating components |
| 2026-01-04 | Sprint complete | All 9 tasks completed |

View File

@@ -2,6 +2,8 @@
> **Core Thesis:** Stella Ops isn't a scanner that outputs findings. It's a platform that outputs **attestable decisions that can be replayed**. That difference survives auditors, regulators, and supply-chain propagation.
> **Looking for the complete feature catalog?** See [`full-features-list.md`](full-features-list.md) for the comprehensive list of all platform capabilities, or [`04_FEATURE_MATRIX.md`](04_FEATURE_MATRIX.md) for tier-by-tier availability.
---
## At a Glance

View File

@@ -1,9 +1,25 @@
# Secret Leak Detection (Scanner Operations)
> **Status:** Preview (Sprint132). Requires `SCANNER-ENG-0007`/`POLICY-READINESS-0001` release bundle and the experimental flag `secret-leak-detection`.
> **Status:** PLANNED - Implementation in progress. See implementation sprints below.
>
> **Previous status:** Preview (Sprint132). Requires `SCANNER-ENG-0007`/`POLICY-READINESS-0001` release bundle and the experimental flag `secret-leak-detection`.
>
> **Audience:** Scanner operators, Security Guild, Docs Guild, Offline Kit maintainers.
## Implementation Status
| Component | Status | Sprint |
|-----------|--------|--------|
| `StellaOps.Scanner.Analyzers.Secrets` plugin | NOT IMPLEMENTED | [SPRINT_20260104_002](../../../implplan/SPRINT_20260104_002_SCANNER_secret_leak_detection_core.md) |
| Rule bundle infrastructure | NOT IMPLEMENTED | [SPRINT_20260104_003](../../../implplan/SPRINT_20260104_003_SCANNER_secret_rule_bundles.md) |
| Policy DSL predicates (`secret.*`) | NOT IMPLEMENTED | [SPRINT_20260104_004](../../../implplan/SPRINT_20260104_004_POLICY_secret_dsl_integration.md) |
| Offline Kit integration | NOT IMPLEMENTED | [SPRINT_20260104_005](../../../implplan/SPRINT_20260104_005_AIRGAP_secret_offline_kit.md) |
| Surface.Secrets (credential delivery) | IMPLEMENTED | N/A (already complete) |
**Note:** The remainder of this document describes the TARGET SPECIFICATION for secret leak detection. The feature is not yet available. Surface.Secrets (operational credential management) is fully implemented and separate from secret leak detection.
---
## 1. Scope & goals
- Introduce the **`StellaOps.Scanner.Analyzers.Secrets`** plug-in, which executes deterministic rule bundles against layer content during scans.
@@ -29,29 +45,107 @@ Rule bundles ship in the Export Center / Offline Kit under `offline/rules/secret
| --- | --- | --- |
| `secrets.ruleset.manifest.json` | Lists rule IDs, versions, severity defaults, and hash digests. | Consume during policy drift audits. |
| `secrets.ruleset.rules.jsonl` | Newline-delimited definitions (regex/entropy metadata, masking hints). | Loaded by the analyzer at startup. |
| `secrets.ruleset.dsse.json` | DSSE envelope (Signer certificate chain + Attestor proof). | Verify before distributing bundles. |
| `secrets.ruleset.dsse.json` | DSSE envelope (HMAC-SHA256 signature). | Verify before distributing bundles. |
Verification checklist (`stella excititor verify` talks to the configured Attestor service):
### 3.1 Creating custom bundles
Organizations can create custom rule bundles with additional detection patterns:
```bash
# Create a bundle from rule definition files
stella secrets bundle create \
--output ./bundles/custom-2026.01 \
--bundle-id custom.secrets.ruleset \
--version 2026.01 \
--rules ./custom-rules/*.json
# Create and sign in one step
stella secrets bundle create \
--output ./bundles/custom-2026.01 \
--bundle-id custom.secrets.ruleset \
--version 2026.01 \
--rules ./custom-rules/*.json \
--sign \
--key-id my-org-secrets-signer \
--shared-secret-file /path/to/signing.key
```
Rule definition files must follow the JSON schema:
```json
{
"id": "myorg.secrets.internal-api-key",
"version": "1.0.0",
"name": "Internal API Key",
"description": "Detects internal API keys with MYORG prefix",
"type": "regex",
"pattern": "MYORG_[A-Z0-9]{32}",
"severity": "high",
"confidence": "high",
"keywords": ["MYORG_"],
"filePatterns": ["*.env", "*.yaml", "*.json"],
"enabled": true
}
```
**Rule ID requirements:**
- Must be namespaced (e.g., `myorg.secrets.rule-name`)
- Must start with lowercase letter
- May contain lowercase letters, digits, dots, and hyphens
### 3.2 Verifying bundles
Verify bundle integrity and signature before deployment:
```bash
# Basic verification (checks SHA-256 integrity)
stella secrets bundle verify \
--bundle ./bundles/2026.01
# Full verification with signature check
stella secrets bundle verify \
--bundle ./bundles/2026.01 \
--shared-secret-file /path/to/signing.key
# Verify with trusted key list
stella secrets bundle verify \
--bundle ./bundles/2026.01 \
--shared-secret-file /path/to/signing.key \
--trusted-key-ids stellaops-secrets-signer,my-org-signer
```
For air-gapped environments, use the local verification mode (no network calls):
```bash
stella secrets bundle verify \
--bundle ./bundles/2026.01 \
--shared-secret-file /path/to/signing.key \
--skip-rekor
```
Alternatively, use `stella excititor verify` for Attestor-based verification:
```bash
stella excititor verify \
--attestation offline/rules/secrets/2025.11/secrets.ruleset.dsse.json \
--digest $(sha256sum offline/rules/secrets/2025.11/secrets.ruleset.rules.jsonl | cut -d' ' -f1)
--attestation offline/rules/secrets/2026.01/secrets.ruleset.dsse.json \
--digest $(sha256sum offline/rules/secrets/2026.01/secrets.ruleset.rules.jsonl | cut -d' ' -f1)
```
For air-gapped environments point the CLI at the Offline Kit Attestor mirror (for example `STELLA_ATTESTOR_URL=http://attestor.offline.local`) before running the command. The Attestor instance validates the DSSE envelope against the mirrored Rekor log and embedded certificate chain; no public network access is required.
### 3.3 Deploying bundles
Once verified, copy the manifest + rules to the worker:
Once verified, copy the bundle to the worker:
```
/opt/stellaops/plugins/scanner/analyzers/secrets/
├── secrets.ruleset.manifest.json
├── secrets.ruleset.rules.jsonl
└── secrets.ruleset.dsse.json
|- secrets.ruleset.manifest.json
|- secrets.ruleset.rules.jsonl
|- secrets.ruleset.dsse.json
```
Restart the worker so the analyzer reloads the updated bundle. Bundles are immutable; upgrading requires replacing all three files and restarting.
See [secrets-bundle-rotation.md](./secrets-bundle-rotation.md) for rotation procedures and rollback instructions.
## 4. Enabling the analyzer
1. **Toggle the feature flag** (WebService + Worker):
@@ -161,6 +255,52 @@ rule low_confidence_warn priority 20 {
| No findings despite seeded secrets | Ensure bundle hash matches manifest. Run worker with `--secrets-trace` (debug build) to log matched rules locally. |
| Policy marks findings as unknown | Upgrade tenant policies to include `secret.*` helpers; older policies silently drop the namespace. |
| Air-gapped verification fails | Ensure `STELLA_ATTESTOR_URL` points to the Offline Kit Attestor mirror and rerun `stella excititor verify --attestation <file> --digest <sha256>`. |
| Signature verification failed | Check shared secret matches signing key. Verify DSSE envelope exists and is not corrupted. See signature troubleshooting below. |
| Bundle integrity check failed | Rules file was modified after signing. Re-download bundle or rebuild from sources. |
| Key not in trusted list | Add signer key ID to `--trusted-key-ids` or update `scanner.secrets.trustedKeyIds` configuration. |
### 7.1 Signature verification troubleshooting
**"Signature verification failed" error:**
1. Verify the shared secret is correct:
```bash
# Check secret file exists and is readable
cat /path/to/signing.key | wc -c
# Should output the key length (typically 32-64 bytes for base64-encoded keys)
```
2. Check the DSSE envelope format:
```bash
# Verify envelope is valid JSON
jq . ./bundles/2026.01/secrets.ruleset.dsse.json
```
3. Confirm manifest matches envelope payload:
```bash
# The envelope payload (base64url-decoded) should match the manifest content
# excluding the signatures field
```
**"Rules file integrity check failed" error:**
1. Recompute the SHA-256 hash:
```bash
sha256sum ./bundles/2026.01/secrets.ruleset.rules.jsonl
```
2. Compare with manifest:
```bash
jq -r '.integrity.rulesSha256' ./bundles/2026.01/secrets.ruleset.manifest.json
```
3. If hashes differ, the rules file was modified. Re-download or rebuild the bundle.
**"Bundle is not signed" error:**
The bundle was created without the `--sign` flag. Either:
- Rebuild with signing: `stella secrets bundle create ... --sign --key-id <key>`
- Skip signature verification: `--skip-signature-verification` (not recommended for production)
## 8. References

View File

@@ -0,0 +1,298 @@
# Secret Detection Bundle Rotation
> **Audience:** Scanner operators, Security Guild, Offline Kit maintainers.
>
> **Related:** [secret-leak-detection.md](./secret-leak-detection.md)
## 1. Overview
Secret detection rule bundles are versioned, immutable artifacts that define the patterns used to detect leaked credentials. This document covers the versioning strategy, rotation procedures, and rollback instructions.
## 2. Versioning strategy
Bundles follow CalVer (Calendar Versioning) with the format `YYYY.MM`:
| Version | Release Type | Notes |
|---------|--------------|-------|
| `2026.01` | Monthly release | Standard monthly update |
| `2026.01.1` | Patch release | Critical rule fix within the month |
| `2026.02` | Monthly release | Next scheduled release |
**Version precedence:**
- `2026.02` > `2026.01.1` > `2026.01`
- Patch versions (`YYYY.MM.N`) are only used for critical fixes
- Monthly releases reset the patch counter
**Custom bundles:**
Organizations creating custom bundles should use a prefix to avoid conflicts:
- `myorg.2026.01` for organization-specific bundles
- Or semantic versioning: `1.0.0`, `1.1.0`, etc.
## 3. Release cadence
| Release Type | Frequency | Notification |
|--------------|-----------|--------------|
| Monthly release | First week of each month | Release notes, changelog |
| Patch release | As needed for critical rules | Security advisory |
| Breaking changes | Major version bump | Migration guide |
## 4. Rotation procedures
### 4.1 Downloading the new bundle
```bash
# From the Export Center or Offline Kit
curl -O https://export.stellaops.io/rules/secrets/2026.02/secrets.ruleset.manifest.json
curl -O https://export.stellaops.io/rules/secrets/2026.02/secrets.ruleset.rules.jsonl
curl -O https://export.stellaops.io/rules/secrets/2026.02/secrets.ruleset.dsse.json
```
For air-gapped environments, obtain the bundle from the Offline Kit media.
### 4.2 Verifying the bundle
Always verify bundles before deployment:
```bash
stella secrets bundle verify \
--bundle ./2026.02 \
--shared-secret-file /etc/stellaops/secrets-signing.key \
--trusted-key-ids stellaops-secrets-signer
```
Expected output:
```
Bundle verified successfully.
Bundle ID: secrets.ruleset
Version: 2026.02
Rule count: 18
Enabled rules: 18
Signed by: stellaops-secrets-signer
Signed at: 2026-02-01T00:00:00Z
```
### 4.3 Staged rollout
For production environments, use a staged rollout:
**Stage 1: Canary (1 worker)**
```bash
# Deploy to canary worker
scp -r ./2026.02/* canary-worker:/opt/stellaops/plugins/scanner/analyzers/secrets/
ssh canary-worker 'systemctl restart stellaops-scanner-worker'
# Monitor for 24 hours
# Check logs, metrics, and finding counts
```
**Stage 2: Ring 1 (10% of workers)**
```bash
# Deploy to ring 1 workers
ansible-playbook -l ring1 deploy-secrets-bundle.yml -e bundle_version=2026.02
```
**Stage 3: Full rollout (all workers)**
```bash
# Deploy to all workers
ansible-playbook deploy-secrets-bundle.yml -e bundle_version=2026.02
```
### 4.4 Atomic deployment
For single-worker deployments or when downtime is acceptable:
```bash
# Stop the worker
systemctl stop stellaops-scanner-worker
# Backup current bundle
cp -r /opt/stellaops/plugins/scanner/analyzers/secrets{,.backup}
# Deploy new bundle
cp -r ./2026.02/* /opt/stellaops/plugins/scanner/analyzers/secrets/
# Start the worker
systemctl start stellaops-scanner-worker
# Verify startup
journalctl -u stellaops-scanner-worker | grep SecretsAnalyzerHost
```
### 4.5 Using symlinks (recommended)
For zero-downtime rotations, use the symlink pattern:
```bash
# Directory structure
/opt/stellaops/plugins/scanner/analyzers/secrets/
bundles/
2026.01/
secrets.ruleset.manifest.json
secrets.ruleset.rules.jsonl
secrets.ruleset.dsse.json
2026.02/
secrets.ruleset.manifest.json
secrets.ruleset.rules.jsonl
secrets.ruleset.dsse.json
current -> bundles/2026.02 # Symlink
```
Rotation with symlinks:
```bash
# Deploy new bundle (no restart needed yet)
cp -r ./2026.02 /opt/stellaops/plugins/scanner/analyzers/secrets/bundles/
# Atomic switch
ln -sfn bundles/2026.02 /opt/stellaops/plugins/scanner/analyzers/secrets/current
# Restart worker to pick up new bundle
systemctl restart stellaops-scanner-worker
```
## 5. Rollback procedures
### 5.1 Quick rollback
If issues are detected after deployment:
```bash
# With symlinks (fastest)
ln -sfn bundles/2026.01 /opt/stellaops/plugins/scanner/analyzers/secrets/current
systemctl restart stellaops-scanner-worker
# Without symlinks
cp -r /opt/stellaops/plugins/scanner/analyzers/secrets.backup/* \
/opt/stellaops/plugins/scanner/analyzers/secrets/
systemctl restart stellaops-scanner-worker
```
### 5.2 Identifying rollback triggers
Roll back immediately if you observe:
| Symptom | Likely Cause | Action |
|---------|--------------|--------|
| Worker fails to start | Bundle corruption or invalid rules | Rollback + investigate |
| Finding count drops to zero | All rules disabled or regex errors | Rollback + check manifest |
| Finding count spikes 10x+ | Overly broad new patterns | Rollback + review rules |
| High CPU usage | Catastrophic regex backtracking | Rollback + report to Security Guild |
| Signature verification failures | Key mismatch or tampering | Rollback + verify bundle source |
### 5.3 Post-rollback verification
After rolling back:
```bash
# Verify worker is healthy
systemctl status stellaops-scanner-worker
# Check bundle version in logs
journalctl -u stellaops-scanner-worker | grep "Loaded bundle"
# Verify finding generation (run a test scan)
stella scan --target test-image:latest --secrets-only
```
## 6. Bundle retention
Retain previous bundle versions for rollback capability:
| Environment | Retention |
|-------------|-----------|
| Production | Last 3 versions |
| Staging | Last 2 versions |
| Development | Latest only |
Cleanup script:
```bash
#!/bin/bash
BUNDLE_DIR=/opt/stellaops/plugins/scanner/analyzers/secrets/bundles
KEEP=3
ls -dt ${BUNDLE_DIR}/*/ | tail -n +$((KEEP+1)) | xargs rm -rf
```
## 7. Monitoring rotation
Key metrics to monitor during rotation:
| Metric | Baseline | Alert Threshold |
|--------|----------|-----------------|
| `scanner.secret.finding_total` | Varies | +/- 50% from baseline |
| `scanner.secret.scan_duration_ms` | < 100ms | > 500ms |
| `scanner.secret.bundle_load_errors` | 0 | > 0 |
| Worker restart success | 100% | < 100% |
Prometheus alert example:
```yaml
- alert: SecretBundleRotationAnomaly
expr: |
abs(
sum(rate(scanner_secret_finding_total[5m]))
- sum(rate(scanner_secret_finding_total[5m] offset 1h))
) / sum(rate(scanner_secret_finding_total[5m] offset 1h)) > 0.5
for: 15m
labels:
severity: warning
annotations:
summary: "Secret finding rate changed significantly after bundle rotation"
```
## 8. Air-gapped rotation
For air-gapped environments:
1. **Obtain bundle from secure media:**
```bash
# Mount offline kit media
mount /dev/sr0 /mnt/offline-kit
# Copy bundle
cp -r /mnt/offline-kit/rules/secrets/2026.02 \
/opt/stellaops/plugins/scanner/analyzers/secrets/bundles/
```
2. **Verify with local secret:**
```bash
stella secrets bundle verify \
--bundle /opt/stellaops/plugins/scanner/analyzers/secrets/bundles/2026.02 \
--shared-secret-file /etc/stellaops/offline-signing.key \
--skip-rekor
```
3. **Follow standard rotation procedure (Section 4).**
## 9. Emergency procedures
### 9.1 Disabling secret detection
If secret detection must be disabled entirely:
```bash
# Disable via configuration
echo 'scanner.features.experimental.secret-leak-detection: false' >> /etc/stellaops/scanner.yaml
# Restart worker
systemctl restart stellaops-scanner-worker
```
### 9.2 Emergency rule disable
To disable a specific problematic rule without full rotation:
1. Edit the manifest to set `enabled: false` for the rule
2. This breaks signature verification (expected)
3. Configure worker to skip signature verification temporarily:
```yaml
scanner:
secrets:
skipSignatureVerification: true # TEMPORARY - re-enable after fix
```
4. Restart worker
5. Request emergency patch release from Security Guild
## 10. References
- [secret-leak-detection.md](./secret-leak-detection.md) - Main secret detection documentation
- [SPRINT_20260104_003_SCANNER_secret_rule_bundles.md](../../../implplan/SPRINT_20260104_003_SCANNER_secret_rule_bundles.md) - Implementation sprint
- [dsse-rekor-operator-guide.md](./dsse-rekor-operator-guide.md) - DSSE and Rekor verification

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.aws-access-key",
"version": "1.0.0",
"name": "AWS Access Key ID",
"description": "Detects AWS Access Key IDs which start with AKIA, ASIA, AIDA, AGPA, AROA, AIPA, ANPA, or ANVA",
"type": "regex",
"pattern": "(?:A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}",
"severity": "high",
"confidence": "high",
"keywords": ["AKIA", "ASIA", "AIDA", "AGPA", "AROA", "AIPA", "ANPA", "ANVA", "aws"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.tf", "*.tfvars", "*.config"],
"enabled": true,
"tags": ["aws", "cloud", "credentials"],
"references": [
"https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.aws-secret-key",
"version": "1.0.0",
"name": "AWS Secret Access Key",
"description": "Detects AWS Secret Access Keys (40-character base64 strings near AWS context)",
"type": "regex",
"pattern": "(?i)(?:aws[_-]?secret[_-]?(?:access[_-]?)?key|secret[_-]?key)['\"]?\\s*[:=]\\s*['\"]?([A-Za-z0-9/+=]{40})['\"]?",
"severity": "critical",
"confidence": "high",
"keywords": ["aws_secret", "secret_key", "secret_access_key", "aws"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.tf", "*.tfvars", "*.config", "*.sh", "*.bash"],
"enabled": true,
"tags": ["aws", "cloud", "credentials"],
"references": [
"https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.azure-storage-key",
"version": "1.0.0",
"name": "Azure Storage Account Key",
"description": "Detects Azure Storage Account access keys (base64 encoded, 88 chars)",
"type": "regex",
"pattern": "(?i)(?:AccountKey|azure[_-]?storage[_-]?key)['\"]?\\s*[:=]\\s*['\"]?([A-Za-z0-9+/]{86}==)['\"]?",
"severity": "critical",
"confidence": "high",
"keywords": ["AccountKey", "azure_storage", "DefaultEndpointsProtocol"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config", "*.tf", "*.tfvars", "appsettings.json", "web.config"],
"enabled": true,
"tags": ["azure", "cloud", "credentials", "storage"],
"references": [
"https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage"
]
}

View File

@@ -0,0 +1,16 @@
{
"id": "stellaops.secrets.database-connection-string",
"version": "1.0.0",
"name": "Database Connection String with Credentials",
"description": "Detects database connection strings containing embedded credentials",
"type": "regex",
"pattern": "(?i)(?:postgres|mysql|mongodb|sqlserver|mssql)://[^:]+:[^@]+@[^/]+",
"severity": "critical",
"confidence": "high",
"keywords": ["postgres://", "mysql://", "mongodb://", "sqlserver://", "connection"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config", "appsettings.json", "*.xml"],
"enabled": true,
"allowlistPatterns": ["localhost", "127\\.0\\.0\\.1", "\\$\\{", "\\{\\{"],
"tags": ["database", "credentials", "connection-string"],
"references": []
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.datadog-api-key",
"version": "1.0.0",
"name": "Datadog API Key",
"description": "Detects Datadog API keys",
"type": "regex",
"pattern": "(?i)(?:datadog[_-]?api[_-]?key|DD_API_KEY)['\"]?\\s*[:=]\\s*['\"]?([a-f0-9]{32})['\"]?",
"severity": "high",
"confidence": "medium",
"keywords": ["DD_API_KEY", "datadog_api_key", "datadog"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config"],
"enabled": true,
"tags": ["datadog", "monitoring", "observability", "credentials", "api-key"],
"references": [
"https://docs.datadoghq.com/account_management/api-app-keys/"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.discord-bot-token",
"version": "1.0.0",
"name": "Discord Bot Token",
"description": "Detects Discord bot tokens",
"type": "regex",
"pattern": "[MN][A-Za-z\\d]{23,}\\.[\\w-]{6}\\.[\\w-]{27,}",
"severity": "high",
"confidence": "high",
"keywords": ["discord", "bot", "token"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config", "*.py", "*.js"],
"enabled": true,
"tags": ["discord", "messaging", "bot", "credentials", "token"],
"references": [
"https://discord.com/developers/docs/topics/oauth2"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.docker-hub-token",
"version": "1.0.0",
"name": "Docker Hub Access Token",
"description": "Detects Docker Hub personal access tokens",
"type": "regex",
"pattern": "dckr_pat_[A-Za-z0-9_-]{27}",
"severity": "high",
"confidence": "high",
"keywords": ["dckr_pat_", "docker", "registry"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.sh", ".docker/config.json"],
"enabled": true,
"tags": ["docker", "container", "registry", "credentials", "token"],
"references": [
"https://docs.docker.com/docker-hub/access-tokens/"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.gcp-service-account",
"version": "1.0.0",
"name": "GCP Service Account Key",
"description": "Detects GCP service account JSON key files by their structure",
"type": "regex",
"pattern": "\"type\"\\s*:\\s*\"service_account\"[\\s\\S]{0,500}\"private_key\"\\s*:\\s*\"-----BEGIN",
"severity": "critical",
"confidence": "high",
"keywords": ["service_account", "private_key", "gcp", "google", "client_email"],
"filePatterns": ["*.json"],
"enabled": true,
"tags": ["gcp", "google", "cloud", "credentials", "service-account"],
"references": [
"https://cloud.google.com/iam/docs/keys-create-delete"
]
}

View File

@@ -0,0 +1,15 @@
{
"id": "stellaops.secrets.generic-api-key",
"version": "1.0.0",
"name": "Generic API Key",
"description": "Detects generic API key patterns in configuration",
"type": "regex",
"pattern": "(?i)(?:api[_-]?key|apikey|api[_-]?secret)['\"]?\\s*[:=]\\s*['\"]?([A-Za-z0-9_-]{20,})['\"]?",
"severity": "medium",
"confidence": "low",
"keywords": ["api_key", "apikey", "api-key", "api_secret"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config"],
"enabled": true,
"tags": ["api-key", "credentials", "generic"],
"references": []
}

View File

@@ -0,0 +1,16 @@
{
"id": "stellaops.secrets.generic-password",
"version": "1.0.0",
"name": "Generic Password Assignment",
"description": "Detects hardcoded password assignments in configuration and code",
"type": "regex",
"pattern": "(?i)(?:password|passwd|pwd)['\"]?\\s*[:=]\\s*['\"]([^'\"\\s]{8,})['\"]",
"severity": "high",
"confidence": "low",
"keywords": ["password", "passwd", "pwd"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config", "*.xml"],
"enabled": true,
"allowlistPatterns": ["\\$\\{", "\\{\\{", "%[A-Z_]+%", "\\$env:", "process\\.env"],
"tags": ["password", "credentials", "generic"],
"references": []
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.github-app-token",
"version": "1.0.0",
"name": "GitHub App Installation Token",
"description": "Detects GitHub App installation access tokens",
"type": "regex",
"pattern": "ghs_[A-Za-z0-9_]{36,255}",
"severity": "high",
"confidence": "high",
"keywords": ["ghs_", "github", "app"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.sh", "*.bash"],
"enabled": true,
"tags": ["github", "vcs", "credentials", "token", "app"],
"references": [
"https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.github-pat",
"version": "1.0.0",
"name": "GitHub Personal Access Token",
"description": "Detects GitHub Personal Access Tokens (classic and fine-grained)",
"type": "regex",
"pattern": "(?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9_]{36,255}",
"severity": "critical",
"confidence": "high",
"keywords": ["ghp_", "gho_", "ghu_", "ghs_", "ghr_", "github"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.sh", "*.bash", "*.md", "*.txt"],
"enabled": true,
"tags": ["github", "vcs", "credentials", "token"],
"references": [
"https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.gitlab-pat",
"version": "1.0.0",
"name": "GitLab Personal Access Token",
"description": "Detects GitLab Personal Access Tokens (glpat- prefix)",
"type": "regex",
"pattern": "glpat-[A-Za-z0-9_-]{20,}",
"severity": "critical",
"confidence": "high",
"keywords": ["glpat-", "gitlab"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.sh", "*.bash", ".gitlab-ci.yml"],
"enabled": true,
"tags": ["gitlab", "vcs", "credentials", "token"],
"references": [
"https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.heroku-api-key",
"version": "1.0.0",
"name": "Heroku API Key",
"description": "Detects Heroku API keys",
"type": "regex",
"pattern": "(?i)(?:heroku[_-]?api[_-]?key|HEROKU_API_KEY)['\"]?\\s*[:=]\\s*['\"]?([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})['\"]?",
"severity": "high",
"confidence": "high",
"keywords": ["HEROKU_API_KEY", "heroku_api_key", "heroku"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.sh", "Procfile"],
"enabled": true,
"tags": ["heroku", "paas", "credentials", "api-key"],
"references": [
"https://devcenter.heroku.com/articles/platform-api-quickstart"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.jwt-secret",
"version": "1.0.0",
"name": "JWT Secret Key",
"description": "Detects JWT secret keys in configuration",
"type": "regex",
"pattern": "(?i)(?:jwt[_-]?secret|jwt[_-]?key|secret[_-]?key)['\"]?\\s*[:=]\\s*['\"]?([A-Za-z0-9+/=_-]{32,})['\"]?",
"severity": "high",
"confidence": "medium",
"keywords": ["jwt_secret", "jwt_key", "secret_key", "JWT"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config", "appsettings.json"],
"enabled": true,
"tags": ["jwt", "authentication", "credentials"],
"references": [
"https://jwt.io/introduction"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.mailchimp-api-key",
"version": "1.0.0",
"name": "Mailchimp API Key",
"description": "Detects Mailchimp API keys",
"type": "regex",
"pattern": "[a-f0-9]{32}-us[0-9]{1,2}",
"severity": "medium",
"confidence": "high",
"keywords": ["mailchimp", "-us", "api_key"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config"],
"enabled": true,
"tags": ["mailchimp", "email", "marketing", "credentials", "api-key"],
"references": [
"https://mailchimp.com/help/about-api-keys/"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.npm-token",
"version": "1.0.0",
"name": "NPM Access Token",
"description": "Detects NPM access tokens",
"type": "regex",
"pattern": "npm_[A-Za-z0-9]{36}",
"severity": "high",
"confidence": "high",
"keywords": ["npm_", "npmrc", "_authToken"],
"filePatterns": [".npmrc", "*.yml", "*.yaml", "*.json", "*.env", "*.sh"],
"enabled": true,
"tags": ["npm", "package-manager", "credentials", "token"],
"references": [
"https://docs.npmjs.com/about-access-tokens"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.nuget-api-key",
"version": "1.0.0",
"name": "NuGet API Key",
"description": "Detects NuGet.org API keys",
"type": "regex",
"pattern": "oy2[a-z0-9]{43}",
"severity": "high",
"confidence": "high",
"keywords": ["oy2", "nuget", "NuGetApiKey"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.config", "nuget.config", "*.csproj", "*.ps1"],
"enabled": true,
"tags": ["nuget", "dotnet", "package-manager", "credentials", "api-key"],
"references": [
"https://docs.microsoft.com/en-us/nuget/nuget-org/publish-a-package#create-api-keys"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.private-key-ec",
"version": "1.0.0",
"name": "EC Private Key",
"description": "Detects EC (Elliptic Curve) private keys in PEM format",
"type": "regex",
"pattern": "-----BEGIN EC PRIVATE KEY-----[\\s\\S]{50,}-----END EC PRIVATE KEY-----",
"severity": "critical",
"confidence": "high",
"keywords": ["BEGIN EC PRIVATE KEY", "END EC PRIVATE KEY"],
"filePatterns": ["*.pem", "*.key", "*.txt", "*.env", "*.yml", "*.yaml", "*.json", "*.config"],
"enabled": true,
"tags": ["cryptography", "private-key", "ecdsa"],
"references": [
"https://www.rfc-editor.org/rfc/rfc5915"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.private-key-generic",
"version": "1.0.0",
"name": "Generic Private Key",
"description": "Detects generic private keys in PEM format (PKCS#8)",
"type": "regex",
"pattern": "-----BEGIN PRIVATE KEY-----[\\s\\S]{100,}-----END PRIVATE KEY-----",
"severity": "critical",
"confidence": "high",
"keywords": ["BEGIN PRIVATE KEY", "END PRIVATE KEY"],
"filePatterns": ["*.pem", "*.key", "*.txt", "*.env", "*.yml", "*.yaml", "*.json", "*.config"],
"enabled": true,
"tags": ["cryptography", "private-key", "pkcs8"],
"references": [
"https://www.rfc-editor.org/rfc/rfc5958"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.private-key-openssh",
"version": "1.0.0",
"name": "OpenSSH Private Key",
"description": "Detects OpenSSH private keys (newer format)",
"type": "regex",
"pattern": "-----BEGIN OPENSSH PRIVATE KEY-----[\\s\\S]{50,}-----END OPENSSH PRIVATE KEY-----",
"severity": "critical",
"confidence": "high",
"keywords": ["BEGIN OPENSSH PRIVATE KEY", "END OPENSSH PRIVATE KEY"],
"filePatterns": ["*.pem", "*.key", "id_rsa", "id_ed25519", "id_ecdsa", "*.txt"],
"enabled": true,
"tags": ["cryptography", "private-key", "ssh", "openssh"],
"references": [
"https://man.openbsd.org/ssh-keygen"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.private-key-rsa",
"version": "1.0.0",
"name": "RSA Private Key",
"description": "Detects RSA private keys in PEM format",
"type": "regex",
"pattern": "-----BEGIN RSA PRIVATE KEY-----[\\s\\S]{100,}-----END RSA PRIVATE KEY-----",
"severity": "critical",
"confidence": "high",
"keywords": ["BEGIN RSA PRIVATE KEY", "END RSA PRIVATE KEY"],
"filePatterns": ["*.pem", "*.key", "*.txt", "*.env", "*.yml", "*.yaml", "*.json", "*.config"],
"enabled": true,
"tags": ["cryptography", "private-key", "rsa"],
"references": [
"https://www.rfc-editor.org/rfc/rfc7468"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.pypi-token",
"version": "1.0.0",
"name": "PyPI API Token",
"description": "Detects PyPI API tokens",
"type": "regex",
"pattern": "pypi-AgEIcHlwaS5vcmc[A-Za-z0-9_-]{50,}",
"severity": "high",
"confidence": "high",
"keywords": ["pypi-", "pypi.org"],
"filePatterns": [".pypirc", "*.yml", "*.yaml", "*.json", "*.env", "*.sh", "*.toml"],
"enabled": true,
"tags": ["pypi", "python", "package-manager", "credentials", "token"],
"references": [
"https://pypi.org/help/#apitoken"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.sendgrid-api-key",
"version": "1.0.0",
"name": "SendGrid API Key",
"description": "Detects SendGrid API keys",
"type": "regex",
"pattern": "SG\\.[A-Za-z0-9_-]{22}\\.[A-Za-z0-9_-]{43}",
"severity": "high",
"confidence": "high",
"keywords": ["SG.", "sendgrid"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config"],
"enabled": true,
"tags": ["sendgrid", "email", "credentials", "api-key"],
"references": [
"https://docs.sendgrid.com/ui/account-and-settings/api-keys"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.slack-token",
"version": "1.0.0",
"name": "Slack Token",
"description": "Detects Slack Bot, User, and Webhook tokens",
"type": "regex",
"pattern": "xox[baprs]-[0-9]{10,13}-[0-9]{10,13}[a-zA-Z0-9-]*",
"severity": "high",
"confidence": "high",
"keywords": ["xoxb-", "xoxa-", "xoxp-", "xoxr-", "xoxs-", "slack"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.sh", "*.bash", "*.config"],
"enabled": true,
"tags": ["slack", "messaging", "credentials", "token"],
"references": [
"https://api.slack.com/authentication/token-types"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.slack-webhook",
"version": "1.0.0",
"name": "Slack Webhook URL",
"description": "Detects Slack incoming webhook URLs",
"type": "regex",
"pattern": "https://hooks\\.slack\\.com/services/T[A-Z0-9]{8,}/B[A-Z0-9]{8,}/[A-Za-z0-9]{24}",
"severity": "medium",
"confidence": "high",
"keywords": ["hooks.slack.com", "webhook", "slack"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.sh", "*.bash", "*.config", "*.md"],
"enabled": true,
"tags": ["slack", "messaging", "webhook"],
"references": [
"https://api.slack.com/messaging/webhooks"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.stripe-restricted-key",
"version": "1.0.0",
"name": "Stripe Restricted Key",
"description": "Detects Stripe restricted API keys",
"type": "regex",
"pattern": "rk_(?:live|test)_[A-Za-z0-9]{24,}",
"severity": "high",
"confidence": "high",
"keywords": ["rk_live_", "rk_test_", "stripe"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config", "*.js", "*.ts", "*.py", "*.rb"],
"enabled": true,
"tags": ["stripe", "payment", "credentials", "api-key"],
"references": [
"https://stripe.com/docs/keys#limit-access"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.stripe-secret-key",
"version": "1.0.0",
"name": "Stripe Secret Key",
"description": "Detects Stripe secret API keys (live and test)",
"type": "regex",
"pattern": "sk_(?:live|test)_[A-Za-z0-9]{24,}",
"severity": "critical",
"confidence": "high",
"keywords": ["sk_live_", "sk_test_", "stripe"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config", "*.js", "*.ts", "*.py", "*.rb"],
"enabled": true,
"tags": ["stripe", "payment", "credentials", "api-key"],
"references": [
"https://stripe.com/docs/keys"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.telegram-bot-token",
"version": "1.0.0",
"name": "Telegram Bot Token",
"description": "Detects Telegram Bot API tokens",
"type": "regex",
"pattern": "[0-9]{8,10}:[A-Za-z0-9_-]{35}",
"severity": "high",
"confidence": "medium",
"keywords": ["telegram", "bot", "api.telegram.org"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config", "*.py", "*.js"],
"enabled": true,
"tags": ["telegram", "messaging", "bot", "credentials", "token"],
"references": [
"https://core.telegram.org/bots/api#authorizing-your-bot"
]
}

View File

@@ -0,0 +1,17 @@
{
"id": "stellaops.secrets.twilio-api-key",
"version": "1.0.0",
"name": "Twilio API Key",
"description": "Detects Twilio API Key SIDs and Auth Tokens",
"type": "regex",
"pattern": "(?:SK[a-f0-9]{32}|AC[a-f0-9]{32})",
"severity": "high",
"confidence": "high",
"keywords": ["SK", "AC", "twilio", "TWILIO"],
"filePatterns": ["*.yml", "*.yaml", "*.json", "*.env", "*.properties", "*.config"],
"enabled": true,
"tags": ["twilio", "messaging", "sms", "credentials", "api-key"],
"references": [
"https://www.twilio.com/docs/iam/keys/api-key"
]
}

View File

@@ -33,6 +33,9 @@ internal static class BinaryCommandGroup
binary.Add(BuildLookupCommand(services, verboseOption, cancellationToken));
binary.Add(BuildFingerprintCommand(services, verboseOption, cancellationToken));
// Sprint: SPRINT_20260104_001_CLI - Binary call graph digest extraction
binary.Add(BuildCallGraphCommand(services, verboseOption, cancellationToken));
return binary;
}
@@ -188,6 +191,70 @@ internal static class BinaryCommandGroup
return command;
}
// CALLGRAPH-01: stella binary callgraph
private static Command BuildCallGraphCommand(
IServiceProvider services,
Option<bool> verboseOption,
CancellationToken cancellationToken)
{
var fileArg = new Argument<string>("file")
{
Description = "Path to binary file to analyze."
};
var formatOption = new Option<string>("--format", ["-f"])
{
Description = "Output format: digest (default), json, summary."
}.SetDefaultValue("digest").FromAmong("digest", "json", "summary");
var outputOption = new Option<string?>("--output", ["-o"])
{
Description = "Output file path (default: stdout)."
};
var emitSbomOption = new Option<string?>("--emit-sbom")
{
Description = "Path to SBOM file to inject callgraph digest as property."
};
var scanIdOption = new Option<string?>("--scan-id")
{
Description = "Scan ID for graph metadata (default: auto-generated)."
};
var command = new Command("callgraph", "Extract call graph and compute deterministic digest.")
{
fileArg,
formatOption,
outputOption,
emitSbomOption,
scanIdOption,
verboseOption
};
command.SetAction(parseResult =>
{
var file = parseResult.GetValue(fileArg)!;
var format = parseResult.GetValue(formatOption)!;
var output = parseResult.GetValue(outputOption);
var emitSbom = parseResult.GetValue(emitSbomOption);
var scanId = parseResult.GetValue(scanIdOption);
var verbose = parseResult.GetValue(verboseOption);
return BinaryCommandHandlers.HandleCallGraphAsync(
services,
file,
format,
output,
emitSbom,
scanId,
verbose,
cancellationToken);
});
return command;
}
private static Command BuildSubmitCommand(
IServiceProvider services,
Option<bool> verboseOption,

View File

@@ -5,10 +5,14 @@
// Description: Command handlers for binary reachability operations.
// -----------------------------------------------------------------------------
using System.Globalization;
using System.Text.Json;
using System.Text.Json.Nodes;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Spectre.Console;
using StellaOps.Scanner.CallGraph;
using StellaOps.Scanner.CallGraph.Binary;
namespace StellaOps.Cli.Commands.Binary;
@@ -632,6 +636,238 @@ internal static class BinaryCommandHandlers
}
}
/// <summary>
/// Handle 'stella binary callgraph' command (CALLGRAPH-01).
/// Extracts call graph from native binary and computes deterministic SHA-256 digest.
/// </summary>
public static async Task<int> HandleCallGraphAsync(
IServiceProvider services,
string filePath,
string format,
string? outputPath,
string? emitSbomPath,
string? scanId,
bool verbose,
CancellationToken cancellationToken)
{
var loggerFactory = services.GetRequiredService<ILoggerFactory>();
var logger = loggerFactory.CreateLogger("binary-callgraph");
try
{
if (!File.Exists(filePath))
{
AnsiConsole.MarkupLine($"[red]Error:[/] File not found: {filePath}");
return ExitCodes.FileNotFound;
}
// Resolve scan ID (auto-generate if not provided)
var effectiveScanId = scanId ?? $"cli-{Path.GetFileName(filePath)}-{DateTime.UtcNow:yyyyMMddHHmmss}";
CallGraphSnapshot snapshot = null!;
await AnsiConsole.Status()
.StartAsync("Extracting binary call graph...", async ctx =>
{
ctx.Status("Loading binary...");
// Get the binary call graph extractor
var timeProvider = services.GetService<TimeProvider>() ?? TimeProvider.System;
var extractorLogger = loggerFactory.CreateLogger<BinaryCallGraphExtractor>();
var extractor = new BinaryCallGraphExtractor(extractorLogger, timeProvider);
ctx.Status("Analyzing symbols and relocations...");
var request = new CallGraphExtractionRequest(
ScanId: effectiveScanId,
Language: "native",
TargetPath: filePath);
snapshot = await extractor.ExtractAsync(request, cancellationToken);
ctx.Status("Computing digest...");
});
// Format output based on requested format
string output;
switch (format)
{
case "digest":
output = snapshot.GraphDigest;
break;
case "json":
output = JsonSerializer.Serialize(new
{
scanId = snapshot.ScanId,
graphDigest = snapshot.GraphDigest,
language = snapshot.Language,
extractedAt = snapshot.ExtractedAt.ToString("O", CultureInfo.InvariantCulture),
nodeCount = snapshot.Nodes.Length,
edgeCount = snapshot.Edges.Length,
entrypointCount = snapshot.EntrypointIds.Length,
nodes = snapshot.Nodes,
edges = snapshot.Edges,
entrypointIds = snapshot.EntrypointIds
}, JsonOptions);
break;
case "summary":
default:
output = string.Join(Environment.NewLine,
[
$"GraphDigest: {snapshot.GraphDigest}",
$"ScanId: {snapshot.ScanId}",
$"Language: {snapshot.Language}",
$"ExtractedAt: {snapshot.ExtractedAt:O}",
$"Nodes: {snapshot.Nodes.Length}",
$"Edges: {snapshot.Edges.Length}",
$"Entrypoints: {snapshot.EntrypointIds.Length}"
]);
break;
}
// Write output
if (!string.IsNullOrWhiteSpace(outputPath))
{
await File.WriteAllTextAsync(outputPath, output, cancellationToken);
AnsiConsole.MarkupLine($"[green]Output written to:[/] {outputPath}");
}
else if (format == "digest")
{
// For digest-only format, just output the digest
Console.WriteLine(output);
}
else
{
AnsiConsole.WriteLine(output);
}
// Inject into SBOM if requested
if (!string.IsNullOrWhiteSpace(emitSbomPath))
{
var sbomResult = await InjectCallGraphDigestIntoSbomAsync(
emitSbomPath,
filePath,
snapshot,
cancellationToken);
if (sbomResult == 0)
{
AnsiConsole.MarkupLine($"[green]Callgraph digest injected into SBOM:[/] {emitSbomPath}");
}
else
{
AnsiConsole.MarkupLine($"[yellow]Warning:[/] Failed to inject digest into SBOM");
}
}
if (verbose)
{
logger.LogInformation(
"Extracted call graph for {Path}: {Nodes} nodes, {Edges} edges, digest={Digest}",
filePath,
snapshot.Nodes.Length,
snapshot.Edges.Length,
snapshot.GraphDigest);
}
return ExitCodes.Success;
}
catch (NotSupportedException ex)
{
AnsiConsole.MarkupLine($"[red]Error:[/] {ex.Message}");
logger.LogError(ex, "Unsupported binary format for {Path}", filePath);
return ExitCodes.InvalidArguments;
}
catch (Exception ex)
{
AnsiConsole.MarkupLine($"[red]Error:[/] {ex.Message}");
logger.LogError(ex, "Failed to extract call graph for {Path}", filePath);
return ExitCodes.GeneralError;
}
}
/// <summary>
/// Injects callgraph digest as a property into a CycloneDX SBOM.
/// </summary>
private static async Task<int> InjectCallGraphDigestIntoSbomAsync(
string sbomPath,
string binaryPath,
CallGraphSnapshot snapshot,
CancellationToken cancellationToken)
{
if (!File.Exists(sbomPath))
{
return 1;
}
try
{
var sbomJson = await File.ReadAllTextAsync(sbomPath, cancellationToken);
var doc = JsonNode.Parse(sbomJson) as JsonObject;
if (doc == null)
{
return 1;
}
// Ensure metadata.properties exists
var metadata = doc["metadata"] as JsonObject;
if (metadata == null)
{
metadata = new JsonObject();
doc["metadata"] = metadata;
}
var properties = metadata["properties"] as JsonArray;
if (properties == null)
{
properties = new JsonArray();
metadata["properties"] = properties;
}
var binaryName = Path.GetFileName(binaryPath);
// Add callgraph properties using stellaops namespace
properties.Add(new JsonObject
{
["name"] = $"stellaops:callgraph:digest:{binaryName}",
["value"] = snapshot.GraphDigest
});
properties.Add(new JsonObject
{
["name"] = $"stellaops:callgraph:nodeCount:{binaryName}",
["value"] = snapshot.Nodes.Length.ToString(CultureInfo.InvariantCulture)
});
properties.Add(new JsonObject
{
["name"] = $"stellaops:callgraph:edgeCount:{binaryName}",
["value"] = snapshot.Edges.Length.ToString(CultureInfo.InvariantCulture)
});
properties.Add(new JsonObject
{
["name"] = $"stellaops:callgraph:entrypointCount:{binaryName}",
["value"] = snapshot.EntrypointIds.Length.ToString(CultureInfo.InvariantCulture)
});
properties.Add(new JsonObject
{
["name"] = $"stellaops:callgraph:extractedAt:{binaryName}",
["value"] = snapshot.ExtractedAt.ToString("O", CultureInfo.InvariantCulture)
});
// Write updated SBOM
var updatedJson = doc.ToJsonString(new JsonSerializerOptions { WriteIndented = true });
await File.WriteAllTextAsync(sbomPath, updatedJson, cancellationToken);
return 0;
}
catch
{
return 1;
}
}
private static string DetectFormat(byte[] header)
{
// ELF magic: 0x7f 'E' 'L' 'F'

View File

@@ -0,0 +1,429 @@
// -----------------------------------------------------------------------------
// CommandHandlers.Secrets.cs
// Sprint: SPRINT_20260104_003_SCANNER (Secret Detection Rule Bundles)
// Tasks: RB-006, RB-007 - Command handlers for secrets bundle operations.
// Description: Implements bundle create, verify, and info commands.
// -----------------------------------------------------------------------------
using System.Text.Json;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Spectre.Console;
using StellaOps.Scanner.Analyzers.Secrets.Bundles;
namespace StellaOps.Cli.Commands;
internal static partial class CommandHandlers
{
private static readonly JsonSerializerOptions SecretsJsonOptions = new()
{
WriteIndented = true,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};
internal static async Task<int> HandleSecretsBundleCreateAsync(
IServiceProvider services,
string sources,
string output,
string id,
string? version,
bool sign,
string? keyId,
string? secret,
string? secretFile,
string format,
bool verbose,
CancellationToken cancellationToken)
{
var logger = services.GetService<ILoggerFactory>()?.CreateLogger("Secrets.Bundle.Create");
if (!Directory.Exists(sources))
{
AnsiConsole.MarkupLine($"[red]Error: Source directory not found: {Markup.Escape(sources)}[/]");
return 1;
}
// Determine version (CalVer if not specified)
var bundleVersion = version ?? DateTimeOffset.UtcNow.ToString("yyyy.MM", System.Globalization.CultureInfo.InvariantCulture);
if (format == "text")
{
AnsiConsole.MarkupLine("[blue]Creating secrets detection rule bundle...[/]");
AnsiConsole.MarkupLine($" Source: [bold]{Markup.Escape(sources)}[/]");
AnsiConsole.MarkupLine($" Output: [bold]{Markup.Escape(output)}[/]");
AnsiConsole.MarkupLine($" ID: [bold]{Markup.Escape(id)}[/]");
AnsiConsole.MarkupLine($" Version: [bold]{Markup.Escape(bundleVersion)}[/]");
}
try
{
// Create output directory
Directory.CreateDirectory(output);
// Build the bundle
var builderLogger = services.GetService<ILoggerFactory>()?.CreateLogger<BundleBuilder>();
var validatorLogger = services.GetService<ILoggerFactory>()?.CreateLogger<RuleValidator>();
var validator = new RuleValidator(validatorLogger ?? Microsoft.Extensions.Logging.Abstractions.NullLogger<RuleValidator>.Instance);
var builder = new BundleBuilder(
validator,
builderLogger ?? Microsoft.Extensions.Logging.Abstractions.NullLogger<BundleBuilder>.Instance);
var buildOptions = new BundleBuildOptions
{
SourceDirectory = sources,
OutputDirectory = output,
BundleId = id,
Version = bundleVersion,
TimeProvider = TimeProvider.System
};
var artifact = await builder.BuildAsync(buildOptions, cancellationToken);
if (format == "text")
{
AnsiConsole.MarkupLine($"[green]Bundle created successfully![/]");
AnsiConsole.MarkupLine($" Manifest: [bold]{Markup.Escape(artifact.ManifestPath)}[/]");
AnsiConsole.MarkupLine($" Rules: [bold]{Markup.Escape(artifact.RulesPath)}[/]");
AnsiConsole.MarkupLine($" Total rules: [bold]{artifact.TotalRules}[/]");
AnsiConsole.MarkupLine($" Enabled rules: [bold]{artifact.EnabledRules}[/]");
AnsiConsole.MarkupLine($" SHA-256: [bold]{artifact.RulesSha256}[/]");
}
// Sign if requested
if (sign)
{
if (string.IsNullOrWhiteSpace(keyId))
{
AnsiConsole.MarkupLine("[red]Error: --key-id is required for signing[/]");
return 1;
}
if (string.IsNullOrWhiteSpace(secret) && string.IsNullOrWhiteSpace(secretFile))
{
AnsiConsole.MarkupLine("[red]Error: --secret or --secret-file is required for signing[/]");
return 1;
}
if (format == "text")
{
AnsiConsole.MarkupLine("[blue]Signing bundle...[/]");
}
var signerLogger = services.GetService<ILoggerFactory>()?.CreateLogger<BundleSigner>();
var signer = new BundleSigner(signerLogger ?? Microsoft.Extensions.Logging.Abstractions.NullLogger<BundleSigner>.Instance);
var signOptions = new BundleSigningOptions
{
KeyId = keyId,
SharedSecret = secret,
SharedSecretFile = secretFile,
TimeProvider = TimeProvider.System
};
var signResult = await signer.SignAsync(artifact, signOptions, cancellationToken);
if (format == "text")
{
AnsiConsole.MarkupLine($"[green]Bundle signed successfully![/]");
AnsiConsole.MarkupLine($" Envelope: [bold]{Markup.Escape(signResult.EnvelopePath)}[/]");
AnsiConsole.MarkupLine($" Key ID: [bold]{Markup.Escape(keyId)}[/]");
}
}
if (format == "json")
{
var result = new
{
success = true,
bundle = new
{
id,
version = bundleVersion,
manifestPath = artifact.ManifestPath,
rulesPath = artifact.RulesPath,
totalRules = artifact.TotalRules,
enabledRules = artifact.EnabledRules,
rulesSha256 = artifact.RulesSha256,
signed = sign
}
};
Console.WriteLine(JsonSerializer.Serialize(result, SecretsJsonOptions));
}
return 0;
}
catch (Exception ex)
{
if (format == "text")
{
AnsiConsole.MarkupLine($"[red]Error: {Markup.Escape(ex.Message)}[/]");
if (verbose)
{
AnsiConsole.WriteException(ex);
}
}
else
{
var result = new { success = false, error = ex.Message };
Console.WriteLine(JsonSerializer.Serialize(result, SecretsJsonOptions));
}
return 1;
}
}
internal static async Task<int> HandleSecretsBundleVerifyAsync(
IServiceProvider services,
string bundle,
string? secret,
string? secretFile,
string[] trustedKeys,
bool requireSignature,
string format,
bool verbose,
CancellationToken cancellationToken)
{
if (!Directory.Exists(bundle))
{
if (format == "text")
{
AnsiConsole.MarkupLine($"[red]Error: Bundle directory not found: {Markup.Escape(bundle)}[/]");
}
else
{
var result = new { success = false, error = $"Bundle directory not found: {bundle}" };
Console.WriteLine(JsonSerializer.Serialize(result, SecretsJsonOptions));
}
return 1;
}
if (format == "text")
{
AnsiConsole.MarkupLine("[blue]Verifying secrets detection rule bundle...[/]");
AnsiConsole.MarkupLine($" Bundle: [bold]{Markup.Escape(bundle)}[/]");
}
try
{
var verifierLogger = services.GetService<ILoggerFactory>()?.CreateLogger<BundleVerifier>();
var verifier = new BundleVerifier(verifierLogger ?? Microsoft.Extensions.Logging.Abstractions.NullLogger<BundleVerifier>.Instance);
var verifyOptions = new BundleVerificationOptions
{
SharedSecret = secret,
SharedSecretFile = secretFile,
TrustedKeyIds = trustedKeys.Length > 0 ? trustedKeys : null,
SkipSignatureVerification = !requireSignature && string.IsNullOrWhiteSpace(secret) && string.IsNullOrWhiteSpace(secretFile),
VerifyIntegrity = true
};
var result = await verifier.VerifyAsync(bundle, verifyOptions, cancellationToken);
if (format == "text")
{
if (result.IsValid)
{
AnsiConsole.MarkupLine("[green]Bundle verification passed![/]");
AnsiConsole.MarkupLine($" Bundle ID: [bold]{Markup.Escape(result.BundleId ?? "unknown")}[/]");
AnsiConsole.MarkupLine($" Version: [bold]{Markup.Escape(result.BundleVersion ?? "unknown")}[/]");
AnsiConsole.MarkupLine($" Rules: [bold]{result.RuleCount ?? 0}[/]");
if (result.SignerKeyId is not null)
{
AnsiConsole.MarkupLine($" Signed by: [bold]{Markup.Escape(result.SignerKeyId)}[/]");
}
if (result.SignedAt.HasValue)
{
AnsiConsole.MarkupLine($" Signed at: [bold]{result.SignedAt.Value:O}[/]");
}
foreach (var warning in result.ValidationWarnings)
{
AnsiConsole.MarkupLine($"[yellow]Warning: {Markup.Escape(warning)}[/]");
}
}
else
{
AnsiConsole.MarkupLine("[red]Bundle verification failed![/]");
foreach (var error in result.ValidationErrors)
{
AnsiConsole.MarkupLine($"[red] - {Markup.Escape(error)}[/]");
}
}
}
else
{
var jsonResult = new
{
success = result.IsValid,
bundleId = result.BundleId,
version = result.BundleVersion,
ruleCount = result.RuleCount,
signerKeyId = result.SignerKeyId,
signedAt = result.SignedAt?.ToString("O"),
errors = result.ValidationErrors,
warnings = result.ValidationWarnings
};
Console.WriteLine(JsonSerializer.Serialize(jsonResult, SecretsJsonOptions));
}
return result.IsValid ? 0 : 1;
}
catch (Exception ex)
{
if (format == "text")
{
AnsiConsole.MarkupLine($"[red]Error: {Markup.Escape(ex.Message)}[/]");
if (verbose)
{
AnsiConsole.WriteException(ex);
}
}
else
{
var result = new { success = false, error = ex.Message };
Console.WriteLine(JsonSerializer.Serialize(result, SecretsJsonOptions));
}
return 1;
}
}
internal static async Task<int> HandleSecretsBundleInfoAsync(
IServiceProvider services,
string bundle,
string format,
bool verbose,
CancellationToken cancellationToken)
{
var manifestPath = Path.Combine(bundle, "secrets.ruleset.manifest.json");
if (!File.Exists(manifestPath))
{
if (format == "text")
{
AnsiConsole.MarkupLine($"[red]Error: Bundle manifest not found: {Markup.Escape(manifestPath)}[/]");
}
else
{
var result = new { success = false, error = $"Bundle manifest not found: {manifestPath}" };
Console.WriteLine(JsonSerializer.Serialize(result, SecretsJsonOptions));
}
return 1;
}
try
{
var manifestJson = await File.ReadAllTextAsync(manifestPath, cancellationToken);
var manifest = JsonSerializer.Deserialize<BundleManifest>(manifestJson, new JsonSerializerOptions
{
PropertyNameCaseInsensitive = true
});
if (manifest is null)
{
if (format == "text")
{
AnsiConsole.MarkupLine("[red]Error: Failed to parse bundle manifest[/]");
}
else
{
var result = new { success = false, error = "Failed to parse bundle manifest" };
Console.WriteLine(JsonSerializer.Serialize(result, SecretsJsonOptions));
}
return 1;
}
if (format == "text")
{
AnsiConsole.MarkupLine("[blue]Bundle Information[/]");
AnsiConsole.MarkupLine($" ID: [bold]{Markup.Escape(manifest.Id)}[/]");
AnsiConsole.MarkupLine($" Version: [bold]{Markup.Escape(manifest.Version)}[/]");
AnsiConsole.MarkupLine($" Schema: [bold]{Markup.Escape(manifest.SchemaVersion)}[/]");
AnsiConsole.MarkupLine($" Created: [bold]{manifest.CreatedAt:O}[/]");
if (!string.IsNullOrWhiteSpace(manifest.Description))
{
AnsiConsole.MarkupLine($" Description: [bold]{Markup.Escape(manifest.Description)}[/]");
}
AnsiConsole.MarkupLine("");
AnsiConsole.MarkupLine("[blue]Integrity[/]");
AnsiConsole.MarkupLine($" Rules file: [bold]{Markup.Escape(manifest.Integrity.RulesFile)}[/]");
AnsiConsole.MarkupLine($" SHA-256: [bold]{Markup.Escape(manifest.Integrity.RulesSha256)}[/]");
AnsiConsole.MarkupLine($" Total rules: [bold]{manifest.Integrity.TotalRules}[/]");
AnsiConsole.MarkupLine($" Enabled rules: [bold]{manifest.Integrity.EnabledRules}[/]");
if (manifest.Signatures is not null)
{
AnsiConsole.MarkupLine("");
AnsiConsole.MarkupLine("[blue]Signature[/]");
AnsiConsole.MarkupLine($" Envelope: [bold]{Markup.Escape(manifest.Signatures.DsseEnvelope)}[/]");
AnsiConsole.MarkupLine($" Key ID: [bold]{Markup.Escape(manifest.Signatures.KeyId ?? "unknown")}[/]");
if (manifest.Signatures.SignedAt.HasValue)
{
AnsiConsole.MarkupLine($" Signed at: [bold]{manifest.Signatures.SignedAt.Value:O}[/]");
}
if (!string.IsNullOrWhiteSpace(manifest.Signatures.RekorLogId))
{
AnsiConsole.MarkupLine($" Rekor log ID: [bold]{Markup.Escape(manifest.Signatures.RekorLogId)}[/]");
}
}
else
{
AnsiConsole.MarkupLine("");
AnsiConsole.MarkupLine("[yellow]Bundle is not signed[/]");
}
if (verbose && manifest.Rules.Length > 0)
{
AnsiConsole.MarkupLine("");
AnsiConsole.MarkupLine("[blue]Rules Summary[/]");
var table = new Table();
table.AddColumn("ID");
table.AddColumn("Version");
table.AddColumn("Severity");
table.AddColumn("Enabled");
foreach (var rule in manifest.Rules.Take(20))
{
table.AddRow(
Markup.Escape(rule.Id),
Markup.Escape(rule.Version),
Markup.Escape(rule.Severity),
rule.Enabled ? "[green]Yes[/]" : "[red]No[/]");
}
if (manifest.Rules.Length > 20)
{
table.AddRow($"... and {manifest.Rules.Length - 20} more", "", "", "");
}
AnsiConsole.Write(table);
}
}
else
{
Console.WriteLine(JsonSerializer.Serialize(manifest, SecretsJsonOptions));
}
return 0;
}
catch (Exception ex)
{
if (format == "text")
{
AnsiConsole.MarkupLine($"[red]Error: {Markup.Escape(ex.Message)}[/]");
if (verbose)
{
AnsiConsole.WriteException(ex);
}
}
else
{
var result = new { success = false, error = ex.Message };
Console.WriteLine(JsonSerializer.Serialize(result, SecretsJsonOptions));
}
return 1;
}
}
}

View File

@@ -0,0 +1,247 @@
// -----------------------------------------------------------------------------
// SecretsCommandGroup.cs
// Sprint: SPRINT_20260104_003_SCANNER (Secret Detection Rule Bundles)
// Tasks: RB-006, RB-007 - CLI commands for secrets rule bundle management.
// Description: CLI commands for building, signing, and verifying secrets bundles.
// -----------------------------------------------------------------------------
using System.CommandLine;
using StellaOps.Cli.Extensions;
namespace StellaOps.Cli.Commands;
internal static class SecretsCommandGroup
{
internal static Command BuildSecretsCommand(
IServiceProvider services,
Option<bool> verboseOption,
CancellationToken cancellationToken)
{
var secrets = new Command("secrets", "Secrets detection rule bundle management.");
secrets.Add(BuildBundleCommand(services, verboseOption, cancellationToken));
return secrets;
}
private static Command BuildBundleCommand(
IServiceProvider services,
Option<bool> verboseOption,
CancellationToken cancellationToken)
{
var bundle = new Command("bundle", "Secrets rule bundle operations.");
bundle.Add(BuildCreateCommand(services, verboseOption, cancellationToken));
bundle.Add(BuildVerifyCommand(services, verboseOption, cancellationToken));
bundle.Add(BuildInfoCommand(services, verboseOption, cancellationToken));
return bundle;
}
private static Command BuildCreateCommand(
IServiceProvider services,
Option<bool> verboseOption,
CancellationToken cancellationToken)
{
var sourcesArg = new Argument<string>("sources")
{
Description = "Path to directory containing rule JSON files"
};
var outputOption = new Option<string>("--output", "-o")
{
Description = "Output directory for the bundle (default: ./bundle)"
};
outputOption.SetDefaultValue("./bundle");
var idOption = new Option<string>("--id")
{
Description = "Bundle identifier (default: stellaops-secrets)"
};
idOption.SetDefaultValue("stellaops-secrets");
var versionOption = new Option<string>("--version", "-v")
{
Description = "Bundle version (CalVer, e.g., 2026.01)"
};
var signOption = new Option<bool>("--sign")
{
Description = "Sign the bundle after creation"
};
var keyIdOption = new Option<string?>("--key-id")
{
Description = "Key identifier for signing"
};
var secretOption = new Option<string?>("--secret")
{
Description = "Shared secret for HMAC signing (base64 or hex)"
};
var secretFileOption = new Option<string?>("--secret-file")
{
Description = "Path to file containing the shared secret"
};
var formatOption = new Option<string>("--format")
{
Description = "Output format: text, json"
}.SetDefaultValue("text").FromAmong("text", "json");
var command = new Command("create", "Create a secrets detection rule bundle from JSON rule definitions.")
{
sourcesArg,
outputOption,
idOption,
versionOption,
signOption,
keyIdOption,
secretOption,
secretFileOption,
formatOption,
verboseOption
};
command.SetAction(parseResult =>
{
var sources = parseResult.GetValue(sourcesArg) ?? string.Empty;
var output = parseResult.GetValue(outputOption) ?? "./bundle";
var id = parseResult.GetValue(idOption) ?? "stellaops-secrets";
var version = parseResult.GetValue(versionOption);
var sign = parseResult.GetValue(signOption);
var keyId = parseResult.GetValue(keyIdOption);
var secret = parseResult.GetValue(secretOption);
var secretFile = parseResult.GetValue(secretFileOption);
var format = parseResult.GetValue(formatOption) ?? "text";
var verbose = parseResult.GetValue(verboseOption);
return CommandHandlers.HandleSecretsBundleCreateAsync(
services,
sources,
output,
id,
version,
sign,
keyId,
secret,
secretFile,
format,
verbose,
cancellationToken);
});
return command;
}
private static Command BuildVerifyCommand(
IServiceProvider services,
Option<bool> verboseOption,
CancellationToken cancellationToken)
{
var bundleArg = new Argument<string>("bundle")
{
Description = "Path to the bundle directory"
};
var secretOption = new Option<string?>("--secret")
{
Description = "Shared secret for HMAC verification (base64 or hex)"
};
var secretFileOption = new Option<string?>("--secret-file")
{
Description = "Path to file containing the shared secret"
};
var trustedKeysOption = new Option<string[]>("--trusted-keys")
{
Description = "List of trusted key IDs for signature verification"
};
var requireSignatureOption = new Option<bool>("--require-signature")
{
Description = "Require a valid signature (fail if unsigned)"
};
var formatOption = new Option<string>("--format")
{
Description = "Output format: text, json"
}.SetDefaultValue("text").FromAmong("text", "json");
var command = new Command("verify", "Verify a secrets detection rule bundle's integrity and signature.")
{
bundleArg,
secretOption,
secretFileOption,
trustedKeysOption,
requireSignatureOption,
formatOption,
verboseOption
};
command.SetAction(parseResult =>
{
var bundle = parseResult.GetValue(bundleArg) ?? string.Empty;
var secret = parseResult.GetValue(secretOption);
var secretFile = parseResult.GetValue(secretFileOption);
var trustedKeys = parseResult.GetValue(trustedKeysOption) ?? Array.Empty<string>();
var requireSignature = parseResult.GetValue(requireSignatureOption);
var format = parseResult.GetValue(formatOption) ?? "text";
var verbose = parseResult.GetValue(verboseOption);
return CommandHandlers.HandleSecretsBundleVerifyAsync(
services,
bundle,
secret,
secretFile,
trustedKeys,
requireSignature,
format,
verbose,
cancellationToken);
});
return command;
}
private static Command BuildInfoCommand(
IServiceProvider services,
Option<bool> verboseOption,
CancellationToken cancellationToken)
{
var bundleArg = new Argument<string>("bundle")
{
Description = "Path to the bundle directory"
};
var formatOption = new Option<string>("--format")
{
Description = "Output format: text, json"
}.SetDefaultValue("text").FromAmong("text", "json");
var command = new Command("info", "Display information about a secrets detection rule bundle.")
{
bundleArg,
formatOption,
verboseOption
};
command.SetAction(parseResult =>
{
var bundle = parseResult.GetValue(bundleArg) ?? string.Empty;
var format = parseResult.GetValue(formatOption) ?? "text";
var verbose = parseResult.GetValue(verboseOption);
return CommandHandlers.HandleSecretsBundleInfoAsync(
services,
bundle,
format,
verbose,
cancellationToken);
});
return command;
}
}

View File

@@ -90,6 +90,10 @@
<ProjectReference Include="../../BinaryIndex/__Libraries/StellaOps.BinaryIndex.Disassembly/StellaOps.BinaryIndex.Disassembly.csproj" />
<ProjectReference Include="../../BinaryIndex/__Libraries/StellaOps.BinaryIndex.Normalization/StellaOps.BinaryIndex.Normalization.csproj" />
<ProjectReference Include="../../BinaryIndex/__Libraries/StellaOps.BinaryIndex.DeltaSig/StellaOps.BinaryIndex.DeltaSig.csproj" />
<!-- Binary Call Graph (SPRINT_20260104_001_CLI) -->
<ProjectReference Include="../../Scanner/__Libraries/StellaOps.Scanner.CallGraph/StellaOps.Scanner.CallGraph.csproj" />
<!-- Secrets Bundle CLI (SPRINT_20260104_003_SCANNER) -->
<ProjectReference Include="../../Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/StellaOps.Scanner.Analyzers.Secrets.csproj" />
</ItemGroup>
<!-- GOST Crypto Plugins (Russia distribution) -->

View File

@@ -9,6 +9,10 @@ Stand up the Policy Engine runtime host that evaluates organization policies aga
- Change stream listeners and scheduler integration for incremental re-evaluation.
- Authority integration enforcing new `policy:*` and `effective:write` scopes.
- Observability: metrics, traces, structured logs, trace sampling.
- **StabilityDampingGate** (Sprint NG-001): Hysteresis-based damping to prevent flip-flopping verdicts:
- Suppresses rapid status oscillations requiring min duration or significant confidence delta
- Upgrades (more severe) bypass damping; downgrades are dampable
- Integrates with VexLens NoiseGate for unified noise-gating
## Expectations
- Keep endpoints deterministic, cancellation-aware, and tenant-scoped.

View File

@@ -0,0 +1,384 @@
// Licensed to StellaOps under the AGPL-3.0-or-later license.
using System.Collections.Concurrent;
using System.Globalization;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace StellaOps.Policy.Engine.Gates;
/// <summary>
/// Gate that applies hysteresis-based stability damping to prevent flip-flopping verdicts.
/// </summary>
/// <remarks>
/// This gate tracks verdict state transitions and suppresses rapid oscillations by requiring:
/// - A minimum duration before a state change is surfaced, OR
/// - A significant confidence delta that justifies immediate surfacing
///
/// This reduces alert fatigue from noisy or unstable feed data while still ensuring
/// significant changes (especially upgrades to more severe states) surface promptly.
/// </remarks>
public interface IStabilityDampingGate
{
/// <summary>
/// Evaluates whether a verdict transition should be surfaced or damped.
/// </summary>
/// <param name="request">The damping evaluation request.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>The damping decision.</returns>
Task<StabilityDampingDecision> EvaluateAsync(
StabilityDampingRequest request,
CancellationToken cancellationToken = default);
/// <summary>
/// Records a verdict state for future damping calculations.
/// </summary>
/// <param name="key">The unique key for this verdict (e.g., "artifact:cve").</param>
/// <param name="state">The verdict state to record.</param>
/// <param name="cancellationToken">Cancellation token.</param>
Task RecordStateAsync(
string key,
VerdictState state,
CancellationToken cancellationToken = default);
/// <summary>
/// Prunes old verdict history records.
/// </summary>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>Number of records pruned.</returns>
Task<int> PruneHistoryAsync(CancellationToken cancellationToken = default);
}
/// <summary>
/// Request for stability damping evaluation.
/// </summary>
public sealed record StabilityDampingRequest
{
/// <summary>
/// Gets the unique key for this verdict (e.g., "artifact:cve" or "sha256:vuln_id").
/// </summary>
public required string Key { get; init; }
/// <summary>
/// Gets the current (proposed) verdict state.
/// </summary>
public required VerdictState ProposedState { get; init; }
/// <summary>
/// Gets the tenant ID for multi-tenant deployments.
/// </summary>
public string? TenantId { get; init; }
}
/// <summary>
/// Represents a verdict state at a point in time.
/// </summary>
public sealed record VerdictState
{
/// <summary>
/// Gets the VEX status (affected, not_affected, fixed, under_investigation).
/// </summary>
public required string Status { get; init; }
/// <summary>
/// Gets the confidence score (0.0 to 1.0).
/// </summary>
public required double Confidence { get; init; }
/// <summary>
/// Gets the timestamp of this state.
/// </summary>
public required DateTimeOffset Timestamp { get; init; }
/// <summary>
/// Gets the rationale class (e.g., "authoritative", "binary", "static").
/// </summary>
public string? RationaleClass { get; init; }
/// <summary>
/// Gets the source that produced this state.
/// </summary>
public string? SourceId { get; init; }
}
/// <summary>
/// Decision from stability damping evaluation.
/// </summary>
public sealed record StabilityDampingDecision
{
/// <summary>
/// Gets whether the transition should be surfaced.
/// </summary>
public required bool ShouldSurface { get; init; }
/// <summary>
/// Gets the reason for the decision.
/// </summary>
public required string Reason { get; init; }
/// <summary>
/// Gets the previous state, if any.
/// </summary>
public VerdictState? PreviousState { get; init; }
/// <summary>
/// Gets how long the previous state has persisted.
/// </summary>
public TimeSpan? StateDuration { get; init; }
/// <summary>
/// Gets the confidence delta from previous to current.
/// </summary>
public double? ConfidenceDelta { get; init; }
/// <summary>
/// Gets whether this is a status upgrade (more severe).
/// </summary>
public bool? IsUpgrade { get; init; }
/// <summary>
/// Gets the timestamp of the decision.
/// </summary>
public required DateTimeOffset DecidedAt { get; init; }
}
/// <summary>
/// Default implementation of <see cref="IStabilityDampingGate"/>.
/// </summary>
public sealed class StabilityDampingGate : IStabilityDampingGate
{
private readonly IOptionsMonitor<StabilityDampingOptions> _options;
private readonly TimeProvider _timeProvider;
private readonly ILogger<StabilityDampingGate> _logger;
private readonly ConcurrentDictionary<string, VerdictState> _stateHistory = new();
// Status severity ordering: higher = more severe
private static readonly Dictionary<string, int> StatusSeverity = new(StringComparer.OrdinalIgnoreCase)
{
["affected"] = 100,
["under_investigation"] = 50,
["fixed"] = 25,
["not_affected"] = 0
};
/// <summary>
/// Initializes a new instance of the <see cref="StabilityDampingGate"/> class.
/// </summary>
public StabilityDampingGate(
IOptionsMonitor<StabilityDampingOptions> options,
TimeProvider timeProvider,
ILogger<StabilityDampingGate> logger)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
_timeProvider = timeProvider ?? throw new ArgumentNullException(nameof(timeProvider));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
/// <inheritdoc/>
public Task<StabilityDampingDecision> EvaluateAsync(
StabilityDampingRequest request,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(request);
var opts = _options.CurrentValue;
var now = _timeProvider.GetUtcNow();
var key = BuildKey(request.TenantId, request.Key);
// If disabled, always surface
if (!opts.Enabled)
{
return Task.FromResult(new StabilityDampingDecision
{
ShouldSurface = true,
Reason = "Stability damping disabled",
DecidedAt = now
});
}
// Check if status is subject to damping
if (!opts.DampedStatuses.Contains(request.ProposedState.Status))
{
return Task.FromResult(new StabilityDampingDecision
{
ShouldSurface = true,
Reason = string.Create(CultureInfo.InvariantCulture,
$"Status '{request.ProposedState.Status}' is not subject to damping"),
DecidedAt = now
});
}
// Get previous state
if (!_stateHistory.TryGetValue(key, out var previousState))
{
// No history - this is a new verdict, surface it
return Task.FromResult(new StabilityDampingDecision
{
ShouldSurface = true,
Reason = "No previous state (new verdict)",
DecidedAt = now
});
}
// Check if status actually changed
if (string.Equals(previousState.Status, request.ProposedState.Status, StringComparison.OrdinalIgnoreCase))
{
// Same status - check confidence delta
var confidenceDelta = Math.Abs(request.ProposedState.Confidence - previousState.Confidence);
if (confidenceDelta >= opts.MinConfidenceDeltaPercent)
{
return Task.FromResult(new StabilityDampingDecision
{
ShouldSurface = true,
Reason = string.Create(CultureInfo.InvariantCulture,
$"Confidence changed by {confidenceDelta:P1} (threshold: {opts.MinConfidenceDeltaPercent:P1})"),
PreviousState = previousState,
ConfidenceDelta = confidenceDelta,
DecidedAt = now
});
}
// No significant change
return Task.FromResult(new StabilityDampingDecision
{
ShouldSurface = false,
Reason = string.Create(CultureInfo.InvariantCulture,
$"Same status, confidence delta {confidenceDelta:P1} below threshold"),
PreviousState = previousState,
ConfidenceDelta = confidenceDelta,
DecidedAt = now
});
}
// Status changed - check if it's an upgrade or downgrade
var isUpgrade = IsStatusUpgrade(previousState.Status, request.ProposedState.Status);
var stateDuration = now - previousState.Timestamp;
// Upgrades (more severe) bypass damping if configured
if (isUpgrade && opts.OnlyDampDowngrades)
{
return Task.FromResult(new StabilityDampingDecision
{
ShouldSurface = true,
Reason = string.Create(CultureInfo.InvariantCulture,
$"Status upgrade ({previousState.Status} -> {request.ProposedState.Status}) surfaces immediately"),
PreviousState = previousState,
StateDuration = stateDuration,
IsUpgrade = true,
DecidedAt = now
});
}
// Check confidence delta for immediate surfacing
var delta = Math.Abs(request.ProposedState.Confidence - previousState.Confidence);
if (delta >= opts.MinConfidenceDeltaPercent)
{
return Task.FromResult(new StabilityDampingDecision
{
ShouldSurface = true,
Reason = string.Create(CultureInfo.InvariantCulture,
$"Confidence delta {delta:P1} exceeds threshold {opts.MinConfidenceDeltaPercent:P1}"),
PreviousState = previousState,
StateDuration = stateDuration,
ConfidenceDelta = delta,
IsUpgrade = isUpgrade,
DecidedAt = now
});
}
// Check duration requirement
if (stateDuration >= opts.MinDurationBeforeChange)
{
return Task.FromResult(new StabilityDampingDecision
{
ShouldSurface = true,
Reason = string.Create(CultureInfo.InvariantCulture,
$"Previous state persisted for {stateDuration.TotalHours:F1}h (threshold: {opts.MinDurationBeforeChange.TotalHours:F1}h)"),
PreviousState = previousState,
StateDuration = stateDuration,
IsUpgrade = isUpgrade,
DecidedAt = now
});
}
// Damped - don't surface yet
var remainingTime = opts.MinDurationBeforeChange - stateDuration;
if (opts.LogDampedTransitions)
{
_logger.LogDebug(
"Damped transition for {Key}: {OldStatus}->{NewStatus}, remaining: {Remaining}",
request.Key,
previousState.Status,
request.ProposedState.Status,
remainingTime);
}
return Task.FromResult(new StabilityDampingDecision
{
ShouldSurface = false,
Reason = string.Create(CultureInfo.InvariantCulture,
$"Damped: state duration {stateDuration.TotalHours:F1}h < {opts.MinDurationBeforeChange.TotalHours:F1}h, " +
$"delta {delta:P1} < {opts.MinConfidenceDeltaPercent:P1}"),
PreviousState = previousState,
StateDuration = stateDuration,
ConfidenceDelta = delta,
IsUpgrade = isUpgrade,
DecidedAt = now
});
}
/// <inheritdoc/>
public Task RecordStateAsync(
string key,
VerdictState state,
CancellationToken cancellationToken = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(key);
ArgumentNullException.ThrowIfNull(state);
_stateHistory[key] = state;
return Task.CompletedTask;
}
/// <inheritdoc/>
public Task<int> PruneHistoryAsync(CancellationToken cancellationToken = default)
{
var opts = _options.CurrentValue;
var cutoff = _timeProvider.GetUtcNow() - opts.HistoryRetention;
var pruned = 0;
foreach (var kvp in _stateHistory)
{
if (kvp.Value.Timestamp < cutoff)
{
if (_stateHistory.TryRemove(kvp.Key, out _))
{
pruned++;
}
}
}
if (pruned > 0)
{
_logger.LogInformation("Pruned {Count} stale verdict state records", pruned);
}
return Task.FromResult(pruned);
}
private static string BuildKey(string? tenantId, string verdictKey)
{
return string.IsNullOrEmpty(tenantId)
? verdictKey
: $"{tenantId}:{verdictKey}";
}
private static bool IsStatusUpgrade(string oldStatus, string newStatus)
{
var oldSeverity = StatusSeverity.GetValueOrDefault(oldStatus, 50);
var newSeverity = StatusSeverity.GetValueOrDefault(newStatus, 50);
return newSeverity > oldSeverity;
}
}

View File

@@ -0,0 +1,81 @@
// Licensed to StellaOps under the AGPL-3.0-or-later license.
using System.ComponentModel.DataAnnotations;
namespace StellaOps.Policy.Engine.Gates;
/// <summary>
/// Configuration options for the stability damping gate.
/// </summary>
/// <remarks>
/// Stability damping prevents flip-flopping verdicts by requiring that:
/// - A verdict must persist for a minimum duration before a change is surfaced, OR
/// - The confidence delta must exceed a minimum threshold
///
/// This reduces notification noise from unstable feed data while still allowing
/// significant changes to surface quickly.
/// </remarks>
public sealed class StabilityDampingOptions
{
/// <summary>
/// Gets or sets whether stability damping is enabled.
/// </summary>
public bool Enabled { get; set; } = true;
/// <summary>
/// Gets or sets the minimum duration a verdict must persist before
/// a change is surfaced, unless the confidence delta exceeds the threshold.
/// </summary>
/// <remarks>
/// Default: 4 hours. Set to TimeSpan.Zero to disable duration-based damping.
/// </remarks>
[Range(typeof(TimeSpan), "00:00:00", "7.00:00:00",
ErrorMessage = "MinDurationBeforeChange must be between 0 and 7 days.")]
public TimeSpan MinDurationBeforeChange { get; set; } = TimeSpan.FromHours(4);
/// <summary>
/// Gets or sets the minimum confidence change percentage required to
/// bypass the duration requirement and surface a change immediately.
/// </summary>
/// <remarks>
/// Default: 15%. A change from 0.70 to 0.85 (15%) would bypass duration damping.
/// Set to 1.0 (100%) to effectively disable delta-based bypass.
/// </remarks>
[Range(0.0, 1.0, ErrorMessage = "MinConfidenceDeltaPercent must be between 0 and 1.")]
public double MinConfidenceDeltaPercent { get; set; } = 0.15;
/// <summary>
/// Gets or sets the VEX statuses to which damping applies.
/// </summary>
/// <remarks>
/// By default, damping applies to affected and not_affected transitions.
/// Transitions to/from under_investigation are typically not damped.
/// </remarks>
public HashSet<string> DampedStatuses { get; set; } =
[
"affected",
"not_affected",
"fixed"
];
/// <summary>
/// Gets or sets whether to apply damping only to downgrade transitions
/// (e.g., affected -> not_affected).
/// </summary>
/// <remarks>
/// When true, upgrades (not_affected -> affected) are surfaced immediately.
/// This is conservative: users are alerted to new risks without delay.
/// </remarks>
public bool OnlyDampDowngrades { get; set; } = true;
/// <summary>
/// Gets or sets the retention period for verdict history.
/// Older records are pruned to prevent unbounded growth.
/// </summary>
public TimeSpan HistoryRetention { get; set; } = TimeSpan.FromDays(30);
/// <summary>
/// Gets or sets whether to log damped transitions for debugging.
/// </summary>
public bool LogDampedTransitions { get; set; } = false;
}

View File

@@ -0,0 +1,347 @@
// Licensed to StellaOps under the AGPL-3.0-or-later license.
using FluentAssertions;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using Microsoft.Extensions.Time.Testing;
using StellaOps.Policy.Engine.Gates;
using Xunit;
namespace StellaOps.Policy.Engine.Tests.Gates;
/// <summary>
/// Unit tests for <see cref="StabilityDampingGate"/>.
/// </summary>
[Trait("Category", "Unit")]
public class StabilityDampingGateTests
{
private readonly FakeTimeProvider _timeProvider;
private readonly StabilityDampingOptions _defaultOptions;
public StabilityDampingGateTests()
{
_timeProvider = new FakeTimeProvider(new DateTimeOffset(2026, 1, 4, 12, 0, 0, TimeSpan.Zero));
_defaultOptions = new StabilityDampingOptions
{
Enabled = true,
MinDurationBeforeChange = TimeSpan.FromHours(4),
MinConfidenceDeltaPercent = 0.15,
OnlyDampDowngrades = true,
DampedStatuses = ["affected", "not_affected", "fixed", "under_investigation"]
};
}
private StabilityDampingGate CreateGate(StabilityDampingOptions? options = null)
{
var opts = options ?? _defaultOptions;
var optionsMonitor = new TestOptionsMonitor<StabilityDampingOptions>(opts);
return new StabilityDampingGate(optionsMonitor, _timeProvider, NullLogger<StabilityDampingGate>.Instance);
}
[Fact]
public async Task EvaluateAsync_NewVerdict_ShouldSurface()
{
// Arrange
var gate = CreateGate();
var request = new StabilityDampingRequest
{
Key = "artifact:CVE-2024-1234",
ProposedState = new VerdictState
{
Status = "affected",
Confidence = 0.85,
Timestamp = _timeProvider.GetUtcNow()
}
};
// Act
var decision = await gate.EvaluateAsync(request);
// Assert
decision.ShouldSurface.Should().BeTrue();
decision.Reason.Should().Contain("new verdict");
}
[Fact]
public async Task EvaluateAsync_WhenDisabled_ShouldAlwaysSurface()
{
// Arrange
var options = new StabilityDampingOptions { Enabled = false };
var gate = CreateGate(options);
var request = new StabilityDampingRequest
{
Key = "artifact:CVE-2024-1234",
ProposedState = new VerdictState
{
Status = "affected",
Confidence = 0.85,
Timestamp = _timeProvider.GetUtcNow()
}
};
// Act
var decision = await gate.EvaluateAsync(request);
// Assert
decision.ShouldSurface.Should().BeTrue();
decision.Reason.Should().Contain("disabled");
}
[Fact]
public async Task EvaluateAsync_StatusUpgrade_ShouldSurfaceImmediately()
{
// Arrange
var gate = CreateGate();
var key = "artifact:CVE-2024-1234";
// Record initial state as not_affected
await gate.RecordStateAsync(key, new VerdictState
{
Status = "not_affected",
Confidence = 0.80,
Timestamp = _timeProvider.GetUtcNow()
});
// Request upgrade to affected (more severe)
var request = new StabilityDampingRequest
{
Key = key,
ProposedState = new VerdictState
{
Status = "affected",
Confidence = 0.85,
Timestamp = _timeProvider.GetUtcNow()
}
};
// Act
var decision = await gate.EvaluateAsync(request);
// Assert
decision.ShouldSurface.Should().BeTrue();
decision.IsUpgrade.Should().BeTrue();
decision.Reason.Should().Contain("upgrade");
}
[Fact]
public async Task EvaluateAsync_StatusDowngrade_WithoutMinDuration_ShouldDamp()
{
// Arrange
var gate = CreateGate();
var key = "artifact:CVE-2024-1234";
// Record initial state as affected
await gate.RecordStateAsync(key, new VerdictState
{
Status = "affected",
Confidence = 0.80,
Timestamp = _timeProvider.GetUtcNow()
});
// Advance time but not enough to meet threshold
_timeProvider.Advance(TimeSpan.FromHours(2));
// Request downgrade to not_affected (less severe)
var request = new StabilityDampingRequest
{
Key = key,
ProposedState = new VerdictState
{
Status = "not_affected",
Confidence = 0.75,
Timestamp = _timeProvider.GetUtcNow()
}
};
// Act
var decision = await gate.EvaluateAsync(request);
// Assert
decision.ShouldSurface.Should().BeFalse();
decision.Reason.Should().Contain("Damped");
}
[Fact]
public async Task EvaluateAsync_StatusDowngrade_AfterMinDuration_ShouldSurface()
{
// Arrange
var gate = CreateGate();
var key = "artifact:CVE-2024-1234";
// Record initial state as affected
await gate.RecordStateAsync(key, new VerdictState
{
Status = "affected",
Confidence = 0.80,
Timestamp = _timeProvider.GetUtcNow()
});
// Advance time past threshold
_timeProvider.Advance(TimeSpan.FromHours(5));
// Request downgrade to not_affected
var request = new StabilityDampingRequest
{
Key = key,
ProposedState = new VerdictState
{
Status = "not_affected",
Confidence = 0.75,
Timestamp = _timeProvider.GetUtcNow()
}
};
// Act
var decision = await gate.EvaluateAsync(request);
// Assert
decision.ShouldSurface.Should().BeTrue();
decision.StateDuration.Should().BeGreaterThan(TimeSpan.FromHours(4));
}
[Fact]
public async Task EvaluateAsync_LargeConfidenceDelta_ShouldSurfaceImmediately()
{
// Arrange
var gate = CreateGate();
var key = "artifact:CVE-2024-1234";
// Record initial state
await gate.RecordStateAsync(key, new VerdictState
{
Status = "affected",
Confidence = 0.50,
Timestamp = _timeProvider.GetUtcNow()
});
// Request with large confidence change (>15%)
var request = new StabilityDampingRequest
{
Key = key,
ProposedState = new VerdictState
{
Status = "affected",
Confidence = 0.90,
Timestamp = _timeProvider.GetUtcNow()
}
};
// Act
var decision = await gate.EvaluateAsync(request);
// Assert
decision.ShouldSurface.Should().BeTrue();
decision.ConfidenceDelta.Should().BeGreaterThan(0.15);
}
[Fact]
public async Task EvaluateAsync_SmallConfidenceDelta_SameStatus_ShouldDamp()
{
// Arrange
var gate = CreateGate();
var key = "artifact:CVE-2024-1234";
// Record initial state
await gate.RecordStateAsync(key, new VerdictState
{
Status = "affected",
Confidence = 0.80,
Timestamp = _timeProvider.GetUtcNow()
});
// Request with small confidence change (<15%)
var request = new StabilityDampingRequest
{
Key = key,
ProposedState = new VerdictState
{
Status = "affected",
Confidence = 0.85,
Timestamp = _timeProvider.GetUtcNow()
}
};
// Act
var decision = await gate.EvaluateAsync(request);
// Assert
decision.ShouldSurface.Should().BeFalse();
decision.ConfidenceDelta.Should().BeLessThan(0.15);
}
[Fact]
public async Task PruneHistoryAsync_ShouldRemoveStaleRecords()
{
// Arrange
var gate = CreateGate();
// Record old state
await gate.RecordStateAsync("old-key", new VerdictState
{
Status = "affected",
Confidence = 0.80,
Timestamp = _timeProvider.GetUtcNow()
});
// Advance time past retention period
_timeProvider.Advance(TimeSpan.FromDays(8)); // Default retention is 7 days
// Record new state (to ensure we have something current)
await gate.RecordStateAsync("new-key", new VerdictState
{
Status = "affected",
Confidence = 0.80,
Timestamp = _timeProvider.GetUtcNow()
});
// Act
var pruned = await gate.PruneHistoryAsync();
// Assert
pruned.Should().BeGreaterThanOrEqualTo(1);
}
[Fact]
public async Task EvaluateAsync_WithTenantId_ShouldIsolateTenants()
{
// Arrange
var gate = CreateGate();
// Record state for tenant-a
await gate.RecordStateAsync("tenant-a:artifact:CVE-2024-1234", new VerdictState
{
Status = "affected",
Confidence = 0.80,
Timestamp = _timeProvider.GetUtcNow()
});
// Request for tenant-b (different tenant, no history)
var request = new StabilityDampingRequest
{
Key = "artifact:CVE-2024-1234",
TenantId = "tenant-b",
ProposedState = new VerdictState
{
Status = "affected",
Confidence = 0.80,
Timestamp = _timeProvider.GetUtcNow()
}
};
// Act
var decision = await gate.EvaluateAsync(request);
// Assert
decision.ShouldSurface.Should().BeTrue();
decision.Reason.Should().Contain("new verdict");
}
private sealed class TestOptionsMonitor<T> : IOptionsMonitor<T>
where T : class
{
public TestOptionsMonitor(T value) => CurrentValue = value;
public T CurrentValue { get; }
public T Get(string? name) => CurrentValue;
public IDisposable? OnChange(Action<T, string?> listener) => null;
}
}

View File

@@ -25,6 +25,12 @@ public sealed class ScannerWorkerMetrics
private readonly Counter<long> _surfacePayloadPersisted;
private readonly Histogram<double> _surfaceManifestPublishDurationMs;
// Secrets analysis metrics (Sprint: SPRINT_20251229_046_BE)
private readonly Counter<long> _secretsAnalysisCompleted;
private readonly Counter<long> _secretsAnalysisFailed;
private readonly Counter<long> _secretFindingsDetected;
private readonly Histogram<double> _secretsAnalysisDurationMs;
public ScannerWorkerMetrics()
{
_queueLatencyMs = ScannerWorkerInstrumentation.Meter.CreateHistogram<double>(
@@ -80,6 +86,21 @@ public sealed class ScannerWorkerMetrics
"scanner_worker_surface_manifest_publish_duration_ms",
unit: "ms",
description: "Duration in milliseconds to persist and publish surface manifests.");
// Secrets analysis metrics (Sprint: SPRINT_20251229_046_BE)
_secretsAnalysisCompleted = ScannerWorkerInstrumentation.Meter.CreateCounter<long>(
"scanner_worker_secrets_analysis_completed_total",
description: "Number of successfully completed secrets analysis runs.");
_secretsAnalysisFailed = ScannerWorkerInstrumentation.Meter.CreateCounter<long>(
"scanner_worker_secrets_analysis_failed_total",
description: "Number of secrets analysis runs that failed.");
_secretFindingsDetected = ScannerWorkerInstrumentation.Meter.CreateCounter<long>(
"scanner_worker_secrets_findings_detected_total",
description: "Number of secret findings detected.");
_secretsAnalysisDurationMs = ScannerWorkerInstrumentation.Meter.CreateHistogram<double>(
"scanner_worker_secrets_analysis_duration_ms",
unit: "ms",
description: "Duration in milliseconds for secrets analysis.");
}
public void RecordQueueLatency(ScanJobContext context, TimeSpan latency)
@@ -343,4 +364,39 @@ public sealed class ScannerWorkerMetrics
// Native analysis metrics are tracked via counters/histograms
// This is a placeholder for when we add dedicated native analysis metrics
}
/// <summary>
/// Records successful secrets analysis completion.
/// Sprint: SPRINT_20251229_046_BE - Secrets Leak Detection
/// </summary>
public void RecordSecretsAnalysisCompleted(
ScanJobContext context,
int findingCount,
int filesScanned,
TimeSpan duration,
TimeProvider timeProvider)
{
var tags = CreateTags(context, stage: ScanStageNames.ScanSecrets);
_secretsAnalysisCompleted.Add(1, tags);
if (findingCount > 0)
{
_secretFindingsDetected.Add(findingCount, tags);
}
if (duration > TimeSpan.Zero)
{
_secretsAnalysisDurationMs.Record(duration.TotalMilliseconds, tags);
}
}
/// <summary>
/// Records secrets analysis failure.
/// Sprint: SPRINT_20251229_046_BE - Secrets Leak Detection
/// </summary>
public void RecordSecretsAnalysisFailed(ScanJobContext context, TimeProvider timeProvider)
{
var tags = CreateTags(context, stage: ScanStageNames.ScanSecrets);
_secretsAnalysisFailed.Add(1, tags);
}
}

View File

@@ -38,6 +38,12 @@ public sealed class ScannerWorkerOptions
public VerdictPushOptions VerdictPush { get; } = new();
/// <summary>
/// Options for secrets leak detection scanning.
/// Sprint: SPRINT_20251229_046_BE - Secrets Leak Detection
/// </summary>
public SecretsOptions Secrets { get; } = new();
public sealed class QueueOptions
{
public int MaxAttempts { get; set; } = 5;
@@ -311,4 +317,43 @@ public sealed class ScannerWorkerOptions
/// </summary>
public bool AllowAnonymousFallback { get; set; } = true;
}
/// <summary>
/// Options for secrets leak detection scanning.
/// Sprint: SPRINT_20251229_046_BE - Secrets Leak Detection
/// </summary>
public sealed class SecretsOptions
{
/// <summary>
/// Enable secrets leak detection scanning.
/// When disabled, the secrets scan stage will be skipped.
/// </summary>
public bool Enabled { get; set; }
/// <summary>
/// Path to the secrets ruleset bundle directory.
/// </summary>
public string RulesetPath { get; set; } = string.Empty;
/// <summary>
/// Maximum file size in bytes to scan for secrets.
/// Files larger than this will be skipped.
/// </summary>
public long MaxFileSizeBytes { get; set; } = 5 * 1024 * 1024; // 5 MB
/// <summary>
/// Maximum number of files to scan per job.
/// </summary>
public int MaxFilesPerJob { get; set; } = 10_000;
/// <summary>
/// Enable entropy-based secret detection.
/// </summary>
public bool EnableEntropyDetection { get; set; } = true;
/// <summary>
/// Minimum entropy threshold for high-entropy string detection.
/// </summary>
public double EntropyThreshold { get; set; } = 4.5;
}
}

View File

@@ -23,6 +23,9 @@ public static class ScanStageNames
// Sprint: SPRINT_20251226_014_BINIDX - Binary Vulnerability Lookup
public const string BinaryLookup = "binary-lookup";
// Sprint: SPRINT_20251229_046_BE - Secrets Leak Detection
public const string ScanSecrets = "scan-secrets";
public static readonly IReadOnlyList<string> Ordered = new[]
{
IngestReplay,
@@ -30,6 +33,7 @@ public static class ScanStageNames
PullLayers,
BuildFilesystem,
ExecuteAnalyzers,
ScanSecrets,
BinaryLookup,
EpssEnrichment,
ComposeArtifacts,

View File

@@ -0,0 +1,235 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.IO;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using StellaOps.Scanner.Analyzers.Secrets;
using StellaOps.Scanner.Core.Contracts;
using StellaOps.Scanner.Worker.Diagnostics;
using StellaOps.Scanner.Worker.Options;
namespace StellaOps.Scanner.Worker.Processing.Secrets;
/// <summary>
/// Stage executor that scans filesystem for hardcoded secrets and credentials.
/// </summary>
internal sealed class SecretsAnalyzerStageExecutor : IScanStageExecutor
{
private static readonly string[] RootFsMetadataKeys =
{
"filesystem.rootfs",
"rootfs.path",
"scanner.rootfs",
};
private readonly ISecretsAnalyzer _secretsAnalyzer;
private readonly ScannerWorkerMetrics _metrics;
private readonly TimeProvider _timeProvider;
private readonly IOptions<ScannerWorkerOptions> _options;
private readonly ILogger<SecretsAnalyzerStageExecutor> _logger;
public SecretsAnalyzerStageExecutor(
ISecretsAnalyzer secretsAnalyzer,
ScannerWorkerMetrics metrics,
TimeProvider timeProvider,
IOptions<ScannerWorkerOptions> options,
ILogger<SecretsAnalyzerStageExecutor> logger)
{
_secretsAnalyzer = secretsAnalyzer ?? throw new ArgumentNullException(nameof(secretsAnalyzer));
_metrics = metrics ?? throw new ArgumentNullException(nameof(metrics));
_timeProvider = timeProvider ?? throw new ArgumentNullException(nameof(timeProvider));
_options = options ?? throw new ArgumentNullException(nameof(options));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public string StageName => ScanStageNames.ScanSecrets;
public async ValueTask ExecuteAsync(ScanJobContext context, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(context);
var secretsOptions = _options.Value.Secrets;
if (!secretsOptions.Enabled)
{
_logger.LogDebug("Secrets scanning is disabled; skipping stage.");
return;
}
// Get file entries from analyzer stage
if (!context.Analysis.TryGet<IReadOnlyList<ScanFileEntry>>(ScanAnalysisKeys.FileEntries, out var files) || files is null)
{
_logger.LogDebug("No file entries available; skipping secrets scan.");
return;
}
var rootfsPath = ResolveRootfsPath(context.Lease.Metadata);
if (string.IsNullOrWhiteSpace(rootfsPath))
{
_logger.LogWarning("No rootfs path found in job metadata; skipping secrets scan for job {JobId}.", context.JobId);
return;
}
var startTime = _timeProvider.GetTimestamp();
var allFindings = new List<SecretFinding>();
try
{
// Filter to text-like files only
var textFiles = files
.Where(f => ShouldScanFile(f))
.ToList();
_logger.LogInformation(
"Scanning {FileCount} files for secrets in job {JobId}.",
textFiles.Count,
context.JobId);
foreach (var file in textFiles)
{
cancellationToken.ThrowIfCancellationRequested();
try
{
var filePath = Path.Combine(rootfsPath, file.Path.TrimStart('/'));
if (!File.Exists(filePath))
{
continue;
}
var content = await File.ReadAllBytesAsync(filePath, cancellationToken).ConfigureAwait(false);
if (content.Length == 0 || content.Length > secretsOptions.MaxFileSizeBytes)
{
continue;
}
var findings = await _secretsAnalyzer.AnalyzeAsync(
content,
file.Path,
cancellationToken).ConfigureAwait(false);
if (findings.Count > 0)
{
allFindings.AddRange(findings);
}
}
catch (Exception ex) when (!cancellationToken.IsCancellationRequested)
{
_logger.LogDebug(ex, "Error scanning file {Path} for secrets: {Message}", file.Path, ex.Message);
}
}
var elapsed = _timeProvider.GetElapsedTime(startTime);
// Store findings in analysis store
var report = new SecretsAnalysisReport
{
JobId = context.JobId,
ScanId = context.ScanId,
Findings = allFindings.ToImmutableArray(),
FilesScanned = textFiles.Count,
RulesetVersion = _secretsAnalyzer.RulesetVersion,
AnalyzedAtUtc = _timeProvider.GetUtcNow(),
ElapsedMilliseconds = elapsed.TotalMilliseconds
};
context.Analysis.Set(ScanAnalysisKeys.SecretFindings, report);
context.Analysis.Set(ScanAnalysisKeys.SecretRulesetVersion, _secretsAnalyzer.RulesetVersion);
_metrics.RecordSecretsAnalysisCompleted(
context,
allFindings.Count,
textFiles.Count,
elapsed,
_timeProvider);
_logger.LogInformation(
"Secrets scan completed for job {JobId}: {FindingCount} findings in {FileCount} files ({ElapsedMs:F0}ms).",
context.JobId,
allFindings.Count,
textFiles.Count,
elapsed.TotalMilliseconds);
}
catch (OperationCanceledException)
{
_logger.LogDebug("Secrets scan cancelled for job {JobId}.", context.JobId);
throw;
}
catch (Exception ex)
{
_metrics.RecordSecretsAnalysisFailed(context, _timeProvider);
_logger.LogError(ex, "Secrets scan failed for job {JobId}: {Message}", context.JobId, ex.Message);
}
}
private static bool ShouldScanFile(ScanFileEntry file)
{
if (file is null || file.SizeBytes == 0)
{
return false;
}
// Skip binary files
if (file.Kind is "elf" or "pe" or "mach-o" or "blob")
{
return false;
}
// Skip very large files
if (file.SizeBytes > 10 * 1024 * 1024)
{
return false;
}
var ext = Path.GetExtension(file.Path).ToLowerInvariant();
// Include common text/config file extensions
return ext is ".json" or ".yaml" or ".yml" or ".xml" or ".properties" or ".conf" or ".config"
or ".env" or ".ini" or ".toml" or ".cfg"
or ".js" or ".ts" or ".jsx" or ".tsx" or ".mjs" or ".cjs"
or ".py" or ".rb" or ".php" or ".go" or ".java" or ".cs" or ".rs" or ".swift" or ".kt"
or ".sh" or ".bash" or ".zsh" or ".ps1" or ".bat" or ".cmd"
or ".sql" or ".graphql" or ".gql"
or ".tf" or ".tfvars" or ".hcl"
or ".dockerfile" or ".dockerignore"
or ".gitignore" or ".npmrc" or ".yarnrc" or ".pypirc"
or ".pem" or ".key" or ".crt" or ".cer"
or ".md" or ".txt" or ".log"
|| string.IsNullOrEmpty(ext);
}
private static string? ResolveRootfsPath(IReadOnlyDictionary<string, string> metadata)
{
if (metadata is null)
{
return null;
}
foreach (var key in RootFsMetadataKeys)
{
if (metadata.TryGetValue(key, out var value) && !string.IsNullOrWhiteSpace(value))
{
return value.Trim();
}
}
return null;
}
}
/// <summary>
/// Report of secrets analysis for a scan job.
/// </summary>
public sealed record SecretsAnalysisReport
{
public required string JobId { get; init; }
public required string ScanId { get; init; }
public required ImmutableArray<SecretFinding> Findings { get; init; }
public required int FilesScanned { get; init; }
public required string RulesetVersion { get; init; }
public required DateTimeOffset AnalyzedAtUtc { get; init; }
public required double ElapsedMilliseconds { get; init; }
}

View File

@@ -26,7 +26,9 @@ using StellaOps.Scanner.Worker.Hosting;
using StellaOps.Scanner.Worker.Options;
using StellaOps.Scanner.Worker.Processing;
using StellaOps.Scanner.Worker.Processing.Entropy;
using StellaOps.Scanner.Worker.Processing.Secrets;
using StellaOps.Scanner.Worker.Determinism;
using StellaOps.Scanner.Analyzers.Secrets;
using StellaOps.Scanner.Worker.Extensions;
using StellaOps.Scanner.Worker.Processing.Surface;
using StellaOps.Scanner.Storage.Extensions;
@@ -167,6 +169,18 @@ builder.Services.AddSingleton<IScanStageExecutor, Reachability.ReachabilityBuild
builder.Services.AddSingleton<IScanStageExecutor, Reachability.ReachabilityPublishStageExecutor>();
builder.Services.AddSingleton<IScanStageExecutor, EntropyStageExecutor>();
// Secrets Leak Detection (Sprint: SPRINT_20251229_046_BE)
if (workerOptions.Secrets.Enabled)
{
builder.Services.AddSecretsAnalyzer(options =>
{
options.RulesetPath = workerOptions.Secrets.RulesetPath;
options.EnableEntropyDetection = workerOptions.Secrets.EnableEntropyDetection;
options.EntropyThreshold = workerOptions.Secrets.EntropyThreshold;
});
builder.Services.AddSingleton<IScanStageExecutor, SecretsAnalyzerStageExecutor>();
}
// Proof of Exposure (Sprint: SPRINT_3500_0001_0001_proof_of_exposure_mvp)
builder.Services.AddOptions<StellaOps.Scanner.Core.Configuration.PoEConfiguration>()
.BindConfiguration("PoE")

View File

@@ -27,6 +27,7 @@
<ProjectReference Include="../__Libraries/StellaOps.Scanner.Surface.Env/StellaOps.Scanner.Surface.Env.csproj" />
<ProjectReference Include="../__Libraries/StellaOps.Scanner.Surface.Validation/StellaOps.Scanner.Surface.Validation.csproj" />
<ProjectReference Include="../__Libraries/StellaOps.Scanner.Surface.Secrets/StellaOps.Scanner.Surface.Secrets.csproj" />
<ProjectReference Include="../__Libraries/StellaOps.Scanner.Analyzers.Secrets/StellaOps.Scanner.Analyzers.Secrets.csproj" />
<ProjectReference Include="../__Libraries/StellaOps.Scanner.Surface.FS/StellaOps.Scanner.Surface.FS.csproj" />
<ProjectReference Include="../__Libraries/StellaOps.Scanner.Storage/StellaOps.Scanner.Storage.csproj" />
<ProjectReference Include="../__Libraries/StellaOps.Scanner.Emit/StellaOps.Scanner.Emit.csproj" />

View File

@@ -0,0 +1,21 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<EnableDefaultItems>false</EnableDefaultItems>
</PropertyGroup>
<ItemGroup>
<Compile Include="**\*.cs" Exclude="obj\**;bin\**" />
<EmbeddedResource Include="**\*.json" Exclude="obj\**;bin\**" />
<None Include="**\*" Exclude="**\*.cs;**\*.json;bin\**;obj\**" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Scanner.Analyzers.Lang\StellaOps.Scanner.Analyzers.Lang.csproj" />
<ProjectReference Include="..\StellaOps.Scanner.Surface.Validation\StellaOps.Scanner.Surface.Validation.csproj" />
</ItemGroup>
</Project>

View File

@@ -0,0 +1,128 @@
# Scanner Secrets Analyzer Guild Charter
## Mission
Detect accidentally committed secrets in container layers during scans using deterministic, DSSE-signed rule bundles. Ensure findings are reproducible, masked before output, and integrated with the Policy Engine for policy-driven decisions.
## Scope
- Secret detection plugin implementing `ILayerAnalyzer`
- Regex and entropy-based detection strategies
- Rule bundle loading, verification, and execution
- Payload masking engine
- Evidence emission (`secret.leak`) for policy integration
- Integration with Scanner Worker pipeline
## Required Reading
- `docs/modules/scanner/operations/secret-leak-detection.md` - Target specification
- `docs/modules/scanner/design/surface-secrets.md` - Credential delivery (different from leak detection)
- `docs/modules/scanner/architecture.md` - Scanner module architecture
- `docs/modules/policy/secret-leak-detection-readiness.md` - Policy integration requirements
- `docs/implplan/SPRINT_20260104_002_SCANNER_secret_leak_detection_core.md` - Implementation sprint
- `docs/implplan/SPRINT_20260104_003_SCANNER_secret_rule_bundles.md` - Bundle infrastructure sprint
- CLAUDE.md Section 8 (Code Quality & Determinism Rules)
## Working Agreement
1. **Status synchronisation**: Update task state in sprint file and local `TASKS.md` when starting or completing work.
2. **Determinism**:
- Sort rules by ID for deterministic execution order
- Use `CultureInfo.InvariantCulture` for all parsing
- Inject `TimeProvider` for timestamps
- Same inputs must produce same outputs
3. **Security posture**:
- NEVER log secret payloads
- Apply masking BEFORE any output or persistence
- Verify bundle signatures on load
- Enforce feature flag for gradual rollout
4. **Testing requirements**:
- Unit tests for all detectors, masking, and rule loading
- Integration tests with Scanner Worker
- Golden fixture tests for determinism verification
- Security tests ensuring secrets are not leaked
5. **Offline readiness**:
- Support local bundle verification without network
- Document Attestor mirror configuration
- Ensure bundles ship with Offline Kit
## Key Interfaces
```csharp
// Detection interface
public interface ISecretDetector
{
string DetectorId { get; }
ValueTask<IReadOnlyList<SecretMatch>> DetectAsync(
ReadOnlyMemory<byte> content,
string filePath,
SecretRule rule,
CancellationToken ct = default);
}
// Masking interface
public interface IPayloadMasker
{
string Mask(ReadOnlySpan<byte> payload, string? hint = null);
}
// Bundle verification
public interface IBundleVerifier
{
Task<BundleVerificationResult> VerifyAsync(
string bundleDirectory,
VerificationOptions options,
CancellationToken ct = default);
}
```
## Metrics
| Metric | Type | Description |
|--------|------|-------------|
| `scanner.secret.finding_total` | Counter | Total findings by tenant, ruleId, severity |
| `scanner.secret.scan_duration_seconds` | Histogram | Detection time per scan |
| `scanner.secret.rules_loaded` | Gauge | Number of active rules |
## Directory Structure
```
StellaOps.Scanner.Analyzers.Secrets/
├── AGENTS.md # This file
├── StellaOps.Scanner.Analyzers.Secrets.csproj
├── Detectors/
│ ├── ISecretDetector.cs
│ ├── RegexDetector.cs
│ ├── EntropyDetector.cs
│ └── CompositeSecretDetector.cs
├── Rules/
│ ├── SecretRule.cs
│ ├── SecretRuleset.cs
│ └── RulesetLoader.cs
├── Bundles/
│ ├── BundleBuilder.cs
│ ├── BundleVerifier.cs
│ └── Schemas/
├── Masking/
│ ├── IPayloadMasker.cs
│ └── PayloadMasker.cs
├── Evidence/
│ ├── SecretLeakEvidence.cs
│ └── SecretFinding.cs
├── SecretsAnalyzer.cs
├── SecretsAnalyzerHost.cs
├── SecretsAnalyzerOptions.cs
└── ServiceCollectionExtensions.cs
```
## Implementation Status
See sprint files for current implementation status:
- SPRINT_20260104_002_SCANNER_secret_leak_detection_core.md
- SPRINT_20260104_003_SCANNER_secret_rule_bundles.md
- SPRINT_20260104_004_POLICY_secret_dsl_integration.md
- SPRINT_20260104_005_AIRGAP_secret_offline_kit.md

View File

@@ -0,0 +1,3 @@
using System.Runtime.CompilerServices;
[assembly: InternalsVisibleTo("StellaOps.Scanner.Analyzers.Secrets.Tests")]

View File

@@ -0,0 +1,345 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
namespace StellaOps.Scanner.Analyzers.Secrets.Bundles;
/// <summary>
/// Builds secrets detection rule bundles from individual rule files.
/// Sprint: SPRINT_20260104_003_SCANNER - Secret Rule Bundles
/// </summary>
public interface IBundleBuilder
{
/// <summary>
/// Creates a bundle from individual rule files.
/// </summary>
Task<BundleArtifact> BuildAsync(
BundleBuildOptions options,
CancellationToken ct = default);
}
/// <summary>
/// Options for bundle creation.
/// </summary>
public sealed record BundleBuildOptions
{
/// <summary>
/// Directory where the bundle will be written.
/// </summary>
public required string OutputDirectory { get; init; }
/// <summary>
/// Bundle identifier (e.g., "secrets.ruleset").
/// </summary>
public required string BundleId { get; init; }
/// <summary>
/// Bundle version (e.g., "2026.01").
/// </summary>
public required string Version { get; init; }
/// <summary>
/// Paths to individual rule JSON files to include.
/// </summary>
public required IReadOnlyList<string> RuleFiles { get; init; }
/// <summary>
/// Optional description for the bundle.
/// </summary>
public string Description { get; init; } = "StellaOps Secret Detection Rules";
/// <summary>
/// Time provider for deterministic timestamps.
/// </summary>
public TimeProvider? TimeProvider { get; init; }
/// <summary>
/// Whether to validate rules during build.
/// </summary>
public bool ValidateRules { get; init; } = true;
/// <summary>
/// Whether to fail on validation warnings.
/// </summary>
public bool FailOnWarnings { get; init; } = false;
}
/// <summary>
/// Result of bundle creation.
/// </summary>
public sealed record BundleArtifact
{
/// <summary>
/// Path to the manifest file.
/// </summary>
public required string ManifestPath { get; init; }
/// <summary>
/// Path to the rules JSONL file.
/// </summary>
public required string RulesPath { get; init; }
/// <summary>
/// SHA-256 hash of the rules file (lowercase hex).
/// </summary>
public required string RulesSha256 { get; init; }
/// <summary>
/// Total number of rules in the bundle.
/// </summary>
public required int TotalRules { get; init; }
/// <summary>
/// Number of enabled rules in the bundle.
/// </summary>
public required int EnabledRules { get; init; }
/// <summary>
/// The generated manifest.
/// </summary>
public required BundleManifest Manifest { get; init; }
}
/// <summary>
/// Default implementation of bundle builder.
/// </summary>
public sealed class BundleBuilder : IBundleBuilder
{
private static readonly JsonSerializerOptions ManifestSerializerOptions = new()
{
WriteIndented = true,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
DefaultIgnoreCondition = System.Text.Json.Serialization.JsonIgnoreCondition.WhenWritingNull
};
private static readonly JsonSerializerOptions RuleSerializerOptions = new()
{
WriteIndented = false,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
DefaultIgnoreCondition = System.Text.Json.Serialization.JsonIgnoreCondition.WhenWritingDefault
};
private static readonly JsonSerializerOptions RuleReaderOptions = new()
{
PropertyNameCaseInsensitive = true,
AllowTrailingCommas = true,
ReadCommentHandling = JsonCommentHandling.Skip
};
private readonly IRuleValidator _validator;
private readonly ILogger<BundleBuilder> _logger;
public BundleBuilder(IRuleValidator validator, ILogger<BundleBuilder> logger)
{
_validator = validator ?? throw new ArgumentNullException(nameof(validator));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public async Task<BundleArtifact> BuildAsync(
BundleBuildOptions options,
CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(options);
var timeProvider = options.TimeProvider ?? TimeProvider.System;
var createdAt = timeProvider.GetUtcNow();
_logger.LogInformation(
"Building bundle {BundleId} v{Version} from {FileCount} rule files",
options.BundleId,
options.Version,
options.RuleFiles.Count);
// Load and validate rules
var rules = new List<SecretRule>();
var validationErrors = new List<string>();
var validationWarnings = new List<string>();
foreach (var ruleFile in options.RuleFiles)
{
ct.ThrowIfCancellationRequested();
if (!File.Exists(ruleFile))
{
validationErrors.Add($"Rule file not found: {ruleFile}");
continue;
}
try
{
var json = await File.ReadAllTextAsync(ruleFile, ct).ConfigureAwait(false);
var rule = JsonSerializer.Deserialize<SecretRule>(json, RuleReaderOptions);
if (rule is null)
{
validationErrors.Add($"Failed to deserialize rule from {ruleFile}");
continue;
}
if (options.ValidateRules)
{
var validation = _validator.Validate(rule);
if (!validation.IsValid)
{
foreach (var error in validation.Errors)
{
validationErrors.Add($"{ruleFile}: {error}");
}
continue;
}
foreach (var warning in validation.Warnings)
{
validationWarnings.Add($"{ruleFile}: {warning}");
}
}
rules.Add(rule);
}
catch (JsonException ex)
{
validationErrors.Add($"JSON parse error in {ruleFile}: {ex.Message}");
}
}
// Handle warnings
if (validationWarnings.Count > 0)
{
_logger.LogWarning(
"Bundle build has {WarningCount} warnings: {Warnings}",
validationWarnings.Count,
string.Join("; ", validationWarnings.Take(5)));
if (options.FailOnWarnings)
{
throw new InvalidOperationException(
$"Bundle build failed due to warnings: {string.Join("; ", validationWarnings)}");
}
}
// Handle errors
if (validationErrors.Count > 0)
{
throw new InvalidOperationException(
$"Bundle build failed with {validationErrors.Count} errors: {string.Join("; ", validationErrors)}");
}
if (rules.Count == 0)
{
throw new InvalidOperationException("No valid rules found to include in bundle.");
}
// Sort rules by ID for deterministic output
rules.Sort((a, b) => string.Compare(a.Id, b.Id, StringComparison.Ordinal));
// Ensure output directory exists
Directory.CreateDirectory(options.OutputDirectory);
// Write rules JSONL file
var rulesPath = Path.Combine(options.OutputDirectory, "secrets.ruleset.rules.jsonl");
await WriteRulesJsonlAsync(rulesPath, rules, ct).ConfigureAwait(false);
// Compute SHA-256 of rules file
var rulesSha256 = await ComputeFileSha256Async(rulesPath, ct).ConfigureAwait(false);
// Build manifest
var manifest = new BundleManifest
{
SchemaVersion = "1.0",
Id = options.BundleId,
Version = options.Version,
CreatedAt = createdAt,
Description = options.Description,
Rules = rules.Select(r => new BundleRuleSummary
{
Id = r.Id,
Version = r.Version,
Severity = r.Severity.ToString().ToLowerInvariant(),
Enabled = r.Enabled
}).ToImmutableArray(),
Integrity = new BundleIntegrity
{
RulesFile = "secrets.ruleset.rules.jsonl",
RulesSha256 = rulesSha256,
TotalRules = rules.Count,
EnabledRules = rules.Count(r => r.Enabled)
}
};
// Write manifest
var manifestPath = Path.Combine(options.OutputDirectory, "secrets.ruleset.manifest.json");
await WriteManifestAsync(manifestPath, manifest, ct).ConfigureAwait(false);
_logger.LogInformation(
"Bundle {BundleId} v{Version} created with {RuleCount} rules ({EnabledCount} enabled)",
options.BundleId,
options.Version,
rules.Count,
rules.Count(r => r.Enabled));
return new BundleArtifact
{
ManifestPath = manifestPath,
RulesPath = rulesPath,
RulesSha256 = rulesSha256,
TotalRules = rules.Count,
EnabledRules = rules.Count(r => r.Enabled),
Manifest = manifest
};
}
private static async Task WriteRulesJsonlAsync(
string path,
IReadOnlyList<SecretRule> rules,
CancellationToken ct)
{
await using var stream = new FileStream(
path,
FileMode.Create,
FileAccess.Write,
FileShare.None,
bufferSize: 4096,
useAsync: true);
await using var writer = new StreamWriter(stream, Encoding.UTF8, leaveOpen: true);
foreach (var rule in rules)
{
ct.ThrowIfCancellationRequested();
var json = JsonSerializer.Serialize(rule, RuleSerializerOptions);
await writer.WriteLineAsync(json).ConfigureAwait(false);
}
}
private static async Task WriteManifestAsync(
string path,
BundleManifest manifest,
CancellationToken ct)
{
var json = JsonSerializer.Serialize(manifest, ManifestSerializerOptions);
await File.WriteAllTextAsync(path, json, Encoding.UTF8, ct).ConfigureAwait(false);
}
private static async Task<string> ComputeFileSha256Async(string path, CancellationToken ct)
{
await using var stream = new FileStream(
path,
FileMode.Open,
FileAccess.Read,
FileShare.Read,
bufferSize: 4096,
useAsync: true);
var hash = await SHA256.HashDataAsync(stream, ct).ConfigureAwait(false);
return Convert.ToHexString(hash).ToLowerInvariant();
}
}

View File

@@ -0,0 +1,151 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Text.Json.Serialization;
namespace StellaOps.Scanner.Analyzers.Secrets.Bundles;
/// <summary>
/// Represents the manifest of a secrets detection rule bundle.
/// Sprint: SPRINT_20260104_003_SCANNER - Secret Rule Bundles
/// </summary>
public sealed record BundleManifest
{
/// <summary>
/// Schema version identifier.
/// </summary>
[JsonPropertyName("schemaVersion")]
public string SchemaVersion { get; init; } = "1.0";
/// <summary>
/// Unique identifier for the bundle (e.g., "secrets.ruleset").
/// </summary>
[JsonPropertyName("id")]
public required string Id { get; init; }
/// <summary>
/// Bundle version using CalVer (e.g., "2026.01").
/// </summary>
[JsonPropertyName("version")]
public required string Version { get; init; }
/// <summary>
/// UTC timestamp when the bundle was created.
/// </summary>
[JsonPropertyName("createdAt")]
public required DateTimeOffset CreatedAt { get; init; }
/// <summary>
/// Human-readable description of the bundle.
/// </summary>
[JsonPropertyName("description")]
public string Description { get; init; } = string.Empty;
/// <summary>
/// Summary of rules included in the bundle.
/// </summary>
[JsonPropertyName("rules")]
public ImmutableArray<BundleRuleSummary> Rules { get; init; } = [];
/// <summary>
/// Integrity information for the bundle.
/// </summary>
[JsonPropertyName("integrity")]
public required BundleIntegrity Integrity { get; init; }
/// <summary>
/// Signature information for the bundle.
/// </summary>
[JsonPropertyName("signatures")]
public BundleSignatures? Signatures { get; init; }
}
/// <summary>
/// Summary of a rule included in the bundle manifest.
/// </summary>
public sealed record BundleRuleSummary
{
/// <summary>
/// Unique rule identifier.
/// </summary>
[JsonPropertyName("id")]
public required string Id { get; init; }
/// <summary>
/// Rule version (SemVer).
/// </summary>
[JsonPropertyName("version")]
public required string Version { get; init; }
/// <summary>
/// Rule severity level.
/// </summary>
[JsonPropertyName("severity")]
public required string Severity { get; init; }
/// <summary>
/// Whether the rule is enabled by default.
/// </summary>
[JsonPropertyName("enabled")]
public bool Enabled { get; init; } = true;
}
/// <summary>
/// Integrity information for bundle verification.
/// </summary>
public sealed record BundleIntegrity
{
/// <summary>
/// Name of the rules file within the bundle.
/// </summary>
[JsonPropertyName("rulesFile")]
public string RulesFile { get; init; } = "secrets.ruleset.rules.jsonl";
/// <summary>
/// SHA-256 hash of the rules file (lowercase hex).
/// </summary>
[JsonPropertyName("rulesSha256")]
public required string RulesSha256 { get; init; }
/// <summary>
/// Total number of rules in the bundle.
/// </summary>
[JsonPropertyName("totalRules")]
public required int TotalRules { get; init; }
/// <summary>
/// Number of enabled rules in the bundle.
/// </summary>
[JsonPropertyName("enabledRules")]
public required int EnabledRules { get; init; }
}
/// <summary>
/// Signature references for the bundle.
/// </summary>
public sealed record BundleSignatures
{
/// <summary>
/// Path to the DSSE envelope file within the bundle.
/// </summary>
[JsonPropertyName("dsseEnvelope")]
public string DsseEnvelope { get; init; } = "secrets.ruleset.dsse.json";
/// <summary>
/// Key ID used for signing (for informational purposes).
/// </summary>
[JsonPropertyName("keyId")]
public string? KeyId { get; init; }
/// <summary>
/// UTC timestamp when the bundle was signed.
/// </summary>
[JsonPropertyName("signedAt")]
public DateTimeOffset? SignedAt { get; init; }
/// <summary>
/// Rekor transparency log entry ID (if applicable).
/// </summary>
[JsonPropertyName("rekorLogId")]
public string? RekorLogId { get; init; }
}

View File

@@ -0,0 +1,349 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.IO;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
namespace StellaOps.Scanner.Analyzers.Secrets.Bundles;
/// <summary>
/// Signs secrets detection rule bundles using DSSE envelopes.
/// Sprint: SPRINT_20260104_003_SCANNER - Secret Rule Bundles
/// </summary>
public interface IBundleSigner
{
/// <summary>
/// Signs a bundle artifact producing a DSSE envelope.
/// </summary>
Task<BundleSigningResult> SignAsync(
BundleArtifact artifact,
BundleSigningOptions options,
CancellationToken ct = default);
}
/// <summary>
/// Options for bundle signing.
/// </summary>
public sealed record BundleSigningOptions
{
/// <summary>
/// Key identifier for the signature.
/// </summary>
public required string KeyId { get; init; }
/// <summary>
/// Signing algorithm (e.g., "HMAC-SHA256", "ES256").
/// </summary>
public string Algorithm { get; init; } = "HMAC-SHA256";
/// <summary>
/// Shared secret for HMAC signing (base64 or hex encoded).
/// Required for HMAC-SHA256 algorithm.
/// </summary>
public string? SharedSecret { get; init; }
/// <summary>
/// Path to file containing the shared secret.
/// </summary>
public string? SharedSecretFile { get; init; }
/// <summary>
/// Payload type for the DSSE envelope.
/// </summary>
public string PayloadType { get; init; } = "application/vnd.stellaops.secrets-ruleset+json";
/// <summary>
/// Time provider for deterministic timestamps.
/// </summary>
public TimeProvider? TimeProvider { get; init; }
}
/// <summary>
/// Result of bundle signing.
/// </summary>
public sealed record BundleSigningResult
{
/// <summary>
/// Path to the generated DSSE envelope file.
/// </summary>
public required string EnvelopePath { get; init; }
/// <summary>
/// The generated DSSE envelope.
/// </summary>
public required DsseEnvelope Envelope { get; init; }
/// <summary>
/// Updated manifest with signature information.
/// </summary>
public required BundleManifest UpdatedManifest { get; init; }
}
/// <summary>
/// DSSE envelope structure for bundle signatures.
/// </summary>
public sealed record DsseEnvelope
{
/// <summary>
/// Base64url-encoded payload.
/// </summary>
public required string Payload { get; init; }
/// <summary>
/// Payload type URI.
/// </summary>
public required string PayloadType { get; init; }
/// <summary>
/// Signatures over the PAE.
/// </summary>
public required ImmutableArray<DsseSignature> Signatures { get; init; }
}
/// <summary>
/// A signature within a DSSE envelope.
/// </summary>
public sealed record DsseSignature
{
/// <summary>
/// Base64url-encoded signature bytes.
/// </summary>
public required string Sig { get; init; }
/// <summary>
/// Key identifier.
/// </summary>
public string? KeyId { get; init; }
}
/// <summary>
/// Default implementation of bundle signing using HMAC-SHA256.
/// </summary>
public sealed class BundleSigner : IBundleSigner
{
private const string DssePrefix = "DSSEv1";
private static readonly JsonSerializerOptions EnvelopeSerializerOptions = new()
{
WriteIndented = true,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};
private readonly ILogger<BundleSigner> _logger;
public BundleSigner(ILogger<BundleSigner> logger)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public async Task<BundleSigningResult> SignAsync(
BundleArtifact artifact,
BundleSigningOptions options,
CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(artifact);
ArgumentNullException.ThrowIfNull(options);
var timeProvider = options.TimeProvider ?? TimeProvider.System;
var signedAt = timeProvider.GetUtcNow();
_logger.LogInformation(
"Signing bundle {BundleId} v{Version} with key {KeyId}",
artifact.Manifest.Id,
artifact.Manifest.Version,
options.KeyId);
// Read manifest as payload
var manifestJson = await File.ReadAllBytesAsync(artifact.ManifestPath, ct).ConfigureAwait(false);
// Encode payload as base64url
var payloadBase64 = ToBase64Url(manifestJson);
// Build PAE (Pre-Authentication Encoding)
var pae = BuildPae(options.PayloadType, manifestJson);
// Sign the PAE
var signature = await SignPaeAsync(pae, options, ct).ConfigureAwait(false);
var signatureBase64 = ToBase64Url(signature);
// Build DSSE envelope
var envelope = new DsseEnvelope
{
Payload = payloadBase64,
PayloadType = options.PayloadType,
Signatures =
[
new DsseSignature
{
Sig = signatureBase64,
KeyId = options.KeyId
}
]
};
// Write envelope to file
var bundleDir = Path.GetDirectoryName(artifact.ManifestPath)!;
var envelopePath = Path.Combine(bundleDir, "secrets.ruleset.dsse.json");
var envelopeJson = JsonSerializer.Serialize(envelope, EnvelopeSerializerOptions);
await File.WriteAllTextAsync(envelopePath, envelopeJson, Encoding.UTF8, ct).ConfigureAwait(false);
// Update manifest with signature info
var updatedManifest = artifact.Manifest with
{
Signatures = new BundleSignatures
{
DsseEnvelope = "secrets.ruleset.dsse.json",
KeyId = options.KeyId,
SignedAt = signedAt
}
};
// Rewrite manifest with signature info
var updatedManifestJson = JsonSerializer.Serialize(updatedManifest, EnvelopeSerializerOptions);
await File.WriteAllTextAsync(artifact.ManifestPath, updatedManifestJson, Encoding.UTF8, ct).ConfigureAwait(false);
_logger.LogInformation(
"Bundle signed successfully. Envelope: {EnvelopePath}",
envelopePath);
return new BundleSigningResult
{
EnvelopePath = envelopePath,
Envelope = envelope,
UpdatedManifest = updatedManifest
};
}
private async Task<byte[]> SignPaeAsync(
byte[] pae,
BundleSigningOptions options,
CancellationToken ct)
{
if (!options.Algorithm.Equals("HMAC-SHA256", StringComparison.OrdinalIgnoreCase))
{
throw new NotSupportedException($"Algorithm '{options.Algorithm}' is not supported. Use HMAC-SHA256.");
}
var secret = await LoadSecretAsync(options, ct).ConfigureAwait(false);
if (secret is null || secret.Length == 0)
{
throw new InvalidOperationException("Shared secret is required for HMAC-SHA256 signing.");
}
using var hmac = new HMACSHA256(secret);
return hmac.ComputeHash(pae);
}
private static async Task<byte[]?> LoadSecretAsync(BundleSigningOptions options, CancellationToken ct)
{
// Try file first
if (!string.IsNullOrWhiteSpace(options.SharedSecretFile) && File.Exists(options.SharedSecretFile))
{
var content = (await File.ReadAllTextAsync(options.SharedSecretFile, ct).ConfigureAwait(false)).Trim();
return DecodeSecret(content);
}
// Then inline secret
if (!string.IsNullOrWhiteSpace(options.SharedSecret))
{
return DecodeSecret(options.SharedSecret);
}
return null;
}
private static byte[] DecodeSecret(string value)
{
// Try base64 first
try
{
return Convert.FromBase64String(value);
}
catch (FormatException)
{
// Not base64
}
// Try hex
if (value.Length % 2 == 0 && IsHexString(value))
{
return Convert.FromHexString(value);
}
// Treat as raw UTF-8
return Encoding.UTF8.GetBytes(value);
}
private static bool IsHexString(string value)
{
foreach (var c in value)
{
if (!char.IsAsciiHexDigit(c))
return false;
}
return true;
}
/// <summary>
/// Builds DSSE v1 Pre-Authentication Encoding.
/// Format: "DSSEv1" SP LEN(type) SP type SP LEN(payload) SP payload
/// </summary>
private static byte[] BuildPae(string payloadType, byte[] payload)
{
var typeBytes = Encoding.UTF8.GetBytes(payloadType);
var typeLenStr = typeBytes.Length.ToString(System.Globalization.CultureInfo.InvariantCulture);
var payloadLenStr = payload.Length.ToString(System.Globalization.CultureInfo.InvariantCulture);
// Calculate total size
var prefixBytes = Encoding.UTF8.GetBytes(DssePrefix);
var typeLenBytes = Encoding.UTF8.GetBytes(typeLenStr);
var payloadLenBytes = Encoding.UTF8.GetBytes(payloadLenStr);
var totalSize = prefixBytes.Length + 1 // prefix + SP
+ typeLenBytes.Length + 1 // type len + SP
+ typeBytes.Length + 1 // type + SP
+ payloadLenBytes.Length + 1 // payload len + SP
+ payload.Length;
var pae = new byte[totalSize];
var offset = 0;
// DSSEv1
Buffer.BlockCopy(prefixBytes, 0, pae, offset, prefixBytes.Length);
offset += prefixBytes.Length;
pae[offset++] = 0x20; // SP
// type length
Buffer.BlockCopy(typeLenBytes, 0, pae, offset, typeLenBytes.Length);
offset += typeLenBytes.Length;
pae[offset++] = 0x20; // SP
// type
Buffer.BlockCopy(typeBytes, 0, pae, offset, typeBytes.Length);
offset += typeBytes.Length;
pae[offset++] = 0x20; // SP
// payload length
Buffer.BlockCopy(payloadLenBytes, 0, pae, offset, payloadLenBytes.Length);
offset += payloadLenBytes.Length;
pae[offset++] = 0x20; // SP
// payload
Buffer.BlockCopy(payload, 0, pae, offset, payload.Length);
return pae;
}
private static string ToBase64Url(byte[] data)
{
return Convert.ToBase64String(data)
.TrimEnd('=')
.Replace('+', '-')
.Replace('/', '_');
}
}

View File

@@ -0,0 +1,527 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.IO;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
namespace StellaOps.Scanner.Analyzers.Secrets.Bundles;
/// <summary>
/// Verifies secrets detection rule bundle signatures and integrity.
/// Sprint: SPRINT_20260104_003_SCANNER - Secret Rule Bundles
/// </summary>
public interface IBundleVerifier
{
/// <summary>
/// Verifies a bundle's DSSE signature and integrity.
/// </summary>
Task<BundleVerificationResult> VerifyAsync(
string bundleDirectory,
BundleVerificationOptions options,
CancellationToken ct = default);
}
/// <summary>
/// Options for bundle verification.
/// </summary>
public sealed record BundleVerificationOptions
{
/// <summary>
/// URL of the attestor service for online verification.
/// </summary>
public string? AttestorUrl { get; init; }
/// <summary>
/// Whether to require Rekor transparency log proof.
/// </summary>
public bool RequireRekorProof { get; init; } = false;
/// <summary>
/// List of trusted key IDs. If empty, any key is accepted.
/// </summary>
public IReadOnlyList<string>? TrustedKeyIds { get; init; }
/// <summary>
/// Shared secret for HMAC verification (base64 or hex encoded).
/// </summary>
public string? SharedSecret { get; init; }
/// <summary>
/// Path to file containing the shared secret.
/// </summary>
public string? SharedSecretFile { get; init; }
/// <summary>
/// Whether to verify file integrity (SHA-256).
/// </summary>
public bool VerifyIntegrity { get; init; } = true;
/// <summary>
/// Whether to skip signature verification (integrity only).
/// </summary>
public bool SkipSignatureVerification { get; init; } = false;
}
/// <summary>
/// Result of bundle verification.
/// </summary>
public sealed record BundleVerificationResult
{
/// <summary>
/// Whether the bundle is valid.
/// </summary>
public required bool IsValid { get; init; }
/// <summary>
/// Bundle version that was verified.
/// </summary>
public string? BundleVersion { get; init; }
/// <summary>
/// Bundle ID that was verified.
/// </summary>
public string? BundleId { get; init; }
/// <summary>
/// UTC timestamp when the bundle was signed.
/// </summary>
public DateTimeOffset? SignedAt { get; init; }
/// <summary>
/// Key ID that signed the bundle.
/// </summary>
public string? SignerKeyId { get; init; }
/// <summary>
/// Rekor transparency log entry ID (if available).
/// </summary>
public string? RekorLogId { get; init; }
/// <summary>
/// Total number of rules in the bundle.
/// </summary>
public int? RuleCount { get; init; }
/// <summary>
/// Validation errors encountered.
/// </summary>
public ImmutableArray<string> ValidationErrors { get; init; } = [];
/// <summary>
/// Validation warnings (non-fatal).
/// </summary>
public ImmutableArray<string> ValidationWarnings { get; init; } = [];
public static BundleVerificationResult Success(BundleManifest manifest, string? keyId) => new()
{
IsValid = true,
BundleId = manifest.Id,
BundleVersion = manifest.Version,
SignedAt = manifest.Signatures?.SignedAt,
SignerKeyId = keyId ?? manifest.Signatures?.KeyId,
RekorLogId = manifest.Signatures?.RekorLogId,
RuleCount = manifest.Integrity.TotalRules
};
public static BundleVerificationResult Failure(params string[] errors) => new()
{
IsValid = false,
ValidationErrors = [.. errors]
};
public static BundleVerificationResult Failure(IEnumerable<string> errors) => new()
{
IsValid = false,
ValidationErrors = [.. errors]
};
}
/// <summary>
/// Default implementation of bundle verification.
/// </summary>
public sealed class BundleVerifier : IBundleVerifier
{
private const string DssePrefix = "DSSEv1";
private static readonly JsonSerializerOptions JsonOptions = new()
{
PropertyNameCaseInsensitive = true,
AllowTrailingCommas = true
};
private readonly ILogger<BundleVerifier> _logger;
public BundleVerifier(ILogger<BundleVerifier> logger)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public async Task<BundleVerificationResult> VerifyAsync(
string bundleDirectory,
BundleVerificationOptions options,
CancellationToken ct = default)
{
ArgumentException.ThrowIfNullOrWhiteSpace(bundleDirectory);
ArgumentNullException.ThrowIfNull(options);
var errors = new List<string>();
var warnings = new List<string>();
_logger.LogDebug("Verifying bundle at {BundleDir}", bundleDirectory);
// Check directory exists
if (!Directory.Exists(bundleDirectory))
{
return BundleVerificationResult.Failure($"Bundle directory not found: {bundleDirectory}");
}
// Load manifest
var manifestPath = Path.Combine(bundleDirectory, "secrets.ruleset.manifest.json");
if (!File.Exists(manifestPath))
{
return BundleVerificationResult.Failure($"Manifest not found: {manifestPath}");
}
BundleManifest manifest;
try
{
var manifestJson = await File.ReadAllTextAsync(manifestPath, ct).ConfigureAwait(false);
manifest = JsonSerializer.Deserialize<BundleManifest>(manifestJson, JsonOptions)!;
if (manifest is null)
{
return BundleVerificationResult.Failure("Failed to parse manifest.");
}
}
catch (JsonException ex)
{
return BundleVerificationResult.Failure($"Invalid manifest JSON: {ex.Message}");
}
_logger.LogDebug(
"Loaded manifest: {BundleId} v{Version} with {RuleCount} rules",
manifest.Id,
manifest.Version,
manifest.Integrity.TotalRules);
// Verify file integrity
if (options.VerifyIntegrity)
{
var rulesPath = Path.Combine(bundleDirectory, manifest.Integrity.RulesFile);
if (!File.Exists(rulesPath))
{
errors.Add($"Rules file not found: {rulesPath}");
}
else
{
var actualHash = await ComputeFileSha256Async(rulesPath, ct).ConfigureAwait(false);
if (!string.Equals(actualHash, manifest.Integrity.RulesSha256, StringComparison.OrdinalIgnoreCase))
{
errors.Add($"Rules file integrity check failed. Expected: {manifest.Integrity.RulesSha256}, Actual: {actualHash}");
}
else
{
_logger.LogDebug("Rules file integrity verified.");
}
}
}
// Verify signature
if (!options.SkipSignatureVerification)
{
if (manifest.Signatures is null)
{
errors.Add("Bundle is not signed.");
}
else
{
var envelopePath = Path.Combine(bundleDirectory, manifest.Signatures.DsseEnvelope);
if (!File.Exists(envelopePath))
{
errors.Add($"DSSE envelope not found: {envelopePath}");
}
else
{
var signatureResult = await VerifySignatureAsync(
manifestPath,
envelopePath,
options,
ct).ConfigureAwait(false);
if (!signatureResult.IsValid)
{
errors.AddRange(signatureResult.Errors);
}
else
{
// Check trusted key IDs
if (options.TrustedKeyIds is { Count: > 0 } trustedKeys)
{
if (signatureResult.KeyId is null || !trustedKeys.Contains(signatureResult.KeyId))
{
errors.Add($"Signature key '{signatureResult.KeyId}' is not in the trusted keys list.");
}
}
_logger.LogDebug("Signature verified with key: {KeyId}", signatureResult.KeyId);
}
}
}
}
// Check Rekor requirement
if (options.RequireRekorProof)
{
if (manifest.Signatures?.RekorLogId is null)
{
errors.Add("Rekor transparency log proof is required but not present.");
}
else
{
// TODO: Implement Rekor verification via Attestor client
warnings.Add("Rekor verification not yet implemented; proof present but not verified.");
}
}
if (errors.Count > 0)
{
_logger.LogWarning(
"Bundle verification failed for {BundleId} v{Version}: {Errors}",
manifest.Id,
manifest.Version,
string.Join("; ", errors));
return new BundleVerificationResult
{
IsValid = false,
BundleId = manifest.Id,
BundleVersion = manifest.Version,
ValidationErrors = [.. errors],
ValidationWarnings = [.. warnings]
};
}
_logger.LogInformation(
"Bundle verified: {BundleId} v{Version} ({RuleCount} rules)",
manifest.Id,
manifest.Version,
manifest.Integrity.TotalRules);
return new BundleVerificationResult
{
IsValid = true,
BundleId = manifest.Id,
BundleVersion = manifest.Version,
SignedAt = manifest.Signatures?.SignedAt,
SignerKeyId = manifest.Signatures?.KeyId,
RekorLogId = manifest.Signatures?.RekorLogId,
RuleCount = manifest.Integrity.TotalRules,
ValidationWarnings = [.. warnings]
};
}
private async Task<SignatureVerificationResult> VerifySignatureAsync(
string manifestPath,
string envelopePath,
BundleVerificationOptions options,
CancellationToken ct)
{
try
{
// Load envelope
var envelopeJson = await File.ReadAllTextAsync(envelopePath, ct).ConfigureAwait(false);
var envelope = JsonSerializer.Deserialize<DsseEnvelope>(envelopeJson, JsonOptions);
if (envelope is null || envelope.Signatures.IsDefaultOrEmpty)
{
return SignatureVerificationResult.Failure("Invalid or empty DSSE envelope.");
}
// Decode payload - this is the original manifest (before signature was added)
var payloadBytes = FromBase64Url(envelope.Payload);
var payloadManifest = JsonSerializer.Deserialize<BundleManifest>(payloadBytes, JsonOptions);
if (payloadManifest is null)
{
return SignatureVerificationResult.Failure("Failed to parse envelope payload as manifest.");
}
// Load current manifest and verify it matches the signed version (ignoring the Signatures field
// which was added after signing)
var manifestJson = await File.ReadAllTextAsync(manifestPath, ct).ConfigureAwait(false);
var currentManifest = JsonSerializer.Deserialize<BundleManifest>(manifestJson, JsonOptions);
if (currentManifest is null)
{
return SignatureVerificationResult.Failure("Failed to parse current manifest.");
}
// Compare all fields except Signatures (which is added after signing)
if (!ManifestsMatchIgnoringSignatures(payloadManifest, currentManifest))
{
return SignatureVerificationResult.Failure("Envelope payload does not match manifest content.");
}
// Build PAE
var pae = BuildPae(envelope.PayloadType, payloadBytes);
// Verify each signature (at least one must be valid)
var secret = await LoadSecretAsync(options, ct).ConfigureAwait(false);
if (secret is null || secret.Length == 0)
{
return SignatureVerificationResult.Failure("Shared secret is required for signature verification.");
}
foreach (var sig in envelope.Signatures)
{
var signatureBytes = FromBase64Url(sig.Sig);
using var hmac = new HMACSHA256(secret);
var expectedSignature = hmac.ComputeHash(pae);
if (CryptographicOperations.FixedTimeEquals(expectedSignature, signatureBytes))
{
return SignatureVerificationResult.Success(sig.KeyId);
}
}
return SignatureVerificationResult.Failure("Signature verification failed.");
}
catch (Exception ex)
{
return SignatureVerificationResult.Failure($"Signature verification error: {ex.Message}");
}
}
private static bool ManifestsMatchIgnoringSignatures(BundleManifest a, BundleManifest b)
{
// Compare all fields except Signatures
return a.SchemaVersion == b.SchemaVersion
&& a.Id == b.Id
&& a.Version == b.Version
&& a.CreatedAt == b.CreatedAt
&& a.Description == b.Description
&& a.Integrity.RulesFile == b.Integrity.RulesFile
&& a.Integrity.RulesSha256 == b.Integrity.RulesSha256
&& a.Integrity.TotalRules == b.Integrity.TotalRules
&& a.Integrity.EnabledRules == b.Integrity.EnabledRules;
}
private static async Task<byte[]?> LoadSecretAsync(BundleVerificationOptions options, CancellationToken ct)
{
// Try file first
if (!string.IsNullOrWhiteSpace(options.SharedSecretFile) && File.Exists(options.SharedSecretFile))
{
var content = (await File.ReadAllTextAsync(options.SharedSecretFile, ct).ConfigureAwait(false)).Trim();
return DecodeSecret(content);
}
// Then inline secret
if (!string.IsNullOrWhiteSpace(options.SharedSecret))
{
return DecodeSecret(options.SharedSecret);
}
return null;
}
private static byte[] DecodeSecret(string value)
{
// Try base64 first
try
{
return Convert.FromBase64String(value);
}
catch (FormatException)
{
// Not base64
}
// Try hex
if (value.Length % 2 == 0 && IsHexString(value))
{
return Convert.FromHexString(value);
}
// Treat as raw UTF-8
return Encoding.UTF8.GetBytes(value);
}
private static bool IsHexString(string value)
{
foreach (var c in value)
{
if (!char.IsAsciiHexDigit(c))
return false;
}
return true;
}
private static byte[] BuildPae(string payloadType, byte[] payload)
{
var typeBytes = Encoding.UTF8.GetBytes(payloadType);
var typeLenStr = typeBytes.Length.ToString(System.Globalization.CultureInfo.InvariantCulture);
var payloadLenStr = payload.Length.ToString(System.Globalization.CultureInfo.InvariantCulture);
var prefixBytes = Encoding.UTF8.GetBytes(DssePrefix);
var typeLenBytes = Encoding.UTF8.GetBytes(typeLenStr);
var payloadLenBytes = Encoding.UTF8.GetBytes(payloadLenStr);
var totalSize = prefixBytes.Length + 1
+ typeLenBytes.Length + 1
+ typeBytes.Length + 1
+ payloadLenBytes.Length + 1
+ payload.Length;
var pae = new byte[totalSize];
var offset = 0;
Buffer.BlockCopy(prefixBytes, 0, pae, offset, prefixBytes.Length);
offset += prefixBytes.Length;
pae[offset++] = 0x20;
Buffer.BlockCopy(typeLenBytes, 0, pae, offset, typeLenBytes.Length);
offset += typeLenBytes.Length;
pae[offset++] = 0x20;
Buffer.BlockCopy(typeBytes, 0, pae, offset, typeBytes.Length);
offset += typeBytes.Length;
pae[offset++] = 0x20;
Buffer.BlockCopy(payloadLenBytes, 0, pae, offset, payloadLenBytes.Length);
offset += payloadLenBytes.Length;
pae[offset++] = 0x20;
Buffer.BlockCopy(payload, 0, pae, offset, payload.Length);
return pae;
}
private static byte[] FromBase64Url(string value)
{
var padded = value.Replace('-', '+').Replace('_', '/');
switch (padded.Length % 4)
{
case 2: padded += "=="; break;
case 3: padded += "="; break;
}
return Convert.FromBase64String(padded);
}
private static async Task<string> ComputeFileSha256Async(string path, CancellationToken ct)
{
await using var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 4096, useAsync: true);
var hash = await SHA256.HashDataAsync(stream, ct).ConfigureAwait(false);
return Convert.ToHexString(hash).ToLowerInvariant();
}
private sealed record SignatureVerificationResult(bool IsValid, string? KeyId, ImmutableArray<string> Errors)
{
public static SignatureVerificationResult Success(string? keyId) => new(true, keyId, []);
public static SignatureVerificationResult Failure(params string[] errors) => new(false, null, [.. errors]);
}
}

View File

@@ -0,0 +1,186 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Text.RegularExpressions;
using Microsoft.Extensions.Logging;
namespace StellaOps.Scanner.Analyzers.Secrets.Bundles;
/// <summary>
/// Validates secret detection rules against the schema requirements.
/// Sprint: SPRINT_20260104_003_SCANNER - Secret Rule Bundles
/// </summary>
public interface IRuleValidator
{
/// <summary>
/// Validates a rule and returns validation errors, if any.
/// </summary>
RuleValidationResult Validate(SecretRule rule);
}
/// <summary>
/// Result of rule validation.
/// </summary>
public sealed record RuleValidationResult
{
/// <summary>
/// Whether the rule is valid.
/// </summary>
public required bool IsValid { get; init; }
/// <summary>
/// Validation errors encountered.
/// </summary>
public ImmutableArray<string> Errors { get; init; } = [];
/// <summary>
/// Validation warnings (non-fatal).
/// </summary>
public ImmutableArray<string> Warnings { get; init; } = [];
public static RuleValidationResult Success() => new() { IsValid = true };
public static RuleValidationResult Failure(params string[] errors) => new()
{
IsValid = false,
Errors = [.. errors]
};
public static RuleValidationResult Failure(IEnumerable<string> errors) => new()
{
IsValid = false,
Errors = [.. errors]
};
}
/// <summary>
/// Default implementation of rule validation.
/// </summary>
public sealed class RuleValidator : IRuleValidator
{
private static readonly Regex NamespacedIdPattern = new(
@"^[a-z][a-z0-9-]*(\.[a-z][a-z0-9-]*)+$",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
private static readonly Regex SemVerPattern = new(
@"^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
private readonly ILogger<RuleValidator> _logger;
public RuleValidator(ILogger<RuleValidator> logger)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public RuleValidationResult Validate(SecretRule rule)
{
ArgumentNullException.ThrowIfNull(rule);
var errors = new List<string>();
var warnings = new List<string>();
// Validate ID
if (string.IsNullOrWhiteSpace(rule.Id))
{
errors.Add("Rule ID is required.");
}
else if (!NamespacedIdPattern.IsMatch(rule.Id))
{
errors.Add($"Rule ID '{rule.Id}' must be namespaced (e.g., 'stellaops.secrets.aws-key').");
}
// Validate version
if (string.IsNullOrWhiteSpace(rule.Version))
{
errors.Add("Rule version is required.");
}
else if (!SemVerPattern.IsMatch(rule.Version))
{
errors.Add($"Rule version '{rule.Version}' must be valid SemVer.");
}
// Validate name
if (string.IsNullOrWhiteSpace(rule.Name))
{
warnings.Add("Rule name is recommended for documentation.");
}
// Validate pattern for regex/composite rules
if (rule.Type is SecretRuleType.Regex or SecretRuleType.Composite)
{
if (string.IsNullOrWhiteSpace(rule.Pattern))
{
errors.Add("Pattern is required for regex/composite rules.");
}
else
{
try
{
// Validate regex compiles
_ = new Regex(rule.Pattern, RegexOptions.Compiled, TimeSpan.FromSeconds(1));
}
catch (RegexParseException ex)
{
errors.Add($"Invalid regex pattern: {ex.Message}");
}
catch (ArgumentException ex)
{
errors.Add($"Invalid regex pattern: {ex.Message}");
}
}
}
// Validate entropy threshold for entropy rules
if (rule.Type is SecretRuleType.Entropy or SecretRuleType.Composite)
{
if (rule.EntropyThreshold <= 0 || rule.EntropyThreshold > 8)
{
warnings.Add($"Entropy threshold {rule.EntropyThreshold} may be out of typical range (3.0-6.0).");
}
}
// Validate min/max length
if (rule.MinLength < 0)
{
errors.Add("MinLength cannot be negative.");
}
if (rule.MaxLength < rule.MinLength)
{
errors.Add("MaxLength must be greater than or equal to MinLength.");
}
// Validate file patterns if provided
foreach (var pattern in rule.FilePatterns)
{
if (string.IsNullOrWhiteSpace(pattern))
{
warnings.Add("Empty file pattern will be ignored.");
}
}
if (errors.Count > 0)
{
_logger.LogDebug("Rule {RuleId} validation failed: {Errors}", rule.Id, string.Join("; ", errors));
return new RuleValidationResult
{
IsValid = false,
Errors = [.. errors],
Warnings = [.. warnings]
};
}
if (warnings.Count > 0)
{
_logger.LogDebug("Rule {RuleId} validated with warnings: {Warnings}", rule.Id, string.Join("; ", warnings));
}
return new RuleValidationResult
{
IsValid = true,
Errors = [],
Warnings = [.. warnings]
};
}
}

View File

@@ -0,0 +1,139 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Combines multiple detection strategies for comprehensive secret detection.
/// </summary>
public sealed class CompositeSecretDetector : ISecretDetector
{
private readonly RegexDetector _regexDetector;
private readonly EntropyDetector _entropyDetector;
private readonly ILogger<CompositeSecretDetector> _logger;
public CompositeSecretDetector(
RegexDetector regexDetector,
EntropyDetector entropyDetector,
ILogger<CompositeSecretDetector> logger)
{
_regexDetector = regexDetector ?? throw new ArgumentNullException(nameof(regexDetector));
_entropyDetector = entropyDetector ?? throw new ArgumentNullException(nameof(entropyDetector));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public string DetectorId => "composite";
public bool CanHandle(SecretRuleType ruleType) => true;
public async ValueTask<IReadOnlyList<SecretMatch>> DetectAsync(
ReadOnlyMemory<byte> content,
string filePath,
SecretRule rule,
CancellationToken ct = default)
{
var results = new List<SecretMatch>();
// Choose detector based on rule type
switch (rule.Type)
{
case SecretRuleType.Regex:
var regexMatches = await _regexDetector.DetectAsync(content, filePath, rule, ct);
results.AddRange(regexMatches);
break;
case SecretRuleType.Entropy:
var entropyMatches = await _entropyDetector.DetectAsync(content, filePath, rule, ct);
results.AddRange(entropyMatches);
break;
case SecretRuleType.Composite:
// Run both detectors and merge results
var regexTask = _regexDetector.DetectAsync(content, filePath, rule, ct);
var entropyTask = _entropyDetector.DetectAsync(content, filePath, rule, ct);
var regexResults = await regexTask;
var entropyResults = await entropyTask;
// Add regex matches
results.AddRange(regexResults);
// Add entropy matches, boosting confidence if they overlap with regex
foreach (var entropyMatch in entropyResults)
{
var overlappingRegex = regexResults.FirstOrDefault(r =>
r.LineNumber == entropyMatch.LineNumber &&
OverlapsColumn(r, entropyMatch));
if (overlappingRegex is not null)
{
// Boost confidence for overlapping matches
results.Add(entropyMatch with
{
ConfidenceScore = Math.Min(0.99, entropyMatch.ConfidenceScore + 0.1)
});
}
else
{
results.Add(entropyMatch);
}
}
break;
}
// Deduplicate overlapping matches
return DeduplicateMatches(results);
}
private static bool OverlapsColumn(SecretMatch a, SecretMatch b)
{
return a.ColumnStart <= b.ColumnEnd && b.ColumnStart <= a.ColumnEnd;
}
private static IReadOnlyList<SecretMatch> DeduplicateMatches(List<SecretMatch> matches)
{
if (matches.Count <= 1)
{
return matches;
}
// Sort by position
matches.Sort((a, b) =>
{
var lineComp = a.LineNumber.CompareTo(b.LineNumber);
return lineComp != 0 ? lineComp : a.ColumnStart.CompareTo(b.ColumnStart);
});
var deduplicated = new List<SecretMatch>();
SecretMatch? previous = null;
foreach (var match in matches)
{
if (previous is null)
{
previous = match;
continue;
}
// Check if this match overlaps with the previous one
if (match.LineNumber == previous.LineNumber && OverlapsColumn(previous, match))
{
// Keep the one with higher confidence
if (match.ConfidenceScore > previous.ConfidenceScore)
{
previous = match;
}
// Otherwise keep previous
}
else
{
deduplicated.Add(previous);
previous = match;
}
}
if (previous is not null)
{
deduplicated.Add(previous);
}
return deduplicated;
}
}

View File

@@ -0,0 +1,161 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Calculates Shannon entropy for detecting high-entropy strings that may be secrets.
/// </summary>
public static class EntropyCalculator
{
/// <summary>
/// Calculates Shannon entropy in bits per character for the given data.
/// </summary>
/// <param name="data">The data to analyze.</param>
/// <returns>Entropy in bits per character (0.0 to 8.0 for byte data).</returns>
public static double Calculate(ReadOnlySpan<byte> data)
{
if (data.IsEmpty)
{
return 0.0;
}
// Count occurrences of each byte value
Span<int> counts = stackalloc int[256];
counts.Clear();
foreach (byte b in data)
{
counts[b]++;
}
// Calculate entropy using Shannon's formula
double entropy = 0.0;
double length = data.Length;
for (int i = 0; i < 256; i++)
{
if (counts[i] > 0)
{
double probability = counts[i] / length;
entropy -= probability * Math.Log2(probability);
}
}
return entropy;
}
/// <summary>
/// Calculates Shannon entropy for a string.
/// </summary>
public static double Calculate(ReadOnlySpan<char> data)
{
if (data.IsEmpty)
{
return 0.0;
}
// For character data, we calculate based on unique characters seen
var counts = new Dictionary<char, int>();
foreach (char c in data)
{
counts.TryGetValue(c, out int count);
counts[c] = count + 1;
}
double entropy = 0.0;
double length = data.Length;
foreach (var count in counts.Values)
{
double probability = count / length;
entropy -= probability * Math.Log2(probability);
}
return entropy;
}
/// <summary>
/// Checks if the data appears to be base64 encoded.
/// </summary>
public static bool IsBase64Like(ReadOnlySpan<char> data)
{
if (data.Length < 4)
{
return false;
}
int validChars = 0;
foreach (char c in data)
{
if (char.IsLetterOrDigit(c) || c is '+' or '/' or '=')
{
validChars++;
}
}
return validChars >= data.Length * 0.9;
}
/// <summary>
/// Checks if the data appears to be hexadecimal.
/// </summary>
public static bool IsHexLike(ReadOnlySpan<char> data)
{
if (data.Length < 8)
{
return false;
}
foreach (char c in data)
{
if (!char.IsAsciiHexDigit(c))
{
return false;
}
}
return true;
}
/// <summary>
/// Determines if a string is likely a secret based on entropy and charset.
/// </summary>
/// <param name="data">The string to check.</param>
/// <param name="threshold">Minimum entropy threshold (default 4.5).</param>
/// <returns>True if the string appears to be a high-entropy secret.</returns>
public static bool IsLikelySecret(ReadOnlySpan<char> data, double threshold = 4.5)
{
if (data.Length < 16)
{
return false;
}
// Skip if it looks like a UUID (common false positive)
if (LooksLikeUuid(data))
{
return false;
}
var entropy = Calculate(data);
return entropy >= threshold;
}
private static bool LooksLikeUuid(ReadOnlySpan<char> data)
{
// UUID format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx (36 chars)
if (data.Length == 36)
{
if (data[8] == '-' && data[13] == '-' && data[18] == '-' && data[23] == '-')
{
return true;
}
}
// UUID without dashes: 32 hex chars
if (data.Length == 32 && IsHexLike(data))
{
return true;
}
return false;
}
}

View File

@@ -0,0 +1,199 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Entropy-based secret detector for high-entropy strings.
/// </summary>
public sealed class EntropyDetector : ISecretDetector
{
private readonly ILogger<EntropyDetector> _logger;
// Regex to find potential secret strings (alphanumeric with common secret characters)
private static readonly Regex CandidatePattern = new(
@"[A-Za-z0-9+/=_\-]{16,}",
RegexOptions.Compiled | RegexOptions.CultureInvariant,
TimeSpan.FromSeconds(5));
public EntropyDetector(ILogger<EntropyDetector> logger)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public string DetectorId => "entropy";
public bool CanHandle(SecretRuleType ruleType) =>
ruleType is SecretRuleType.Entropy or SecretRuleType.Composite;
public ValueTask<IReadOnlyList<SecretMatch>> DetectAsync(
ReadOnlyMemory<byte> content,
string filePath,
SecretRule rule,
CancellationToken ct = default)
{
if (ct.IsCancellationRequested)
{
return ValueTask.FromResult<IReadOnlyList<SecretMatch>>([]);
}
// Decode content as UTF-8
string text;
try
{
text = Encoding.UTF8.GetString(content.Span);
}
catch (DecoderFallbackException)
{
return ValueTask.FromResult<IReadOnlyList<SecretMatch>>([]);
}
var matches = new List<SecretMatch>();
var lineStarts = ComputeLineStarts(text);
var threshold = rule.EntropyThreshold > 0 ? rule.EntropyThreshold : 4.5;
var minLength = rule.MinLength > 0 ? rule.MinLength : 16;
var maxLength = rule.MaxLength > 0 ? rule.MaxLength : 1000;
try
{
foreach (Match candidate in CandidatePattern.Matches(text))
{
if (ct.IsCancellationRequested)
{
break;
}
var value = candidate.Value.AsSpan();
// Check length constraints
if (value.Length < minLength || value.Length > maxLength)
{
continue;
}
// Skip common false positives
if (ShouldSkip(value))
{
continue;
}
// Calculate entropy
var entropy = EntropyCalculator.Calculate(value);
if (entropy < threshold)
{
continue;
}
var (lineNumber, columnStart) = GetLineAndColumn(lineStarts, candidate.Index);
var matchBytes = Encoding.UTF8.GetBytes(candidate.Value);
// Adjust confidence based on entropy level
var confidenceScore = CalculateConfidence(entropy, threshold);
matches.Add(new SecretMatch
{
Rule = rule,
FilePath = filePath,
LineNumber = lineNumber,
ColumnStart = columnStart,
ColumnEnd = columnStart + candidate.Length - 1,
RawMatch = matchBytes,
ConfidenceScore = confidenceScore,
DetectorId = DetectorId,
Entropy = entropy
});
}
}
catch (RegexMatchTimeoutException)
{
_logger.LogWarning(
"Entropy detection timeout on file '{FilePath}'",
filePath);
}
return ValueTask.FromResult<IReadOnlyList<SecretMatch>>(matches);
}
private static bool ShouldSkip(ReadOnlySpan<char> value)
{
// Skip UUIDs
if (EntropyCalculator.IsHexLike(value) && value.Length == 32)
{
return true;
}
// Skip if it looks like a UUID with dashes
if (value.Length == 36 && value[8] == '-')
{
return true;
}
// Skip common hash prefixes that aren't secrets
if (value.StartsWith("sha256:", StringComparison.OrdinalIgnoreCase) ||
value.StartsWith("sha512:", StringComparison.OrdinalIgnoreCase) ||
value.StartsWith("md5:", StringComparison.OrdinalIgnoreCase))
{
return true;
}
// Skip if it's all the same character repeated
char first = value[0];
bool allSame = true;
for (int i = 1; i < value.Length; i++)
{
if (value[i] != first)
{
allSame = false;
break;
}
}
if (allSame)
{
return true;
}
return false;
}
private static double CalculateConfidence(double entropy, double threshold)
{
// Scale confidence based on how far above threshold
// entropy >= threshold + 1.5 => 0.95 (high)
// entropy >= threshold + 0.5 => 0.75 (medium)
// entropy >= threshold => 0.5 (low)
var excess = entropy - threshold;
return excess switch
{
>= 1.5 => 0.95,
>= 0.5 => 0.75,
_ => 0.5
};
}
private static List<int> ComputeLineStarts(string text)
{
var lineStarts = new List<int> { 0 };
for (int i = 0; i < text.Length; i++)
{
if (text[i] == '\n')
{
lineStarts.Add(i + 1);
}
}
return lineStarts;
}
private static (int lineNumber, int column) GetLineAndColumn(List<int> lineStarts, int position)
{
int line = 1;
for (int i = 1; i < lineStarts.Count; i++)
{
if (lineStarts[i] > position)
{
break;
}
line = i + 1;
}
int lineStart = lineStarts[line - 1];
int column = position - lineStart + 1;
return (line, column);
}
}

View File

@@ -0,0 +1,32 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Contract for secret detection strategies.
/// Implementations must be thread-safe and deterministic.
/// </summary>
public interface ISecretDetector
{
/// <summary>
/// Unique identifier for this detector (e.g., "regex", "entropy").
/// </summary>
string DetectorId { get; }
/// <summary>
/// Detects secrets in the provided content using the specified rule.
/// </summary>
/// <param name="content">The file content to scan.</param>
/// <param name="filePath">The file path (for reporting).</param>
/// <param name="rule">The rule to apply.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>List of matches found.</returns>
ValueTask<IReadOnlyList<SecretMatch>> DetectAsync(
ReadOnlyMemory<byte> content,
string filePath,
SecretRule rule,
CancellationToken ct = default);
/// <summary>
/// Checks if this detector can handle the specified rule type.
/// </summary>
bool CanHandle(SecretRuleType ruleType);
}

View File

@@ -0,0 +1,137 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Regex-based secret detector.
/// </summary>
public sealed class RegexDetector : ISecretDetector
{
private readonly ILogger<RegexDetector> _logger;
public RegexDetector(ILogger<RegexDetector> logger)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public string DetectorId => "regex";
public bool CanHandle(SecretRuleType ruleType) =>
ruleType is SecretRuleType.Regex or SecretRuleType.Composite;
public ValueTask<IReadOnlyList<SecretMatch>> DetectAsync(
ReadOnlyMemory<byte> content,
string filePath,
SecretRule rule,
CancellationToken ct = default)
{
if (ct.IsCancellationRequested)
{
return ValueTask.FromResult<IReadOnlyList<SecretMatch>>([]);
}
var regex = rule.GetCompiledPattern();
if (regex is null)
{
_logger.LogWarning("Rule '{RuleId}' has invalid regex pattern", rule.Id);
return ValueTask.FromResult<IReadOnlyList<SecretMatch>>([]);
}
// Decode content as UTF-8
string text;
try
{
text = Encoding.UTF8.GetString(content.Span);
}
catch (DecoderFallbackException)
{
// Not valid UTF-8, skip
return ValueTask.FromResult<IReadOnlyList<SecretMatch>>([]);
}
// Apply keyword pre-filter
if (!rule.MightMatch(text.AsSpan()))
{
return ValueTask.FromResult<IReadOnlyList<SecretMatch>>([]);
}
var matches = new List<SecretMatch>();
var lineStarts = ComputeLineStarts(text);
try
{
foreach (Match match in regex.Matches(text))
{
if (ct.IsCancellationRequested)
{
break;
}
if (match.Length < rule.MinLength || match.Length > rule.MaxLength)
{
continue;
}
var (lineNumber, columnStart) = GetLineAndColumn(lineStarts, match.Index);
var matchBytes = Encoding.UTF8.GetBytes(match.Value);
matches.Add(new SecretMatch
{
Rule = rule,
FilePath = filePath,
LineNumber = lineNumber,
ColumnStart = columnStart,
ColumnEnd = columnStart + match.Length - 1,
RawMatch = matchBytes,
ConfidenceScore = MapConfidenceToScore(rule.Confidence),
DetectorId = DetectorId
});
}
}
catch (RegexMatchTimeoutException)
{
_logger.LogWarning(
"Regex timeout for rule '{RuleId}' on file '{FilePath}'",
rule.Id,
filePath);
}
return ValueTask.FromResult<IReadOnlyList<SecretMatch>>(matches);
}
private static List<int> ComputeLineStarts(string text)
{
var lineStarts = new List<int> { 0 };
for (int i = 0; i < text.Length; i++)
{
if (text[i] == '\n')
{
lineStarts.Add(i + 1);
}
}
return lineStarts;
}
private static (int lineNumber, int column) GetLineAndColumn(List<int> lineStarts, int position)
{
int line = 1;
for (int i = 1; i < lineStarts.Count; i++)
{
if (lineStarts[i] > position)
{
break;
}
line = i + 1;
}
int lineStart = lineStarts[line - 1];
int column = position - lineStart + 1;
return (line, column);
}
private static double MapConfidenceToScore(SecretConfidence confidence) => confidence switch
{
SecretConfidence.Low => 0.5,
SecretConfidence.Medium => 0.75,
SecretConfidence.High => 0.95,
_ => 0.5
};
}

View File

@@ -0,0 +1,57 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Represents a potential secret match found by a detector.
/// </summary>
public sealed record SecretMatch
{
/// <summary>
/// The rule that produced this match.
/// </summary>
public required SecretRule Rule { get; init; }
/// <summary>
/// The file path where the match was found.
/// </summary>
public required string FilePath { get; init; }
/// <summary>
/// The 1-based line number of the match.
/// </summary>
public required int LineNumber { get; init; }
/// <summary>
/// The 1-based column where the match starts.
/// </summary>
public required int ColumnStart { get; init; }
/// <summary>
/// The 1-based column where the match ends.
/// </summary>
public required int ColumnEnd { get; init; }
/// <summary>
/// The raw matched content (will be masked before output).
/// </summary>
public required ReadOnlyMemory<byte> RawMatch { get; init; }
/// <summary>
/// Confidence score from 0.0 to 1.0.
/// </summary>
public required double ConfidenceScore { get; init; }
/// <summary>
/// The detector that found this match.
/// </summary>
public required string DetectorId { get; init; }
/// <summary>
/// Optional entropy value if entropy-based detection was used.
/// </summary>
public double? Entropy { get; init; }
/// <summary>
/// Gets the length of the matched content.
/// </summary>
public int MatchLength => RawMatch.Length;
}

View File

@@ -0,0 +1,52 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Aggregated finding for storage in ScanAnalysisStore.
/// </summary>
public sealed record SecretFinding
{
/// <summary>
/// Unique identifier for this finding.
/// </summary>
public required Guid Id { get; init; }
/// <summary>
/// The evidence record.
/// </summary>
public required SecretLeakEvidence Evidence { get; init; }
/// <summary>
/// The scan that produced this finding.
/// </summary>
public required string ScanId { get; init; }
/// <summary>
/// The tenant that owns this finding.
/// </summary>
public required string TenantId { get; init; }
/// <summary>
/// The artifact digest (container image or other artifact).
/// </summary>
public required string ArtifactDigest { get; init; }
/// <summary>
/// Creates a new finding from evidence.
/// </summary>
public static SecretFinding Create(
SecretLeakEvidence evidence,
string scanId,
string tenantId,
string artifactDigest,
Guid? id = null)
{
return new SecretFinding
{
Id = id ?? Guid.NewGuid(),
Evidence = evidence,
ScanId = scanId,
TenantId = tenantId,
ArtifactDigest = artifactDigest
};
}
}

View File

@@ -0,0 +1,136 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Evidence record for a detected secret leak.
/// This record is emitted to the policy engine for decision-making.
/// </summary>
public sealed record SecretLeakEvidence
{
/// <summary>
/// The evidence type identifier.
/// </summary>
public const string EvidenceType = "secret.leak";
/// <summary>
/// The rule ID that produced this finding.
/// </summary>
public required string RuleId { get; init; }
/// <summary>
/// The rule version.
/// </summary>
public required string RuleVersion { get; init; }
/// <summary>
/// The severity of the finding.
/// </summary>
public required SecretSeverity Severity { get; init; }
/// <summary>
/// The confidence level of the finding.
/// </summary>
public required SecretConfidence Confidence { get; init; }
/// <summary>
/// The file path where the secret was found (relative to scan root).
/// </summary>
public required string FilePath { get; init; }
/// <summary>
/// The 1-based line number.
/// </summary>
public required int LineNumber { get; init; }
/// <summary>
/// The 1-based column number.
/// </summary>
public int ColumnNumber { get; init; }
/// <summary>
/// The masked payload (e.g., "AKIA****B7"). Never contains the actual secret.
/// </summary>
public required string Mask { get; init; }
/// <summary>
/// The bundle ID that contained the rule.
/// </summary>
public required string BundleId { get; init; }
/// <summary>
/// The bundle version.
/// </summary>
public required string BundleVersion { get; init; }
/// <summary>
/// When this finding was detected.
/// </summary>
public required DateTimeOffset DetectedAt { get; init; }
/// <summary>
/// The detector that found this secret.
/// </summary>
public required string DetectorId { get; init; }
/// <summary>
/// Entropy value if entropy-based detection was used.
/// </summary>
public double? Entropy { get; init; }
/// <summary>
/// Additional metadata for the finding.
/// </summary>
public ImmutableDictionary<string, string> Metadata { get; init; } =
ImmutableDictionary<string, string>.Empty;
/// <summary>
/// Creates evidence from a secret match and masker.
/// </summary>
public static SecretLeakEvidence FromMatch(
SecretMatch match,
IPayloadMasker masker,
SecretRuleset ruleset,
TimeProvider timeProvider)
{
ArgumentNullException.ThrowIfNull(match);
ArgumentNullException.ThrowIfNull(masker);
ArgumentNullException.ThrowIfNull(ruleset);
ArgumentNullException.ThrowIfNull(timeProvider);
var masked = masker.Mask(match.RawMatch.Span, match.Rule.MaskingHint);
return new SecretLeakEvidence
{
RuleId = match.Rule.Id,
RuleVersion = match.Rule.Version,
Severity = match.Rule.Severity,
Confidence = MapScoreToConfidence(match.ConfidenceScore, match.Rule.Confidence),
FilePath = match.FilePath,
LineNumber = match.LineNumber,
ColumnNumber = match.ColumnStart,
Mask = masked,
BundleId = ruleset.Id,
BundleVersion = ruleset.Version,
DetectedAt = timeProvider.GetUtcNow(),
DetectorId = match.DetectorId,
Entropy = match.Entropy
};
}
private static SecretConfidence MapScoreToConfidence(double score, SecretConfidence ruleDefault)
{
// Adjust confidence based on detection score
if (score >= 0.9)
{
return SecretConfidence.High;
}
if (score >= 0.7)
{
return SecretConfidence.Medium;
}
if (score >= 0.5)
{
return ruleDefault;
}
return SecretConfidence.Low;
}
}

View File

@@ -0,0 +1,10 @@
global using System.Collections.Immutable;
global using System.Diagnostics.CodeAnalysis;
global using System.Globalization;
global using System.Text;
global using System.Text.Json;
global using System.Text.RegularExpressions;
global using Microsoft.Extensions.DependencyInjection;
global using Microsoft.Extensions.DependencyInjection.Extensions;
global using Microsoft.Extensions.Logging;
global using Microsoft.Extensions.Options;

View File

@@ -0,0 +1,23 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Contract for secret payload masking.
/// </summary>
public interface IPayloadMasker
{
/// <summary>
/// Masks a secret payload, preserving prefix/suffix for identification.
/// </summary>
/// <param name="payload">The raw secret bytes.</param>
/// <param name="hint">Optional masking hint (e.g., "prefix:4,suffix:2").</param>
/// <returns>Masked string (e.g., "AKIA****B7").</returns>
string Mask(ReadOnlySpan<byte> payload, string? hint = null);
/// <summary>
/// Masks a secret string, preserving prefix/suffix for identification.
/// </summary>
/// <param name="payload">The raw secret string.</param>
/// <param name="hint">Optional masking hint (e.g., "prefix:4,suffix:2").</param>
/// <returns>Masked string (e.g., "AKIA****B7").</returns>
string Mask(ReadOnlySpan<char> payload, string? hint = null);
}

View File

@@ -0,0 +1,151 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Default implementation of payload masking for secrets.
/// </summary>
public sealed class PayloadMasker : IPayloadMasker
{
/// <summary>
/// Default number of characters to preserve at the start.
/// </summary>
public const int DefaultPrefixLength = 4;
/// <summary>
/// Default number of characters to preserve at the end.
/// </summary>
public const int DefaultSuffixLength = 2;
/// <summary>
/// Maximum number of mask characters to use.
/// </summary>
public const int MaxMaskLength = 8;
/// <summary>
/// Minimum output length for masked values.
/// </summary>
public const int MinOutputLength = 8;
/// <summary>
/// Maximum total characters to expose (prefix + suffix).
/// </summary>
public const int MaxExposedChars = 6;
/// <summary>
/// The character used for masking.
/// </summary>
public const char MaskChar = '*';
public string Mask(ReadOnlySpan<byte> payload, string? hint = null)
{
if (payload.IsEmpty)
{
return string.Empty;
}
// Try to decode as UTF-8
try
{
var text = Encoding.UTF8.GetString(payload);
return Mask(text.AsSpan(), hint);
}
catch (DecoderFallbackException)
{
// Not valid UTF-8, represent as hex
var hex = Convert.ToHexString(payload);
return Mask(hex.AsSpan(), hint);
}
}
public string Mask(ReadOnlySpan<char> payload, string? hint = null)
{
if (payload.IsEmpty)
{
return string.Empty;
}
var (prefixLen, suffixLen) = ParseHint(hint);
// Enforce maximum exposed characters
if (prefixLen + suffixLen > MaxExposedChars)
{
var ratio = (double)prefixLen / (prefixLen + suffixLen);
prefixLen = (int)(MaxExposedChars * ratio);
suffixLen = MaxExposedChars - prefixLen;
}
// Handle short payloads
if (payload.Length <= prefixLen + suffixLen)
{
// Too short to mask meaningfully, just return masked placeholder
return new string(MaskChar, Math.Min(payload.Length, MinOutputLength));
}
// Calculate mask length
var middleLength = payload.Length - prefixLen - suffixLen;
var maskLength = Math.Min(middleLength, MaxMaskLength);
// Build masked output
var sb = new StringBuilder(prefixLen + maskLength + suffixLen);
// Prefix
if (prefixLen > 0)
{
sb.Append(payload[..prefixLen]);
}
// Mask
sb.Append(MaskChar, maskLength);
// Suffix
if (suffixLen > 0)
{
sb.Append(payload[^suffixLen..]);
}
// Ensure minimum length
while (sb.Length < MinOutputLength)
{
sb.Insert(prefixLen, MaskChar);
}
return sb.ToString();
}
private static (int prefix, int suffix) ParseHint(string? hint)
{
if (string.IsNullOrWhiteSpace(hint))
{
return (DefaultPrefixLength, DefaultSuffixLength);
}
int prefix = DefaultPrefixLength;
int suffix = DefaultSuffixLength;
// Parse hint format: "prefix:4,suffix:2"
var parts = hint.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
foreach (var part in parts)
{
var kv = part.Split(':', StringSplitOptions.TrimEntries);
if (kv.Length != 2)
{
continue;
}
if (!int.TryParse(kv[1], NumberStyles.Integer, CultureInfo.InvariantCulture, out var value))
{
continue;
}
if (kv[0].Equals("prefix", StringComparison.OrdinalIgnoreCase))
{
prefix = Math.Max(0, Math.Min(value, MaxExposedChars));
}
else if (kv[0].Equals("suffix", StringComparison.OrdinalIgnoreCase))
{
suffix = Math.Max(0, Math.Min(value, MaxExposedChars));
}
}
return (prefix, suffix);
}
}

View File

@@ -0,0 +1,29 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Contract for loading secret detection rulesets.
/// </summary>
public interface IRulesetLoader
{
/// <summary>
/// Loads a ruleset from a directory containing bundle files.
/// </summary>
/// <param name="bundlePath">Path to the bundle directory.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The loaded ruleset.</returns>
ValueTask<SecretRuleset> LoadAsync(string bundlePath, CancellationToken ct = default);
/// <summary>
/// Loads a ruleset from a JSONL stream.
/// </summary>
/// <param name="rulesStream">Stream containing NDJSON rule definitions.</param>
/// <param name="bundleId">The bundle identifier.</param>
/// <param name="bundleVersion">The bundle version.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The loaded ruleset.</returns>
ValueTask<SecretRuleset> LoadFromJsonlAsync(
Stream rulesStream,
string bundleId,
string bundleVersion,
CancellationToken ct = default);
}

View File

@@ -0,0 +1,227 @@
using System.Security.Cryptography;
using System.Text.Json.Serialization;
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Loads secret detection rulesets from bundle files.
/// </summary>
public sealed class RulesetLoader : IRulesetLoader
{
private readonly ILogger<RulesetLoader> _logger;
private readonly TimeProvider _timeProvider;
private static readonly JsonSerializerOptions JsonOptions = new()
{
PropertyNameCaseInsensitive = true,
Converters =
{
new JsonStringEnumConverter(JsonNamingPolicy.CamelCase)
}
};
public RulesetLoader(ILogger<RulesetLoader> logger, TimeProvider timeProvider)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_timeProvider = timeProvider ?? throw new ArgumentNullException(nameof(timeProvider));
}
public async ValueTask<SecretRuleset> LoadAsync(string bundlePath, CancellationToken ct = default)
{
if (string.IsNullOrWhiteSpace(bundlePath))
{
throw new ArgumentException("Bundle path is required", nameof(bundlePath));
}
if (!Directory.Exists(bundlePath))
{
throw new DirectoryNotFoundException($"Bundle directory not found: {bundlePath}");
}
// Load manifest
var manifestPath = Path.Combine(bundlePath, "secrets.ruleset.manifest.json");
if (!File.Exists(manifestPath))
{
throw new FileNotFoundException("Bundle manifest not found", manifestPath);
}
var manifestJson = await File.ReadAllTextAsync(manifestPath, ct);
var manifest = JsonSerializer.Deserialize<BundleManifest>(manifestJson, JsonOptions)
?? throw new InvalidOperationException("Failed to parse bundle manifest");
// Load rules
var rulesPath = Path.Combine(bundlePath, "secrets.ruleset.rules.jsonl");
if (!File.Exists(rulesPath))
{
throw new FileNotFoundException("Bundle rules file not found", rulesPath);
}
await using var rulesStream = File.OpenRead(rulesPath);
var ruleset = await LoadFromJsonlAsync(
rulesStream,
manifest.Id ?? "secrets.ruleset",
manifest.Version ?? "unknown",
ct);
// Verify integrity if digest is available
if (!string.IsNullOrEmpty(manifest.Integrity?.RulesSha256))
{
rulesStream.Position = 0;
var actualDigest = await ComputeSha256Async(rulesStream, ct);
if (!string.Equals(actualDigest, manifest.Integrity.RulesSha256, StringComparison.OrdinalIgnoreCase))
{
throw new InvalidOperationException(
$"Rules file integrity check failed. Expected: {manifest.Integrity.RulesSha256}, Actual: {actualDigest}");
}
}
_logger.LogInformation(
"Loaded secrets ruleset '{BundleId}' version {Version} with {RuleCount} rules ({EnabledCount} enabled)",
ruleset.Id,
ruleset.Version,
ruleset.Rules.Length,
ruleset.EnabledRuleCount);
return ruleset;
}
public async ValueTask<SecretRuleset> LoadFromJsonlAsync(
Stream rulesStream,
string bundleId,
string bundleVersion,
CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(rulesStream);
var rules = new List<SecretRule>();
using var reader = new StreamReader(rulesStream, Encoding.UTF8, leaveOpen: true);
int lineNumber = 0;
string? line;
while ((line = await reader.ReadLineAsync(ct)) is not null)
{
ct.ThrowIfCancellationRequested();
lineNumber++;
if (string.IsNullOrWhiteSpace(line))
{
continue;
}
try
{
var ruleJson = JsonSerializer.Deserialize<RuleJson>(line, JsonOptions);
if (ruleJson is null)
{
_logger.LogWarning("Skipping null rule at line {LineNumber}", lineNumber);
continue;
}
var rule = MapToRule(ruleJson);
rules.Add(rule);
}
catch (JsonException ex)
{
_logger.LogWarning(ex, "Failed to parse rule at line {LineNumber}", lineNumber);
}
}
// Sort rules by ID for deterministic ordering
rules.Sort((a, b) => string.Compare(a.Id, b.Id, StringComparison.Ordinal));
return new SecretRuleset
{
Id = bundleId,
Version = bundleVersion,
CreatedAt = _timeProvider.GetUtcNow(),
Rules = [.. rules]
};
}
private static SecretRule MapToRule(RuleJson json)
{
return new SecretRule
{
Id = json.Id ?? throw new InvalidOperationException("Rule ID is required"),
Version = json.Version ?? "1.0.0",
Name = json.Name ?? json.Id ?? "Unknown",
Description = json.Description ?? string.Empty,
Type = ParseRuleType(json.Type),
Pattern = json.Pattern ?? throw new InvalidOperationException("Rule pattern is required"),
Severity = ParseSeverity(json.Severity),
Confidence = ParseConfidence(json.Confidence),
MaskingHint = json.MaskingHint,
Keywords = json.Keywords?.ToImmutableArray() ?? [],
FilePatterns = json.FilePatterns?.ToImmutableArray() ?? [],
Enabled = json.Enabled ?? true,
EntropyThreshold = json.EntropyThreshold ?? 4.5,
MinLength = json.MinLength ?? 16,
MaxLength = json.MaxLength ?? 1000,
Metadata = json.Metadata?.ToImmutableDictionary() ?? ImmutableDictionary<string, string>.Empty
};
}
private static SecretRuleType ParseRuleType(string? type) => type?.ToLowerInvariant() switch
{
"regex" => SecretRuleType.Regex,
"entropy" => SecretRuleType.Entropy,
"composite" => SecretRuleType.Composite,
_ => SecretRuleType.Regex
};
private static SecretSeverity ParseSeverity(string? severity) => severity?.ToLowerInvariant() switch
{
"low" => SecretSeverity.Low,
"medium" => SecretSeverity.Medium,
"high" => SecretSeverity.High,
"critical" => SecretSeverity.Critical,
_ => SecretSeverity.Medium
};
private static SecretConfidence ParseConfidence(string? confidence) => confidence?.ToLowerInvariant() switch
{
"low" => SecretConfidence.Low,
"medium" => SecretConfidence.Medium,
"high" => SecretConfidence.High,
_ => SecretConfidence.Medium
};
private static async Task<string> ComputeSha256Async(Stream stream, CancellationToken ct)
{
var hash = await SHA256.HashDataAsync(stream, ct);
return Convert.ToHexString(hash).ToLowerInvariant();
}
// JSON deserialization models
private sealed class BundleManifest
{
public string? Id { get; set; }
public string? Version { get; set; }
public IntegrityInfo? Integrity { get; set; }
}
private sealed class IntegrityInfo
{
public string? RulesSha256 { get; set; }
}
private sealed class RuleJson
{
public string? Id { get; set; }
public string? Version { get; set; }
public string? Name { get; set; }
public string? Description { get; set; }
public string? Type { get; set; }
public string? Pattern { get; set; }
public string? Severity { get; set; }
public string? Confidence { get; set; }
public string? MaskingHint { get; set; }
public List<string>? Keywords { get; set; }
public List<string>? FilePatterns { get; set; }
public bool? Enabled { get; set; }
public double? EntropyThreshold { get; set; }
public int? MinLength { get; set; }
public int? MaxLength { get; set; }
public Dictionary<string, string>? Metadata { get; set; }
}
}

View File

@@ -0,0 +1,22 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Confidence level for a secret detection finding.
/// </summary>
public enum SecretConfidence
{
/// <summary>
/// Low confidence - may be a false positive.
/// </summary>
Low = 0,
/// <summary>
/// Medium confidence - likely a real secret but requires verification.
/// </summary>
Medium = 1,
/// <summary>
/// High confidence - almost certainly a real secret.
/// </summary>
High = 2
}

View File

@@ -0,0 +1,191 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// A single secret detection rule defining patterns and metadata for identifying secrets.
/// </summary>
public sealed record SecretRule
{
/// <summary>
/// Unique rule identifier (e.g., "stellaops.secrets.aws-access-key").
/// </summary>
public required string Id { get; init; }
/// <summary>
/// Rule version in SemVer format (e.g., "1.0.0").
/// </summary>
public required string Version { get; init; }
/// <summary>
/// Human-readable rule name.
/// </summary>
public required string Name { get; init; }
/// <summary>
/// Detailed description of what this rule detects.
/// </summary>
public required string Description { get; init; }
/// <summary>
/// The detection strategy type.
/// </summary>
public required SecretRuleType Type { get; init; }
/// <summary>
/// The detection pattern (regex pattern for Regex type, entropy config for Entropy type).
/// </summary>
public required string Pattern { get; init; }
/// <summary>
/// Default severity for findings from this rule.
/// </summary>
public required SecretSeverity Severity { get; init; }
/// <summary>
/// Default confidence level for findings from this rule.
/// </summary>
public required SecretConfidence Confidence { get; init; }
/// <summary>
/// Optional masking hint (e.g., "prefix:4,suffix:2") for payload masking.
/// </summary>
public string? MaskingHint { get; init; }
/// <summary>
/// Pre-filter keywords for fast rejection of non-matching content.
/// </summary>
public ImmutableArray<string> Keywords { get; init; } = [];
/// <summary>
/// Glob patterns for files this rule should be applied to.
/// Empty means all text files.
/// </summary>
public ImmutableArray<string> FilePatterns { get; init; } = [];
/// <summary>
/// Whether this rule is enabled.
/// </summary>
public bool Enabled { get; init; } = true;
/// <summary>
/// Minimum entropy threshold for entropy-based detection.
/// Only used when Type is Entropy or Composite.
/// </summary>
public double EntropyThreshold { get; init; } = 4.5;
/// <summary>
/// Minimum string length for entropy-based detection.
/// </summary>
public int MinLength { get; init; } = 16;
/// <summary>
/// Maximum string length for detection (prevents matching entire files).
/// </summary>
public int MaxLength { get; init; } = 1000;
/// <summary>
/// Optional metadata for the rule.
/// </summary>
public ImmutableDictionary<string, string> Metadata { get; init; } =
ImmutableDictionary<string, string>.Empty;
/// <summary>
/// The compiled regex pattern, created lazily.
/// </summary>
private Regex? _compiledPattern;
/// <summary>
/// Gets the compiled regex for this rule. Returns null if the pattern is invalid.
/// </summary>
public Regex? GetCompiledPattern()
{
if (Type == SecretRuleType.Entropy)
{
return null;
}
if (_compiledPattern is not null)
{
return _compiledPattern;
}
try
{
_compiledPattern = new Regex(
Pattern,
RegexOptions.Compiled | RegexOptions.CultureInvariant,
TimeSpan.FromSeconds(5));
return _compiledPattern;
}
catch (ArgumentException)
{
return null;
}
}
/// <summary>
/// Checks if the content might match this rule based on keywords.
/// Returns true if no keywords are defined or if any keyword is found.
/// </summary>
public bool MightMatch(ReadOnlySpan<char> content)
{
if (Keywords.IsDefaultOrEmpty)
{
return true;
}
foreach (var keyword in Keywords)
{
if (content.Contains(keyword.AsSpan(), StringComparison.OrdinalIgnoreCase))
{
return true;
}
}
return false;
}
/// <summary>
/// Checks if this rule should be applied to the given file path.
/// </summary>
public bool AppliesToFile(string filePath)
{
if (FilePatterns.IsDefaultOrEmpty)
{
return true;
}
var fileName = Path.GetFileName(filePath);
foreach (var pattern in FilePatterns)
{
if (MatchesGlob(fileName, pattern) || MatchesGlob(filePath, pattern))
{
return true;
}
}
return false;
}
private static bool MatchesGlob(string path, string pattern)
{
// Simple glob matching for common patterns
if (pattern.StartsWith("**", StringComparison.Ordinal))
{
var suffix = pattern[2..].TrimStart('/').TrimStart('\\');
if (suffix.StartsWith("*.", StringComparison.Ordinal))
{
var extension = suffix[1..];
return path.EndsWith(extension, StringComparison.OrdinalIgnoreCase);
}
return path.Contains(suffix, StringComparison.OrdinalIgnoreCase);
}
if (pattern.StartsWith("*.", StringComparison.Ordinal))
{
var extension = pattern[1..];
return path.EndsWith(extension, StringComparison.OrdinalIgnoreCase);
}
return path.Equals(pattern, StringComparison.OrdinalIgnoreCase);
}
}

View File

@@ -0,0 +1,22 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// The type of detection strategy used by a secret rule.
/// </summary>
public enum SecretRuleType
{
/// <summary>
/// Regex-based pattern matching.
/// </summary>
Regex = 0,
/// <summary>
/// Shannon entropy-based detection for high-entropy strings.
/// </summary>
Entropy = 1,
/// <summary>
/// Combined regex and entropy detection.
/// </summary>
Composite = 2
}

View File

@@ -0,0 +1,115 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// A versioned collection of secret detection rules.
/// </summary>
public sealed record SecretRuleset
{
/// <summary>
/// Bundle identifier (e.g., "secrets.ruleset").
/// </summary>
public required string Id { get; init; }
/// <summary>
/// Bundle version in YYYY.MM format (e.g., "2026.01").
/// </summary>
public required string Version { get; init; }
/// <summary>
/// When this bundle was created.
/// </summary>
public required DateTimeOffset CreatedAt { get; init; }
/// <summary>
/// The rules in this bundle.
/// </summary>
public required ImmutableArray<SecretRule> Rules { get; init; }
/// <summary>
/// SHA-256 digest of the rules file for integrity verification.
/// </summary>
public string? Sha256Digest { get; init; }
/// <summary>
/// Optional description of this bundle.
/// </summary>
public string? Description { get; init; }
/// <summary>
/// Gets only the enabled rules from this bundle.
/// </summary>
public IEnumerable<SecretRule> EnabledRules => Rules.Where(r => r.Enabled);
/// <summary>
/// Gets the count of enabled rules.
/// </summary>
public int EnabledRuleCount => Rules.Count(r => r.Enabled);
/// <summary>
/// Creates an empty ruleset.
/// </summary>
public static SecretRuleset Empty { get; } = new()
{
Id = "empty",
Version = "0.0",
CreatedAt = DateTimeOffset.MinValue,
Rules = []
};
/// <summary>
/// Validates that all rules in this bundle have valid patterns.
/// </summary>
/// <returns>A list of validation errors, empty if valid.</returns>
public IReadOnlyList<string> Validate()
{
var errors = new List<string>();
if (string.IsNullOrWhiteSpace(Id))
{
errors.Add("Bundle ID is required");
}
if (string.IsNullOrWhiteSpace(Version))
{
errors.Add("Bundle version is required");
}
var seenIds = new HashSet<string>(StringComparer.Ordinal);
foreach (var rule in Rules)
{
if (string.IsNullOrWhiteSpace(rule.Id))
{
errors.Add("Rule ID is required");
continue;
}
if (!seenIds.Add(rule.Id))
{
errors.Add($"Duplicate rule ID: {rule.Id}");
}
if (string.IsNullOrWhiteSpace(rule.Pattern))
{
errors.Add($"Rule '{rule.Id}' has no pattern");
}
if (rule.Type is SecretRuleType.Regex or SecretRuleType.Composite)
{
if (rule.GetCompiledPattern() is null)
{
errors.Add($"Rule '{rule.Id}' has invalid regex pattern");
}
}
}
return errors;
}
/// <summary>
/// Gets rules that apply to the specified file.
/// </summary>
public IEnumerable<SecretRule> GetRulesForFile(string filePath)
{
return EnabledRules.Where(r => r.AppliesToFile(filePath));
}
}

View File

@@ -0,0 +1,27 @@
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Severity level for a secret detection rule.
/// </summary>
public enum SecretSeverity
{
/// <summary>
/// Low severity - informational or low-risk credentials.
/// </summary>
Low = 0,
/// <summary>
/// Medium severity - credentials with limited scope or short lifespan.
/// </summary>
Medium = 1,
/// <summary>
/// High severity - production credentials with broad access.
/// </summary>
High = 2,
/// <summary>
/// Critical severity - highly privileged credentials requiring immediate action.
/// </summary>
Critical = 3
}

View File

@@ -0,0 +1,234 @@
using StellaOps.Scanner.Analyzers.Lang;
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Analyzer that detects accidentally committed secrets in container layers.
/// </summary>
public sealed class SecretsAnalyzer : ILanguageAnalyzer
{
private readonly IOptions<SecretsAnalyzerOptions> _options;
private readonly CompositeSecretDetector _detector;
private readonly IPayloadMasker _masker;
private readonly ILogger<SecretsAnalyzer> _logger;
private readonly TimeProvider _timeProvider;
private SecretRuleset? _ruleset;
public SecretsAnalyzer(
IOptions<SecretsAnalyzerOptions> options,
CompositeSecretDetector detector,
IPayloadMasker masker,
ILogger<SecretsAnalyzer> logger,
TimeProvider timeProvider)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
_detector = detector ?? throw new ArgumentNullException(nameof(detector));
_masker = masker ?? throw new ArgumentNullException(nameof(masker));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_timeProvider = timeProvider ?? throw new ArgumentNullException(nameof(timeProvider));
}
public string Id => "secrets";
public string DisplayName => "Secret Leak Detector";
/// <summary>
/// Gets whether the analyzer is enabled and has a valid ruleset.
/// </summary>
public bool IsEnabled => _options.Value.Enabled && _ruleset is not null;
/// <summary>
/// Gets the currently loaded ruleset.
/// </summary>
public SecretRuleset? Ruleset => _ruleset;
/// <summary>
/// Sets the ruleset to use for detection.
/// Called by SecretsAnalyzerHost after loading the bundle.
/// </summary>
internal void SetRuleset(SecretRuleset ruleset)
{
_ruleset = ruleset ?? throw new ArgumentNullException(nameof(ruleset));
}
public async ValueTask AnalyzeAsync(
LanguageAnalyzerContext context,
LanguageComponentWriter writer,
CancellationToken cancellationToken)
{
if (!IsEnabled)
{
_logger.LogDebug("Secrets analyzer is disabled or has no ruleset");
return;
}
var options = _options.Value;
var findings = new List<SecretLeakEvidence>();
var filesScanned = 0;
// Scan all text files in the root
foreach (var filePath in EnumerateTextFiles(context.RootPath, options))
{
cancellationToken.ThrowIfCancellationRequested();
if (findings.Count >= options.MaxFindingsPerScan)
{
_logger.LogWarning(
"Maximum findings limit ({MaxFindings}) reached, stopping scan",
options.MaxFindingsPerScan);
break;
}
var fileFindings = await ScanFileAsync(context, filePath, options, cancellationToken);
findings.AddRange(fileFindings);
filesScanned++;
}
_logger.LogInformation(
"Secrets scan complete: {FileCount} files scanned, {FindingCount} findings",
filesScanned,
findings.Count);
// Store findings in analysis store if available
if (context.AnalysisStore is not null && findings.Count > 0)
{
await StoreFindings(context.AnalysisStore, findings, cancellationToken);
}
}
private async ValueTask<List<SecretLeakEvidence>> ScanFileAsync(
LanguageAnalyzerContext context,
string filePath,
SecretsAnalyzerOptions options,
CancellationToken ct)
{
var findings = new List<SecretLeakEvidence>();
try
{
var fileInfo = new FileInfo(filePath);
if (fileInfo.Length > options.MaxFileSizeBytes)
{
_logger.LogDebug("Skipping large file: {FilePath} ({Size} bytes)", filePath, fileInfo.Length);
return findings;
}
var content = await File.ReadAllBytesAsync(filePath, ct);
var relativePath = context.GetRelativePath(filePath);
foreach (var rule in _ruleset!.GetRulesForFile(relativePath))
{
ct.ThrowIfCancellationRequested();
var matches = await _detector.DetectAsync(content, relativePath, rule, ct);
foreach (var match in matches)
{
// Check confidence threshold
var confidence = MapScoreToConfidence(match.ConfidenceScore);
if (confidence < options.MinConfidence)
{
continue;
}
var evidence = SecretLeakEvidence.FromMatch(match, _masker, _ruleset, _timeProvider);
findings.Add(evidence);
_logger.LogDebug(
"Found secret: Rule={RuleId}, File={FilePath}:{Line}, Mask={Mask}",
rule.Id,
relativePath,
match.LineNumber,
evidence.Mask);
}
}
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
_logger.LogWarning(ex, "Error scanning file: {FilePath}", filePath);
}
return findings;
}
private static IEnumerable<string> EnumerateTextFiles(string rootPath, SecretsAnalyzerOptions options)
{
var searchOptions = new EnumerationOptions
{
RecurseSubdirectories = true,
IgnoreInaccessible = true,
AttributesToSkip = FileAttributes.System | FileAttributes.Hidden
};
foreach (var file in Directory.EnumerateFiles(rootPath, "*", searchOptions))
{
var extension = Path.GetExtension(file).ToLowerInvariant();
// Check exclusions
if (options.ExcludeExtensions.Contains(extension))
{
continue;
}
// Check if directory is excluded
var relativePath = Path.GetRelativePath(rootPath, file).Replace('\\', '/');
if (IsExcludedDirectory(relativePath, options.ExcludeDirectories))
{
continue;
}
// Check inclusions if specified
if (options.IncludeExtensions.Count > 0 && !options.IncludeExtensions.Contains(extension))
{
continue;
}
yield return file;
}
}
private static bool IsExcludedDirectory(string relativePath, HashSet<string> patterns)
{
foreach (var pattern in patterns)
{
if (MatchesGlobPattern(relativePath, pattern))
{
return true;
}
}
return false;
}
private static bool MatchesGlobPattern(string path, string pattern)
{
if (pattern.StartsWith("**/", StringComparison.Ordinal))
{
var suffix = pattern[3..];
if (suffix.EndsWith("/**", StringComparison.Ordinal))
{
var middle = suffix[..^3];
return path.Contains(middle, StringComparison.OrdinalIgnoreCase);
}
return path.EndsWith(suffix, StringComparison.OrdinalIgnoreCase);
}
return path.StartsWith(pattern, StringComparison.OrdinalIgnoreCase);
}
private static SecretConfidence MapScoreToConfidence(double score) => score switch
{
>= 0.9 => SecretConfidence.High,
>= 0.7 => SecretConfidence.Medium,
_ => SecretConfidence.Low
};
private async ValueTask StoreFindings(
object analysisStore,
List<SecretLeakEvidence> findings,
CancellationToken ct)
{
// TODO: Store findings in ScanAnalysisStore when interface is defined
// For now, just log that we would store them
_logger.LogDebug("Would store {Count} secret findings in analysis store", findings.Count);
await ValueTask.CompletedTask;
}
}

View File

@@ -0,0 +1,186 @@
using Microsoft.Extensions.Hosting;
using StellaOps.Scanner.Analyzers.Secrets.Bundles;
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Hosted service that manages the lifecycle of the secrets analyzer.
/// Loads and validates the rule bundle on startup with optional signature verification.
/// Sprint: SPRINT_20260104_003_SCANNER - Secret Rule Bundles
/// </summary>
public sealed class SecretsAnalyzerHost : IHostedService
{
private readonly SecretsAnalyzer _analyzer;
private readonly IRulesetLoader _rulesetLoader;
private readonly IBundleVerifier? _bundleVerifier;
private readonly IOptions<SecretsAnalyzerOptions> _options;
private readonly ILogger<SecretsAnalyzerHost> _logger;
public SecretsAnalyzerHost(
SecretsAnalyzer analyzer,
IRulesetLoader rulesetLoader,
IOptions<SecretsAnalyzerOptions> options,
ILogger<SecretsAnalyzerHost> logger,
IBundleVerifier? bundleVerifier = null)
{
_analyzer = analyzer ?? throw new ArgumentNullException(nameof(analyzer));
_rulesetLoader = rulesetLoader ?? throw new ArgumentNullException(nameof(rulesetLoader));
_bundleVerifier = bundleVerifier;
_options = options ?? throw new ArgumentNullException(nameof(options));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
/// <summary>
/// Gets the bundle verification result from the last startup, if available.
/// </summary>
public BundleVerificationResult? LastVerificationResult { get; private set; }
/// <summary>
/// Gets whether the analyzer is enabled and has loaded successfully.
/// </summary>
public bool IsEnabled => _analyzer.IsEnabled;
/// <summary>
/// Gets the loaded bundle version, if available.
/// </summary>
public string? BundleVersion => _analyzer.Ruleset?.Version;
public async Task StartAsync(CancellationToken cancellationToken)
{
var options = _options.Value;
if (!options.Enabled)
{
_logger.LogInformation("SecretsAnalyzerHost: Secret leak detection is disabled");
return;
}
_logger.LogInformation("SecretsAnalyzerHost: Loading secrets rule bundle from {Path}", options.RulesetPath);
try
{
// Verify bundle signature if required
if (options.RequireSignatureVerification || _bundleVerifier is not null)
{
await VerifyBundleAsync(options, cancellationToken);
}
var ruleset = await _rulesetLoader.LoadAsync(options.RulesetPath, cancellationToken);
// Validate the ruleset
var errors = ruleset.Validate();
if (errors.Count > 0)
{
_logger.LogError(
"SecretsAnalyzerHost: Bundle validation failed with {ErrorCount} errors: {Errors}",
errors.Count,
string.Join(", ", errors));
if (options.FailOnInvalidBundle)
{
throw new InvalidOperationException(
$"Secret detection bundle validation failed: {string.Join(", ", errors)}");
}
return;
}
// Set the ruleset on the analyzer
_analyzer.SetRuleset(ruleset);
_logger.LogInformation(
"SecretsAnalyzerHost: Loaded bundle '{BundleId}' version {Version} with {RuleCount} rules ({EnabledCount} enabled)",
ruleset.Id,
ruleset.Version,
ruleset.Rules.Length,
ruleset.EnabledRuleCount);
}
catch (DirectoryNotFoundException ex)
{
_logger.LogWarning(ex, "SecretsAnalyzerHost: Bundle directory not found, analyzer disabled");
if (options.FailOnInvalidBundle)
{
throw;
}
}
catch (FileNotFoundException ex)
{
_logger.LogWarning(ex, "SecretsAnalyzerHost: Bundle file not found, analyzer disabled");
if (options.FailOnInvalidBundle)
{
throw;
}
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
_logger.LogError(ex, "SecretsAnalyzerHost: Failed to load bundle");
if (options.FailOnInvalidBundle)
{
throw;
}
}
}
public Task StopAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("SecretsAnalyzerHost: Shutting down");
return Task.CompletedTask;
}
private async Task VerifyBundleAsync(SecretsAnalyzerOptions options, CancellationToken ct)
{
if (_bundleVerifier is null)
{
if (options.RequireSignatureVerification)
{
throw new InvalidOperationException(
"Signature verification is required but no IBundleVerifier is registered.");
}
return;
}
var verificationOptions = new BundleVerificationOptions
{
RequireRekorProof = options.RequireRekorProof,
TrustedKeyIds = options.TrustedKeyIds.Count > 0 ? [.. options.TrustedKeyIds] : null,
SharedSecret = options.SignatureSecret,
SharedSecretFile = options.SignatureSecretFile,
VerifyIntegrity = true,
SkipSignatureVerification = !options.RequireSignatureVerification
};
_logger.LogDebug("SecretsAnalyzerHost: Verifying bundle signature");
var result = await _bundleVerifier.VerifyAsync(
options.RulesetPath,
verificationOptions,
ct);
LastVerificationResult = result;
if (!result.IsValid)
{
var errorMessage = $"Bundle verification failed: {string.Join("; ", result.ValidationErrors)}";
_logger.LogError("SecretsAnalyzerHost: {Error}", errorMessage);
if (options.FailOnInvalidBundle)
{
throw new InvalidOperationException(errorMessage);
}
// Allow loading but log prominently
_logger.LogWarning(
"SecretsAnalyzerHost: Continuing with unverified bundle. " +
"Set RequireSignatureVerification=true to enforce verification.");
return;
}
_logger.LogInformation(
"SecretsAnalyzerHost: Bundle verified - signed by {KeyId} at {SignedAt}",
result.SignerKeyId ?? "unknown",
result.SignedAt?.ToString("o") ?? "unknown");
}
}

View File

@@ -0,0 +1,118 @@
using System.ComponentModel.DataAnnotations;
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Configuration options for the secrets analyzer.
/// </summary>
public sealed class SecretsAnalyzerOptions
{
/// <summary>
/// Configuration section name.
/// </summary>
public const string SectionName = "Scanner:Analyzers:Secrets";
/// <summary>
/// Enable secret leak detection (experimental feature).
/// Default: false (opt-in).
/// </summary>
public bool Enabled { get; set; } = false;
/// <summary>
/// Path to the ruleset bundle directory.
/// </summary>
[Required]
public string RulesetPath { get; set; } = "/opt/stellaops/plugins/scanner/analyzers/secrets";
/// <summary>
/// Minimum confidence level to report findings.
/// </summary>
public SecretConfidence MinConfidence { get; set; } = SecretConfidence.Low;
/// <summary>
/// Maximum findings per scan (circuit breaker).
/// </summary>
[Range(1, 10000)]
public int MaxFindingsPerScan { get; set; } = 1000;
/// <summary>
/// Maximum file size to scan in bytes.
/// Files larger than this are skipped.
/// </summary>
[Range(1, 100 * 1024 * 1024)]
public long MaxFileSizeBytes { get; set; } = 10 * 1024 * 1024; // 10MB
/// <summary>
/// Enable entropy-based detection.
/// </summary>
public bool EnableEntropyDetection { get; set; } = true;
/// <summary>
/// Default entropy threshold (bits per character).
/// </summary>
[Range(3.0, 8.0)]
public double EntropyThreshold { get; set; } = 4.5;
/// <summary>
/// File extensions to scan. Empty means all text files.
/// </summary>
public HashSet<string> IncludeExtensions { get; set; } = [];
/// <summary>
/// File extensions to exclude from scanning.
/// </summary>
public HashSet<string> ExcludeExtensions { get; set; } =
[
".png", ".jpg", ".jpeg", ".gif", ".ico", ".webp",
".zip", ".tar", ".gz", ".bz2", ".xz",
".exe", ".dll", ".so", ".dylib",
".pdf", ".doc", ".docx", ".xls", ".xlsx",
".mp3", ".mp4", ".avi", ".mov", ".mkv",
".ttf", ".woff", ".woff2", ".eot",
".min.js", ".min.css"
];
/// <summary>
/// Directories to exclude from scanning (glob patterns).
/// </summary>
public HashSet<string> ExcludeDirectories { get; set; } =
[
"**/node_modules/**",
"**/.git/**",
"**/vendor/**",
"**/__pycache__/**",
"**/bin/**",
"**/obj/**"
];
/// <summary>
/// Whether to fail the scan if the bundle cannot be loaded.
/// </summary>
public bool FailOnInvalidBundle { get; set; } = false;
/// <summary>
/// Whether to require DSSE signature verification for bundles.
/// </summary>
public bool RequireSignatureVerification { get; set; } = false;
/// <summary>
/// Shared secret for HMAC signature verification (base64 or hex).
/// </summary>
public string? SignatureSecret { get; set; }
/// <summary>
/// Path to file containing the shared secret.
/// </summary>
public string? SignatureSecretFile { get; set; }
/// <summary>
/// List of trusted key IDs for signature verification.
/// If empty, any key is accepted.
/// </summary>
public HashSet<string> TrustedKeyIds { get; set; } = [];
/// <summary>
/// Whether to require Rekor transparency log proof.
/// </summary>
public bool RequireRekorProof { get; set; } = false;
}

View File

@@ -0,0 +1,90 @@
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using StellaOps.Scanner.Analyzers.Lang;
using StellaOps.Scanner.Analyzers.Secrets.Bundles;
namespace StellaOps.Scanner.Analyzers.Secrets;
/// <summary>
/// Extension methods for registering secrets analyzer services.
/// </summary>
public static class ServiceCollectionExtensions
{
/// <summary>
/// Adds the secrets analyzer services to the service collection.
/// </summary>
/// <param name="services">The service collection.</param>
/// <param name="configuration">The configuration.</param>
/// <returns>The service collection for chaining.</returns>
public static IServiceCollection AddSecretsAnalyzer(
this IServiceCollection services,
IConfiguration configuration)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentNullException.ThrowIfNull(configuration);
// Register options
services.AddOptions<SecretsAnalyzerOptions>()
.Bind(configuration.GetSection(SecretsAnalyzerOptions.SectionName))
.ValidateDataAnnotations()
.ValidateOnStart();
// Register TimeProvider if not already registered
services.TryAddSingleton(TimeProvider.System);
RegisterCoreServices(services);
return services;
}
/// <summary>
/// Adds the secrets analyzer services with custom options.
/// </summary>
/// <param name="services">The service collection.</param>
/// <param name="configureOptions">Action to configure options.</param>
/// <returns>The service collection for chaining.</returns>
public static IServiceCollection AddSecretsAnalyzer(
this IServiceCollection services,
Action<SecretsAnalyzerOptions> configureOptions)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentNullException.ThrowIfNull(configureOptions);
// Register options
services.AddOptions<SecretsAnalyzerOptions>()
.Configure(configureOptions)
.ValidateDataAnnotations()
.ValidateOnStart();
RegisterCoreServices(services);
return services;
}
private static void RegisterCoreServices(IServiceCollection services)
{
// Register TimeProvider if not already registered
services.TryAddSingleton(TimeProvider.System);
// Register core services
services.AddSingleton<IPayloadMasker, PayloadMasker>();
services.AddSingleton<IRulesetLoader, RulesetLoader>();
// Register detectors
services.AddSingleton<RegexDetector>();
services.AddSingleton<EntropyDetector>();
services.AddSingleton<CompositeSecretDetector>();
// Register bundle infrastructure (Sprint: SPRINT_20260104_003_SCANNER)
services.AddSingleton<IRuleValidator, RuleValidator>();
services.AddSingleton<IBundleBuilder, BundleBuilder>();
services.AddSingleton<IBundleSigner, BundleSigner>();
services.AddSingleton<IBundleVerifier, BundleVerifier>();
// Register analyzer
services.AddSingleton<SecretsAnalyzer>();
services.AddSingleton<ILanguageAnalyzer>(sp => sp.GetRequiredService<SecretsAnalyzer>());
// Register hosted service
services.AddSingleton<SecretsAnalyzerHost>();
services.AddHostedService(sp => sp.GetRequiredService<SecretsAnalyzerHost>());
}
}

View File

@@ -0,0 +1,32 @@
<?xml version='1.0' encoding='utf-8'?>
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<EnableDefaultItems>false</EnableDefaultItems>
</PropertyGroup>
<ItemGroup>
<Compile Include="**\*.cs" Exclude="obj\**;bin\**" />
<EmbeddedResource Include="**\*.json" Exclude="obj\**;bin\**" />
<None Include="**\*" Exclude="**\*.cs;**\*.json;bin\**;obj\**" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="../StellaOps.Scanner.Core/StellaOps.Scanner.Core.csproj" />
<ProjectReference Include="../StellaOps.Scanner.Analyzers.Lang/StellaOps.Scanner.Analyzers.Lang.csproj" />
<ProjectReference Include="../../../__Libraries/StellaOps.Evidence.Core/StellaOps.Evidence.Core.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Hosting.Abstractions" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" />
<PackageReference Include="Microsoft.Extensions.Options" />
<PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" />
<PackageReference Include="Microsoft.Extensions.Options.DataAnnotations" />
</ItemGroup>
</Project>

View File

@@ -6,6 +6,7 @@
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using StellaOps.Scanner.CallGraph.Binary;
using StellaOps.Scanner.CallGraph.Caching;
using StellaOps.Scanner.CallGraph.DotNet;
using StellaOps.Scanner.CallGraph.Go;
@@ -40,6 +41,7 @@ public static class CallGraphServiceCollectionExtensions
services.AddSingleton<ICallGraphExtractor, NodeCallGraphExtractor>(); // Node.js/JavaScript via Babel
services.AddSingleton<ICallGraphExtractor, PythonCallGraphExtractor>(); // Python via AST analysis
services.AddSingleton<ICallGraphExtractor, GoCallGraphExtractor>(); // Go via SSA analysis
services.AddSingleton<ICallGraphExtractor, BinaryCallGraphExtractor>(); // Native ELF/PE/Mach-O binaries
// Register the extractor registry for language-based lookup
services.AddSingleton<ICallGraphExtractorRegistry, CallGraphExtractorRegistry>();

View File

@@ -50,4 +50,8 @@ public static class ScanAnalysisKeys
public const string VulnerabilityMatches = "analysis.poe.vulnerability.matches";
public const string PoEResults = "analysis.poe.results";
public const string PoEConfiguration = "analysis.poe.configuration";
// Sprint: SPRINT_20251229_046_BE - Secrets Leak Detection
public const string SecretFindings = "analysis.secrets.findings";
public const string SecretRulesetVersion = "analysis.secrets.ruleset.version";
}

View File

@@ -0,0 +1,264 @@
// -----------------------------------------------------------------------------
// BundleBuilderTests.cs
// Sprint: SPRINT_20260104_003_SCANNER (Secret Detection Rule Bundles)
// Task: RB-011 - Unit tests for bundle building.
// -----------------------------------------------------------------------------
using System.Text.Json;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Time.Testing;
using StellaOps.Scanner.Analyzers.Secrets;
using StellaOps.Scanner.Analyzers.Secrets.Bundles;
using Xunit;
namespace StellaOps.Scanner.Analyzers.Secrets.Tests.Bundles;
[Trait("Category", "Unit")]
public class BundleBuilderTests : IDisposable
{
private readonly string _tempDir;
private readonly string _sourceDir;
private readonly string _outputDir;
private readonly BundleBuilder _sut;
private readonly FakeTimeProvider _timeProvider;
public BundleBuilderTests()
{
_tempDir = Path.Combine(Path.GetTempPath(), $"bundle-test-{Guid.NewGuid():N}");
_sourceDir = Path.Combine(_tempDir, "sources");
_outputDir = Path.Combine(_tempDir, "output");
Directory.CreateDirectory(_sourceDir);
Directory.CreateDirectory(_outputDir);
var validator = new RuleValidator(NullLogger<RuleValidator>.Instance);
_sut = new BundleBuilder(validator, NullLogger<BundleBuilder>.Instance);
_timeProvider = new FakeTimeProvider(new DateTimeOffset(2026, 1, 4, 12, 0, 0, TimeSpan.Zero));
}
public void Dispose()
{
if (Directory.Exists(_tempDir))
{
Directory.Delete(_tempDir, recursive: true);
}
}
[Fact]
public async Task BuildAsync_ValidRules_CreatesBundle()
{
// Arrange
var rule1Path = CreateRuleFile("rule1.json", new SecretRule
{
Id = "stellaops.secrets.test-rule1",
Version = "1.0.0",
Name = "Test Rule 1",
Description = "A test rule for validation",
Type = SecretRuleType.Regex,
Pattern = "[A-Z]{10}",
Severity = SecretSeverity.High,
Confidence = SecretConfidence.High,
Enabled = true
});
var rule2Path = CreateRuleFile("rule2.json", new SecretRule
{
Id = "stellaops.secrets.test-rule2",
Version = "1.0.0",
Name = "Test Rule 2",
Description = "Another test rule",
Type = SecretRuleType.Regex,
Pattern = "[0-9]{8}",
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium,
Enabled = true
});
var options = new BundleBuildOptions
{
RuleFiles = new[] { rule1Path, rule2Path },
OutputDirectory = _outputDir,
BundleId = "test-bundle",
Version = "2026.01",
TimeProvider = _timeProvider
};
// Act
var artifact = await _sut.BuildAsync(options, TestContext.Current.CancellationToken);
// Assert
Assert.NotNull(artifact);
Assert.Equal("test-bundle", artifact.Manifest.Id);
Assert.Equal("2026.01", artifact.Manifest.Version);
Assert.Equal(2, artifact.TotalRules);
Assert.Equal(2, artifact.EnabledRules);
Assert.True(File.Exists(artifact.ManifestPath));
Assert.True(File.Exists(artifact.RulesPath));
}
[Fact]
public async Task BuildAsync_SortsRulesById()
{
// Arrange
var zebraPath = CreateRuleFile("z-rule.json", new SecretRule
{
Id = "stellaops.secrets.zebra",
Version = "1.0.0",
Name = "Zebra Rule",
Description = "Rule that should sort last",
Type = SecretRuleType.Regex,
Pattern = "zebra",
Severity = SecretSeverity.Low,
Confidence = SecretConfidence.Medium,
Enabled = true
});
var alphaPath = CreateRuleFile("a-rule.json", new SecretRule
{
Id = "stellaops.secrets.alpha",
Version = "1.0.0",
Name = "Alpha Rule",
Description = "Rule that should sort first",
Type = SecretRuleType.Regex,
Pattern = "alpha",
Severity = SecretSeverity.High,
Confidence = SecretConfidence.High,
Enabled = true
});
var options = new BundleBuildOptions
{
RuleFiles = new[] { zebraPath, alphaPath },
OutputDirectory = _outputDir,
BundleId = "sorted-bundle",
Version = "1.0.0",
TimeProvider = _timeProvider
};
// Act
var artifact = await _sut.BuildAsync(options, TestContext.Current.CancellationToken);
// Assert - check the manifest rules array is sorted (manifest is already built)
Assert.Equal(2, artifact.Manifest.Rules.Length);
Assert.Equal("stellaops.secrets.alpha", artifact.Manifest.Rules[0].Id);
Assert.Equal("stellaops.secrets.zebra", artifact.Manifest.Rules[1].Id);
}
[Fact]
public async Task BuildAsync_ComputesCorrectSha256()
{
// Arrange
var rulePath = CreateRuleFile("rule.json", new SecretRule
{
Id = "stellaops.secrets.hash-test",
Version = "1.0.0",
Name = "Hash Test",
Description = "Rule for testing SHA-256 computation",
Type = SecretRuleType.Regex,
Pattern = "test123",
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium,
Enabled = true
});
var options = new BundleBuildOptions
{
RuleFiles = new[] { rulePath },
OutputDirectory = _outputDir,
BundleId = "hash-bundle",
Version = "1.0.0",
TimeProvider = _timeProvider
};
// Act
var artifact = await _sut.BuildAsync(options, TestContext.Current.CancellationToken);
// Assert
Assert.NotEmpty(artifact.RulesSha256);
Assert.Matches("^[a-f0-9]{64}$", artifact.RulesSha256);
// Verify hash matches file content
await using var stream = File.OpenRead(artifact.RulesPath);
var hash = await System.Security.Cryptography.SHA256.HashDataAsync(stream, TestContext.Current.CancellationToken);
var expectedHash = Convert.ToHexString(hash).ToLowerInvariant();
Assert.Equal(expectedHash, artifact.RulesSha256);
}
[Fact]
public async Task BuildAsync_InvalidRule_ThrowsException()
{
// Arrange - create an invalid rule (id not properly namespaced)
var invalidRulePath = Path.Combine(_sourceDir, "invalid-rule.json");
var invalidRuleJson = JsonSerializer.Serialize(new
{
id = "invalid", // Not namespaced with stellaops.secrets
version = "1.0.0",
name = "Invalid Rule",
description = "This rule has an invalid ID",
type = "regex",
pattern = "test",
severity = "medium",
confidence = "medium"
}, new JsonSerializerOptions { WriteIndented = true });
await File.WriteAllTextAsync(invalidRulePath, invalidRuleJson, TestContext.Current.CancellationToken);
var options = new BundleBuildOptions
{
RuleFiles = new[] { invalidRulePath },
OutputDirectory = _outputDir,
BundleId = "invalid-bundle",
Version = "1.0.0",
TimeProvider = _timeProvider
};
// Act & Assert
await Assert.ThrowsAsync<InvalidOperationException>(
() => _sut.BuildAsync(options, TestContext.Current.CancellationToken));
}
[Fact]
public async Task BuildAsync_EmptyRuleFiles_ThrowsException()
{
// Arrange
var options = new BundleBuildOptions
{
RuleFiles = Array.Empty<string>(),
OutputDirectory = _outputDir,
BundleId = "empty-bundle",
Version = "1.0.0"
};
// Act & Assert
await Assert.ThrowsAsync<InvalidOperationException>(
() => _sut.BuildAsync(options, TestContext.Current.CancellationToken));
}
[Fact]
public async Task BuildAsync_NonexistentRuleFile_ThrowsException()
{
// Arrange
var options = new BundleBuildOptions
{
RuleFiles = new[] { Path.Combine(_tempDir, "nonexistent.json") },
OutputDirectory = _outputDir,
BundleId = "missing-bundle",
Version = "1.0.0"
};
// Act & Assert
await Assert.ThrowsAsync<InvalidOperationException>(
() => _sut.BuildAsync(options, TestContext.Current.CancellationToken));
}
private string CreateRuleFile(string filename, SecretRule rule)
{
var json = JsonSerializer.Serialize(rule, new JsonSerializerOptions
{
WriteIndented = true,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
});
var path = Path.Combine(_sourceDir, filename);
File.WriteAllText(path, json);
return path;
}
}

View File

@@ -0,0 +1,217 @@
// -----------------------------------------------------------------------------
// BundleSignerTests.cs
// Sprint: SPRINT_20260104_003_SCANNER (Secret Detection Rule Bundles)
// Task: RB-011 - Unit tests for bundle signing.
// -----------------------------------------------------------------------------
using System.Text.Json;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Time.Testing;
using StellaOps.Scanner.Analyzers.Secrets;
using StellaOps.Scanner.Analyzers.Secrets.Bundles;
using Xunit;
namespace StellaOps.Scanner.Analyzers.Secrets.Tests.Bundles;
[Trait("Category", "Unit")]
public class BundleSignerTests : IDisposable
{
private readonly string _tempDir;
private readonly BundleSigner _sut;
private readonly FakeTimeProvider _timeProvider;
public BundleSignerTests()
{
_tempDir = Path.Combine(Path.GetTempPath(), $"signer-test-{Guid.NewGuid():N}");
Directory.CreateDirectory(_tempDir);
_sut = new BundleSigner(NullLogger<BundleSigner>.Instance);
_timeProvider = new FakeTimeProvider(new DateTimeOffset(2026, 1, 4, 12, 0, 0, TimeSpan.Zero));
}
public void Dispose()
{
if (Directory.Exists(_tempDir))
{
Directory.Delete(_tempDir, recursive: true);
}
}
[Fact]
public async Task SignAsync_ValidArtifact_CreatesDsseEnvelope()
{
// Arrange
var artifact = CreateTestArtifact();
var options = new BundleSigningOptions
{
KeyId = "test-key-001",
SharedSecret = Convert.ToBase64String(new byte[32]), // 256-bit key
TimeProvider = _timeProvider
};
// Act
var result = await _sut.SignAsync(artifact, options, TestContext.Current.CancellationToken);
// Assert
Assert.NotNull(result);
Assert.True(File.Exists(result.EnvelopePath));
Assert.NotNull(result.Envelope);
Assert.Single(result.Envelope.Signatures);
Assert.Equal("test-key-001", result.Envelope.Signatures[0].KeyId);
}
[Fact]
public async Task SignAsync_UpdatesManifestWithSignatureInfo()
{
// Arrange
var artifact = CreateTestArtifact();
var options = new BundleSigningOptions
{
KeyId = "signer-key",
SharedSecret = Convert.ToBase64String(new byte[32]),
TimeProvider = _timeProvider
};
// Act
var result = await _sut.SignAsync(artifact, options, TestContext.Current.CancellationToken);
// Assert
Assert.NotNull(result.UpdatedManifest.Signatures);
Assert.Equal("signer-key", result.UpdatedManifest.Signatures.KeyId);
Assert.Equal("secrets.ruleset.dsse.json", result.UpdatedManifest.Signatures.DsseEnvelope);
Assert.Equal(_timeProvider.GetUtcNow(), result.UpdatedManifest.Signatures.SignedAt);
}
[Fact]
public async Task SignAsync_EnvelopeContainsBase64UrlPayload()
{
// Arrange
var artifact = CreateTestArtifact();
var options = new BundleSigningOptions
{
KeyId = "test-key",
SharedSecret = Convert.ToBase64String(new byte[32]),
TimeProvider = _timeProvider
};
// Act
var result = await _sut.SignAsync(artifact, options, TestContext.Current.CancellationToken);
// Assert
Assert.NotEmpty(result.Envelope.Payload);
// Base64url should not contain +, /, or =
Assert.DoesNotContain("+", result.Envelope.Payload);
Assert.DoesNotContain("/", result.Envelope.Payload);
Assert.DoesNotContain("=", result.Envelope.Payload);
}
[Fact]
public async Task SignAsync_WithSecretFile_LoadsSecret()
{
// Arrange
var artifact = CreateTestArtifact();
var secretFile = Path.Combine(_tempDir, "secret.key");
var secret = Convert.ToBase64String(new byte[32]);
await File.WriteAllTextAsync(secretFile, secret, TestContext.Current.CancellationToken);
var options = new BundleSigningOptions
{
KeyId = "file-key",
SharedSecretFile = secretFile,
TimeProvider = _timeProvider
};
// Act
var result = await _sut.SignAsync(artifact, options, TestContext.Current.CancellationToken);
// Assert
Assert.NotNull(result);
Assert.NotNull(result.Envelope);
}
[Fact]
public async Task SignAsync_WithoutSecret_ThrowsException()
{
// Arrange
var artifact = CreateTestArtifact();
var options = new BundleSigningOptions
{
KeyId = "no-secret-key",
TimeProvider = _timeProvider
};
// Act & Assert
await Assert.ThrowsAsync<InvalidOperationException>(
() => _sut.SignAsync(artifact, options, TestContext.Current.CancellationToken));
}
[Fact]
public async Task SignAsync_UnsupportedAlgorithm_ThrowsException()
{
// Arrange
var artifact = CreateTestArtifact();
var options = new BundleSigningOptions
{
KeyId = "test-key",
SharedSecret = Convert.ToBase64String(new byte[32]),
Algorithm = "ES256", // Not supported
TimeProvider = _timeProvider
};
// Act & Assert
await Assert.ThrowsAsync<NotSupportedException>(
() => _sut.SignAsync(artifact, options, TestContext.Current.CancellationToken));
}
[Fact]
public async Task SignAsync_HexEncodedSecret_Works()
{
// Arrange
var artifact = CreateTestArtifact();
var hexSecret = new string('a', 64); // 32 bytes as hex
var options = new BundleSigningOptions
{
KeyId = "hex-key",
SharedSecret = hexSecret,
TimeProvider = _timeProvider
};
// Act
var result = await _sut.SignAsync(artifact, options, TestContext.Current.CancellationToken);
// Assert
Assert.NotNull(result);
}
private BundleArtifact CreateTestArtifact()
{
var manifest = new BundleManifest
{
Id = "test-bundle",
Version = "1.0.0",
CreatedAt = _timeProvider.GetUtcNow(),
Integrity = new BundleIntegrity
{
RulesSha256 = new string('0', 64),
TotalRules = 1,
EnabledRules = 1
}
};
var manifestPath = Path.Combine(_tempDir, "secrets.ruleset.manifest.json");
var rulesPath = Path.Combine(_tempDir, "secrets.ruleset.rules.jsonl");
File.WriteAllText(manifestPath, JsonSerializer.Serialize(manifest, new JsonSerializerOptions { WriteIndented = true }));
File.WriteAllText(rulesPath, "{\"id\":\"test.rule\"}");
return new BundleArtifact
{
ManifestPath = manifestPath,
RulesPath = rulesPath,
RulesSha256 = manifest.Integrity.RulesSha256,
TotalRules = 1,
EnabledRules = 1,
Manifest = manifest
};
}
}

View File

@@ -0,0 +1,328 @@
// -----------------------------------------------------------------------------
// BundleVerifierTests.cs
// Sprint: SPRINT_20260104_003_SCANNER (Secret Detection Rule Bundles)
// Task: RB-011 - Unit tests for bundle verification.
// -----------------------------------------------------------------------------
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Time.Testing;
using StellaOps.Scanner.Analyzers.Secrets;
using StellaOps.Scanner.Analyzers.Secrets.Bundles;
using Xunit;
namespace StellaOps.Scanner.Analyzers.Secrets.Tests.Bundles;
[Trait("Category", "Unit")]
public class BundleVerifierTests : IDisposable
{
private readonly string _tempDir;
private readonly BundleVerifier _sut;
private readonly FakeTimeProvider _timeProvider;
private readonly byte[] _testSecret;
public BundleVerifierTests()
{
_tempDir = Path.Combine(Path.GetTempPath(), $"verifier-test-{Guid.NewGuid():N}");
Directory.CreateDirectory(_tempDir);
_sut = new BundleVerifier(NullLogger<BundleVerifier>.Instance);
_timeProvider = new FakeTimeProvider(new DateTimeOffset(2026, 1, 4, 12, 0, 0, TimeSpan.Zero));
_testSecret = new byte[32];
RandomNumberGenerator.Fill(_testSecret);
}
public void Dispose()
{
if (Directory.Exists(_tempDir))
{
Directory.Delete(_tempDir, recursive: true);
}
}
[Fact]
public async Task VerifyAsync_ValidBundle_ReturnsValid()
{
// Arrange
var bundleDir = await CreateSignedBundleAsync(TestContext.Current.CancellationToken);
var options = new BundleVerificationOptions
{
SharedSecret = Convert.ToBase64String(_testSecret),
VerifyIntegrity = true
};
// Act
var result = await _sut.VerifyAsync(bundleDir, options, TestContext.Current.CancellationToken);
// Assert
Assert.True(result.IsValid);
Assert.Equal("test-bundle", result.BundleId);
Assert.Equal("1.0.0", result.BundleVersion);
Assert.Empty(result.ValidationErrors);
}
[Fact]
public async Task VerifyAsync_TamperedRulesFile_ReturnsInvalid()
{
// Arrange
var bundleDir = await CreateSignedBundleAsync(TestContext.Current.CancellationToken);
// Tamper with the rules file
var rulesPath = Path.Combine(bundleDir, "secrets.ruleset.rules.jsonl");
await File.AppendAllTextAsync(rulesPath, "\n{\"id\":\"injected.rule\"}", TestContext.Current.CancellationToken);
var options = new BundleVerificationOptions
{
SharedSecret = Convert.ToBase64String(_testSecret),
VerifyIntegrity = true
};
// Act
var result = await _sut.VerifyAsync(bundleDir, options, TestContext.Current.CancellationToken);
// Assert
Assert.False(result.IsValid);
Assert.Contains(result.ValidationErrors, e => e.Contains("integrity", StringComparison.OrdinalIgnoreCase));
}
[Fact]
public async Task VerifyAsync_WrongSecret_ReturnsInvalid()
{
// Arrange
var bundleDir = await CreateSignedBundleAsync(TestContext.Current.CancellationToken);
var wrongSecret = new byte[32];
RandomNumberGenerator.Fill(wrongSecret);
var options = new BundleVerificationOptions
{
SharedSecret = Convert.ToBase64String(wrongSecret),
VerifyIntegrity = true
};
// Act
var result = await _sut.VerifyAsync(bundleDir, options, TestContext.Current.CancellationToken);
// Assert
Assert.False(result.IsValid);
Assert.Contains(result.ValidationErrors, e => e.Contains("signature", StringComparison.OrdinalIgnoreCase));
}
[Fact]
public async Task VerifyAsync_MissingManifest_ReturnsInvalid()
{
// Arrange
var bundleDir = Path.Combine(_tempDir, "missing-manifest");
Directory.CreateDirectory(bundleDir);
var options = new BundleVerificationOptions
{
VerifyIntegrity = true
};
// Act
var result = await _sut.VerifyAsync(bundleDir, options, TestContext.Current.CancellationToken);
// Assert
Assert.False(result.IsValid);
Assert.Contains(result.ValidationErrors, e => e.Contains("manifest", StringComparison.OrdinalIgnoreCase));
}
[Fact]
public async Task VerifyAsync_NonexistentDirectory_ReturnsInvalid()
{
// Arrange
var options = new BundleVerificationOptions();
// Act
var result = await _sut.VerifyAsync(Path.Combine(_tempDir, "nonexistent"), options, TestContext.Current.CancellationToken);
// Assert
Assert.False(result.IsValid);
Assert.Contains(result.ValidationErrors, e => e.Contains("not found", StringComparison.OrdinalIgnoreCase));
}
[Fact]
public async Task VerifyAsync_SkipSignatureVerification_OnlyChecksIntegrity()
{
// Arrange
var bundleDir = await CreateUnsignedBundleAsync(TestContext.Current.CancellationToken);
var options = new BundleVerificationOptions
{
SkipSignatureVerification = true,
VerifyIntegrity = true
};
// Act
var result = await _sut.VerifyAsync(bundleDir, options, TestContext.Current.CancellationToken);
// Assert
Assert.True(result.IsValid);
}
[Fact]
public async Task VerifyAsync_UntrustedKeyId_ReturnsInvalid()
{
// Arrange
var bundleDir = await CreateSignedBundleAsync(TestContext.Current.CancellationToken);
var options = new BundleVerificationOptions
{
SharedSecret = Convert.ToBase64String(_testSecret),
TrustedKeyIds = new[] { "other-trusted-key" },
VerifyIntegrity = true
};
// Act
var result = await _sut.VerifyAsync(bundleDir, options, TestContext.Current.CancellationToken);
// Assert
Assert.False(result.IsValid);
Assert.Contains(result.ValidationErrors, e => e.Contains("trusted", StringComparison.OrdinalIgnoreCase));
}
[Fact]
public async Task VerifyAsync_TrustedKeyId_ReturnsValid()
{
// Arrange
var bundleDir = await CreateSignedBundleAsync(TestContext.Current.CancellationToken);
var options = new BundleVerificationOptions
{
SharedSecret = Convert.ToBase64String(_testSecret),
TrustedKeyIds = new[] { "test-key-001" },
VerifyIntegrity = true
};
// Act
var result = await _sut.VerifyAsync(bundleDir, options, TestContext.Current.CancellationToken);
// Assert
Assert.True(result.IsValid);
Assert.Equal("test-key-001", result.SignerKeyId);
}
[Fact]
public async Task VerifyAsync_RequireRekorProof_ReturnsWarningWhenNotVerified()
{
// Arrange
var bundleDir = await CreateSignedBundleWithRekorAsync(TestContext.Current.CancellationToken);
var options = new BundleVerificationOptions
{
SharedSecret = Convert.ToBase64String(_testSecret),
RequireRekorProof = true,
VerifyIntegrity = true
};
// Act
var result = await _sut.VerifyAsync(bundleDir, options, TestContext.Current.CancellationToken);
// Assert (Rekor verification not implemented, should have warning)
Assert.NotEmpty(result.ValidationWarnings);
Assert.Contains(result.ValidationWarnings, w => w.Contains("Rekor", StringComparison.OrdinalIgnoreCase));
}
private async Task<string> CreateUnsignedBundleAsync(CancellationToken ct = default)
{
var bundleDir = Path.Combine(_tempDir, $"bundle-{Guid.NewGuid():N}");
Directory.CreateDirectory(bundleDir);
// Create rules file
var rulesPath = Path.Combine(bundleDir, "secrets.ruleset.rules.jsonl");
var ruleJson = JsonSerializer.Serialize(new SecretRule
{
Id = "stellaops.secrets.test-rule",
Version = "1.0.0",
Name = "Test",
Description = "A test rule for verification tests",
Type = SecretRuleType.Regex,
Pattern = "test",
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium
});
await File.WriteAllTextAsync(rulesPath, ruleJson, ct);
// Compute hash
await using var stream = File.OpenRead(rulesPath);
var hash = await SHA256.HashDataAsync(stream, ct);
var hashHex = Convert.ToHexString(hash).ToLowerInvariant();
// Create manifest
var manifest = new BundleManifest
{
Id = "test-bundle",
Version = "1.0.0",
CreatedAt = _timeProvider.GetUtcNow(),
Integrity = new BundleIntegrity
{
RulesSha256 = hashHex,
TotalRules = 1,
EnabledRules = 1
}
};
var manifestPath = Path.Combine(bundleDir, "secrets.ruleset.manifest.json");
await File.WriteAllTextAsync(manifestPath,
JsonSerializer.Serialize(manifest, new JsonSerializerOptions { WriteIndented = true }), ct);
return bundleDir;
}
private async Task<string> CreateSignedBundleAsync(CancellationToken ct = default)
{
var bundleDir = await CreateUnsignedBundleAsync(ct);
// Sign the bundle
var signer = new BundleSigner(NullLogger<BundleSigner>.Instance);
var manifestPath = Path.Combine(bundleDir, "secrets.ruleset.manifest.json");
var manifestJson = await File.ReadAllTextAsync(manifestPath, ct);
var manifest = JsonSerializer.Deserialize<BundleManifest>(manifestJson,
new JsonSerializerOptions { PropertyNameCaseInsensitive = true })!;
var artifact = new BundleArtifact
{
ManifestPath = manifestPath,
RulesPath = Path.Combine(bundleDir, "secrets.ruleset.rules.jsonl"),
RulesSha256 = manifest.Integrity.RulesSha256,
TotalRules = 1,
EnabledRules = 1,
Manifest = manifest
};
await signer.SignAsync(artifact, new BundleSigningOptions
{
KeyId = "test-key-001",
SharedSecret = Convert.ToBase64String(_testSecret),
TimeProvider = _timeProvider
}, ct);
return bundleDir;
}
private async Task<string> CreateSignedBundleWithRekorAsync(CancellationToken ct = default)
{
var bundleDir = await CreateSignedBundleAsync(ct);
// Update manifest to include Rekor log ID
var manifestPath = Path.Combine(bundleDir, "secrets.ruleset.manifest.json");
var manifestJson = await File.ReadAllTextAsync(manifestPath, ct);
var manifest = JsonSerializer.Deserialize<BundleManifest>(manifestJson,
new JsonSerializerOptions { PropertyNameCaseInsensitive = true })!;
var updatedManifest = manifest with
{
Signatures = manifest.Signatures! with
{
RekorLogId = "rekor-log-entry-123456"
}
};
await File.WriteAllTextAsync(manifestPath,
JsonSerializer.Serialize(updatedManifest, new JsonSerializerOptions { WriteIndented = true }), ct);
return bundleDir;
}
}

View File

@@ -0,0 +1,228 @@
// -----------------------------------------------------------------------------
// RuleValidatorTests.cs
// Sprint: SPRINT_20260104_003_SCANNER (Secret Detection Rule Bundles)
// Task: RB-011 - Unit tests for rule validation.
// -----------------------------------------------------------------------------
using Microsoft.Extensions.Logging.Abstractions;
using StellaOps.Scanner.Analyzers.Secrets;
using StellaOps.Scanner.Analyzers.Secrets.Bundles;
using Xunit;
namespace StellaOps.Scanner.Analyzers.Secrets.Tests.Bundles;
[Trait("Category", "Unit")]
public class RuleValidatorTests
{
private readonly RuleValidator _sut;
public RuleValidatorTests()
{
_sut = new RuleValidator(NullLogger<RuleValidator>.Instance);
}
[Fact]
public void Validate_ValidRule_ReturnsValid()
{
// Arrange
var rule = new SecretRule
{
Id = "stellaops.secrets.test-rule",
Version = "1.0.0",
Name = "Test Rule",
Description = "A test rule for validation",
Type = SecretRuleType.Regex,
Pattern = "[A-Z]{10}",
Severity = SecretSeverity.High,
Confidence = SecretConfidence.High
};
// Act
var result = _sut.Validate(rule);
// Assert
Assert.True(result.IsValid);
Assert.Empty(result.Errors);
}
[Theory]
[InlineData("")]
[InlineData("invalid-id")] // No namespace separator (no dots)
[InlineData("InvalidCase.rule")] // Starts with uppercase
public void Validate_InvalidId_ReturnsError(string invalidId)
{
// Arrange
var rule = new SecretRule
{
Id = invalidId,
Version = "1.0.0",
Name = "Test Rule",
Description = "Test description",
Type = SecretRuleType.Regex,
Pattern = "[A-Z]{10}",
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium
};
// Act
var result = _sut.Validate(rule);
// Assert
Assert.False(result.IsValid);
Assert.Contains(result.Errors, e => e.Contains("ID"));
}
[Theory]
[InlineData("")]
[InlineData("invalid")]
[InlineData("1.0")]
[InlineData("v1.0.0")]
public void Validate_InvalidVersion_ReturnsError(string invalidVersion)
{
// Arrange
var rule = new SecretRule
{
Id = "stellaops.secrets.test",
Version = invalidVersion,
Name = "Test Rule",
Description = "Test description",
Type = SecretRuleType.Regex,
Pattern = "[A-Z]{10}",
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium
};
// Act
var result = _sut.Validate(rule);
// Assert
Assert.False(result.IsValid);
Assert.Contains(result.Errors, e => e.Contains("version", StringComparison.OrdinalIgnoreCase));
}
[Theory]
[InlineData("[")]
[InlineData("(unclosed")]
[InlineData("(?invalid)")]
public void Validate_InvalidRegex_ReturnsError(string invalidPattern)
{
// Arrange
var rule = new SecretRule
{
Id = "stellaops.secrets.test",
Version = "1.0.0",
Name = "Test Rule",
Description = "Test description",
Type = SecretRuleType.Regex,
Pattern = invalidPattern,
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium
};
// Act
var result = _sut.Validate(rule);
// Assert
Assert.False(result.IsValid);
Assert.Contains(result.Errors, e => e.Contains("regex", StringComparison.OrdinalIgnoreCase) ||
e.Contains("pattern", StringComparison.OrdinalIgnoreCase));
}
[Fact]
public void Validate_EmptyPattern_ReturnsError()
{
// Arrange
var rule = new SecretRule
{
Id = "stellaops.secrets.test",
Version = "1.0.0",
Name = "Test Rule",
Description = "Test description",
Type = SecretRuleType.Regex,
Pattern = "",
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium
};
// Act
var result = _sut.Validate(rule);
// Assert
Assert.False(result.IsValid);
Assert.Contains(result.Errors, e => e.Contains("pattern", StringComparison.OrdinalIgnoreCase));
}
[Fact]
public void Validate_ValidEntropyRule_ReturnsValid()
{
// Arrange
var rule = new SecretRule
{
Id = "stellaops.secrets.entropy-test",
Version = "1.0.0",
Name = "Entropy Test",
Description = "Detects high-entropy strings",
Type = SecretRuleType.Entropy,
Pattern = "", // Pattern can be empty for entropy rules
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium,
EntropyThreshold = 4.5
};
// Act
var result = _sut.Validate(rule);
// Assert
Assert.True(result.IsValid);
}
[Fact]
public void Validate_EntropyRuleWithDefaultThreshold_ReturnsValid()
{
// Arrange - using the default entropy threshold (4.5) which is in valid range
var rule = new SecretRule
{
Id = "stellaops.secrets.entropy-test",
Version = "1.0.0",
Name = "Entropy Test",
Description = "Detects high-entropy strings with default threshold",
Type = SecretRuleType.Entropy,
Pattern = "", // Pattern can be empty for entropy rules
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium
// Default entropy threshold is 4.5, which is in the valid range
};
// Act
var result = _sut.Validate(rule);
// Assert - default threshold (4.5) is valid, no warnings expected
Assert.True(result.IsValid);
Assert.Empty(result.Warnings);
}
[Fact]
public void Validate_EntropyRuleWithOutOfRangeThreshold_ReturnsWarning()
{
// Arrange - using an out-of-range threshold
var rule = new SecretRule
{
Id = "stellaops.secrets.entropy-test",
Version = "1.0.0",
Name = "Entropy Test",
Description = "Detects high-entropy strings with extreme threshold",
Type = SecretRuleType.Entropy,
Pattern = "",
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium,
EntropyThreshold = 0 // Zero triggers <= 0 warning condition
};
// Act
var result = _sut.Validate(rule);
// Assert - valid but with warning about unusual threshold
Assert.True(result.IsValid);
Assert.Contains(result.Warnings, w => w.Contains("entropy", StringComparison.OrdinalIgnoreCase));
}
}

View File

@@ -0,0 +1,110 @@
using FluentAssertions;
using StellaOps.Scanner.Analyzers.Secrets;
namespace StellaOps.Scanner.Analyzers.Secrets.Tests;
[Trait("Category", "Unit")]
public sealed class EntropyCalculatorTests
{
[Fact]
public void Calculate_EmptyString_ReturnsZero()
{
var entropy = EntropyCalculator.Calculate(string.Empty);
entropy.Should().Be(0.0);
}
[Fact]
public void Calculate_SingleCharacter_ReturnsZero()
{
var entropy = EntropyCalculator.Calculate("a");
entropy.Should().Be(0.0);
}
[Fact]
public void Calculate_RepeatedCharacter_ReturnsZero()
{
var entropy = EntropyCalculator.Calculate("aaaaaaaaaa");
entropy.Should().Be(0.0);
}
[Fact]
public void Calculate_TwoDistinctCharacters_ReturnsOne()
{
var entropy = EntropyCalculator.Calculate("ababababab");
entropy.Should().BeApproximately(1.0, 0.01);
}
[Fact]
public void Calculate_FourDistinctCharacters_ReturnsTwo()
{
var entropy = EntropyCalculator.Calculate("abcdabcdabcd");
entropy.Should().BeApproximately(2.0, 0.01);
}
[Fact]
public void Calculate_HighEntropyString_ReturnsHighValue()
{
var highEntropyString = "aB1cD2eF3gH4iJ5kL6mN7oP8qR9sT0uV";
var entropy = EntropyCalculator.Calculate(highEntropyString);
entropy.Should().BeGreaterThan(4.0);
}
[Fact]
public void Calculate_LowEntropyPassword_ReturnsLowValue()
{
var lowEntropyString = "password";
var entropy = EntropyCalculator.Calculate(lowEntropyString);
entropy.Should().BeLessThan(3.0);
}
[Fact]
public void Calculate_AwsAccessKeyPattern_ReturnsHighEntropy()
{
var awsKey = "AKIAIOSFODNN7EXAMPLE";
var entropy = EntropyCalculator.Calculate(awsKey);
entropy.Should().BeGreaterThan(3.5);
}
[Fact]
public void Calculate_Base64String_ReturnsHighEntropy()
{
var base64 = "SGVsbG8gV29ybGQhIFRoaXMgaXMgYSB0ZXN0";
var entropy = EntropyCalculator.Calculate(base64);
entropy.Should().BeGreaterThan(4.0);
}
[Fact]
public void Calculate_IsDeterministic()
{
var input = "TestString123!@#";
var entropy1 = EntropyCalculator.Calculate(input);
var entropy2 = EntropyCalculator.Calculate(input);
entropy1.Should().Be(entropy2);
}
[Theory]
[InlineData("0123456789", 3.32)]
[InlineData("abcdefghij", 3.32)]
[InlineData("ABCDEFGHIJ", 3.32)]
public void Calculate_KnownPatterns_ReturnsExpectedEntropy(string input, double expectedEntropy)
{
var entropy = EntropyCalculator.Calculate(input);
entropy.Should().BeApproximately(expectedEntropy, 0.1);
}
}

View File

@@ -0,0 +1,171 @@
using FluentAssertions;
using StellaOps.Scanner.Analyzers.Secrets;
namespace StellaOps.Scanner.Analyzers.Secrets.Tests;
[Trait("Category", "Unit")]
public sealed class PayloadMaskerTests
{
private readonly PayloadMasker _masker = new();
[Fact]
public void Mask_EmptySpan_ReturnsEmpty()
{
_masker.Mask(ReadOnlySpan<char>.Empty).Should().BeEmpty();
}
[Fact]
public void Mask_ShortValue_ReturnsMaskChars()
{
// Values shorter than prefix+suffix get masked placeholder
var result = _masker.Mask("abc".AsSpan());
result.Should().Contain("*");
}
[Fact]
public void Mask_StandardValue_PreservesPrefixAndSuffix()
{
var result = _masker.Mask("1234567890".AsSpan());
// Default: 4 char prefix, 2 char suffix
result.Should().StartWith("1234");
result.Should().EndWith("90");
result.Should().Contain("****");
}
[Fact]
public void Mask_AwsAccessKey_PreservesPrefix()
{
var awsKey = "AKIAIOSFODNN7EXAMPLE";
var result = _masker.Mask(awsKey.AsSpan());
result.Should().StartWith("AKIA");
result.Should().EndWith("LE");
result.Should().Contain("****");
}
[Fact]
public void Mask_WithPrefixHint_UsesCustomPrefixLength()
{
var apiKey = "sk-proj-abcdefghijklmnop";
// MaxExposedChars is 6, so prefix:8 + suffix:2 gets scaled down
var result = _masker.Mask(apiKey.AsSpan(), "prefix:4,suffix:2");
result.Should().StartWith("sk-p");
result.Should().Contain("****");
}
[Fact]
public void Mask_LongValue_MasksMiddle()
{
var longSecret = "verylongsecretthatexceeds100characters" +
"andshouldbemaskkedproperlywithoutexpo" +
"singtheentirecontentstoanyoneviewingit";
var result = _masker.Mask(longSecret.AsSpan());
// Should contain mask characters and be shorter than original
result.Should().Contain("****");
result.Length.Should().BeLessThan(longSecret.Length);
}
[Fact]
public void Mask_IsDeterministic()
{
var secret = "AKIAIOSFODNN7EXAMPLE";
var result1 = _masker.Mask(secret.AsSpan());
var result2 = _masker.Mask(secret.AsSpan());
result1.Should().Be(result2);
}
[Fact]
public void Mask_NeverExposesFullSecret()
{
var secret = "supersecretkey123";
var result = _masker.Mask(secret.AsSpan());
result.Should().NotBe(secret);
result.Should().Contain("*");
}
[Theory]
[InlineData("prefix:6,suffix:0")]
[InlineData("prefix:0,suffix:6")]
[InlineData("prefix:3,suffix:3")]
public void Mask_WithVariousHints_RespectsTotalLimit(string hint)
{
var secret = "abcdefghijklmnopqrstuvwxyz";
var result = _masker.Mask(secret.AsSpan(), hint);
var visibleChars = result.Replace("*", "").Length;
visibleChars.Should().BeLessThanOrEqualTo(PayloadMasker.MaxExposedChars);
}
[Fact]
public void Mask_EnforcesMinOutputLength()
{
var secret = "abcdefghijklmnop";
var result = _masker.Mask(secret.AsSpan());
result.Length.Should().BeGreaterThanOrEqualTo(PayloadMasker.MinOutputLength);
}
[Fact]
public void Mask_ByteOverload_DecodesUtf8()
{
var text = "secretpassword123";
var bytes = System.Text.Encoding.UTF8.GetBytes(text);
var result = _masker.Mask(bytes.AsSpan());
result.Should().Contain("****");
result.Should().StartWith("secr");
}
[Fact]
public void Mask_EmptyByteSpan_ReturnsEmpty()
{
_masker.Mask(ReadOnlySpan<byte>.Empty).Should().BeEmpty();
}
[Fact]
public void Mask_InvalidHint_UsesDefaults()
{
var secret = "abcdefghijklmnop";
var result1 = _masker.Mask(secret.AsSpan(), "invalid:hint:format");
var result2 = _masker.Mask(secret.AsSpan());
result1.Should().Be(result2);
}
[Fact]
public void Mask_UsesCorrectMaskChar()
{
var secret = "abcdefghijklmnop";
var result = _masker.Mask(secret.AsSpan());
result.Should().Contain(PayloadMasker.MaskChar.ToString());
}
[Fact]
public void Mask_MaskLengthLimited()
{
var longSecret = new string('x', 100);
var result = _masker.Mask(longSecret.AsSpan());
// Count mask characters
var maskCount = result.Count(c => c == PayloadMasker.MaskChar);
maskCount.Should().BeLessThanOrEqualTo(PayloadMasker.MaxMaskLength);
}
}

View File

@@ -0,0 +1,235 @@
using System.Collections.Immutable;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Logging.Abstractions;
using StellaOps.Scanner.Analyzers.Secrets;
using Xunit;
namespace StellaOps.Scanner.Analyzers.Secrets.Tests;
[Trait("Category", "Unit")]
public sealed class RegexDetectorTests
{
private readonly RegexDetector _detector = new(NullLogger<RegexDetector>.Instance);
[Fact]
public void DetectorId_ReturnsRegex()
{
_detector.DetectorId.Should().Be("regex");
}
[Fact]
public void CanHandle_RegexType_ReturnsTrue()
{
_detector.CanHandle(SecretRuleType.Regex).Should().BeTrue();
}
[Fact]
public void CanHandle_EntropyType_ReturnsFalse()
{
_detector.CanHandle(SecretRuleType.Entropy).Should().BeFalse();
}
[Fact]
public void CanHandle_CompositeType_ReturnsTrue()
{
// RegexDetector handles both Regex and Composite types
_detector.CanHandle(SecretRuleType.Composite).Should().BeTrue();
}
[Fact]
public async Task DetectAsync_NoMatch_ReturnsEmpty()
{
var rule = CreateRule(@"AKIA[0-9A-Z]{16}");
var content = Encoding.UTF8.GetBytes("no aws key here");
var matches = await _detector.DetectAsync(
content.AsMemory(),
"test.txt",
rule,
TestContext.Current.CancellationToken);
matches.Should().BeEmpty();
}
[Fact]
public async Task DetectAsync_SingleMatch_ReturnsOne()
{
var rule = CreateRule(@"AKIA[0-9A-Z]{16}");
var content = Encoding.UTF8.GetBytes("aws_key = AKIAIOSFODNN7EXAMPLE");
var matches = await _detector.DetectAsync(
content.AsMemory(),
"test.txt",
rule,
TestContext.Current.CancellationToken);
matches.Should().HaveCount(1);
matches[0].Rule.Id.Should().Be("test-rule");
}
[Fact]
public async Task DetectAsync_MultipleMatches_ReturnsAll()
{
var rule = CreateRule(@"AKIA[0-9A-Z]{16}");
var content = Encoding.UTF8.GetBytes(
"key1 = AKIAIOSFODNN7EXAMPLE\n" +
"key2 = AKIABCDEFGHIJKLMNOP1");
var matches = await _detector.DetectAsync(
content.AsMemory(),
"test.txt",
rule,
TestContext.Current.CancellationToken);
matches.Should().HaveCount(2);
}
[Fact]
public async Task DetectAsync_ReportsCorrectLineNumber()
{
var rule = CreateRule(@"secret_key\s*=\s*\S+");
var content = Encoding.UTF8.GetBytes(
"# config file\n" +
"debug = true\n" +
"secret_key = mysecretvalue\n" +
"port = 8080");
var matches = await _detector.DetectAsync(
content.AsMemory(),
"config.txt",
rule,
TestContext.Current.CancellationToken);
matches.Should().HaveCount(1);
matches[0].LineNumber.Should().Be(3);
}
[Fact]
public async Task DetectAsync_ReportsCorrectColumn()
{
var rule = CreateRule(@"secret_key");
var content = Encoding.UTF8.GetBytes("config: secret_key = value");
var matches = await _detector.DetectAsync(
content.AsMemory(),
"test.txt",
rule,
TestContext.Current.CancellationToken);
matches.Should().HaveCount(1);
// "secret_key" starts at index 8 (0-based), column 9 (1-based)
matches[0].ColumnStart.Should().Be(9);
}
[Fact]
public async Task DetectAsync_HandlesMultilineContent()
{
var rule = CreateRule(@"API_KEY\s*=\s*\w+");
var content = Encoding.UTF8.GetBytes(
"line1\n" +
"line2\n" +
"API_KEY = abc123\n" +
"line4\n" +
"API_KEY = xyz789");
var matches = await _detector.DetectAsync(
content.AsMemory(),
"test.txt",
rule,
TestContext.Current.CancellationToken);
matches.Should().HaveCount(2);
matches[0].LineNumber.Should().Be(3);
matches[1].LineNumber.Should().Be(5);
}
[Fact]
public async Task DetectAsync_DisabledRule_StillProcesses()
{
// Note: The detector doesn't filter by Enabled status.
// Filtering disabled rules is the caller's responsibility (e.g., SecretsAnalyzer)
var rule = CreateRule(@"AKIA[0-9A-Z]{16}", enabled: false);
var content = Encoding.UTF8.GetBytes("AKIAIOSFODNN7EXAMPLE");
var matches = await _detector.DetectAsync(
content.AsMemory(),
"test.txt",
rule,
TestContext.Current.CancellationToken);
// Detector processes regardless of Enabled flag
matches.Should().HaveCount(1);
}
[Fact]
public async Task DetectAsync_RespectsCancellation()
{
var rule = CreateRule(@"test");
var content = Encoding.UTF8.GetBytes("test content");
using var cts = new CancellationTokenSource();
await cts.CancelAsync();
// When cancellation is already requested, detector returns empty (doesn't throw)
var matches = await _detector.DetectAsync(content.AsMemory(), "test.txt", rule, cts.Token);
matches.Should().BeEmpty();
}
[Fact]
public async Task DetectAsync_IncludesFilePath()
{
var rule = CreateRule(@"secret");
var content = Encoding.UTF8.GetBytes("mysecret");
var matches = await _detector.DetectAsync(
content.AsMemory(),
"path/to/file.txt",
rule,
TestContext.Current.CancellationToken);
matches.Should().HaveCount(1);
matches[0].FilePath.Should().Be("path/to/file.txt");
}
[Fact]
public async Task DetectAsync_LargeFile_HandlesEfficiently()
{
var rule = CreateRule(@"SECRET_KEY");
var lines = Enumerable.Range(0, 10000)
.Select(i => i == 5000 ? "SECRET_KEY = value" : $"line {i}")
.ToArray();
var content = Encoding.UTF8.GetBytes(string.Join("\n", lines));
var matches = await _detector.DetectAsync(
content.AsMemory(),
"large.txt",
rule,
TestContext.Current.CancellationToken);
matches.Should().HaveCount(1);
matches[0].LineNumber.Should().Be(5001);
}
private static SecretRule CreateRule(string pattern, bool enabled = true)
{
return new SecretRule
{
Id = "test-rule",
Version = "1.0.0",
Name = "Test Rule",
Description = "Test rule for unit tests",
Type = SecretRuleType.Regex,
Pattern = pattern,
Severity = SecretSeverity.High,
Confidence = SecretConfidence.High,
Keywords = ImmutableArray<string>.Empty,
FilePatterns = ImmutableArray<string>.Empty,
Enabled = enabled,
EntropyThreshold = 0,
MinLength = 0,
MaxLength = 1000,
Metadata = ImmutableDictionary<string, string>.Empty
};
}
}

View File

@@ -0,0 +1,348 @@
using System.Security.Cryptography;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Time.Testing;
using StellaOps.Scanner.Analyzers.Secrets;
using Xunit;
namespace StellaOps.Scanner.Analyzers.Secrets.Tests;
[Trait("Category", "Unit")]
public sealed class RulesetLoaderTests : IAsyncLifetime
{
private readonly string _testDir;
private readonly FakeTimeProvider _timeProvider;
private readonly RulesetLoader _loader;
public RulesetLoaderTests()
{
_testDir = Path.Combine(Path.GetTempPath(), $"secrets-test-{Guid.NewGuid():N}");
Directory.CreateDirectory(_testDir);
_timeProvider = new FakeTimeProvider(new DateTimeOffset(2026, 1, 4, 12, 0, 0, TimeSpan.Zero));
_loader = new RulesetLoader(NullLogger<RulesetLoader>.Instance, _timeProvider);
}
public ValueTask InitializeAsync() => ValueTask.CompletedTask;
public ValueTask DisposeAsync()
{
if (Directory.Exists(_testDir))
{
Directory.Delete(_testDir, recursive: true);
}
return ValueTask.CompletedTask;
}
[Fact]
public async Task LoadAsync_ValidBundle_LoadsRuleset()
{
await CreateValidBundleAsync(TestContext.Current.CancellationToken);
var ruleset = await _loader.LoadAsync(_testDir, TestContext.Current.CancellationToken);
ruleset.Should().NotBeNull();
ruleset.Id.Should().Be("test-secrets");
ruleset.Version.Should().Be("1.0.0");
ruleset.Rules.Should().HaveCount(2);
}
[Fact]
public async Task LoadAsync_MissingDirectory_ThrowsDirectoryNotFound()
{
var nonExistentPath = Path.Combine(_testDir, "does-not-exist");
await Assert.ThrowsAsync<DirectoryNotFoundException>(
() => _loader.LoadAsync(nonExistentPath, TestContext.Current.CancellationToken).AsTask());
}
[Fact]
public async Task LoadAsync_MissingManifest_ThrowsFileNotFound()
{
await File.WriteAllTextAsync(
Path.Combine(_testDir, "secrets.ruleset.rules.jsonl"),
"""{"id":"rule1","pattern":"test"}""",
TestContext.Current.CancellationToken);
await Assert.ThrowsAsync<FileNotFoundException>(
() => _loader.LoadAsync(_testDir, TestContext.Current.CancellationToken).AsTask());
}
[Fact]
public async Task LoadAsync_MissingRulesFile_ThrowsFileNotFound()
{
await File.WriteAllTextAsync(
Path.Combine(_testDir, "secrets.ruleset.manifest.json"),
"""{"id":"test","version":"1.0.0"}""",
TestContext.Current.CancellationToken);
await Assert.ThrowsAsync<FileNotFoundException>(
() => _loader.LoadAsync(_testDir, TestContext.Current.CancellationToken).AsTask());
}
[Fact]
public async Task LoadAsync_InvalidIntegrity_ThrowsException()
{
await CreateBundleWithBadIntegrityAsync(TestContext.Current.CancellationToken);
await Assert.ThrowsAsync<InvalidOperationException>(
() => _loader.LoadAsync(_testDir, TestContext.Current.CancellationToken).AsTask());
}
[Fact]
public async Task LoadAsync_SortsRulesById()
{
await CreateBundleWithUnorderedRulesAsync(TestContext.Current.CancellationToken);
var ruleset = await _loader.LoadAsync(_testDir, TestContext.Current.CancellationToken);
ruleset.Rules.Select(r => r.Id).Should().BeInAscendingOrder();
}
[Fact]
public async Task LoadAsync_SkipsBlankLines()
{
await CreateBundleWithBlankLinesAsync(TestContext.Current.CancellationToken);
var ruleset = await _loader.LoadAsync(_testDir, TestContext.Current.CancellationToken);
ruleset.Rules.Should().HaveCount(2);
}
[Fact]
public async Task LoadAsync_SkipsInvalidJsonLines()
{
await CreateBundleWithInvalidJsonAsync(TestContext.Current.CancellationToken);
var ruleset = await _loader.LoadAsync(_testDir, TestContext.Current.CancellationToken);
// JSONL processes each line independently - invalid lines are skipped but don't stop processing
// So we get rule1 and rule2 (2 rules), with the invalid line skipped
ruleset.Rules.Should().HaveCount(2);
}
[Fact]
public async Task LoadAsync_SetsCreatedAt()
{
await CreateValidBundleAsync(TestContext.Current.CancellationToken);
var ruleset = await _loader.LoadAsync(_testDir, TestContext.Current.CancellationToken);
ruleset.CreatedAt.Should().Be(_timeProvider.GetUtcNow());
}
[Fact]
public async Task LoadFromJsonlAsync_ValidStream_LoadsRules()
{
var jsonl = """
{"id":"rule1","version":"1.0","name":"Rule 1","type":"regex","pattern":"secret","severity":"high","confidence":"high"}
{"id":"rule2","version":"1.0","name":"Rule 2","type":"regex","pattern":"password","severity":"medium","confidence":"medium"}
""";
using var stream = new MemoryStream(Encoding.UTF8.GetBytes(jsonl));
var ruleset = await _loader.LoadFromJsonlAsync(
stream,
"test-bundle",
"1.0.0",
TestContext.Current.CancellationToken);
ruleset.Should().NotBeNull();
ruleset.Id.Should().Be("test-bundle");
ruleset.Version.Should().Be("1.0.0");
ruleset.Rules.Should().HaveCount(2);
}
[Fact]
public async Task LoadFromJsonlAsync_DefaultValues_AppliedCorrectly()
{
var jsonl = """{"id":"minimal-rule","pattern":"test"}""";
using var stream = new MemoryStream(Encoding.UTF8.GetBytes(jsonl));
var ruleset = await _loader.LoadFromJsonlAsync(
stream,
"test",
"1.0",
TestContext.Current.CancellationToken);
var rule = ruleset.Rules[0];
rule.Version.Should().Be("1.0.0");
rule.Enabled.Should().BeTrue();
rule.Severity.Should().Be(SecretSeverity.Medium);
rule.Confidence.Should().Be(SecretConfidence.Medium);
rule.Type.Should().Be(SecretRuleType.Regex);
rule.EntropyThreshold.Should().Be(4.5);
rule.MinLength.Should().Be(16);
rule.MaxLength.Should().Be(1000);
}
[Theory]
[InlineData("regex", SecretRuleType.Regex)]
[InlineData("entropy", SecretRuleType.Entropy)]
[InlineData("composite", SecretRuleType.Composite)]
[InlineData("REGEX", SecretRuleType.Regex)]
[InlineData("unknown", SecretRuleType.Regex)]
public async Task LoadFromJsonlAsync_ParsesRuleType(string typeString, SecretRuleType expected)
{
var jsonl = $$$"""{"id":"rule1","pattern":"test","type":"{{{typeString}}}"}""";
using var stream = new MemoryStream(Encoding.UTF8.GetBytes(jsonl));
var ruleset = await _loader.LoadFromJsonlAsync(
stream,
"test",
"1.0",
TestContext.Current.CancellationToken);
ruleset.Rules[0].Type.Should().Be(expected);
}
[Theory]
[InlineData("low", SecretSeverity.Low)]
[InlineData("medium", SecretSeverity.Medium)]
[InlineData("high", SecretSeverity.High)]
[InlineData("critical", SecretSeverity.Critical)]
[InlineData("HIGH", SecretSeverity.High)]
public async Task LoadFromJsonlAsync_ParsesSeverity(string severityString, SecretSeverity expected)
{
var jsonl = $$$"""{"id":"rule1","pattern":"test","severity":"{{{severityString}}}"}""";
using var stream = new MemoryStream(Encoding.UTF8.GetBytes(jsonl));
var ruleset = await _loader.LoadFromJsonlAsync(
stream,
"test",
"1.0",
TestContext.Current.CancellationToken);
ruleset.Rules[0].Severity.Should().Be(expected);
}
[Fact]
public async Task LoadFromJsonlAsync_ParsesKeywords()
{
var jsonl = """{"id":"rule1","pattern":"test","keywords":["aws","key","secret"]}""";
using var stream = new MemoryStream(Encoding.UTF8.GetBytes(jsonl));
var ruleset = await _loader.LoadFromJsonlAsync(
stream,
"test",
"1.0",
TestContext.Current.CancellationToken);
ruleset.Rules[0].Keywords.Should().BeEquivalentTo(["aws", "key", "secret"]);
}
[Fact]
public async Task LoadFromJsonlAsync_ParsesMetadata()
{
var jsonl = """{"id":"rule1","pattern":"test","metadata":{"source":"gitleaks","category":"api-key"}}""";
using var stream = new MemoryStream(Encoding.UTF8.GetBytes(jsonl));
var ruleset = await _loader.LoadFromJsonlAsync(
stream,
"test",
"1.0",
TestContext.Current.CancellationToken);
ruleset.Rules[0].Metadata.Should().Contain("source", "gitleaks");
ruleset.Rules[0].Metadata.Should().Contain("category", "api-key");
}
private async Task CreateValidBundleAsync(CancellationToken ct)
{
var rules = """
{"id":"aws-key","version":"1.0","name":"AWS Access Key","type":"regex","pattern":"AKIA[0-9A-Z]{16}","severity":"critical","confidence":"high"}
{"id":"generic-secret","version":"1.0","name":"Generic Secret","type":"regex","pattern":"secret[_-]?key","severity":"medium","confidence":"medium"}
""";
var rulesPath = Path.Combine(_testDir, "secrets.ruleset.rules.jsonl");
await File.WriteAllTextAsync(rulesPath, rules, ct);
var hash = await ComputeHashAsync(rulesPath, ct);
var manifest = $$$"""{"id":"test-secrets","version":"1.0.0","integrity":{"rulesSha256":"{{{hash}}}"}}""";
await File.WriteAllTextAsync(
Path.Combine(_testDir, "secrets.ruleset.manifest.json"),
manifest,
ct);
}
private async Task CreateBundleWithBadIntegrityAsync(CancellationToken ct)
{
var rules = """{"id":"rule1","pattern":"test"}""";
var rulesPath = Path.Combine(_testDir, "secrets.ruleset.rules.jsonl");
await File.WriteAllTextAsync(rulesPath, rules, ct);
// Use a known bad hash (clearly different from any real SHA-256)
const string badHash = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
var manifest = $$$"""{"id":"test","version":"1.0","integrity":{"rulesSha256":"{{{badHash}}}"}}""";
await File.WriteAllTextAsync(
Path.Combine(_testDir, "secrets.ruleset.manifest.json"),
manifest,
ct);
}
private async Task CreateBundleWithUnorderedRulesAsync(CancellationToken ct)
{
var rules = """
{"id":"z-rule","pattern":"z"}
{"id":"a-rule","pattern":"a"}
{"id":"m-rule","pattern":"m"}
""";
await File.WriteAllTextAsync(
Path.Combine(_testDir, "secrets.ruleset.rules.jsonl"),
rules,
ct);
var manifest = """{"id":"test","version":"1.0"}""";
await File.WriteAllTextAsync(
Path.Combine(_testDir, "secrets.ruleset.manifest.json"),
manifest,
ct);
}
private async Task CreateBundleWithBlankLinesAsync(CancellationToken ct)
{
var rules = """
{"id":"rule1","pattern":"test1"}
{"id":"rule2","pattern":"test2"}
""";
await File.WriteAllTextAsync(
Path.Combine(_testDir, "secrets.ruleset.rules.jsonl"),
rules,
ct);
var manifest = """{"id":"test","version":"1.0"}""";
await File.WriteAllTextAsync(
Path.Combine(_testDir, "secrets.ruleset.manifest.json"),
manifest,
ct);
}
private async Task CreateBundleWithInvalidJsonAsync(CancellationToken ct)
{
var rules = """
{"id":"rule1","pattern":"valid"}
not valid json at all
{"id":"rule2","pattern":"also valid but will be skipped due to earlier error?"}
""";
await File.WriteAllTextAsync(
Path.Combine(_testDir, "secrets.ruleset.rules.jsonl"),
rules,
ct);
var manifest = """{"id":"test","version":"1.0"}""";
await File.WriteAllTextAsync(
Path.Combine(_testDir, "secrets.ruleset.manifest.json"),
manifest,
ct);
}
private static async Task<string> ComputeHashAsync(string filePath, CancellationToken ct)
{
await using var stream = File.OpenRead(filePath);
var hash = await SHA256.HashDataAsync(stream, ct);
return Convert.ToHexString(hash).ToLowerInvariant();
}
}

View File

@@ -0,0 +1,173 @@
using System.Collections.Immutable;
using FluentAssertions;
using StellaOps.Scanner.Analyzers.Secrets;
namespace StellaOps.Scanner.Analyzers.Secrets.Tests;
[Trait("Category", "Unit")]
public sealed class SecretRuleTests
{
[Fact]
public void GetCompiledPattern_ValidRegex_ReturnsRegex()
{
var rule = CreateRule(@"AKIA[0-9A-Z]{16}");
var regex = rule.GetCompiledPattern();
regex.Should().NotBeNull();
regex!.IsMatch("AKIAIOSFODNN7EXAMPLE").Should().BeTrue();
}
[Fact]
public void GetCompiledPattern_InvalidRegex_ReturnsNull()
{
var rule = CreateRule(@"[invalid(regex");
var regex = rule.GetCompiledPattern();
regex.Should().BeNull();
}
[Fact]
public void GetCompiledPattern_IsCached()
{
var rule = CreateRule(@"test\d+");
var regex1 = rule.GetCompiledPattern();
var regex2 = rule.GetCompiledPattern();
regex1.Should().BeSameAs(regex2);
}
[Fact]
public void GetCompiledPattern_EntropyType_ReturnsNull()
{
var rule = new SecretRule
{
Id = "entropy-rule",
Version = "1.0.0",
Name = "Entropy Rule",
Description = "Test",
Type = SecretRuleType.Entropy,
Pattern = string.Empty,
Severity = SecretSeverity.Medium,
Confidence = SecretConfidence.Medium,
Keywords = ImmutableArray<string>.Empty,
FilePatterns = ImmutableArray<string>.Empty,
Enabled = true,
EntropyThreshold = 4.5,
MinLength = 16,
MaxLength = 100,
Metadata = ImmutableDictionary<string, string>.Empty
};
var regex = rule.GetCompiledPattern();
regex.Should().BeNull();
}
[Fact]
public void AppliesToFile_NoPatterns_MatchesAll()
{
var rule = CreateRule(@"test");
rule.AppliesToFile("any/path/file.txt").Should().BeTrue();
rule.AppliesToFile("config.json").Should().BeTrue();
rule.AppliesToFile("secrets.yaml").Should().BeTrue();
}
[Fact]
public void AppliesToFile_WithExtensionPattern_FiltersByExtension()
{
var rule = CreateRuleWithFilePatterns(@"test", "*.json", "*.yaml");
rule.AppliesToFile("config.json").Should().BeTrue();
rule.AppliesToFile("config.yaml").Should().BeTrue();
rule.AppliesToFile("config.xml").Should().BeFalse();
rule.AppliesToFile("config.txt").Should().BeFalse();
}
[Fact]
public void MightMatch_NoKeywords_ReturnsTrue()
{
var rule = CreateRule(@"test");
rule.MightMatch("any content here".AsSpan()).Should().BeTrue();
}
[Fact]
public void MightMatch_WithKeywords_MatchesIfKeywordFound()
{
var rule = CreateRuleWithKeywords(@"test", "secret", "password");
rule.MightMatch("contains secret here".AsSpan()).Should().BeTrue();
rule.MightMatch("contains password here".AsSpan()).Should().BeTrue();
rule.MightMatch("no matching content".AsSpan()).Should().BeFalse();
}
private static SecretRule CreateRule(string pattern)
{
return new SecretRule
{
Id = "test-rule",
Version = "1.0.0",
Name = "Test Rule",
Description = "Test rule",
Type = SecretRuleType.Regex,
Pattern = pattern,
Severity = SecretSeverity.High,
Confidence = SecretConfidence.High,
Keywords = ImmutableArray<string>.Empty,
FilePatterns = ImmutableArray<string>.Empty,
Enabled = true,
EntropyThreshold = 0,
MinLength = 0,
MaxLength = 1000,
Metadata = ImmutableDictionary<string, string>.Empty
};
}
private static SecretRule CreateRuleWithFilePatterns(string pattern, params string[] filePatterns)
{
return new SecretRule
{
Id = "test-rule",
Version = "1.0.0",
Name = "Test Rule",
Description = "Test rule",
Type = SecretRuleType.Regex,
Pattern = pattern,
Severity = SecretSeverity.High,
Confidence = SecretConfidence.High,
Keywords = ImmutableArray<string>.Empty,
FilePatterns = [..filePatterns],
Enabled = true,
EntropyThreshold = 0,
MinLength = 0,
MaxLength = 1000,
Metadata = ImmutableDictionary<string, string>.Empty
};
}
private static SecretRule CreateRuleWithKeywords(string pattern, params string[] keywords)
{
return new SecretRule
{
Id = "test-rule",
Version = "1.0.0",
Name = "Test Rule",
Description = "Test rule",
Type = SecretRuleType.Regex,
Pattern = pattern,
Severity = SecretSeverity.High,
Confidence = SecretConfidence.High,
Keywords = [..keywords],
FilePatterns = ImmutableArray<string>.Empty,
Enabled = true,
EntropyThreshold = 0,
MinLength = 0,
MaxLength = 1000,
Metadata = ImmutableDictionary<string, string>.Empty
};
}
}

View File

@@ -0,0 +1,208 @@
using System.Collections.Immutable;
using FluentAssertions;
using StellaOps.Scanner.Analyzers.Secrets;
namespace StellaOps.Scanner.Analyzers.Secrets.Tests;
[Trait("Category", "Unit")]
public sealed class SecretRulesetTests
{
[Fact]
public void EnabledRuleCount_ReturnsCorrectCount()
{
var ruleset = CreateRuleset(
CreateRule("rule1", enabled: true),
CreateRule("rule2", enabled: true),
CreateRule("rule3", enabled: false));
ruleset.EnabledRuleCount.Should().Be(2);
}
[Fact]
public void EnabledRuleCount_AllDisabled_ReturnsZero()
{
var ruleset = CreateRuleset(
CreateRule("rule1", enabled: false),
CreateRule("rule2", enabled: false));
ruleset.EnabledRuleCount.Should().Be(0);
}
[Fact]
public void EnabledRuleCount_AllEnabled_ReturnsTotal()
{
var ruleset = CreateRuleset(
CreateRule("rule1", enabled: true),
CreateRule("rule2", enabled: true),
CreateRule("rule3", enabled: true));
ruleset.EnabledRuleCount.Should().Be(3);
}
[Fact]
public void GetRulesForFile_ReturnsEnabledMatchingRules()
{
var ruleset = CreateRuleset(
CreateRuleWithPattern("json-rule", "*.json", enabled: true),
CreateRuleWithPattern("yaml-rule", "*.yaml", enabled: true),
CreateRuleWithPattern("disabled-rule", "*.json", enabled: false));
var rules = ruleset.GetRulesForFile("config.json").ToList();
rules.Should().HaveCount(1);
rules[0].Id.Should().Be("json-rule");
}
[Fact]
public void GetRulesForFile_NoMatchingPatterns_ReturnsRulesWithNoPatterns()
{
var ruleset = CreateRuleset(
CreateRule("generic-rule", enabled: true),
CreateRuleWithPattern("json-rule", "*.json", enabled: true));
var rules = ruleset.GetRulesForFile("config.xml").ToList();
rules.Should().HaveCount(1);
rules[0].Id.Should().Be("generic-rule");
}
[Fact]
public void Validate_ValidRuleset_ReturnsEmpty()
{
var ruleset = CreateRuleset(
CreateRule("rule1", enabled: true),
CreateRule("rule2", enabled: true));
var errors = ruleset.Validate();
errors.Should().BeEmpty();
}
[Fact]
public void Validate_DuplicateIds_ReturnsError()
{
var ruleset = CreateRuleset(
CreateRule("same-id", enabled: true),
CreateRule("same-id", enabled: true));
var errors = ruleset.Validate();
errors.Should().Contain(e => e.Contains("Duplicate", StringComparison.OrdinalIgnoreCase));
}
[Fact]
public void Validate_InvalidRegex_ReturnsError()
{
var rule = new SecretRule
{
Id = "bad-regex",
Version = "1.0.0",
Name = "Bad Regex",
Description = "Invalid regex pattern",
Type = SecretRuleType.Regex,
Pattern = "[invalid(regex",
Severity = SecretSeverity.High,
Confidence = SecretConfidence.High,
Keywords = ImmutableArray<string>.Empty,
FilePatterns = ImmutableArray<string>.Empty,
Enabled = true,
EntropyThreshold = 0,
MinLength = 0,
MaxLength = 1000,
Metadata = ImmutableDictionary<string, string>.Empty
};
var ruleset = new SecretRuleset
{
Id = "test",
Version = "1.0.0",
CreatedAt = DateTimeOffset.UtcNow,
Rules = [rule]
};
var errors = ruleset.Validate();
errors.Should().Contain(e => e.Contains("bad-regex", StringComparison.OrdinalIgnoreCase) &&
e.Contains("invalid", StringComparison.OrdinalIgnoreCase));
}
[Fact]
public void EnabledRules_ReturnsOnlyEnabled()
{
var ruleset = CreateRuleset(
CreateRule("rule1", enabled: true),
CreateRule("rule2", enabled: false),
CreateRule("rule3", enabled: true));
var enabled = ruleset.EnabledRules.ToList();
enabled.Should().HaveCount(2);
enabled.Select(r => r.Id).Should().BeEquivalentTo(["rule1", "rule3"]);
}
[Fact]
public void Empty_ReturnsEmptyRuleset()
{
var empty = SecretRuleset.Empty;
empty.Id.Should().Be("empty");
empty.Version.Should().Be("0.0");
empty.Rules.Should().BeEmpty();
empty.EnabledRuleCount.Should().Be(0);
}
private static SecretRuleset CreateRuleset(params SecretRule[] rules)
{
return new SecretRuleset
{
Id = "test-ruleset",
Version = "1.0.0",
CreatedAt = DateTimeOffset.UtcNow,
Rules = [..rules]
};
}
private static SecretRule CreateRule(string id, bool enabled)
{
return new SecretRule
{
Id = id,
Version = "1.0.0",
Name = $"Rule {id}",
Description = "Test rule",
Type = SecretRuleType.Regex,
Pattern = "test",
Severity = SecretSeverity.High,
Confidence = SecretConfidence.High,
Keywords = ImmutableArray<string>.Empty,
FilePatterns = ImmutableArray<string>.Empty,
Enabled = enabled,
EntropyThreshold = 0,
MinLength = 0,
MaxLength = 1000,
Metadata = ImmutableDictionary<string, string>.Empty
};
}
private static SecretRule CreateRuleWithPattern(string id, string filePattern, bool enabled)
{
return new SecretRule
{
Id = id,
Version = "1.0.0",
Name = $"Rule {id}",
Description = "Test rule",
Type = SecretRuleType.Regex,
Pattern = "test",
Severity = SecretSeverity.High,
Confidence = SecretConfidence.High,
Keywords = ImmutableArray<string>.Empty,
FilePatterns = [filePattern],
Enabled = enabled,
EntropyThreshold = 0,
MinLength = 0,
MaxLength = 1000,
Metadata = ImmutableDictionary<string, string>.Empty
};
}
}

View File

@@ -0,0 +1,32 @@
<?xml version='1.0' encoding='utf-8'?>
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<IsPackable>false</IsPackable>
<IsTestProject>true</IsTestProject>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="../../__Libraries/StellaOps.Scanner.Analyzers.Secrets/StellaOps.Scanner.Analyzers.Secrets.csproj" />
<ProjectReference Include="../../../__Libraries/StellaOps.TestKit/StellaOps.TestKit.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" />
<PackageReference Include="xunit.v3" />
<PackageReference Include="xunit.runner.visualstudio" />
<PackageReference Include="Moq" />
<PackageReference Include="FluentAssertions" />
<PackageReference Include="Microsoft.Extensions.TimeProvider.Testing" />
</ItemGroup>
<ItemGroup>
<Content Include="Fixtures/**/*">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>
</Project>

Some files were not shown because too many files have changed in this diff Show More