git.stella-ops.org/docs/modules/scanner/operations/secret-leak-detection.md

# Secret Leak Detection (Scanner Operations)

> **Status:** IMPLEMENTED (2026-01-04). Feature is production-ready.
>
> **Audience:** Scanner operators, Security Guild, Docs Guild, Offline Kit maintainers.

## Implementation Status

| Component | Status | Sprint (Archived) |
|-----------|--------|-------------------|
| `StellaOps.Scanner.Analyzers.Secrets` plugin | IMPLEMENTED | [SPRINT_20260104_002](../../../implplan/archived/2026-01-04-secret-detection/SPRINT_20260104_002_SCANNER_secret_leak_detection_core.md) |
| Rule bundle infrastructure | IMPLEMENTED | [SPRINT_20260104_003](../../../implplan/archived/2026-01-04-secret-detection/SPRINT_20260104_003_SCANNER_secret_rule_bundles.md) |
| Policy DSL predicates (`secret.*`) | IMPLEMENTED | [SPRINT_20260104_004](../../../implplan/archived/2026-01-04-secret-detection/SPRINT_20260104_004_POLICY_secret_dsl_integration.md) |
| Offline Kit integration | IMPLEMENTED | [SPRINT_20260104_005](../../../implplan/archived/2026-01-04-secret-detection/SPRINT_20260104_005_AIRGAP_secret_offline_kit.md) |
| Configuration API | IMPLEMENTED | [SPRINT_20260104_006](../../../implplan/archived/2026-01-04-secret-detection/SPRINT_20260104_006_BE_secret_detection_config_api.md) |
| Alert Integration | IMPLEMENTED | [SPRINT_20260104_007](../../../implplan/archived/2026-01-04-secret-detection/SPRINT_20260104_007_BE_secret_detection_alerts.md) |
| UI Components | IMPLEMENTED | [SPRINT_20260104_008](../../../implplan/archived/2026-01-04-secret-detection/SPRINT_20260104_008_FE_secret_detection_ui.md) |
| Surface.Secrets (credential delivery) | IMPLEMENTED | N/A (already complete) |

**Note:** All secret leak detection components are now fully implemented and production-ready. Surface.Secrets (operational credential management) remains a separate, independent feature.

---

## 1. Scope & goals

- Introduce the **`StellaOps.Scanner.Analyzers.Secrets`** plug-in, which executes deterministic rule bundles against layer content during scans.
- Ensure every finding is reproducible: rule bundles are DSSE-signed, versioned, and shipped with the Offline Kit.
- Surface policy-ready evidence (`secret.leak`) so tenants can enforce block/warn flows using `stella-dsl@1` predicates.
- Preserve sovereignty: rule bundles install locally, no outbound telemetry, masking is enforced before data leaves the worker.

## 2. Prerequisites

| Requirement | Notes |
| --- | --- |
| Analyzer binaries | Deploy `StellaOps.Scanner.Analyzers.Secrets` alongside Scanner Worker (packaged with the standard container images). |
| Surface libraries | `Surface.Secrets`, `Surface.Validation`, and `Surface.Env` must already be configured (see `surface-secrets.md`). |
| Experimental flag | Enable `scanner.features.experimental["secret-leak-detection"] = true` on both WebService and Worker. |
| Policy readiness | Import predicates from `docs/modules/policy/secret-leak-detection-readiness.md` into tenant policy packs. |
| Offline Kit | Update to an Offline Kit that includes `offline/rules/secrets/<release>/` before enabling production scans. |

## 3. Rule bundle lifecycle

Rule bundles ship in the Export Center / Offline Kit under `offline/rules/secrets/<release>/`.

| File | Purpose | Notes |
| --- | --- | --- |
| `secrets.ruleset.manifest.json` | Lists rule IDs, versions, severity defaults, and hash digests. | Consume during policy drift audits. |
| `secrets.ruleset.rules.jsonl` | Newline-delimited definitions (regex/entropy metadata, masking hints). | Loaded by the analyzer at startup. |
| `secrets.ruleset.dsse.json` | DSSE envelope (HMAC-SHA256 signature). | Verify before distributing bundles. |

### 3.1 Creating custom bundles

Organizations can create custom rule bundles with additional detection patterns:

```bash
# Create a bundle from rule definition files
stella secrets bundle create \
  --output ./bundles/custom-2026.01 \
  --bundle-id custom.secrets.ruleset \
  --version 2026.01 \
  --rules ./custom-rules/*.json

# Create and sign in one step
stella secrets bundle create \
  --output ./bundles/custom-2026.01 \
  --bundle-id custom.secrets.ruleset \
  --version 2026.01 \
  --rules ./custom-rules/*.json \
  --sign \
  --key-id my-org-secrets-signer \
  --shared-secret-file /path/to/signing.key
```

Rule definition files must follow the JSON schema:

```json
{
  "id": "myorg.secrets.internal-api-key",
  "version": "1.0.0",
  "name": "Internal API Key",
  "description": "Detects internal API keys with MYORG prefix",
  "type": "regex",
  "pattern": "MYORG_[A-Z0-9]{32}",
  "severity": "high",
  "confidence": "high",
  "keywords": ["MYORG_"],
  "filePatterns": ["*.env", "*.yaml", "*.json"],
  "enabled": true
}
```

**Rule ID requirements:**
- Must be namespaced (e.g., `myorg.secrets.rule-name`)
- Must start with lowercase letter
- May contain lowercase letters, digits, dots, and hyphens

### 3.2 Verifying bundles

Verify bundle integrity and signature before deployment:

```bash
# Basic verification (checks SHA-256 integrity)
stella secrets bundle verify \
  --bundle ./bundles/2026.01

# Full verification with signature check
stella secrets bundle verify \
  --bundle ./bundles/2026.01 \
  --shared-secret-file /path/to/signing.key

# Verify with trusted key list
stella secrets bundle verify \
  --bundle ./bundles/2026.01 \
  --shared-secret-file /path/to/signing.key \
  --trusted-key-ids stellaops-secrets-signer,my-org-signer
```

For air-gapped environments, use the local verification mode (no network calls):

```bash
stella secrets bundle verify \
  --bundle ./bundles/2026.01 \
  --shared-secret-file /path/to/signing.key \
  --skip-rekor
```

Alternatively, use `stella excititor verify` for Attestor-based verification:

```bash
stella excititor verify \
  --attestation offline/rules/secrets/2026.01/secrets.ruleset.dsse.json \
  --digest $(sha256sum offline/rules/secrets/2026.01/secrets.ruleset.rules.jsonl | cut -d' ' -f1)
```

### 3.3 Deploying bundles

Once verified, copy the bundle to the worker:

```
/opt/stellaops/plugins/scanner/analyzers/secrets/
  |- secrets.ruleset.manifest.json
  |- secrets.ruleset.rules.jsonl
  |- secrets.ruleset.dsse.json
```

Restart the worker so the analyzer reloads the updated bundle. Bundles are immutable; upgrading requires replacing all three files and restarting.

See [secrets-bundle-rotation.md](./secrets-bundle-rotation.md) for rotation procedures and rollback instructions.

## 4. Enabling the analyzer

1. **Toggle the feature flag** (WebService + Worker):

   ```yaml
   scanner:
     features:
       experimental:
         secret-leak-detection: true
   ```

   (Environment alternative: `SCANNER__FEATURES__EXPERIMENTAL__secret-leak-detection=true`.)

2. **Configure retention** (WebService):

   ```yaml
   scanner:
     storage:
       migrations:
         - Scanner.Analysis.SecretFindingsTtl
   ```

   The migration adds `secretFindings` records to `ScanAnalysisStore` with the standard TTL (default 90 days). Adjust PostgreSQL retention policy via the deployment overlay if longer retention is required.

3. **Activate policy ingestion** (WebService):

   ```yaml
   scanner:
     runtime:
       enableSecretFindings: true
   ```

   (Experimental builds gate secret evidence behind this toggle to avoid surprising downstream consumers.)

4. **Roll scanner hosts**. Apply the configuration, roll WebService first, then Workers. Verify the startup logs contain `SecretsAnalyzerHost` and `SecretLeakDetection: Enabled`.

## 5. Configuration API

The secret detection feature provides a REST API for per-tenant configuration:

### 5.1 Settings Endpoints

```
GET    /api/v1/tenants/{tenantId}/secrets/config/settings
PUT    /api/v1/tenants/{tenantId}/secrets/config/settings
PATCH  /api/v1/tenants/{tenantId}/secrets/config/settings
```

### 5.2 Exception Pattern Endpoints

```
GET    /api/v1/tenants/{tenantId}/secrets/config/exceptions
POST   /api/v1/tenants/{tenantId}/secrets/config/exceptions
DELETE /api/v1/tenants/{tenantId}/secrets/config/exceptions/{exceptionId}
```

### 5.3 Revelation Policy

Control how detected secrets appear in different contexts:

| Policy | Display | Use Case |
|--------|---------|----------|
| `FullMask` | `[REDACTED]` | Maximum security, compliance reports |
| `PartialReveal` | `AKIA****WXYZ` | Default for UI, allows identification |
| `FullReveal` | Full value | Incident response (requires elevated permissions) |

### 5.4 Alert Configuration

Configure alerting for secret findings via the Notify service:

- **Destinations**: Slack, Teams, Email, Webhook, PagerDuty
- **Rate Limiting**: Max alerts per scan (default: 10)
- **Deduplication**: 24-hour window to prevent duplicate alerts
- **Severity Routing**: Route critical findings to different channels

## 6. Policy patterns

The analyzer emits `secret.leak` evidence with the shape:

```json
{
  "ruleId": "stellaops.secrets.aws-access-key",
  "ruleVersion": "2026.01.0",
  "severity": "high",
  "confidence": "high",
  "file": "/app/config.yml",
  "line": 42,
  "mask": "AKIA********B7",
  "bundleId": "secrets.ruleset",
  "bundleVersion": "2026.01"
}
```

Policy DSL helpers introduced with this release:

| Helper | Description |
| --- | --- |
| `secret.hasFinding(ruleId?, severity?, confidence?)` | Returns true if any finding matches the filter. |
| `secret.bundle.version(requiredVersion)` | Ensures the active bundle meets or exceeds a version. |
| `secret.match.count(ruleId?)` | Returns the number of findings (useful for thresholds). |
| `secret.mask.applied` | Returns true if masking was successfully applied. |
| `secret.path.allowlist(patterns)` | Returns true if all findings are in allowed paths. |

Sample policy (`policies/secret-blocker.stella`):

```dsl
policy "Secret Leak Guard" syntax "stella-dsl@1" {
  metadata {
    description = "Block high-confidence secret leaks"
    tags = ["secrets","compliance"]
  }

  rule block_high_confidence priority 10 {
    when secret.hasFinding(severity: "high", confidence: "high")
    then escalate to "block";
    because "High severity secret leak detected";
  }

  rule require_current_bundle priority 5 {
    when not secret.bundle.version("2026.01")
    then warn message "Secret leak bundle out of date";
  }
}
```

Tenants that prefer staged rollout can downgrade low-confidence findings:

```dsl
rule low_confidence_warn priority 20 {
  when secret.hasFinding(confidence: "low")
  then annotate decision.notes := "Investigate masked payload";
  else ignore;
}
```

## 7. UI Components

The secret detection UI is available at `/tenants/{tenantId}/secrets/`:

### 7.1 Settings Page

- **General Tab**: Enable/disable detection, revelation policy, rule categories
- **Exceptions Tab**: Manage allowlist patterns for false positive suppression
- **Alerts Tab**: Configure alert destinations and thresholds

### 7.2 Findings List

- Filterable by severity, status, rule category
- Masked value display with conditional reveal
- Pagination and export support

### 7.3 Exception Manager

- Create/edit/delete exception patterns
- Regex validation with test mode
- Expiration dates for temporary exceptions

## 8. Observability & reporting

- **Metrics:** `scanner.secret.finding_total{tenant,ruleId,severity,confidence}` increments per finding. Add Prometheus alerts for spikes.
- **Logs:** `SecretsAnalyzerHost` logs bundle version on load and emits warnings when masking fails (payload never leaves memory).
- **Traces:** Each analyzer run adds a `scanner.secrets.scan` span with rule counts and wall-clock timing.
- **Reports / CLI:** Scan reports include a `secretFindings` array; CLI diff/export surfaces render masked snippets plus remediation guidance.

## 9. Troubleshooting

| Symptom | Resolution |
| --- | --- |
| Analyzer disabled at startup | Confirm feature flag and bundle files exist; check `plugins/scanner/analyzers/secrets` permissions (`640`). |
| No findings despite seeded secrets | Ensure bundle hash matches manifest. Run worker with `--secrets-trace` (debug build) to log matched rules locally. |
| Policy marks findings as unknown | Upgrade tenant policies to include `secret.*` helpers; older policies silently drop the namespace. |
| Air-gapped verification fails | Ensure `STELLA_ATTESTOR_URL` points to the Offline Kit Attestor mirror and rerun `stella excititor verify --attestation <file> --digest <sha256>`. |
| Signature verification failed | Check shared secret matches signing key. Verify DSSE envelope exists and is not corrupted. See signature troubleshooting below. |
| Bundle integrity check failed | Rules file was modified after signing. Re-download bundle or rebuild from sources. |
| Key not in trusted list | Add signer key ID to `--trusted-key-ids` or update `scanner.secrets.trustedKeyIds` configuration. |

### 9.1 Signature verification troubleshooting

**"Signature verification failed" error:**

1. Verify the shared secret is correct:
   ```bash
   # Check secret file exists and is readable
   cat /path/to/signing.key | wc -c
   # Should output the key length (typically 32-64 bytes for base64-encoded keys)
   ```

2. Check the DSSE envelope format:
   ```bash
   # Verify envelope is valid JSON
   jq . ./bundles/2026.01/secrets.ruleset.dsse.json
   ```

3. Confirm manifest matches envelope payload:
   ```bash
   # The envelope payload (base64url-decoded) should match the manifest content
   # excluding the signatures field
   ```

**"Rules file integrity check failed" error:**

1. Recompute the SHA-256 hash:
   ```bash
   sha256sum ./bundles/2026.01/secrets.ruleset.rules.jsonl
   ```

2. Compare with manifest:
   ```bash
   jq -r '.integrity.rulesSha256' ./bundles/2026.01/secrets.ruleset.manifest.json
   ```

3. If hashes differ, the rules file was modified. Re-download or rebuild the bundle.

**"Bundle is not signed" error:**

The bundle was created without the `--sign` flag. Either:
- Rebuild with signing: `stella secrets bundle create ... --sign --key-id <key>`
- Skip signature verification: `--skip-signature-verification` (not recommended for production)

## 10. References

- `docs/modules/policy/secret-leak-detection-readiness.md`
- `docs/benchmarks/scanner/deep-dives/secrets.md`
- `docs/modules/scanner/design/surface-secrets.md`
- `docs/ARCHITECTURE_OVERVIEW.md` - Runtime inventory (Scanner)
- [Secrets Bundle Rotation](./secrets-bundle-rotation.md)