git.stella-ops.org/docs/modules/scanner/operations/secret-leak-detection.md

# Secret Leak Detection (Scanner Operations)

> **Status:** Preview (Sprint 132). Requires `SCANNER-ENG-0007`/`POLICY-READINESS-0001` release bundle and the experimental flag `secret-leak-detection`.
>
> **Audience:** Scanner operators, Security Guild, Docs Guild, Offline Kit maintainers.

## 1. Scope & goals

- Introduce the **`StellaOps.Scanner.Analyzers.Secrets`** plug-in, which executes deterministic rule bundles against layer content during scans.
- Ensure every finding is reproducible: rule bundles are DSSE-signed, versioned, and shipped with the Offline Kit.
- Surface policy-ready evidence (`secret.leak`) so tenants can enforce block/warn flows using `stella-dsl@1` predicates.
- Preserve sovereignty: rule bundles install locally, no outbound telemetry, masking is enforced before data leaves the worker.

## 2. Prerequisites

| Requirement | Notes |
| --- | --- |
| Analyzer binaries | Deploy `StellaOps.Scanner.Analyzers.Secrets` alongside Scanner Worker (packaged with the standard container images). |
| Surface libraries | `Surface.Secrets`, `Surface.Validation`, and `Surface.Env` must already be configured (see `surface-secrets.md`). |
| Experimental flag | Enable `scanner.features.experimental["secret-leak-detection"] = true` on both WebService and Worker. |
| Policy readiness | Import predicates from `docs/modules/policy/secret-leak-detection-readiness.md` into tenant policy packs. |
| Offline Kit | Update to an Offline Kit that includes `offline/rules/secrets/<release>/` before enabling production scans. |

## 3. Rule bundle lifecycle

Rule bundles ship in the Export Center / Offline Kit under `offline/rules/secrets/<release>/`.

| File | Purpose | Notes |
| --- | --- | --- |
| `secrets.ruleset.manifest.json` | Lists rule IDs, versions, severity defaults, and hash digests. | Consume during policy drift audits. |
| `secrets.ruleset.rules.jsonl` | Newline-delimited definitions (regex/entropy metadata, masking hints). | Loaded by the analyzer at startup. |
| `secrets.ruleset.dsse.json` | DSSE envelope (Signer certificate chain + Attestor proof). | Verify before distributing bundles. |

Verification checklist (`stella excititor verify` talks to the configured Attestor service):

```
stella excititor verify \
  --attestation offline/rules/secrets/2025.11/secrets.ruleset.dsse.json \
  --digest $(sha256sum offline/rules/secrets/2025.11/secrets.ruleset.rules.jsonl | cut -d' ' -f1)
```

For air-gapped environments point the CLI at the Offline Kit Attestor mirror (for example `STELLA_ATTESTOR_URL=http://attestor.offline.local`) before running the command. The Attestor instance validates the DSSE envelope against the mirrored Rekor log and embedded certificate chain; no public network access is required.

Once verified, copy the manifest + rules to the worker:

```
/opt/stellaops/plugins/scanner/analyzers/secrets/
  ├── secrets.ruleset.manifest.json
  ├── secrets.ruleset.rules.jsonl
  └── secrets.ruleset.dsse.json
```

Restart the worker so the analyzer reloads the updated bundle. Bundles are immutable; upgrading requires replacing all three files and restarting.

## 4. Enabling the analyzer

1. **Toggle the feature flag** (WebService + Worker):

   ```yaml
   scanner:
     features:
       experimental:
         secret-leak-detection: true
   ```

   (Environment alternative: `SCANNER__FEATURES__EXPERIMENTAL__secret-leak-detection=true`.)

2. **Configure retention** (WebService):

   ```yaml
   scanner:
     storage:
       migrations:
         - Scanner.Analysis.SecretFindingsTtl
   ```

   The migration adds `secretFindings` documents to `ScanAnalysisStore` with the standard TTL (default 90 days). Adjust Mongo TTL via the deployment overlay if longer retention is required.

3. **Activate policy ingestion** (WebService):

   ```yaml
   scanner:
     runtime:
       enableSecretFindings: true
   ```

   (Experimental builds gate secret evidence behind this toggle to avoid surprising downstream consumers.)

4. **Roll scanner hosts**. Apply the configuration, roll WebService first, then Workers. Verify the startup logs contain `SecretsAnalyzerHost` and `SecretLeakDetection: Enabled`.

## 5. Policy patterns

The analyzer emits `secret.leak` evidence with the shape:

```json
{
  "ruleId": "stellaops.secrets.aws-access-key",
  "ruleVersion": "2025.11.0",
  "severity": "high",
  "confidence": "high",
  "file": "/app/config.yml",
  "line": 42,
  "mask": "AKIA********B7",
  "bundleId": "secrets.ruleset",
  "bundleVersion": "2025.11"
}
```

Policy DSL helpers introduced with this release:

| Helper | Description |
| --- | --- |
| `secret.hasFinding(ruleId?, severity?, confidence?)` | Returns true if any finding matches the filter. |
| `secret.bundle.version(requiredVersion)` | Ensures the active bundle meets or exceeds a version. |
| `secret.match.count(ruleId?)` | Returns the number of findings (useful for thresholds). |

Sample policy (`policies/secret-blocker.stella`):

```dsl
policy "Secret Leak Guard" syntax "stella-dsl@1" {
  metadata {
    description = "Block high-confidence secret leaks"
    tags = ["secrets","compliance"]
  }

  rule block_high_confidence priority 10 {
    when secret.hasFinding(severity: "high", confidence: "high")
    then escalate to "block";
    because "High severity secret leak detected";
  }

  rule require_current_bundle priority 5 {
    when not secret.bundle.version("2025.11")
    then warn message "Secret leak bundle out of date";
  }
}
```

Tenants that prefer staged rollout can downgrade low-confidence findings:

```dsl
rule low_confidence_warn priority 20 {
  when secret.hasFinding(confidence: "low")
  then annotate decision.notes := "Investigate masked payload";
  else ignore;
}
```

## 6. Observability & reporting

- **Metrics:** `scanner.secret.finding_total{tenant,ruleId,severity,confidence}` increments per finding. Add Prometheus alerts for spikes.
- **Logs:** `SecretsAnalyzerHost` logs bundle version on load and emits warnings when masking fails (payload never leaves memory).
- **Traces:** Each analyzer run adds a `scanner.secrets.scan` span with rule counts and wall-clock timing.
- **Reports / CLI:** Scan reports include a `secretFindings` array; CLI diff/export surfaces render masked snippets plus remediation guidance.

## 7. Troubleshooting

| Symptom | Resolution |
| --- | --- |
| Analyzer disabled at startup | Confirm feature flag and bundle files exist; check `plugins/scanner/analyzers/secrets` permissions (`640`). |
| No findings despite seeded secrets | Ensure bundle hash matches manifest. Run worker with `--secrets-trace` (debug build) to log matched rules locally. |
| Policy marks findings as unknown | Upgrade tenant policies to include `secret.*` helpers; older policies silently drop the namespace. |
| Air-gapped verification fails | Ensure `STELLA_ATTESTOR_URL` points to the Offline Kit Attestor mirror and rerun `stella excititor verify --attestation <file> --digest <sha256>`. |

## 8. References

- `docs/modules/policy/secret-leak-detection-readiness.md`
- `docs/benchmarks/scanner/deep-dives/secrets.md`
- `docs/modules/scanner/design/surface-secrets.md`
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md` §1.1 Runtime inventory (Scanner)