git.stella-ops.org/docs/modules/scanner/operations/secret-leak-detection.md

Secret Leak Detection (Scanner Operations)

Status: Preview (Sprint 132). Requires SCANNER-ENG-0007/POLICY-READINESS-0001 release bundle and the experimental flag secret-leak-detection.

Audience: Scanner operators, Security Guild, Docs Guild, Offline Kit maintainers.

1. Scope & goals

• Introduce the StellaOps.Scanner.Analyzers.Secrets plug-in, which executes deterministic rule bundles against layer content during scans.
• Ensure every finding is reproducible: rule bundles are DSSE-signed, versioned, and shipped with the Offline Kit.
• Surface policy-ready evidence (secret.leak) so tenants can enforce block/warn flows using stella-dsl@1 predicates.
• Preserve sovereignty: rule bundles install locally, no outbound telemetry, masking is enforced before data leaves the worker.

2. Prerequisites

Requirement	Notes
Analyzer binaries	Deploy StellaOps.Scanner.Analyzers.Secrets alongside Scanner Worker (packaged with the standard container images).
Surface libraries	Surface.Secrets, Surface.Validation, and Surface.Env must already be configured (see surface-secrets.md).
Experimental flag	Enable scanner.features.experimental["secret-leak-detection"] = true on both WebService and Worker.
Policy readiness	Import predicates from docs/modules/policy/secret-leak-detection-readiness.md into tenant policy packs.
Offline Kit	Update to an Offline Kit that includes offline/rules/secrets/<release>/ before enabling production scans.

3. Rule bundle lifecycle

Rule bundles ship in the Export Center / Offline Kit under offline/rules/secrets/<release>/.

File	Purpose	Notes
secrets.ruleset.manifest.json	Lists rule IDs, versions, severity defaults, and hash digests.	Consume during policy drift audits.
secrets.ruleset.rules.jsonl	Newline-delimited definitions (regex/entropy metadata, masking hints).	Loaded by the analyzer at startup.
secrets.ruleset.dsse.json	DSSE envelope (Signer certificate chain + Attestor proof).	Verify before distributing bundles.

Verification checklist (stella excititor verify talks to the configured Attestor service):

stella excititor verify \
  --attestation offline/rules/secrets/2025.11/secrets.ruleset.dsse.json \
  --digest $(sha256sum offline/rules/secrets/2025.11/secrets.ruleset.rules.jsonl | cut -d' ' -f1)

For air-gapped environments point the CLI at the Offline Kit Attestor mirror (for example STELLA_ATTESTOR_URL=http://attestor.offline.local) before running the command. The Attestor instance validates the DSSE envelope against the mirrored Rekor log and embedded certificate chain; no public network access is required.

Once verified, copy the manifest + rules to the worker:

/opt/stellaops/plugins/scanner/analyzers/secrets/
  ├── secrets.ruleset.manifest.json
  ├── secrets.ruleset.rules.jsonl
  └── secrets.ruleset.dsse.json

Restart the worker so the analyzer reloads the updated bundle. Bundles are immutable; upgrading requires replacing all three files and restarting.

4. Enabling the analyzer

1. Toggle the feature flag (WebService + Worker):

   scanner:
     features:
       experimental:
         secret-leak-detection: true
   

   (Environment alternative: SCANNER__FEATURES__EXPERIMENTAL__secret-leak-detection=true.)

2. Configure retention (WebService):

   scanner:
     storage:
       migrations:
         - Scanner.Analysis.SecretFindingsTtl
   

   The migration adds secretFindings documents to ScanAnalysisStore with the standard TTL (default 90 days). Adjust Mongo TTL via the deployment overlay if longer retention is required.

3. Activate policy ingestion (WebService):

   scanner:
     runtime:
       enableSecretFindings: true
   

   (Experimental builds gate secret evidence behind this toggle to avoid surprising downstream consumers.)

4. Roll scanner hosts. Apply the configuration, roll WebService first, then Workers. Verify the startup logs contain SecretsAnalyzerHost and SecretLeakDetection: Enabled.

5. Policy patterns

The analyzer emits secret.leak evidence with the shape:

{
  "ruleId": "stellaops.secrets.aws-access-key",
  "ruleVersion": "2025.11.0",
  "severity": "high",
  "confidence": "high",
  "file": "/app/config.yml",
  "line": 42,
  "mask": "AKIA********B7",
  "bundleId": "secrets.ruleset",
  "bundleVersion": "2025.11"
}

Policy DSL helpers introduced with this release:

Helper	Description
secret.hasFinding(ruleId?, severity?, confidence?)	Returns true if any finding matches the filter.
secret.bundle.version(requiredVersion)	Ensures the active bundle meets or exceeds a version.
secret.match.count(ruleId?)	Returns the number of findings (useful for thresholds).

Sample policy (policies/secret-blocker.stella):

policy "Secret Leak Guard" syntax "stella-dsl@1" {
  metadata {
    description = "Block high-confidence secret leaks"
    tags = ["secrets","compliance"]
  }

  rule block_high_confidence priority 10 {
    when secret.hasFinding(severity: "high", confidence: "high")
    then escalate to "block";
    because "High severity secret leak detected";
  }

  rule require_current_bundle priority 5 {
    when not secret.bundle.version("2025.11")
    then warn message "Secret leak bundle out of date";
  }
}

Tenants that prefer staged rollout can downgrade low-confidence findings:

rule low_confidence_warn priority 20 {
  when secret.hasFinding(confidence: "low")
  then annotate decision.notes := "Investigate masked payload";
  else ignore;
}

6. Observability & reporting

• Metrics: scanner.secret.finding_total{tenant,ruleId,severity,confidence} increments per finding. Add Prometheus alerts for spikes.
• Logs: SecretsAnalyzerHost logs bundle version on load and emits warnings when masking fails (payload never leaves memory).
• Traces: Each analyzer run adds a scanner.secrets.scan span with rule counts and wall-clock timing.
• Reports / CLI: Scan reports include a secretFindings array; CLI diff/export surfaces render masked snippets plus remediation guidance.

7. Troubleshooting

Symptom	Resolution
Analyzer disabled at startup	Confirm feature flag and bundle files exist; check plugins/scanner/analyzers/secrets permissions (640).
No findings despite seeded secrets	Ensure bundle hash matches manifest. Run worker with --secrets-trace (debug build) to log matched rules locally.
Policy marks findings as unknown	Upgrade tenant policies to include secret.* helpers; older policies silently drop the namespace.
Air-gapped verification fails	Ensure STELLA_ATTESTOR_URL points to the Offline Kit Attestor mirror and rerun stella excititor verify --attestation <file> --digest <sha256>.

8. References

• docs/modules/policy/secret-leak-detection-readiness.md
• docs/benchmarks/scanner/deep-dives/secrets.md
• docs/modules/scanner/design/surface-secrets.md
• docs/07_HIGH_LEVEL_ARCHITECTURE.md §1.1 Runtime inventory (Scanner)