save progress
This commit is contained in:
StellaOps Bot
@@ -1,9 +1,25 @@
|
||||
# Secret Leak Detection (Scanner Operations)
|
||||
|
||||
> **Status:** Preview (Sprint 132). Requires `SCANNER-ENG-0007`/`POLICY-READINESS-0001` release bundle and the experimental flag `secret-leak-detection`.
|
||||
> **Status:** PLANNED - Implementation in progress. See implementation sprints below.
|
||||
>
|
||||
> **Previous status:** Preview (Sprint 132). Requires `SCANNER-ENG-0007`/`POLICY-READINESS-0001` release bundle and the experimental flag `secret-leak-detection`.
|
||||
>
|
||||
> **Audience:** Scanner operators, Security Guild, Docs Guild, Offline Kit maintainers.
|
||||
|
||||
## Implementation Status
|
||||
|
||||
| Component | Status | Sprint |
|
||||
|-----------|--------|--------|
|
||||
| `StellaOps.Scanner.Analyzers.Secrets` plugin | NOT IMPLEMENTED | [SPRINT_20260104_002](../../../implplan/SPRINT_20260104_002_SCANNER_secret_leak_detection_core.md) |
|
||||
| Rule bundle infrastructure | NOT IMPLEMENTED | [SPRINT_20260104_003](../../../implplan/SPRINT_20260104_003_SCANNER_secret_rule_bundles.md) |
|
||||
| Policy DSL predicates (`secret.*`) | NOT IMPLEMENTED | [SPRINT_20260104_004](../../../implplan/SPRINT_20260104_004_POLICY_secret_dsl_integration.md) |
|
||||
| Offline Kit integration | NOT IMPLEMENTED | [SPRINT_20260104_005](../../../implplan/SPRINT_20260104_005_AIRGAP_secret_offline_kit.md) |
|
||||
| Surface.Secrets (credential delivery) | IMPLEMENTED | N/A (already complete) |
|
||||
|
||||
**Note:** The remainder of this document describes the TARGET SPECIFICATION for secret leak detection. The feature is not yet available. Surface.Secrets (operational credential management) is fully implemented and separate from secret leak detection.
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope & goals
|
||||
|
||||
- Introduce the **`StellaOps.Scanner.Analyzers.Secrets`** plug-in, which executes deterministic rule bundles against layer content during scans.
|
||||
@@ -29,29 +45,107 @@ Rule bundles ship in the Export Center / Offline Kit under `offline/rules/secret
|
||||
| --- | --- | --- |
|
||||
| `secrets.ruleset.manifest.json` | Lists rule IDs, versions, severity defaults, and hash digests. | Consume during policy drift audits. |
|
||||
| `secrets.ruleset.rules.jsonl` | Newline-delimited definitions (regex/entropy metadata, masking hints). | Loaded by the analyzer at startup. |
|
||||
| `secrets.ruleset.dsse.json` | DSSE envelope (Signer certificate chain + Attestor proof). | Verify before distributing bundles. |
|
||||
| `secrets.ruleset.dsse.json` | DSSE envelope (HMAC-SHA256 signature). | Verify before distributing bundles. |
|
||||
|
||||
Verification checklist (`stella excititor verify` talks to the configured Attestor service):
|
||||
### 3.1 Creating custom bundles
|
||||
|
||||
Organizations can create custom rule bundles with additional detection patterns:
|
||||
|
||||
```bash
|
||||
# Create a bundle from rule definition files
|
||||
stella secrets bundle create \
|
||||
--output ./bundles/custom-2026.01 \
|
||||
--bundle-id custom.secrets.ruleset \
|
||||
--version 2026.01 \
|
||||
--rules ./custom-rules/*.json
|
||||
|
||||
# Create and sign in one step
|
||||
stella secrets bundle create \
|
||||
--output ./bundles/custom-2026.01 \
|
||||
--bundle-id custom.secrets.ruleset \
|
||||
--version 2026.01 \
|
||||
--rules ./custom-rules/*.json \
|
||||
--sign \
|
||||
--key-id my-org-secrets-signer \
|
||||
--shared-secret-file /path/to/signing.key
|
||||
```
|
||||
|
||||
Rule definition files must follow the JSON schema:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "myorg.secrets.internal-api-key",
|
||||
"version": "1.0.0",
|
||||
"name": "Internal API Key",
|
||||
"description": "Detects internal API keys with MYORG prefix",
|
||||
"type": "regex",
|
||||
"pattern": "MYORG_[A-Z0-9]{32}",
|
||||
"severity": "high",
|
||||
"confidence": "high",
|
||||
"keywords": ["MYORG_"],
|
||||
"filePatterns": ["*.env", "*.yaml", "*.json"],
|
||||
"enabled": true
|
||||
}
|
||||
```
|
||||
|
||||
**Rule ID requirements:**
|
||||
- Must be namespaced (e.g., `myorg.secrets.rule-name`)
|
||||
- Must start with lowercase letter
|
||||
- May contain lowercase letters, digits, dots, and hyphens
|
||||
|
||||
### 3.2 Verifying bundles
|
||||
|
||||
Verify bundle integrity and signature before deployment:
|
||||
|
||||
```bash
|
||||
# Basic verification (checks SHA-256 integrity)
|
||||
stella secrets bundle verify \
|
||||
--bundle ./bundles/2026.01
|
||||
|
||||
# Full verification with signature check
|
||||
stella secrets bundle verify \
|
||||
--bundle ./bundles/2026.01 \
|
||||
--shared-secret-file /path/to/signing.key
|
||||
|
||||
# Verify with trusted key list
|
||||
stella secrets bundle verify \
|
||||
--bundle ./bundles/2026.01 \
|
||||
--shared-secret-file /path/to/signing.key \
|
||||
--trusted-key-ids stellaops-secrets-signer,my-org-signer
|
||||
```
|
||||
|
||||
For air-gapped environments, use the local verification mode (no network calls):
|
||||
|
||||
```bash
|
||||
stella secrets bundle verify \
|
||||
--bundle ./bundles/2026.01 \
|
||||
--shared-secret-file /path/to/signing.key \
|
||||
--skip-rekor
|
||||
```
|
||||
|
||||
Alternatively, use `stella excititor verify` for Attestor-based verification:
|
||||
|
||||
```bash
|
||||
stella excititor verify \
|
||||
--attestation offline/rules/secrets/2025.11/secrets.ruleset.dsse.json \
|
||||
--digest $(sha256sum offline/rules/secrets/2025.11/secrets.ruleset.rules.jsonl | cut -d' ' -f1)
|
||||
--attestation offline/rules/secrets/2026.01/secrets.ruleset.dsse.json \
|
||||
--digest $(sha256sum offline/rules/secrets/2026.01/secrets.ruleset.rules.jsonl | cut -d' ' -f1)
|
||||
```
|
||||
|
||||
For air-gapped environments point the CLI at the Offline Kit Attestor mirror (for example `STELLA_ATTESTOR_URL=http://attestor.offline.local`) before running the command. The Attestor instance validates the DSSE envelope against the mirrored Rekor log and embedded certificate chain; no public network access is required.
|
||||
### 3.3 Deploying bundles
|
||||
|
||||
Once verified, copy the manifest + rules to the worker:
|
||||
Once verified, copy the bundle to the worker:
|
||||
|
||||
```
|
||||
/opt/stellaops/plugins/scanner/analyzers/secrets/
|
||||
├── secrets.ruleset.manifest.json
|
||||
├── secrets.ruleset.rules.jsonl
|
||||
└── secrets.ruleset.dsse.json
|
||||
|- secrets.ruleset.manifest.json
|
||||
|- secrets.ruleset.rules.jsonl
|
||||
|- secrets.ruleset.dsse.json
|
||||
```
|
||||
|
||||
Restart the worker so the analyzer reloads the updated bundle. Bundles are immutable; upgrading requires replacing all three files and restarting.
|
||||
|
||||
See [secrets-bundle-rotation.md](./secrets-bundle-rotation.md) for rotation procedures and rollback instructions.
|
||||
|
||||
## 4. Enabling the analyzer
|
||||
|
||||
1. **Toggle the feature flag** (WebService + Worker):
|
||||
@@ -161,6 +255,52 @@ rule low_confidence_warn priority 20 {
|
||||
| No findings despite seeded secrets | Ensure bundle hash matches manifest. Run worker with `--secrets-trace` (debug build) to log matched rules locally. |
|
||||
| Policy marks findings as unknown | Upgrade tenant policies to include `secret.*` helpers; older policies silently drop the namespace. |
|
||||
| Air-gapped verification fails | Ensure `STELLA_ATTESTOR_URL` points to the Offline Kit Attestor mirror and rerun `stella excititor verify --attestation <file> --digest <sha256>`. |
|
||||
| Signature verification failed | Check shared secret matches signing key. Verify DSSE envelope exists and is not corrupted. See signature troubleshooting below. |
|
||||
| Bundle integrity check failed | Rules file was modified after signing. Re-download bundle or rebuild from sources. |
|
||||
| Key not in trusted list | Add signer key ID to `--trusted-key-ids` or update `scanner.secrets.trustedKeyIds` configuration. |
|
||||
|
||||
### 7.1 Signature verification troubleshooting
|
||||
|
||||
**"Signature verification failed" error:**
|
||||
|
||||
1. Verify the shared secret is correct:
|
||||
```bash
|
||||
# Check secret file exists and is readable
|
||||
cat /path/to/signing.key | wc -c
|
||||
# Should output the key length (typically 32-64 bytes for base64-encoded keys)
|
||||
```
|
||||
|
||||
2. Check the DSSE envelope format:
|
||||
```bash
|
||||
# Verify envelope is valid JSON
|
||||
jq . ./bundles/2026.01/secrets.ruleset.dsse.json
|
||||
```
|
||||
|
||||
3. Confirm manifest matches envelope payload:
|
||||
```bash
|
||||
# The envelope payload (base64url-decoded) should match the manifest content
|
||||
# excluding the signatures field
|
||||
```
|
||||
|
||||
**"Rules file integrity check failed" error:**
|
||||
|
||||
1. Recompute the SHA-256 hash:
|
||||
```bash
|
||||
sha256sum ./bundles/2026.01/secrets.ruleset.rules.jsonl
|
||||
```
|
||||
|
||||
2. Compare with manifest:
|
||||
```bash
|
||||
jq -r '.integrity.rulesSha256' ./bundles/2026.01/secrets.ruleset.manifest.json
|
||||
```
|
||||
|
||||
3. If hashes differ, the rules file was modified. Re-download or rebuild the bundle.
|
||||
|
||||
**"Bundle is not signed" error:**
|
||||
|
||||
The bundle was created without the `--sign` flag. Either:
|
||||
- Rebuild with signing: `stella secrets bundle create ... --sign --key-id <key>`
|
||||
- Skip signature verification: `--skip-signature-verification` (not recommended for production)
|
||||
|
||||
## 8. References
|
||||
|
||||
|
||||
Reference in New Issue
Block a user