tests fixes and some product advisories tunes ups

This commit is contained in:
master
2026-01-30 07:57:43 +02:00
parent 644887997c
commit 55744f6a39
345 changed files with 26290 additions and 2267 deletions

View File

@@ -0,0 +1,244 @@
# Verifying Binary Patches with Stella Micro-Witnesses
This guide explains how to use binary micro-witnesses to verify that shipped binaries contain specific security patches.
## Overview
Binary micro-witnesses provide cryptographic proof of patch status at the binary level. Unlike source-level attestations, they verify what's actually deployed, not what should be deployed.
### Use Cases
- **Procurement**: Verify vendor claims that CVEs are fixed in delivered binaries
- **Audit**: Provide evidence of patch status for compliance
- **Incident Response**: Quickly determine exposure across binary inventory
- **Supply Chain**: Validate that build outputs match expected patch state
## Quick Start
### Generate a Witness
```bash
stella witness generate /path/to/libssl.so.3 \
--cve CVE-2024-0567 \
--sbom sbom.cdx.json \
--sign \
--rekor \
--output witness.json
```
### Verify a Witness
```bash
# Online verification (checks Rekor)
stella witness verify witness.json
# Offline verification (air-gapped)
stella witness verify witness.json --offline
```
### Create Portable Bundle
```bash
stella witness bundle witness.json --output ./audit-bundle
```
## Understanding Verdicts
| Verdict | Meaning |
|---------|---------|
| `patched` | Binary matches patched version signature |
| `vulnerable` | Binary matches vulnerable version signature |
| `inconclusive` | Unable to determine (insufficient evidence) |
| `partial` | Some functions patched, others not |
### Confidence Scores
Confidence ranges from 0.0 to 1.0:
- **0.95+**: High confidence - multiple functions matched with strong evidence
- **0.80-0.95**: Medium confidence - some functions matched
- **<0.80**: Low confidence - limited evidence or compiler variation
## Evidence Types
Micro-witnesses include function-level evidence:
```json
{
"evidence": [
{
"function": "SSL_CTX_new",
"state": "patched",
"score": 0.97,
"method": "semantic_ksg"
}
]
}
```
### Match Methods
| Method | Description | Robustness |
|--------|-------------|------------|
| `byte_exact` | Exact byte-level match | Fragile to recompilation |
| `cfg_structural` | Control flow graph structure | Moderate |
| `semantic_ksg` | Semantic Key-Semantics Graph | Robust to optimization |
| `ir_semantic` | IR-level semantic comparison | Most robust |
## Offline Verification
For air-gapped environments:
1. **Create bundle** with Rekor proof included:
```bash
stella witness bundle witness.json --output ./bundle
```
2. **Transfer bundle** to air-gapped system
3. **Verify offline**:
```bash
# PowerShell
.\verify.ps1
# Bash
./verify.sh
```
The bundle includes:
- Witness predicate (JSON)
- Verification scripts (cross-platform)
- SBOM reference (if included)
- Rekor tile proof (when available)
## Integration with SBOMs
Micro-witnesses can reference SBOM components:
```json
{
"sbomRef": {
"sbomDigest": "sha256:...",
"purl": "pkg:deb/debian/openssl@3.0.11",
"bomRef": "openssl-3.0.11"
}
}
```
This links binary verification to your software inventory.
## Predicate Schema
Full schema: `https://stellaops.dev/predicates/binary-micro-witness@v1`
```json
{
"schemaVersion": "1.0.0",
"binary": {
"digest": "sha256:...",
"purl": "pkg:...",
"arch": "linux-amd64",
"filename": "libssl.so.3"
},
"cve": {
"id": "CVE-2024-0567",
"advisory": "https://...",
"patchCommit": "abc123"
},
"verdict": "patched",
"confidence": 0.95,
"evidence": [...],
"deltaSigDigest": "sha256:...",
"sbomRef": {...},
"tooling": {
"binaryIndexVersion": "2.1.0",
"lifter": "b2r2",
"matchAlgorithm": "semantic_ksg"
},
"computedAt": "2026-01-28T12:00:00Z"
}
```
## Limitations
### What Micro-Witnesses Prove
- A specific binary was analyzed
- Function signatures were compared against known patterns
- A verdict was computed with a confidence score
### What They Do NOT Prove
- Binary authenticity (use SBOM attestations)
- Absence of other vulnerabilities (only specific CVE)
- Build provenance (use SLSA attestations)
### Technical Limitations
- Heavy inlining may hide patched functions
- Stripped symbols reduce match accuracy
- Obfuscated binaries yield "inconclusive"
- Very old binaries may not have ground-truth signatures
## Transparency Logging
When `--rekor` is specified, witnesses are logged to the Rekor transparency log:
- Provides tamper-evidence
- Enables auditors to verify witness wasn't backdated
- Supports v2 tile-based inclusion proofs
Offline bundles include tile proofs for air-gapped verification.
## CLI Reference
### `stella witness generate`
```
stella witness generate <binary> --cve <id> [options]
Arguments:
binary Path to binary file to analyze
Options:
-c, --cve <id> CVE identifier (required)
-s, --sbom <path> Path to SBOM file
-o, --output <path> Output file (default: stdout)
--sign Sign the witness
--rekor Log to Rekor transparency log
-f, --format Output format: json, envelope (default: json)
-v, --verbose Enable verbose output
```
### `stella witness verify`
```
stella witness verify <witness> [options]
Arguments:
witness Path to witness file
Options:
--offline Verify without network access
-s, --sbom <path> Validate SBOM reference
-f, --format Output format: text, json (default: text)
-v, --verbose Enable verbose output
```
### `stella witness bundle`
```
stella witness bundle <witness> --output <dir> [options]
Arguments:
witness Path to witness file
Options:
-o, --output <dir> Output directory (required)
--include-binary Include analyzed binary
--include-sbom Include SBOM file
-v, --verbose Enable verbose output
```
## Related Documentation
- [BinaryIndex Architecture](../modules/binaryindex/architecture.md)
- [Attestor Module](../modules/attestor/architecture.md)
- [Delta-Sig Predicate Schema](../schemas/predicates/deltasig-v2.schema.json)

View File

@@ -1,378 +0,0 @@
# Sprint 0127 · OCI Referrer Bundle Export (Critical Gap Closure)
## Topic & Scope
- **Critical gap**: Mirror bundle and offline kit exports do NOT discover or include OCI referrer artifacts (SBOMs, attestations, signatures) linked to images via the OCI 1.1 referrers API.
- Integrate existing `OciReferrerDiscovery` infrastructure into `MirrorAdapter`, `MirrorBundleBuilder`, and `OfflineKitPackager` flows.
- Ensure `ImportValidator` verifies referrer artifacts are present for each subject image.
- Support fallback tag-based discovery for registries without OCI 1.1 API (e.g., GHCR).
- **Working directory:** `src/ExportCenter/`, `src/AirGap/`
- **Expected evidence:** Unit tests, integration tests with Testcontainers, deterministic bundle output verification.
## Dependencies & Concurrency
- Upstream: `OciReferrerDiscovery` and `OciReferrerFallback` already implemented in `src/ExportCenter/.../Distribution/Oci/`.
- No blocking dependencies; can proceed immediately.
- Concurrency: Tasks 1-3 can proceed in parallel; Task 4-6 depend on 1-3.
## Documentation Prerequisites
- `docs/modules/export-center/architecture.md` (update with referrer discovery flow)
- `docs/modules/airgap/guides/offline-bundle-format.md` (update bundle structure)
- Advisory source: OCI v1.1 referrers API specification and registry compatibility matrix.
---
## Delivery Tracker
### REF-EXPORT-01 - Add Referrer Discovery to MirrorAdapter
Status: DONE
Dependency: None
Owners: ExportCenter Guild
Task description:
Modify `MirrorAdapter.CollectDataSourcesAsync()` to detect image references in items and automatically discover their OCI referrer artifacts.
For each item that represents a container image (identifiable by digest pattern `sha256:*` or image reference format):
1. Call `OciReferrerDiscovery.ListReferrersAsync()` with the image digest
2. If referrers API returns 404/empty, call `OciReferrerFallback.DiscoverViaTagsAsync()` to check for `sha256-{digest}.*` tags
3. For each discovered referrer (SBOM, attestation, signature, VEX), fetch the artifact content
4. Add discovered artifacts to the data sources list with appropriate `MirrorBundleDataCategory`
Inject `IOciReferrerDiscovery` and `IOciReferrerFallback` via DI into `MirrorAdapter`.
Handle errors gracefully: if referrer discovery fails for a single image, log warning and continue with other images.
Implementation completed:
- Created `IReferrerDiscoveryService` interface in Core with `DiscoverReferrersAsync` and `GetReferrerContentAsync`
- Created `ReferrerDiscoveryResult`, `DiscoveredReferrer`, `ReferrerLayer` models
- Added `NullReferrerDiscoveryService` for when discovery is disabled
- Modified `MirrorAdapter` to inject `IReferrerDiscoveryService` (optional)
- Added `IsImageReference()` detection and `DiscoverAndCollectReferrersAsync()` method
- Added artifact type to category mapping (SBOM, VEX, Attestation, DSSE, SLSA, etc.)
- Created `OciReferrerDiscoveryService` wrapper in WebService to implement `IReferrerDiscoveryService`
- Updated DI registration in `ExportAdapterRegistry`
- Added 21 unit tests for MirrorAdapter referrer discovery
- Added 15 unit tests for OciReferrerDiscoveryService
Completion criteria:
- [x] `MirrorAdapter.CollectDataSourcesAsync()` calls `OciReferrerDiscovery.ListReferrersAsync()` for image items
- [x] Fallback tag discovery is invoked when native API returns 404 (via OciReferrerDiscovery)
- [x] Discovered SBOMs are added with category `Sbom`
- [x] Discovered attestations are added with category `Attestation`
- [x] Discovered VEX statements are added with category `Vex`
- [x] Unit tests verify discovery flow with mocked HTTP handlers (36 tests passing)
- [ ] Integration test with Testcontainers `registry:2` verifies end-to-end flow (deferred)
---
### REF-EXPORT-02 - Extend MirrorBundleBuilder for Referrer Metadata
Status: DONE
Dependency: None
Owners: ExportCenter Guild
Task description:
Update `MirrorBundleBuilder` to track the relationship between subject images and their referrer artifacts in the bundle manifest.
Add to `manifest.yaml`:
```yaml
referrers:
- subject: "sha256:abc123..."
artifacts:
- digest: "sha256:def456..."
artifactType: "application/vnd.cyclonedx+json"
mediaType: "application/vnd.oci.image.manifest.v1+json"
size: 12345
annotations:
org.opencontainers.image.created: "2026-01-27T10:00:00Z"
- digest: "sha256:ghi789..."
artifactType: "application/vnd.in-toto+json"
...
```
Update bundle structure to include referrer artifacts under `referrers/` directory:
```
bundle.tgz
├── manifest.yaml # Updated with referrers section
├── images/
│ └── sha256-abc123/
│ └── manifest.json
├── referrers/
│ └── sha256-abc123/ # Keyed by subject digest
│ ├── sha256-def456.json # SBOM
│ └── sha256-ghi789.json # Attestation
└── checksums.txt
```
Implementation completed:
- Added `Attestation = 8` and `Referrer = 9` to `MirrorBundleDataCategory` enum
- Updated `MirrorBundleManifestCounts` to include `Attestations` and `Referrers` fields
- Updated `MirrorBundleBuilder.ComputeBundlePath()` to handle referrer categories under `referrers/{subject-digest}/`
- Updated `SerializeManifestToYaml()` to include attestation and referrer counts
- Updated `BuildReadme()` to include attestation and referrer counts
- Added `indexes/attestations.index.json` and `indexes/referrers.index.json` placeholder files
- Created referrer metadata models in `MirrorBundleModels.cs`:
- `MirrorBundleReferrersSection`, `MirrorBundleSubjectReferrers`, `MirrorBundleReferrerArtifact`
- `MirrorBundleReferrerCounts`, `MirrorBundleReferrerDataSource`
- All 13 existing MirrorBundleBuilder tests continue to pass
Completion criteria:
- [x] `MirrorBundleBuilder` accepts referrer metadata in build request
- [x] `manifest.yaml` includes counts for attestations and referrers
- [x] Referrer artifacts stored under `referrers/{subject-digest}/` directory
- [x] `checksums.txt` includes referrer artifact hashes (existing behavior)
- [x] Bundle structure is deterministic (sorted by digest)
- [x] Unit tests verify manifest structure (existing tests pass)
- [x] Existing tests continue to pass (13/13 pass)
---
### REF-EXPORT-03 - Extend OfflineKitPackager for Referrer Artifacts
Status: DONE
Dependency: None
Owners: ExportCenter Guild · AirGap Guild
Task description:
Update `OfflineKitPackager` to propagate referrer artifacts from mirror bundles into offline kits.
When packaging an offline kit from mirror bundles:
1. Detect `referrers/` directory in source mirror bundle
2. Copy referrer artifacts to offline kit with same structure
3. Update offline kit manifest to include referrer metadata
4. Add verification for referrer presence in `verify-offline-kit.sh`
Update `OfflineKitManifest` to include:
```csharp
public IReadOnlyList<OfflineKitReferrerEntry> Referrers { get; init; }
public record OfflineKitReferrerEntry
{
public required string SubjectDigest { get; init; }
public required IReadOnlyList<OfflineKitReferrerArtifact> Artifacts { get; init; }
}
public record OfflineKitReferrerArtifact
{
public required string Digest { get; init; }
public required string ArtifactType { get; init; }
public required string MediaType { get; init; }
public required long SizeBytes { get; init; }
public required string RelativePath { get; init; }
}
```
Implementation completed:
- Added `OfflineKitReferrersSummary` record with counts for subjects, artifacts, SBOMs, attestations, VEX, other
- Updated `OfflineKitMirrorEntry` to include optional `Referrers` summary field
- Updated `OfflineKitMirrorRequest` to accept optional `Referrers` parameter
- Updated `OfflineKitPackager.CreateMirrorEntry()` to include referrer summary in manifest entry
- Note: Referrer artifacts are already inside the mirror bundle (tar.gz), so no separate copying needed
- All 27 existing OfflineKitPackager tests continue to pass
Completion criteria:
- [x] `OfflineKitPackager` propagates referrer summary from request to manifest
- [x] Offline kit manifest includes referrer metadata summary (counts, API support)
- [ ] `verify-offline-kit.sh` validates referrer artifact presence (deferred - inside bundle)
- [x] Unit tests verify referrer handling (existing tests pass)
- [ ] Integration test packages kit with referrers and verifies structure (deferred)
---
### REF-EXPORT-04 - Add Referrer Verification to ImportValidator
Status: DONE
Dependency: REF-EXPORT-02, REF-EXPORT-03
Owners: AirGap Guild
Task description:
Update `ImportValidator` to verify that all referrer artifacts declared in the manifest are present in the bundle.
In `ImportValidator.ValidateAsync()`:
1. Parse `referrers` section from manifest
2. For each subject image:
- Verify all declared referrer artifacts exist at expected paths
- Verify artifact checksums match declared values
- Verify artifact sizes match declared values
3. Add validation result entries for:
- `ReferrerMissing`: Declared artifact not found in bundle
- `ReferrerChecksumMismatch`: Artifact checksum doesn't match
- `ReferrerSizeMismatch`: Artifact size doesn't match
- `OrphanedReferrer`: Artifact exists but not declared (warning only)
Update `BundleValidationResult` to include referrer validation summary:
```csharp
public record ReferrerValidationSummary
{
public int TotalSubjects { get; init; }
public int TotalReferrers { get; init; }
public int ValidReferrers { get; init; }
public int MissingReferrers { get; init; }
public int ChecksumMismatches { get; init; }
public IReadOnlyList<ReferrerValidationIssue> Issues { get; init; }
}
```
Implementation completed:
- Created `ReferrerValidator` class with `Validate()` method that parses referrers section from manifest JSON
- Created `ReferrerValidationSummary`, `ReferrerValidationIssue`, `ReferrerValidationIssueType`, `ReferrerValidationSeverity` types
- Updated `BundleValidationResult` to include optional `ReferrerSummary` property
- Integrated `ReferrerValidator` into `ImportValidator` as optional dependency
- Added validation for missing artifacts, checksum mismatches, size mismatches
- Orphaned referrers (files in referrers/ not declared in manifest) produce warnings only
- Added `IsBundleTypeWithReferrers()` to enable validation only for mirror-bundle and offline-kit types
- Created 17 unit tests for ReferrerValidator
- Created 2 integration tests for ImportValidator with referrer validation
Completion criteria:
- [x] `ImportValidator` parses and validates referrer section
- [x] Missing referrer artifacts fail validation
- [x] Checksum mismatches fail validation
- [x] Orphaned referrers produce warnings (not failures)
- [x] `BundleValidationResult` includes referrer summary
- [x] Unit tests cover all validation scenarios (17 tests in ReferrerValidatorTests.cs + 2 in ImportValidatorTests.cs)
- [ ] Integration test imports bundle with intentional errors and verifies detection (deferred)
---
### REF-EXPORT-05 - Add Registry Capability Probing to Export Flow
Status: DONE
Dependency: REF-EXPORT-01
Owners: ExportCenter Guild
Task description:
Before discovering referrers for an image, probe the registry to determine the best discovery strategy.
Use `OciReferrerFallback.ProbeCapabilitiesAsync()` to detect:
- `SupportsReferrersApi`: Native OCI 1.1 referrers API available
- `DistributionVersion`: OCI Distribution spec version
- `SupportsArtifactType`: Registry supports artifactType field
Cache capabilities per registry host (already implemented with 1-hour TTL).
Log registry capabilities at start of export:
```
[INFO] Registry registry.example.com: OCI 1.1 (referrers API supported)
[WARN] Registry ghcr.io: OCI 1.0 (using fallback tag discovery)
```
Add export metrics:
- `export_registry_capabilities_probed_total{registry,api_supported}`
- `export_referrer_discovery_method_total{method=native|fallback}`
Implementation completed:
- Added `ProbeRegistryCapabilitiesAsync` to `IReferrerDiscoveryService` interface and `RegistryCapabilitiesInfo` record
- Updated `OciReferrerDiscoveryService` to probe capabilities using `IOciReferrerFallback.ProbeCapabilitiesAsync()` with caching
- Updated `MirrorAdapter.DiscoverAndCollectReferrersAsync()` to probe all unique registries before starting discovery
- Added logging at export start: "Probing {RegistryCount} registries for OCI referrer capabilities before export"
- Added capability logging: "Registry {Registry}: OCI 1.1 (referrers API supported, version={Version}, probe_ms={ProbeMs})" or warning for fallback
- Using existing `ExportTelemetry` metrics: `RegistryCapabilitiesProbedTotal`, `ReferrerDiscoveryMethodTotal`, `ReferrersDiscoveredTotal`, `ReferrerDiscoveryFailuresTotal`
- Added 3 new unit tests for probe-then-discover flow in `MirrorAdapterReferrerDiscoveryTests.cs`
Completion criteria:
- [x] Export flow probes registry capabilities before discovery
- [x] Capabilities are logged at export start
- [x] Metrics track probe results and discovery methods
- [x] Fallback is automatically used for registries without API support
- [x] Unit tests verify probe-then-discover flow
- [ ] Integration test with `registry:2` verifies native API path (deferred)
---
### REF-EXPORT-06 - Update Documentation and Architecture Docs
Status: DONE
Dependency: REF-EXPORT-01, REF-EXPORT-02, REF-EXPORT-03, REF-EXPORT-04
Owners: Documentation Guild
Task description:
Update documentation to reflect new referrer discovery and bundle handling.
Files to update:
1. `docs/modules/export-center/architecture.md`:
- Add section on OCI referrer discovery
- Document fallback mechanism for non-OCI-1.1 registries
- Add sequence diagram for referrer discovery flow
2. `docs/modules/airgap/guides/offline-bundle-format.md`:
- Update bundle structure to show `referrers/` directory
- Document referrer manifest format
- Add example with SBOM and attestation referrers
3. `docs/runbooks/registry-referrer-troubleshooting.md` (new):
- How to diagnose referrer discovery issues
- Registry compatibility matrix (brief, links to detailed doc)
- Common issues and solutions
4. `docs/modules/export-center/registry-compatibility.md` (new):
- Detailed registry compatibility matrix
- Per-registry quirks and workarounds
- Includes: GHCR, ACR, ECR, GCR, Harbor, Quay, JFrog
Implementation completed:
- Updated `architecture.md` with "OCI Referrer Discovery" section including:
- Discovery flow diagram (ASCII)
- Capability probing explanation
- Telemetry metrics table
- Artifact type mapping table
- Error handling notes
- Links to related docs
- Updated `offline-bundle-format.md` with "OCI Referrer Artifacts" section including:
- Referrer directory structure
- Manifest referrers section YAML example
- Referrer validation table
- Artifact types table
- Registry compatibility note
- Created `registry-referrer-troubleshooting.md` runbook with:
- Quick reference table
- Registry compatibility quick reference
- Diagnostic steps (logs, metrics, connectivity tests)
- Common issues and solutions
- Validation commands
- Escalation process
- Created `registry-compatibility.md` with:
- Compatibility summary table
- Detection behavior explanation
- Per-registry details (Docker Hub, GHCR, GCR, ECR, ACR, Harbor, Quay, JFrog)
- Fallback tag discovery documentation
- Testing instructions
Completion criteria:
- [x] Architecture doc updated with referrer discovery flow
- [x] Bundle format doc updated with referrer structure
- [x] New runbook created for troubleshooting
- [x] New compatibility matrix doc created
- [x] All docs link to each other appropriately
- [x] Code comments reference relevant docs (via doc links)
---
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-01-27 | Sprint created from OCI v1.1 referrers advisory review; critical gap identified in mirror bundle export. | Planning |
| 2026-01-27 | REF-EXPORT-01 DONE: Created IReferrerDiscoveryService interface, integrated into MirrorAdapter, added OciReferrerDiscoveryService wrapper, 36 tests passing. | Implementation |
| 2026-01-27 | REF-EXPORT-02 DONE: Added attestation/referrer counts to manifest YAML and README, added index placeholders, all 13 existing tests pass. | Implementation |
| 2026-01-27 | REF-EXPORT-03 DONE: Added OfflineKitReferrersSummary, updated OfflineKitMirrorEntry/Request, all 27 existing tests pass. | Implementation |
| 2026-01-27 | Core implementation complete (01, 02, 03). REF-EXPORT-04, 05, 06 deferred for follow-up. Total: 76 tests passing across 10 new/modified files. | Implementation |
| 2026-01-27 | REF-EXPORT-04 DONE: Created ReferrerValidator with validation logic, integrated into ImportValidator, updated BundleValidationResult with ReferrerSummary. 19 new tests (17 ReferrerValidator + 2 ImportValidator). | Implementation |
| 2026-01-27 | REF-EXPORT-05 verified TODO: ProbeCapabilitiesAsync infrastructure exists in OciReferrerFallback.cs with 1-hour cache, but MirrorAdapter does not call it before discovery. No metrics implemented. Fallback works automatically via OciReferrerDiscovery.ListReferrersAsync(). | Verification |
| 2026-01-27 | REF-EXPORT-06 verified TODO: Checked architecture.md and offline-bundle-format.md - no referrer mentions. registry-compatibility.md and registry-referrer-troubleshooting.md do not exist. | Verification |
| 2026-01-27 | REF-EXPORT-05 DONE: Added ProbeRegistryCapabilitiesAsync to IReferrerDiscoveryService, updated OciReferrerDiscoveryService with probing and metrics, updated MirrorAdapter to probe before discovery. 3 new tests. | Implementation |
| 2026-01-27 | REF-EXPORT-06 DONE: Updated architecture.md and offline-bundle-format.md with OCI referrer sections. Created registry-referrer-troubleshooting.md runbook and registry-compatibility.md with detailed per-registry info. All docs cross-linked. | Documentation |
| 2026-01-27 | Sprint COMPLETE: All 6 tasks DONE. Core implementation (01-04) + capability probing (05) + documentation (06). Integration tests deferred as noted in criteria. | Milestone |
## Decisions & Risks
| Item | Status / Decision | Notes |
| --- | --- | --- |
| Critical gap confirmation | CONFIRMED | `MirrorAdapter` does not call `OciReferrerDiscovery`; artifacts silently dropped. |
| Referrer storage structure | PROPOSED | `referrers/{subject-digest}/` hierarchy; to be confirmed during implementation. |
| Fallback tag pattern | USE EXISTING | `sha256-{digest}.*` pattern already in `OciReferrerFallback`. |
### Risk table
| Risk | Severity | Mitigation / Owner |
| --- | --- | --- |
| Referrer discovery significantly increases export time | Medium | Add parallelism, cache registry probes; measure in integration tests. |
| Large referrer artifacts bloat bundles | Medium | Add size limits/warnings; document recommended max sizes. |
| Fallback tag discovery misses artifacts | Low | Comprehensive testing with GHCR-like behavior. |
## Next Checkpoints
| Date (UTC) | Session / Owner | Target outcome | Fallback / Escalation |
| --- | --- | --- | --- |
| 2026-02-03 | REF-EXPORT-01/02/03 completion | Core referrer discovery and bundle integration complete. | If blocked, escalate registry access issues. |
| 2026-02-07 | REF-EXPORT-04/05 completion | Validation and capability probing complete. | Defer non-critical enhancements if needed. |
| 2026-02-10 | Sprint completion + docs | All tasks DONE, documentation updated. | Archive sprint; carry forward any blockers. |

File diff suppressed because it is too large Load Diff

View File

@@ -739,54 +739,213 @@ sequenceDiagram
- Health endpoints: `/health/liveness`, `/health/readiness`, `/status`; verification probe `/api/attestations/verify` once demo bundle is available (see runbook).
- Alert hints: signing latency > 1s p99, verification failure spikes, tlog submission lag >10s, key rotation age over policy threshold, backlog above configured threshold.
---
## 17) Rekor Entry Events
> Sprint: SPRINT_20260112_007_ATTESTOR_rekor_entry_events
Attestor emits deterministic events when DSSE bundles are logged to Rekor and inclusion proofs become available. These events drive policy reanalysis.
### Event Types
| Event Type | Constant | Description |
|------------|----------|-------------|
| `rekor.entry.logged` | `RekorEventTypes.EntryLogged` | Bundle successfully logged with inclusion proof |
| `rekor.entry.queued` | `RekorEventTypes.EntryQueued` | Bundle queued for logging (async mode) |
| `rekor.entry.inclusion_verified` | `RekorEventTypes.InclusionVerified` | Inclusion proof independently verified |
| `rekor.entry.failed` | `RekorEventTypes.EntryFailed` | Logging or verification failed |
### RekorEntryEvent Schema
```jsonc
{
"eventId": "rekor-evt-sha256:...",
"eventType": "rekor.entry.logged",
"tenant": "default",
"bundleDigest": "sha256:abc123...",
"artifactDigest": "sha256:def456...",
"predicateType": "StellaOps.ScanResults@1",
"rekorEntry": {
"uuid": "24296fb24b8ad77a...",
"logIndex": 123456789,
"logUrl": "https://rekor.sigstore.dev",
"integratedTime": "2026-01-15T10:30:02Z"
},
"reanalysisHints": {
"cveIds": ["CVE-2026-1234"],
"productKeys": ["pkg:npm/lodash@4.17.21"],
"mayAffectDecision": true,
"reanalysisScope": "immediate"
},
"occurredAtUtc": "2026-01-15T10:30:05Z"
}
```
### Offline Mode Behavior
When operating in offline/air-gapped mode:
1. Events are not emitted when Rekor is unreachable
2. Bundles are queued locally for later submission
3. Verification uses bundled checkpoints
---
## 17) Rekor Entry Events
> Sprint: SPRINT_20260112_007_ATTESTOR_rekor_entry_events
Attestor emits deterministic events when DSSE bundles are logged to Rekor and inclusion proofs become available. These events drive policy reanalysis.
### Event Types
| Event Type | Constant | Description |
|------------|----------|-------------|
| `rekor.entry.logged` | `RekorEventTypes.EntryLogged` | Bundle successfully logged with inclusion proof |
| `rekor.entry.queued` | `RekorEventTypes.EntryQueued` | Bundle queued for logging (async mode) |
| `rekor.entry.inclusion_verified` | `RekorEventTypes.InclusionVerified` | Inclusion proof independently verified |
| `rekor.entry.failed` | `RekorEventTypes.EntryFailed` | Logging or verification failed |
### RekorEntryEvent Schema
```jsonc
{
"eventId": "rekor-evt-sha256:...",
"eventType": "rekor.entry.logged",
"tenant": "default",
"bundleDigest": "sha256:abc123...",
"artifactDigest": "sha256:def456...",
"predicateType": "StellaOps.ScanResults@1",
"rekorEntry": {
"uuid": "24296fb24b8ad77a...",
"logIndex": 123456789,
"logUrl": "https://rekor.sigstore.dev",
"integratedTime": "2026-01-15T10:30:02Z"
},
"reanalysisHints": {
"cveIds": ["CVE-2026-1234"],
"productKeys": ["pkg:npm/lodash@4.17.21"],
"mayAffectDecision": true,
"reanalysisScope": "immediate"
},
"occurredAtUtc": "2026-01-15T10:30:05Z"
}
```
### Offline Mode Behavior
When operating in offline/air-gapped mode:
1. Events are not emitted when Rekor is unreachable
2. Bundles are queued locally for later submission
3. Verification uses bundled checkpoints
4. Events are generated when connectivity is restored
---
## 18) Identity Watchlist & Monitoring
> Sprint: SPRINT_0129_001_ATTESTOR_identity_watchlist_alerting
The Attestor provides proactive monitoring for signing identities appearing in transparency logs. Organizations can define watchlists to receive alerts when specific identities sign artifacts.
### Purpose
- **Credential compromise detection**: Alert when your signing identity appears unexpectedly
- **Third-party monitoring**: Watch for specific vendors or dependencies signing artifacts
- **Compliance auditing**: Track all signing activity for specific issuers
### Watchlist Entry Model
```jsonc
{
"id": "uuid",
"tenantId": "tenant-123",
"scope": "tenant", // tenant | global | system
"displayName": "GitHub Actions Signer",
"description": "Watch for GitHub Actions OIDC tokens",
// Identity fields (at least one required)
"issuer": "https://token.actions.githubusercontent.com",
"subjectAlternativeName": "repo:org/repo:*", // glob pattern
"keyId": null,
"matchMode": "glob", // exact | prefix | glob | regex
// Alert configuration
"severity": "warning", // info | warning | critical
"enabled": true,
"channelOverrides": ["slack-security"],
"suppressDuplicatesMinutes": 60,
"tags": ["github", "ci-cd"],
"createdAt": "2026-01-29T10:00:00Z",
"createdBy": "admin@example.com"
}
```
### Matching Modes
| Mode | Behavior | Example Pattern | Matches |
|------|----------|-----------------|---------|
| `exact` | Case-insensitive equality | `alice@example.com` | `Alice@example.com` |
| `prefix` | Starts-with match | `https://accounts.google.com/` | Any Google OIDC issuer |
| `glob` | Glob pattern (`*`, `?`) | `*@example.com` | `alice@example.com`, `bob@example.com` |
| `regex` | Full regex (with timeout) | `repo:org/(frontend\|backend):.*` | `repo:org/frontend:ref:main` |
### Scope Hierarchy
| Scope | Visibility | Who Can Create |
|-------|------------|----------------|
| `tenant` | Owning tenant only | Tenant admins |
| `global` | All tenants | Platform admins |
| `system` | All tenants (read-only) | System bootstrap |
### Event Flow
```
New AttestorEntry persisted
→ SignerIdentityDescriptor extracted
→ IIdentityMatcher.MatchAsync()
→ For each match:
→ Check dedup window (default 60 min)
→ Emit attestor.identity.matched event
→ Route via Notifier rules → Slack/Email/Webhook
```
### Event Schema (IdentityAlertEvent)
```jsonc
{
"eventId": "uuid",
"eventKind": "attestor.identity.matched",
"tenantId": "tenant-123",
"watchlistEntryId": "uuid",
"watchlistEntryName": "GitHub Actions Signer",
"matchedIdentity": {
"issuer": "https://token.actions.githubusercontent.com",
"subjectAlternativeName": "repo:org/repo:ref:refs/heads/main",
"keyId": null
},
"rekorEntry": {
"uuid": "24296fb24b8ad77a...",
"logIndex": 123456789,
"artifactSha256": "sha256:abc123...",
"integratedTimeUtc": "2026-01-29T10:30:00Z"
},
"severity": "warning",
"occurredAtUtc": "2026-01-29T10:30:05Z",
"suppressedCount": 0
}
```
### API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/api/v1/watchlist` | Create watchlist entry |
| `GET` | `/api/v1/watchlist` | List entries (tenant + optional global) |
| `GET` | `/api/v1/watchlist/{id}` | Get single entry |
| `PUT` | `/api/v1/watchlist/{id}` | Update entry |
| `DELETE` | `/api/v1/watchlist/{id}` | Delete entry |
| `POST` | `/api/v1/watchlist/{id}/test` | Test pattern against sample identity |
| `GET` | `/api/v1/watchlist/alerts` | List recent alerts (paginated) |
### CLI Commands
```bash
# Add a watchlist entry
stella watchlist add --issuer "https://token.actions.githubusercontent.com" \
--san "repo:org/*" --match-mode glob --severity warning
# List entries
stella watchlist list --include-global
# Test a pattern
stella watchlist test <id> --issuer "https://..." --san "repo:org/repo:ref:main"
# View recent alerts
stella watchlist alerts --since 24h --severity warning
```
### Metrics
| Metric | Description |
|--------|-------------|
| `attestor.watchlist.entries_scanned_total` | Entries processed by monitor |
| `attestor.watchlist.matches_total{severity}` | Pattern matches by severity |
| `attestor.watchlist.alerts_emitted_total` | Alerts sent to notification system |
| `attestor.watchlist.alerts_suppressed_total` | Alerts deduplicated |
| `attestor.watchlist.scan_latency_seconds` | Per-entry scan duration |
### Configuration
```yaml
attestor:
watchlist:
enabled: true
monitorMode: "changefeed" # changefeed | polling
pollingIntervalSeconds: 5 # only for polling mode
maxEventsPerSecond: 100 # rate limit
defaultDedupWindowMinutes: 60
regexTimeoutMs: 100 # safety limit
maxWatchlistEntriesPerTenant: 1000
```
### Offline Mode
In air-gapped environments:
- Polling mode used instead of Postgres NOTIFY
- Alerts queued locally if notification channels unavailable
- Alerts delivered when connectivity restored

View File

@@ -0,0 +1,237 @@
# Identity Watchlist User Guide
This guide covers how to use the identity watchlist feature in Stella Ops to monitor signing activity in transparency logs.
## Overview
The identity watchlist enables proactive alerting when specific signing identities appear in Rekor transparency log entries. Use cases include:
- **Threat Detection**: Monitor for known malicious or compromised signing identities
- **Compliance Monitoring**: Track when specific OIDC issuers sign artifacts
- **Operational Awareness**: Get notified when CI/CD workflows from specific repositories sign releases
- **Key Tracking**: Monitor usage of specific signing keys across your organization
## Core Concepts
### Watchlist Entries
A watchlist entry defines an identity pattern to monitor and alert configuration:
| Field | Description | Required |
|-------|-------------|----------|
| `displayName` | Human-readable label | Yes |
| `description` | Why this identity is watched | No |
| `issuer` | OIDC issuer URL pattern | At least one |
| `subjectAlternativeName` | Certificate SAN pattern | of these |
| `keyId` | Key identifier for keyful signing | three |
| `matchMode` | Pattern matching mode | No (default: `exact`) |
| `severity` | Alert severity level | No (default: `warning`) |
| `enabled` | Whether entry is active | No (default: `true`) |
| `scope` | Visibility scope | No (default: `tenant`) |
### Match Modes
| Mode | Description | Example Pattern | Matches |
|------|-------------|-----------------|---------|
| `exact` | Case-insensitive equality | `https://accounts.google.com` | Only that exact URL |
| `prefix` | Starts with pattern | `https://token.actions.` | Any GitHub Actions issuer |
| `glob` | Wildcard matching | `*@example.com` | Any email at example.com |
| `regex` | Full regex (advanced) | `user\d+@.*` | user123@domain.com |
> **Note**: Regex mode has a 100ms timeout and pattern validation. Use sparingly for performance-critical environments.
### Severity Levels
| Level | Use Case | Notification Priority |
|-------|----------|----------------------|
| `info` | Informational tracking | Low |
| `warning` | Unexpected but not critical (default) | Medium |
| `critical` | Immediate attention required | High |
### Scope Levels
| Scope | Description | Who Can Create |
|-------|-------------|----------------|
| `tenant` | Visible only to owning tenant | Any user with `watchlist:write` |
| `global` | Shared across all tenants | Administrators only |
| `system` | System-managed entries | System only |
## CLI Usage
### Adding a Watchlist Entry
Monitor a specific OIDC issuer:
```bash
stella watchlist add \
--issuer "https://token.actions.githubusercontent.com" \
--name "GitHub Actions" \
--severity warning \
--description "Track all GitHub Actions signatures"
```
Monitor email patterns with glob matching:
```bash
stella watchlist add \
--san "*@malicious-domain.com" \
--match-mode glob \
--severity critical \
--name "Malicious Domain"
```
### Listing Entries
List all entries including global ones:
```bash
stella watchlist list --include-global
```
Output as JSON for scripting:
```bash
stella watchlist list --format json
```
### Testing Patterns
Test if an identity would trigger an alert:
```bash
stella watchlist test <entry-id> \
--issuer "https://token.actions.githubusercontent.com" \
--san "repo:org/repo:ref:refs/heads/main"
```
### Managing Entries
Update an entry:
```bash
stella watchlist update <entry-id> --enabled false
stella watchlist update <entry-id> --severity critical
```
Delete an entry:
```bash
stella watchlist remove <entry-id>
stella watchlist remove <entry-id> --force # Skip confirmation
```
### Viewing Alerts
List recent alerts:
```bash
stella watchlist alerts --since 24h
stella watchlist alerts --severity critical --limit 50
```
## API Usage
### Create Entry
```http
POST /api/v1/watchlist
Content-Type: application/json
{
"displayName": "GitHub Actions Monitor",
"issuer": "https://token.actions.githubusercontent.com",
"matchMode": "prefix",
"severity": "warning",
"description": "Monitor all GitHub Actions signatures",
"enabled": true,
"suppressDuplicatesMinutes": 60
}
```
### List Entries
```http
GET /api/v1/watchlist?includeGlobal=true
```
### Test Pattern
```http
POST /api/v1/watchlist/{id}/test
Content-Type: application/json
{
"issuer": "https://token.actions.githubusercontent.com",
"subjectAlternativeName": "repo:org/repo:ref:refs/heads/main"
}
```
## Alert Deduplication
To prevent alert storms, the watchlist system deduplicates alerts based on:
1. **Watchlist Entry ID** - Which entry triggered the alert
2. **Identity Hash** - SHA-256 of the matched identity fields
Configure the deduplication window per entry with `suppressDuplicatesMinutes` (default: 60 minutes).
When an alert is suppressed, subsequent alerts will include a `suppressedCount` indicating how many similar alerts were skipped.
## Best Practices
### Pattern Design
1. **Start specific, broaden as needed**: Begin with exact matches before using wildcards
2. **Test before deploying**: Use the test endpoint to validate patterns
3. **Document why**: Always include a description explaining the monitoring purpose
4. **Use appropriate match modes**:
- `exact` for known bad actors
- `prefix` for issuer families (e.g., all Google issuers)
- `glob` for email/SAN patterns
- `regex` only when simpler modes can't express the pattern
### Performance Considerations
- Limit regex patterns to < 256 characters
- Avoid catastrophic backtracking patterns (e.g., `(a+)+`)
- Monitor the `attestor.watchlist.scan_latency_seconds` metric
- Keep watchlist size reasonable (< 1000 entries per tenant)
### Alert Management
1. **Set appropriate severity**: Reserve `critical` for genuine emergencies
2. **Configure dedup windows**: Use shorter windows (5-15 min) for critical entries
3. **Review suppressed counts**: High counts may indicate a pattern is too broad
4. **Route appropriately**: Use channel overrides for sensitive entries
## Monitoring & Metrics
The following OpenTelemetry metrics are exposed:
| Metric | Description |
|--------|-------------|
| `attestor.watchlist.entries_scanned_total` | Total Rekor entries processed |
| `attestor.watchlist.matches_total{severity}` | Pattern matches by severity |
| `attestor.watchlist.alerts_emitted_total` | Alerts sent to notification system |
| `attestor.watchlist.alerts_suppressed_total` | Alerts suppressed by deduplication |
| `attestor.watchlist.scan_latency_seconds` | Per-entry scan duration |
## Troubleshooting
### Entry Not Matching Expected Identity
1. Verify the match mode is appropriate for your pattern
2. Use the test endpoint with verbose output
3. Check if the entry is enabled
4. Verify case sensitivity (all modes are case-insensitive)
### Too Many Alerts
1. Increase `suppressDuplicatesMinutes`
2. Narrow the pattern (more specific issuer, SAN prefix)
3. Consider if the pattern is too broad
### Missing Alerts
1. Verify the entry is enabled
2. Check if alerts are being suppressed (view suppressed count)
3. Verify notification routing is configured
4. Check service logs for matching errors
## Related Documentation
- [Attestor Architecture](../architecture.md)
- [Notification Templates](../../notify/templates.md)
- [Watchlist Monitoring Runbook](../../operations/watchlist-monitoring-runbook.md)

View File

@@ -1496,5 +1496,101 @@ A mismatch fails the blob replay verification step.
---
*Document Version: 1.2.0*
*Last Updated: 2026-01-21*
## 12. Binary Micro-Witnesses
Binary micro-witnesses provide cryptographic proof of patch status at the binary level. They formalize the output of BinaryIndex's semantic diffing capabilities into an auditor-friendly, portable format.
### 12.1 Overview
A micro-witness is a DSSE (Dead Simple Signing Envelope) predicate that captures:
- Subject binary digest (SHA-256)
- CVE/patch reference
- Function-level evidence with confidence scores
- Delta-Sig fingerprint hash
- Tool versions and analysis metadata
- Optional SBOM component mapping
### 12.2 Predicate Schema
**Predicate Type:** `https://stellaops.dev/predicates/binary-micro-witness@v1`
```json
{
"schemaVersion": "1.0.0",
"binary": {
"digest": "sha256:...",
"purl": "pkg:deb/debian/openssl@3.0.11",
"arch": "linux-amd64",
"filename": "libssl.so.3"
},
"cve": {
"id": "CVE-2024-0567",
"advisory": "https://...",
"patchCommit": "abc123"
},
"verdict": "patched",
"confidence": 0.95,
"evidence": [
{
"function": "SSL_CTX_new",
"state": "patched",
"score": 0.97,
"method": "semantic_ksg",
"hash": "sha256:..."
}
],
"deltaSigDigest": "sha256:...",
"sbomRef": {
"sbomDigest": "sha256:...",
"purl": "pkg:...",
"bomRef": "component-ref"
},
"tooling": {
"binaryIndexVersion": "2.1.0",
"lifter": "b2r2",
"matchAlgorithm": "semantic_ksg"
},
"computedAt": "2026-01-28T12:00:00Z"
}
```
### 12.3 Verdicts
| Verdict | Meaning |
|---------|---------|
| `patched` | Binary matches patched version signature |
| `vulnerable` | Binary matches vulnerable version signature |
| `inconclusive` | Unable to determine (insufficient evidence) |
| `partial` | Some functions patched, others not |
### 12.4 CLI Commands
```bash
# Generate a micro-witness
stella witness generate /path/to/binary --cve CVE-2024-0567 --sbom sbom.json --output witness.json
# Verify a micro-witness
stella witness verify witness.json --offline
# Create portable bundle for air-gapped verification
stella witness bundle witness.json --output ./audit-bundle
```
### 12.5 Integration with Rekor
When `--rekor` is specified during generation, witnesses are logged to the Rekor transparency log using v2 tile-based inclusion proofs. This provides tamper-evidence and enables auditors to verify witnesses weren't backdated.
Offline verification bundles include tile proofs for air-gapped environments.
### 12.6 Related Documentation
- **Auditor Guide:** `docs/guides/binary-micro-witness-verification.md`
- **Predicate Schema:** `src/Attestor/StellaOps.Attestor.Types/schemas/stellaops-binary-micro-witness.v1.schema.json`
- **CLI Commands:** `src/Cli/StellaOps.Cli/Commands/Witness/`
- **Demo Bundle:** `demos/binary-micro-witness/`
- **Sprint:** `docs-archived/implplan/SPRINT_0128_001_BinaryIndex_binary_micro_witness.md`
---
*Document Version: 1.3.0*
*Last Updated: 2026-01-28*

View File

@@ -995,6 +995,70 @@ Group: `/api/v1/policy/interop` with tag `PolicyInterop`
| POST | `/evaluate` | `platform.policy.evaluate` | Evaluate policy against input |
| GET | `/formats` | `platform.policy.read` | List supported formats |
### 13.9 · OPA Supply Chain Evidence Input
> **Sprint:** SPRINT_0129_001_Policy_supply_chain_evidence_input
OPA policies can optionally access comprehensive supply chain evidence beyond basic VEX merge results. When `PolicyGateContext.SupplyChainEvidence` is populated, the following fields become available in the OPA input:
| Input Field | Type | Description |
|-------------|------|-------------|
| `artifact.digest` | string | Artifact digest (e.g., `sha256:abc...`) |
| `artifact.mediaType` | string | OCI media type |
| `artifact.reference` | string | Full artifact reference |
| `sbom.digest` | string | SBOM content hash |
| `sbom.format` | string | Format identifier (e.g., `cyclonedx-1.7`, `spdx-3.0.1`) |
| `sbom.componentCount` | int | Number of components |
| `sbom.content` | object | Optional inline SBOM JSON |
| `attestations[]` | array | Attestation references |
| `attestations[].digest` | string | DSSE envelope digest |
| `attestations[].predicateType` | string | in-toto predicate type URI |
| `attestations[].signatureVerified` | bool | Signature verification status |
| `attestations[].rekorLogIndex` | long | Transparency log index |
| `transparency.rekor[]` | array | Rekor receipts |
| `transparency.rekor[].logId` | string | Log identifier |
| `transparency.rekor[].uuid` | string | Entry UUID |
| `transparency.rekor[].logIndex` | long | Log position |
| `transparency.rekor[].integratedTime` | long | Unix timestamp |
| `transparency.rekor[].verified` | bool | Receipt verification status |
| `vex.mergeDecision` | object | VEX merge decision |
| `vex.mergeDecision.algorithm` | string | Merge algorithm (e.g., `trust-weighted-lattice-v1`) |
| `vex.mergeDecision.inputs[]` | array | Source documents with trust weights |
| `vex.mergeDecision.decisions[]` | array | Per-vulnerability decisions with provenance |
**Code locations:**
- Evidence models: `src/Policy/__Libraries/StellaOps.Policy/Gates/Opa/OpaEvidenceModels.cs`
- Context extension: `src/Policy/__Libraries/StellaOps.Policy/Gates/PolicyGateAbstractions.cs`
- Input builder: `src/Policy/__Libraries/StellaOps.Policy/Gates/Opa/OpaGateAdapter.cs`
**Example Rego policy using evidence:**
```rego
package stella.supply_chain
default allow = false
# Require SBOM presence
allow {
input.sbom.digest != ""
input.sbom.componentCount > 0
}
# Require verified attestation with SLSA provenance
allow {
some att in input.attestations
att.predicateType == "https://slsa.dev/provenance/v1"
att.signatureVerified == true
}
# Require transparency log entry within 24 hours
allow {
some receipt in input.transparency.rekor
receipt.verified == true
time.now_ns() - (receipt.integratedTime * 1000000000) < 86400000000000
}
```
---
*Last updated: 2026-01-23 (Sprint 041).*
*Last updated: 2026-01-29 (Sprint 0129_001).*

View File

@@ -0,0 +1,214 @@
# Identity Watchlist Monitoring Runbook
This runbook covers operational procedures for the Stella Ops identity watchlist monitoring system.
## Service Overview
The identity watchlist monitor is a background service that:
1. Monitors new Attestor entries in real-time (or via polling)
2. Matches signer identities against configured watchlist patterns
3. Emits alerts through the notification system
4. Applies deduplication to prevent alert storms
### Configuration
```yaml
# appsettings.json
{
"Attestor": {
"Watchlist": {
"Enabled": true,
"Mode": "ChangeFeed", # or "Polling" for air-gap
"PollingInterval": "00:00:05", # 5 seconds
"MaxEventsPerSecond": 100,
"DefaultDedupWindowMinutes": 60,
"RegexTimeoutMs": 100,
"MaxWatchlistEntriesPerTenant": 1000,
"PatternCacheSize": 1000,
"InitialDelay": "00:00:10",
"NotifyChannelName": "attestor_entries_inserted"
}
}
}
```
## Alert Triage Procedures
### Critical Severity Alert
**Response Time**: Immediate (< 15 minutes)
1. **Acknowledge** the alert in your incident management system
2. **Verify** the matched identity in Rekor:
```bash
rekor-cli get --uuid <rekor-uuid>
```
3. **Determine impact**:
- What artifact was signed?
- Is this a known/expected signer?
- What systems consume this artifact?
4. **Escalate** if malicious activity is confirmed
5. **Document** findings in incident record
### Warning Severity Alert
**Response Time**: Within 1 hour
1. **Review** the alert details
2. **Check context**:
- Is this a new legitimate workflow?
- Is the pattern too broad?
3. **Adjust** watchlist entry if needed:
```bash
stella watchlist update <id> --severity info
# or
stella watchlist update <id> --enabled false
```
4. **Document** decision rationale
### Info Severity Alert
**Response Time**: Next business day
1. **Review** for patterns or trends
2. **Consider** if alert should be disabled or tuned
3. **Archive** after review
## Performance Tuning
### High Scan Latency
**Symptom**: `attestor.watchlist.scan_latency_seconds` > 10ms
**Investigation**:
1. Check pattern cache hit rate:
```sql
SELECT COUNT(*) FROM attestor.identity_watchlist WHERE enabled = true;
```
2. Review regex patterns for complexity
3. Check tenant watchlist count
**Resolution**:
- Increase `PatternCacheSize` if cache misses are high
- Simplify complex regex patterns
- Consider splitting overly broad patterns
### High Alert Volume
**Symptom**: `attestor.watchlist.alerts_emitted_total` growing rapidly
**Investigation**:
1. Identify top-triggering entries:
```bash
stella watchlist alerts --since 1h --format json | jq 'group_by(.watchlistEntryId) | map({id: .[0].watchlistEntryId, count: length}) | sort_by(-.count)'
```
2. Check if pattern is too broad
**Resolution**:
- Narrow pattern scope
- Increase dedup window
- Reduce severity if appropriate
### Database Performance
**Symptom**: Slow list/match queries
**Investigation**:
```sql
EXPLAIN ANALYZE
SELECT * FROM attestor.identity_watchlist
WHERE enabled = true AND (tenant_id = 'tenant-1' OR scope IN ('Global', 'System'));
```
**Resolution**:
- Verify indexes exist:
```sql
SELECT indexname FROM pg_indexes WHERE tablename = 'identity_watchlist';
```
- Run VACUUM ANALYZE if needed
- Consider partitioning for large deployments
## Deduplication Table Maintenance
### Cleanup Expired Records
Run periodically (daily recommended):
```sql
DELETE FROM attestor.identity_alert_dedup
WHERE last_alert_at < NOW() - INTERVAL '7 days';
```
### Check Dedup Effectiveness
```sql
SELECT
watchlist_id,
COUNT(*) as suppressed_identities,
SUM(alert_count) as total_suppressions
FROM attestor.identity_alert_dedup
GROUP BY watchlist_id
ORDER BY total_suppressions DESC
LIMIT 10;
```
## Air-Gap Operation
For environments without network access to PostgreSQL LISTEN/NOTIFY:
1. Set `Mode: Polling` in configuration
2. Adjust `PollingInterval` based on acceptable delay (default: 5s)
3. Ensure sufficient database connection pool size
4. Monitor for missed entries during polling gaps
## Disaster Recovery
### Service Restart
1. Entries are processed based on `IntegratedTimeUtc`
2. On restart, the service resumes from last checkpoint
3. Some duplicate alerts may occur during recovery (handled by dedup)
### Database Failover
1. Service will retry connections automatically
2. Pattern cache survives in-memory during brief outages
3. Long outages may require service restart
### Watchlist Export/Import
Export:
```bash
stella watchlist list --include-global --format json > watchlist-backup.json
```
Import (manual):
```bash
# Process each entry and recreate
jq -c '.[]' watchlist-backup.json | while read entry; do
# Extract fields and call stella watchlist add
done
```
## Metrics Reference
| Metric | Description | Alert Threshold |
|--------|-------------|-----------------|
| `attestor.watchlist.entries_scanned_total` | Processing volume | N/A (informational) |
| `attestor.watchlist.matches_total` | Match frequency | > 100/min (review patterns) |
| `attestor.watchlist.alerts_emitted_total` | Alert volume | > 50/min (check notification capacity) |
| `attestor.watchlist.alerts_suppressed_total` | Dedup effectiveness | High ratio = good dedup working |
| `attestor.watchlist.scan_latency_seconds` | Performance | p99 > 50ms (tune cache/patterns) |
## Escalation Contacts
| Severity | Contact | Response SLA |
|----------|---------|--------------|
| Critical | On-call Security | 15 minutes |
| Warning | Security Team | 1 hour |
| Info | Security Analyst | Next business day |
## Related Documents
- [Identity Watchlist User Guide](../modules/attestor/guides/identity-watchlist.md)
- [Attestor Architecture](../modules/attestor/architecture.md)
- [Notification System](../modules/notify/architecture.md)

View File

@@ -1,84 +1,171 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://spdx.org/rdf/3.0.1/spdx-json-schema.json",
"title": "SPDX 3.0.1 JSON-LD Schema",
"description": "Schema for SPDX 3.0.1 documents in JSON-LD format",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://spdx.org/schema/3.0.1/spdx-json-schema.json",
"title": "SPDX 3.0.1 JSON-LD Schema (Structural Subset)",
"description": "Focused structural validation for SPDX 3.0.1 JSON-LD documents. Validates key requirements per SPDX 3.0.1 specification.",
"type": "object",
"required": ["@context", "@graph"],
"properties": {
"@context": {
"type": "string",
"description": "JSON-LD context for SPDX 3.0"
"const": "https://spdx.org/rdf/3.0.1/spdx-context.jsonld"
},
"@graph": {
"type": "array",
"description": "Array of SPDX elements",
"minItems": 1,
"items": {
"type": "object",
"properties": {
"@type": { "type": "string" },
"@id": { "type": "string" },
"spdxId": { "type": "string" },
"name": { "type": "string" },
"description": { "type": "string" },
"comment": { "type": "string" },
"creationInfo": { "type": "object" },
"verifiedUsing": { "type": "array" },
"externalRef": { "type": "array" },
"externalIdentifier": { "type": "array" },
"extension": { "type": "array" }
}
"$ref": "#/$defs/GraphElement"
}
},
"spdxVersion": {
"type": "string",
"pattern": "^SPDX-3\\.[0-9]+(\\.[0-9]+)?$"
},
"creationInfo": {
}
},
"additionalProperties": false,
"$defs": {
"GraphElement": {
"type": "object",
"required": ["type"],
"properties": {
"@type": { "type": "string" },
"specVersion": { "type": "string" },
"created": { "type": "string", "format": "date-time" },
"createdBy": { "type": "array" },
"createdUsing": { "type": "array" },
"profile": { "type": "array" },
"dataLicense": { "type": "string" }
}
},
"rootElement": {
"type": "array",
"items": { "type": "string" }
},
"element": {
"type": "array",
"items": {
"type": "object",
"properties": {
"@type": { "type": "string" },
"spdxId": { "type": "string" },
"name": { "type": "string" },
"summary": { "type": "string" },
"description": { "type": "string" },
"comment": { "type": "string" },
"verifiedUsing": { "type": "array" },
"externalRef": { "type": "array" },
"externalIdentifier": { "type": "array" }
"type": {
"type": "string",
"minLength": 1
},
"@id": {
"type": "string"
},
"spdxId": {
"type": "string"
},
"creationInfo": {
"type": "string"
},
"name": {
"type": "string"
},
"created": {
"type": "string"
},
"specVersion": {
"type": "string"
},
"createdBy": {
"type": "array",
"items": { "type": "string" }
},
"createdUsing": {
"type": "array",
"items": { "type": "string" }
},
"rootElement": {
"type": "array",
"items": { "type": "string" }
},
"element": {
"type": "array",
"items": { "type": "string" }
},
"profileConformance": {
"type": "array",
"items": { "type": "string" }
},
"software_sbomType": {
"type": "array",
"items": { "type": "string" }
},
"software_packageVersion": { "type": "string" },
"software_packageUrl": { "type": "string" },
"software_downloadLocation": { "type": "string" },
"software_primaryPurpose": { "type": "string" },
"software_copyrightText": { "type": "string" },
"simplelicensing_licenseExpression": { "type": "string" },
"from": { "type": "string" },
"relationshipType": { "type": "string" },
"to": {
"type": "array",
"items": { "type": "string" }
}
}
},
"namespaceMap": {
"type": "array",
"items": {
"type": "object",
"properties": {
"prefix": { "type": "string" },
"namespace": { "type": "string", "format": "uri" }
},
"allOf": [
{
"if": {
"properties": { "type": { "const": "CreationInfo" } },
"required": ["type"]
},
"then": {
"required": ["@id", "created", "specVersion"]
}
},
{
"if": {
"properties": { "type": { "const": "SpdxDocument" } },
"required": ["type"]
},
"then": {
"required": ["spdxId", "creationInfo", "rootElement", "element"]
}
},
{
"if": {
"properties": { "type": { "const": "software_Sbom" } },
"required": ["type"]
},
"then": {
"required": ["spdxId", "creationInfo", "rootElement", "element"]
}
},
{
"if": {
"properties": { "type": { "const": "software_Package" } },
"required": ["type"]
},
"then": {
"required": ["spdxId", "creationInfo", "name"]
}
},
{
"if": {
"properties": { "type": { "const": "software_File" } },
"required": ["type"]
},
"then": {
"required": ["spdxId", "creationInfo", "name"]
}
},
{
"if": {
"properties": { "type": { "const": "Relationship" } },
"required": ["type"]
},
"then": {
"required": ["spdxId", "creationInfo", "from", "relationshipType", "to"]
}
},
{
"if": {
"properties": { "type": { "const": "Tool" } },
"required": ["type"]
},
"then": {
"required": ["spdxId", "creationInfo", "name"]
}
},
{
"if": {
"properties": { "type": { "const": "Organization" } },
"required": ["type"]
},
"then": {
"required": ["spdxId", "creationInfo", "name"]
}
},
{
"if": {
"properties": { "type": { "const": "Person" } },
"required": ["type"]
},
"then": {
"required": ["spdxId", "creationInfo", "name"]
}
}
}
},
"imports": {
"type": "array",
"items": { "type": "string", "format": "uri" }
],
"additionalProperties": true
}
}
}