8768c27f30
- Introduced `ReachabilityState`, `RuntimeHit`, `ExploitabilitySignal`, `ReachabilitySignal`, `SignalEnvelope`, `SignalType`, `TrustSignal`, and `UnknownSymbolSignal` records to define various signal types and their properties. - Implemented JSON serialization attributes for proper data interchange. - Created project files for the new signal contracts library and corresponding test projects. - Added deterministic test fixtures for micro-interaction testing. - Included cryptographic keys for secure operations with cosign.
318 lines
8.5 KiB
Markdown
318 lines
8.5 KiB
Markdown
# Feed Snapshot Freeze/Staleness Thresholds (SP6)
|
|
|
|
Status: Draft · Date: 2025-12-04
|
|
Scope: Codify feed snapshot governance including freshness budgets, staleness thresholds, freeze procedures, and validation checks.
|
|
|
|
## Objectives
|
|
|
|
- Define maximum age for vulnerability feed snapshots.
|
|
- Establish freeze/thaw procedures for reproducible scans.
|
|
- Set staleness detection and alerting thresholds.
|
|
- Enable deterministic feed state for replay scenarios.
|
|
|
|
## Freshness Budgets
|
|
|
|
### Feed Types
|
|
|
|
| Feed Type | Source | Max Age | Staleness Threshold | Critical Threshold |
|
|
|-----------|--------|---------|---------------------|-------------------|
|
|
| NVD | NIST NVD | 24 hours | 48 hours | 7 days |
|
|
| OSV | OSV.dev | 12 hours | 24 hours | 3 days |
|
|
| GHSA | GitHub | 12 hours | 24 hours | 3 days |
|
|
| Alpine | Alpine SecDB | 24 hours | 48 hours | 7 days |
|
|
| Debian | Debian Security Tracker | 24 hours | 48 hours | 7 days |
|
|
| RHEL | Red Hat OVAL | 24 hours | 72 hours | 7 days |
|
|
| Ubuntu | Ubuntu CVE Tracker | 24 hours | 48 hours | 7 days |
|
|
|
|
### Definitions
|
|
|
|
- **Max Age**: Maximum time since last successful sync before warnings
|
|
- **Staleness Threshold**: Time after which scans emit staleness warnings
|
|
- **Critical Threshold**: Time after which scans are blocked without override
|
|
|
|
## Snapshot States
|
|
|
|
```
|
|
┌─────────────┐
|
|
│ SYNCING │ ──────────────────────────────────┐
|
|
└─────────────┘ │
|
|
│ │
|
|
▼ ▼
|
|
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
|
│ FRESH │ ──► │ STALE │ ──► │ CRITICAL │
|
|
└─────────────┘ └─────────────┘ └─────────────┘
|
|
│ │ │
|
|
▼ ▼ ▼
|
|
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
|
│ FROZEN │ │ FROZEN │ │ FROZEN │
|
|
│ (fresh) │ │ (stale) │ │ (critical) │
|
|
└─────────────┘ └─────────────┘ └─────────────┘
|
|
```
|
|
|
|
### State Transitions
|
|
|
|
| Current State | Trigger | New State | Action |
|
|
|---------------|---------|-----------|--------|
|
|
| SYNCING | Sync complete | FRESH | Record snapshot hash |
|
|
| FRESH | Max age exceeded | STALE | Emit warning |
|
|
| STALE | Staleness threshold exceeded | CRITICAL | Block scans |
|
|
| CRITICAL | Manual override | CRITICAL (override) | Log override |
|
|
| Any | Freeze command | FROZEN | Lock snapshot |
|
|
| FROZEN | Thaw command | Previous state | Unlock snapshot |
|
|
|
|
## Snapshot Schema
|
|
|
|
```json
|
|
{
|
|
"id": "nvd-2025-12-04T00-00-00Z",
|
|
"feed": "nvd",
|
|
"source": "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz",
|
|
"syncedAt": "2025-12-04T00:00:00Z",
|
|
"hash": {
|
|
"blake3": "...",
|
|
"sha256": "..."
|
|
},
|
|
"state": "fresh",
|
|
"advisoryCount": 250000,
|
|
"lastModified": "2025-12-03T23:45:00Z",
|
|
"frozen": false,
|
|
"frozenAt": null,
|
|
"frozenBy": null,
|
|
"thresholds": {
|
|
"maxAge": "PT24H",
|
|
"staleness": "PT48H",
|
|
"critical": "P7D"
|
|
}
|
|
}
|
|
```
|
|
|
|
## Freeze Procedures
|
|
|
|
### Manual Freeze
|
|
|
|
```bash
|
|
# Freeze current snapshot for reproducible scans
|
|
stellaops feed freeze --feed nvd --reason "Audit baseline 2025-Q4"
|
|
|
|
# Freeze with specific snapshot
|
|
stellaops feed freeze --feed nvd --snapshot-id nvd-2025-12-04T00-00-00Z
|
|
|
|
# List frozen snapshots
|
|
stellaops feed list --frozen
|
|
```
|
|
|
|
### Automatic Freeze (Replay Mode)
|
|
|
|
When recording a scan for replay:
|
|
|
|
1. Scanner captures current snapshot IDs for all feeds
|
|
2. Snapshot hashes recorded in replay manifest
|
|
3. Replay execution uses frozen snapshots from manifest
|
|
|
|
```json
|
|
{
|
|
"replay": {
|
|
"feeds": {
|
|
"nvd": {
|
|
"snapshotId": "nvd-2025-12-04T00-00-00Z",
|
|
"hash": "b3:...",
|
|
"state": "frozen"
|
|
},
|
|
"osv": {
|
|
"snapshotId": "osv-2025-12-04T00-00-00Z",
|
|
"hash": "b3:...",
|
|
"state": "frozen"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Thaw Procedures
|
|
|
|
```bash
|
|
# Thaw a frozen snapshot (resume normal sync)
|
|
stellaops feed thaw --feed nvd --snapshot-id nvd-2025-12-04T00-00-00Z
|
|
|
|
# Force thaw all frozen snapshots
|
|
stellaops feed thaw --all --force
|
|
```
|
|
|
|
## Validation Checks
|
|
|
|
### Pre-Scan Validation
|
|
|
|
```python
|
|
# scripts/scanner/validate_feeds.py
|
|
def validate_feed_freshness(feed_states: dict) -> ValidationResult:
|
|
errors = []
|
|
warnings = []
|
|
|
|
for feed, state in feed_states.items():
|
|
age = datetime.utcnow() - state['syncedAt']
|
|
thresholds = state['thresholds']
|
|
|
|
if age > parse_duration(thresholds['critical']):
|
|
if not state.get('override'):
|
|
errors.append(f"{feed}: Critical staleness ({age})")
|
|
elif age > parse_duration(thresholds['staleness']):
|
|
warnings.append(f"{feed}: Stale ({age})")
|
|
elif age > parse_duration(thresholds['maxAge']):
|
|
warnings.append(f"{feed}: Approaching staleness ({age})")
|
|
|
|
return ValidationResult(
|
|
valid=len(errors) == 0,
|
|
errors=errors,
|
|
warnings=warnings
|
|
)
|
|
```
|
|
|
|
### Scan Output Metadata
|
|
|
|
```json
|
|
{
|
|
"scan": {
|
|
"id": "scan-12345",
|
|
"feedState": {
|
|
"nvd": {
|
|
"snapshotId": "nvd-2025-12-04T00-00-00Z",
|
|
"state": "fresh",
|
|
"age": "PT12H",
|
|
"hash": "b3:..."
|
|
},
|
|
"osv": {
|
|
"snapshotId": "osv-2025-12-04T00-00-00Z",
|
|
"state": "stale",
|
|
"age": "PT36H",
|
|
"hash": "b3:...",
|
|
"warning": "Feed approaching staleness threshold"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## API Endpoints
|
|
|
|
### Feed Status
|
|
|
|
```http
|
|
GET /api/v1/feeds/status
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"feeds": {
|
|
"nvd": {
|
|
"state": "fresh",
|
|
"snapshotId": "nvd-2025-12-04T00-00-00Z",
|
|
"syncedAt": "2025-12-04T00:00:00Z",
|
|
"age": "PT12H",
|
|
"frozen": false,
|
|
"thresholds": {
|
|
"maxAge": "PT24H",
|
|
"staleness": "PT48H",
|
|
"critical": "P7D"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Freeze Feed
|
|
|
|
```http
|
|
POST /api/v1/feeds/{feed}/freeze
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"snapshotId": "nvd-2025-12-04T00-00-00Z",
|
|
"reason": "Audit baseline"
|
|
}
|
|
```
|
|
|
|
### Override Staleness
|
|
|
|
```http
|
|
POST /api/v1/feeds/{feed}/override
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"snapshotId": "nvd-2025-12-04T00-00-00Z",
|
|
"reason": "Emergency scan required",
|
|
"approvedBy": "security-admin@example.com",
|
|
"expiresAt": "2025-12-05T00:00:00Z"
|
|
}
|
|
```
|
|
|
|
## Alerting
|
|
|
|
### Alert Conditions
|
|
|
|
| Condition | Severity | Action |
|
|
|-----------|----------|--------|
|
|
| Feed sync failed | Warning | Retry with backoff |
|
|
| Feed approaching staleness | Info | Log, notify ops |
|
|
| Feed stale | Warning | Notify ops, scan warnings |
|
|
| Feed critical | Critical | Block scans, page on-call |
|
|
| Feed frozen > 30 days | Warning | Review freeze necessity |
|
|
|
|
### Alert Payload
|
|
|
|
```json
|
|
{
|
|
"alert": {
|
|
"type": "feed_staleness",
|
|
"feed": "nvd",
|
|
"severity": "warning",
|
|
"snapshotId": "nvd-2025-12-04T00-00-00Z",
|
|
"age": "PT50H",
|
|
"threshold": "PT48H",
|
|
"message": "NVD feed exceeds staleness threshold (50h > 48h)",
|
|
"timestamp": "2025-12-06T02:00:00Z"
|
|
}
|
|
}
|
|
```
|
|
|
|
## Determinism Integration
|
|
|
|
### Replay Requirements
|
|
|
|
For deterministic replay:
|
|
|
|
1. All feeds MUST be frozen at recorded state
|
|
2. Snapshot hashes MUST match manifest
|
|
3. No network fetch during replay
|
|
4. Staleness validation skipped (uses recorded state)
|
|
|
|
### Validation Script
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
# scripts/scanner/verify_feed_hashes.sh
|
|
|
|
MANIFEST="replay-manifest.json"
|
|
|
|
for feed in $(jq -r '.replay.feeds | keys[]' "${MANIFEST}"); do
|
|
expected_hash=$(jq -r ".replay.feeds.${feed}.hash" "${MANIFEST}")
|
|
snapshot_id=$(jq -r ".replay.feeds.${feed}.snapshotId" "${MANIFEST}")
|
|
|
|
actual_hash=$(stellaops feed hash --snapshot-id "${snapshot_id}")
|
|
|
|
if [[ "${actual_hash}" != "${expected_hash}" ]]; then
|
|
echo "FAIL: ${feed} snapshot hash mismatch"
|
|
echo " expected: ${expected_hash}"
|
|
echo " actual: ${actual_hash}"
|
|
exit 1
|
|
fi
|
|
echo "PASS: ${feed} snapshot ${snapshot_id}"
|
|
done
|
|
```
|
|
|
|
## Links
|
|
|
|
- Sprint: `docs/implplan/SPRINT_0186_0001_0001_record_deterministic_execution.md` (SP6)
|
|
- Spine Versioning: `docs/modules/policy/contracts/spine-versioning-plan.md` (SP1)
|
|
- Replay: `docs/replay/DETERMINISTIC_REPLAY.md`
|