git.stella-ops.org/bench/unknowns/README.md

# Unknowns Tracking Benchmark Suite

> **Purpose:** Verify epistemic uncertainty tracking and unknown state management.
> **Status:** Active
> **Sprint:** SPRINT_3850_0001_0001 (Competitive Gap Closure)

## Overview

StellaOps tracks "unknowns" - gaps in knowledge that affect confidence:
- Missing SBOM components
- Unmatched CVEs
- Stale feed data
- Zero-day windows
- Analysis limitations

## What Gets Tested

### Unknown State Lifecycle
1. Detection of unknown conditions
2. Propagation to affected findings
3. Score penalty application
4. Resolution tracking

### Unknown Categories
- `SBOM_GAP`: Component not in SBOM
- `CVE_UNMATCHED`: CVE without component mapping
- `FEED_STALE`: Feed data older than threshold
- `ZERO_DAY_WINDOW`: Time between disclosure and feed update
- `ANALYSIS_LIMIT`: Depth/timeout constraints

### Score Impact
- Each unknown type has a penalty weight
- Penalties reduce overall confidence
- Resolved unknowns restore confidence

## Test Cases

### TC-001: SBOM Gap Detection

```json
{
  "scenario": "Package in image not in SBOM",
  "input": {
    "image_packages": ["openssl@3.0.1", "curl@7.86"],
    "sbom_packages": ["openssl@3.0.1"]
  },
  "expected": {
    "unknowns": [{ "type": "SBOM_GAP", "package": "curl@7.86" }],
    "confidence_penalty": 0.15
  }
}
```

### TC-002: Zero-Day Window Tracking

```json
{
  "scenario": "CVE disclosed before feed update",
  "input": {
    "cve_disclosure": "2025-01-01T00:00:00Z",
    "feed_update": "2025-01-03T00:00:00Z",
    "scan_time": "2025-01-02T12:00:00Z"
  },
  "expected": {
    "unknowns": [{
      "type": "ZERO_DAY_WINDOW",
      "cve": "CVE-2025-0001",
      "window_hours": 36
    }],
    "risk_note": "Scan occurred during zero-day window"
  }
}
```

### TC-003: Feed Staleness

```json
{
  "scenario": "NVD feed older than 24 hours",
  "input": {
    "feed_last_update": "2025-01-01T00:00:00Z",
    "scan_time": "2025-01-02T12:00:00Z",
    "staleness_threshold_hours": 24
  },
  "expected": {
    "unknowns": [{
      "type": "FEED_STALE",
      "feed": "nvd",
      "age_hours": 36
    }]
  }
}
```

### TC-004: Score Penalty Application

```json
{
  "scenario": "Multiple unknowns compound penalty",
  "input": {
    "base_confidence": 0.95,
    "unknowns": [
      { "type": "SBOM_GAP", "penalty": 0.15 },
      { "type": "FEED_STALE", "penalty": 0.10 }
    ]
  },
  "expected": {
    "final_confidence": 0.70,
    "penalty_formula": "0.95 * (1 - 0.15) * (1 - 0.10)"
  }
}
```

## Fixtures

```
fixtures/
├── sbom-gaps/
│   ├── single-missing.json
│   ├── multiple-missing.json
│   └── layer-specific.json
├── zero-day/
│   ├── within-window.json
│   ├── after-window.json
│   └── ongoing.json
├── feed-staleness/
│   ├── nvd-stale.json
│   ├── osv-stale.json
│   └── multiple-stale.json
└── expected/
    └── all-tests.results.json
```

## Running the Suite

```bash
# Run unknowns tests
dotnet test tests/StellaOps.Unknowns.Tests

# Run penalty calculation tests
./run-penalty-tests.sh

# Run full benchmark
./run-benchmark.sh --all
```

## Metrics

| Metric | Target | Description |
|--------|--------|-------------|
| Detection rate | 100% | All unknown conditions detected |
| Penalty accuracy | ±1% | Penalties match expected values |
| Resolution tracking | 100% | All resolutions properly logged |

## UI Integration

Unknowns appear as:
- Chips in findings table
- Warning banners on scan results
- Confidence reduction indicators
- Triage action suggestions

## Integration with CI

```yaml
# .gitea/workflows/bench-unknowns.yaml
name: Unknowns Benchmark
on:
  push:
    paths:
      - 'src/Unknowns/**'
      - 'bench/unknowns/**'

jobs:
  unknowns:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run Unknowns Tests
        run: dotnet test tests/StellaOps.Unknowns.Tests
      - name: Run Benchmark
        run: ./bench/unknowns/run-benchmark.sh
```