git.stella-ops.org/docs/dev/SCANNER_CACHE_CONFIGURATION.md

# Scanner Cache Configuration Guide

The scanner cache stores layer-level SBOM fragments and file content that can be reused across scans. This document explains how to configure and operate the cache subsystem introduced in Sprint 10 (Group SP10-G5).

## 1. Overview

- **Layer cache** persists SBOM fragments per layer digest under `<root>/layers/<digest>/` with deterministic metadata (`meta.json`).
- **File CAS** (content-addressable store) keeps deduplicated blobs (e.g., analyzer fixtures, imported SBOM layers) under `<root>/cas/<prefix>/<hash>/`.
- **Maintenance** runs via `ScannerCacheMaintenanceService`, evicting expired entries and compacting the cache to stay within size limits.
- **Metrics** emit on the `StellaOps.Scanner.Cache` meter with counters for hits, misses, evictions, and byte histograms.
- **Offline workflows** use the CAS import/export helpers to package cache warmups inside the Offline Kit.

## 2. Configuration keys (`scanner:cache`)

| Key | Default | Description |
| --- | --- | --- |
| `enabled` | `true` | Globally disable cache if `false`. |
| `rootPath` | `cache/scanner` | Base directory for cache data. Use an SSD-backed path for best warm-scan latency. |
| `layersDirectoryName` | `layers` | Subdirectory for layer cache entries. |
| `fileCasDirectoryName` | `cas` | Subdirectory for file CAS entries. |
| `layerTtl` | `45.00:00:00` | Time-to-live for layer cache entries (`TimeSpan`). `0` disables TTL eviction. |
| `fileTtl` | `30.00:00:00` | Time-to-live for CAS entries. `0` disables TTL eviction. |
| `maxBytes` | `5368709120` (5 GiB) | Hard cap for combined cache footprint. Compaction trims data back to `warmBytesThreshold`. |
| `warmBytesThreshold` | `maxBytes / 5` | Target size after compaction. |
| `coldBytesThreshold` | `maxBytes * 0.8` | Upper bound that triggers compaction. |
| `enableAutoEviction` | `true` | If `false`, callers must invoke `ILayerCacheStore.CompactAsync` / `IFileContentAddressableStore.CompactAsync` manually. |
| `maintenanceInterval` | `00:15:00` | Interval for the maintenance hosted service. |
| `enableFileCas` | `true` | Disable to prevent CAS usage (APIs throw on `PutAsync`). |
| `importDirectory` / `exportDirectory` | `null` | Optional defaults for offline import/export tooling. |

> **Tip:** configure `scanner:cache:rootPath` to a dedicated volume and mount it into worker containers when running in Kubernetes or Nomad.

## 3. Metrics

Instrumentation lives in `ScannerCacheMetrics` on meter `StellaOps.Scanner.Cache`.

| Instrument | Unit | Description |
| --- | --- | --- |
| `scanner.layer_cache_hits_total` | count | Layer cache hit counter. Tag: `layer`. |
| `scanner.layer_cache_misses_total` | count | Layer cache miss counter. Tag: `layer`. |
| `scanner.layer_cache_evictions_total` | count | Layer entries evicted due to TTL or compaction. Tag: `layer`. |
| `scanner.layer_cache_bytes` | bytes | Histogram of per-entry payload size when stored. |
| `scanner.file_cas_hits_total` | count | File CAS hit counter. Tag: `sha256`. |
| `scanner.file_cas_misses_total` | count | File CAS miss counter. Tag: `sha256`. |
| `scanner.file_cas_evictions_total` | count | CAS eviction counter. Tag: `sha256`. |
| `scanner.file_cas_bytes` | bytes | Histogram of CAS payload sizes on insert. |

## 4. Import / Export workflow

1. **Export warm cache**
   ```bash
   dotnet tool run stellaops-cache export --destination ./offline-kit/cache
   ```
   Internally this calls `IFileContentAddressableStore.ExportAsync` which copies each CAS entry (metadata + `content.bin`).

2. **Import on air-gapped hosts**
   ```bash
   dotnet tool run stellaops-cache import --source ./offline-kit/cache
   ```
   The import API merges newer metadata and skips older snapshots automatically.

3. **Layer cache seeding**
   Layer cache entries are deterministic and can be packaged the same way (copy `<root>/layers`). For now we keep seeding optional because layers are larger; follow-up tooling can compress directories as needed.

## 5. Hosted maintenance loop

`ScannerCacheMaintenanceService` runs as a background service within Scanner Worker or WebService hosts when `AddScannerCache` is registered. Behaviour:

- At startup it performs an immediate eviction/compaction run.
- Every `maintenanceInterval` it triggers:
  - `ILayerCacheStore.EvictExpiredAsync`
  - `ILayerCacheStore.CompactAsync`
  - `IFileContentAddressableStore.EvictExpiredAsync`
  - `IFileContentAddressableStore.CompactAsync`
- Failures are logged at `Error` with preserved stack traces; the next tick continues normally.

Set `enableAutoEviction=false` when hosting the cache inside ephemeral build pipelines that want to drive eviction explicitly.

## 6. API surface summary

```csharp
public interface ILayerCacheStore
{
    ValueTask<LayerCacheEntry?> TryGetAsync(string layerDigest, CancellationToken ct = default);
    Task<LayerCacheEntry> PutAsync(LayerCachePutRequest request, CancellationToken ct = default);
    Task RemoveAsync(string layerDigest, CancellationToken ct = default);
    Task<int> EvictExpiredAsync(CancellationToken ct = default);
    Task<int> CompactAsync(CancellationToken ct = default);
    Task<Stream?> OpenArtifactAsync(string layerDigest, string artifactName, CancellationToken ct = default);
}

public interface IFileContentAddressableStore
{
    ValueTask<FileCasEntry?> TryGetAsync(string sha256, CancellationToken ct = default);
    Task<FileCasEntry> PutAsync(FileCasPutRequest request, CancellationToken ct = default);
    Task<bool> RemoveAsync(string sha256, CancellationToken ct = default);
    Task<int> EvictExpiredAsync(CancellationToken ct = default);
    Task<int> CompactAsync(CancellationToken ct = default);
    Task<int> ExportAsync(string destinationDirectory, CancellationToken ct = default);
    Task<int> ImportAsync(string sourceDirectory, CancellationToken ct = default);
}
```

Register both stores via `services.AddScannerCache(configuration);` in WebService or Worker hosts.

---

_Last updated: 2025-10-19_