Scanner Cache Configuration Guide

The scanner cache stores layer-level SBOM fragments and file content that can be reused across scans. This document explains how to configure and operate the cache subsystem introduced in Sprint 10 (Group SP10-G5).

1. Overview

Layer cache persists SBOM fragments per layer digest under <root>/layers/<digest>/ with deterministic metadata (meta.json).
File CAS (content-addressable store) keeps deduplicated blobs (e.g., analyzer fixtures, imported SBOM layers) under <root>/cas/<prefix>/<hash>/.
Maintenance runs via ScannerCacheMaintenanceService, evicting expired entries and compacting the cache to stay within size limits.
Metrics emit on the StellaOps.Scanner.Cache meter with counters for hits, misses, evictions, and byte histograms.
Offline workflows use the CAS import/export helpers to package cache warmups inside the Offline Kit.

2. Configuration keys (`scanner:cache`)

Key	Default	Description
`enabled`	`true`	Globally disable cache if `false`.
`rootPath`	`cache/scanner`	Base directory for cache data. Use an SSD-backed path for best warm-scan latency.
`layersDirectoryName`	`layers`	Subdirectory for layer cache entries.
`fileCasDirectoryName`	`cas`	Subdirectory for file CAS entries.
`layerTtl`	`45.00:00:00`	Time-to-live for layer cache entries (`TimeSpan`). `0` disables TTL eviction.
`fileTtl`	`30.00:00:00`	Time-to-live for CAS entries. `0` disables TTL eviction.
`maxBytes`	`5368709120` (5 GiB)	Hard cap for combined cache footprint. Compaction trims data back to `warmBytesThreshold`.
`warmBytesThreshold`	`maxBytes / 5`	Target size after compaction.
`coldBytesThreshold`	`maxBytes * 0.8`	Upper bound that triggers compaction.
`enableAutoEviction`	`true`	If `false`, callers must invoke `ILayerCacheStore.CompactAsync` / `IFileContentAddressableStore.CompactAsync` manually.
`maintenanceInterval`	`00:15:00`	Interval for the maintenance hosted service.
`enableFileCas`	`true`	Disable to prevent CAS usage (APIs throw on `PutAsync`).
`importDirectory` / `exportDirectory`	`null`	Optional defaults for offline import/export tooling.

Tip: configure scanner:cache:rootPath to a dedicated volume and mount it into worker containers when running in Kubernetes or Nomad.

3. Metrics

Instrumentation lives in ScannerCacheMetrics on meter StellaOps.Scanner.Cache.

Instrument	Unit	Description
`scanner.layer_cache_hits_total`	count	Layer cache hit counter. Tag: `layer`.
`scanner.layer_cache_misses_total`	count	Layer cache miss counter. Tag: `layer`.
`scanner.layer_cache_evictions_total`	count	Layer entries evicted due to TTL or compaction. Tag: `layer`.
`scanner.layer_cache_bytes`	bytes	Histogram of per-entry payload size when stored.
`scanner.file_cas_hits_total`	count	File CAS hit counter. Tag: `sha256`.
`scanner.file_cas_misses_total`	count	File CAS miss counter. Tag: `sha256`.
`scanner.file_cas_evictions_total`	count	CAS eviction counter. Tag: `sha256`.
`scanner.file_cas_bytes`	bytes	Histogram of CAS payload sizes on insert.

4. Import / Export workflow

Export warm cache
```
dotnet tool run stellaops-cache export --destination ./offline-kit/cache
```
Internally this calls IFileContentAddressableStore.ExportAsync which copies each CAS entry (metadata + content.bin).
Import on air-gapped hosts
```
dotnet tool run stellaops-cache import --source ./offline-kit/cache
```
The import API merges newer metadata and skips older snapshots automatically.
Layer cache seeding Layer cache entries are deterministic and can be packaged the same way (copy <root>/layers). For now we keep seeding optional because layers are larger; follow-up tooling can compress directories as needed.

5. Hosted maintenance loop

ScannerCacheMaintenanceService runs as a background service within Scanner Worker or WebService hosts when AddScannerCache is registered. Behaviour:

At startup it performs an immediate eviction/compaction run.
Every maintenanceInterval it triggers:
- ILayerCacheStore.EvictExpiredAsync
- ILayerCacheStore.CompactAsync
- IFileContentAddressableStore.EvictExpiredAsync
- IFileContentAddressableStore.CompactAsync
Failures are logged at Error with preserved stack traces; the next tick continues normally.

Set enableAutoEviction=false when hosting the cache inside ephemeral build pipelines that want to drive eviction explicitly.

6. API surface summary

public interface ILayerCacheStore
{
    ValueTask<LayerCacheEntry?> TryGetAsync(string layerDigest, CancellationToken ct = default);
    Task<LayerCacheEntry> PutAsync(LayerCachePutRequest request, CancellationToken ct = default);
    Task RemoveAsync(string layerDigest, CancellationToken ct = default);
    Task<int> EvictExpiredAsync(CancellationToken ct = default);
    Task<int> CompactAsync(CancellationToken ct = default);
    Task<Stream?> OpenArtifactAsync(string layerDigest, string artifactName, CancellationToken ct = default);
}

public interface IFileContentAddressableStore
{
    ValueTask<FileCasEntry?> TryGetAsync(string sha256, CancellationToken ct = default);
    Task<FileCasEntry> PutAsync(FileCasPutRequest request, CancellationToken ct = default);
    Task<bool> RemoveAsync(string sha256, CancellationToken ct = default);
    Task<int> EvictExpiredAsync(CancellationToken ct = default);
    Task<int> CompactAsync(CancellationToken ct = default);
    Task<int> ExportAsync(string destinationDirectory, CancellationToken ct = default);
    Task<int> ImportAsync(string sourceDirectory, CancellationToken ct = default);
}

Last updated: 2025-10-19

6.0 KiB Raw Blame History