git.stella-ops.org/docs/dev/SCANNER_CACHE_CONFIGURATION.md

Scanner Cache Configuration Guide

The scanner cache stores layer-level SBOM fragments and file content that can be reused across scans. This document explains how to configure and operate the cache subsystem introduced in Sprint 10 (Group SP10-G5).

1. Overview

• Layer cache persists SBOM fragments per layer digest under <root>/layers/<digest>/ with deterministic metadata (meta.json).
• File CAS (content-addressable store) keeps deduplicated blobs (e.g., analyzer fixtures, imported SBOM layers) under <root>/cas/<prefix>/<hash>/.
• Maintenance runs via ScannerCacheMaintenanceService, evicting expired entries and compacting the cache to stay within size limits.
• Metrics emit on the StellaOps.Scanner.Cache meter with counters for hits, misses, evictions, and byte histograms.
• Offline workflows use the CAS import/export helpers to package cache warmups inside the Offline Kit.

2. Configuration keys (scanner:cache)

Key	Default	Description
enabled	true	Globally disable cache if false.
rootPath	cache/scanner	Base directory for cache data. Use an SSD-backed path for best warm-scan latency.
layersDirectoryName	layers	Subdirectory for layer cache entries.
fileCasDirectoryName	cas	Subdirectory for file CAS entries.
layerTtl	45.00:00:00	Time-to-live for layer cache entries (TimeSpan). 0 disables TTL eviction.
fileTtl	30.00:00:00	Time-to-live for CAS entries. 0 disables TTL eviction.
maxBytes	5368709120 (5 GiB)	Hard cap for combined cache footprint. Compaction trims data back to warmBytesThreshold.
warmBytesThreshold	maxBytes / 5	Target size after compaction.
coldBytesThreshold	maxBytes * 0.8	Upper bound that triggers compaction.
enableAutoEviction	true	If false, callers must invoke ILayerCacheStore.CompactAsync / IFileContentAddressableStore.CompactAsync manually.
maintenanceInterval	00:15:00	Interval for the maintenance hosted service.
enableFileCas	true	Disable to prevent CAS usage (APIs throw on PutAsync).
importDirectory / exportDirectory	null	Optional defaults for offline import/export tooling.

Tip: configure scanner:cache:rootPath to a dedicated volume and mount it into worker containers when running in Kubernetes or Nomad.

3. Metrics

Instrumentation lives in ScannerCacheMetrics on meter StellaOps.Scanner.Cache.

Instrument	Unit	Description
scanner.layer_cache_hits_total	count	Layer cache hit counter. Tag: layer.
scanner.layer_cache_misses_total	count	Layer cache miss counter. Tag: layer.
scanner.layer_cache_evictions_total	count	Layer entries evicted due to TTL or compaction. Tag: layer.
scanner.layer_cache_bytes	bytes	Histogram of per-entry payload size when stored.
scanner.file_cas_hits_total	count	File CAS hit counter. Tag: sha256.
scanner.file_cas_misses_total	count	File CAS miss counter. Tag: sha256.
scanner.file_cas_evictions_total	count	CAS eviction counter. Tag: sha256.
scanner.file_cas_bytes	bytes	Histogram of CAS payload sizes on insert.

4. Import / Export workflow

1. Export warm cache

   dotnet tool run stellaops-cache export --destination ./offline-kit/cache
   

   Internally this calls IFileContentAddressableStore.ExportAsync which copies each CAS entry (metadata + content.bin).

2. Import on air-gapped hosts

   dotnet tool run stellaops-cache import --source ./offline-kit/cache
   

   The import API merges newer metadata and skips older snapshots automatically.

3. Layer cache seeding Layer cache entries are deterministic and can be packaged the same way (copy <root>/layers). For now we keep seeding optional because layers are larger; follow-up tooling can compress directories as needed.

5. Hosted maintenance loop

ScannerCacheMaintenanceService runs as a background service within Scanner Worker or WebService hosts when AddScannerCache is registered. Behaviour:

• At startup it performs an immediate eviction/compaction run.
• Every maintenanceInterval it triggers:
  • ILayerCacheStore.EvictExpiredAsync
  • ILayerCacheStore.CompactAsync
  • IFileContentAddressableStore.EvictExpiredAsync
  • IFileContentAddressableStore.CompactAsync
• Failures are logged at Error with preserved stack traces; the next tick continues normally.

Set enableAutoEviction=false when hosting the cache inside ephemeral build pipelines that want to drive eviction explicitly.

6. API surface summary

public interface ILayerCacheStore
{
    ValueTask<LayerCacheEntry?> TryGetAsync(string layerDigest, CancellationToken ct = default);
    Task<LayerCacheEntry> PutAsync(LayerCachePutRequest request, CancellationToken ct = default);
    Task RemoveAsync(string layerDigest, CancellationToken ct = default);
    Task<int> EvictExpiredAsync(CancellationToken ct = default);
    Task<int> CompactAsync(CancellationToken ct = default);
    Task<Stream?> OpenArtifactAsync(string layerDigest, string artifactName, CancellationToken ct = default);
}

public interface IFileContentAddressableStore
{
    ValueTask<FileCasEntry?> TryGetAsync(string sha256, CancellationToken ct = default);
    Task<FileCasEntry> PutAsync(FileCasPutRequest request, CancellationToken ct = default);
    Task<bool> RemoveAsync(string sha256, CancellationToken ct = default);
    Task<int> EvictExpiredAsync(CancellationToken ct = default);
    Task<int> CompactAsync(CancellationToken ct = default);
    Task<int> ExportAsync(string destinationDirectory, CancellationToken ct = default);
    Task<int> ImportAsync(string sourceDirectory, CancellationToken ct = default);
}

Register both stores via services.AddScannerCache(configuration); in WebService or Worker hosts.

---

Last updated: 2025-10-19