# Surface.FS Workflow Guide

> **Version:** 1.0 (2025-11-28)
>
> **Audience:** Scanner Worker/WebService integrators, Zastava operators, Offline Kit builders

## Overview

Surface.FS provides a content-addressable storage layer for Scanner-derived artefacts. This guide covers the end-to-end workflow from artefact generation to consumption, including offline bundle handling.

## Workflow Stages

```
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Scanner Worker │───▶│  Surface.FS     │───▶│  Consumers      │
│  - Scan image   │    │  - Store manifest│   │  - WebService   │
│  - Generate     │    │  - Store payload │   │  - Zastava      │
│    artefacts    │    │  - Local cache   │   │  - CLI          │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                      │                      │
         ▼                      ▼                      ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Generate:      │    │  Store:         │    │  Consume:       │
│  - Layer frags  │    │  - RustFS/S3    │    │  - Report API   │
│  - EntryTrace   │    │  - Local disk   │    │  - Drift detect │
│  - SBOM frags   │    │  - Offline kit  │    │  - Rescan plan  │
└─────────────────┘    └─────────────────┘    └─────────────────┘
```

## Stage 1: Artefact Generation (Scanner Worker)

### 1.1 Configure Surface.FS

```csharp
// In Scanner Worker startup
builder.Services.AddSurfaceFileCache();
builder.Services.AddSurfaceManifestStore();
```

Environment variables (see [Surface.Env guide](../design/surface-env.md)):
```bash
SCANNER_SURFACE_FS_ENDPOINT=http://rustfs:8080
SCANNER_SURFACE_FS_BUCKET=surface-cache
SCANNER_SURFACE_CACHE_ROOT=/var/lib/stellaops/surface
SCANNER_SURFACE_TENANT=default
```

### 1.2 Generate and Publish Artefacts

```csharp
public async Task<ScanResult> ExecuteScanAsync(ScanJob job, CancellationToken ct)
{
    // 1. Run analyzers to generate artefacts
    var layerFragments = await AnalyzeLayersAsync(job.Image, ct);
    var entryTrace = await AnalyzeEntryPointsAsync(job.Image, ct);
    var sbomFragments = await GenerateSbomAsync(job.Image, ct);

    // 2. Create manifest document
    var manifest = new SurfaceManifestDocument
    {
        Schema = "stellaops.surface.manifest@1",
        Tenant = _environment.Settings.Tenant,
        ImageDigest = job.Image.Digest,
        ScanId = job.Id,
        GeneratedAt = DateTimeOffset.UtcNow,
        Source = new SurfaceManifestSource
        {
            Component = "scanner.worker",
            Version = _version,
            WorkerInstance = Environment.MachineName,
            Attempt = job.Attempt
        },
        Artifacts = new List<SurfaceManifestArtifact>()
    };

    // 3. Add artefacts to manifest
    foreach (var fragment in layerFragments)
    {
        var payloadUri = await _manifestWriter.StorePayloadAsync(
            fragment.Content,
            "layer.fragments",
            ct);

        manifest.Artifacts.Add(new SurfaceManifestArtifact
        {
            Kind = "layer.fragments",
            Uri = payloadUri,
            Digest = fragment.Digest,
            MediaType = "application/vnd.stellaops.layer-fragments+json",
            Format = "json",
            SizeBytes = fragment.Content.Length
        });
    }

    // 4. Publish manifest
    var result = await _manifestWriter.PublishAsync(manifest, ct);

    _logger.LogInformation(
        "Published manifest {Digest} with {Count} artefacts",
        result.Digest,
        manifest.Artifacts.Count);

    return new ScanResult
    {
        ManifestUri = result.Uri,
        ManifestDigest = result.Digest
    };
}
```

### 1.3 Cache EntryTrace Results

```csharp
public async Task<EntryTraceGraph?> GetOrComputeEntryTraceAsync(
    ImageReference image,
    EntryTraceOptions options,
    CancellationToken ct)
{
    // Create deterministic cache key
    var cacheKey = new SurfaceCacheKey(
        @namespace: "entrytrace.graph",
        tenant: _environment.Settings.Tenant,
        digest: ComputeOptionsHash(options, image.Digest));

    // Try cache first
    var cached = await _cache.TryGetAsync<EntryTraceGraph>(cacheKey, ct);
    if (cached is not null)
    {
        _logger.LogDebug("EntryTrace cache hit for {Key}", cacheKey);
        return cached;
    }

    // Compute and cache
    var graph = await ComputeEntryTraceAsync(image, options, ct);
    await _cache.SetAsync(cacheKey, graph, ct);

    return graph;
}
```

## Stage 2: Storage (Surface.FS)

### 2.1 Manifest Storage Layout

```
<bucket>/
├── manifests/
│   └── <tenant>/
│       └── <digest[0..1]>/
│           └── <digest[2..3]>/
│               └── <digest>.json
└── payloads/
    └── <tenant>/
        └── <kind>/
            └── sha256/
                └── <digest[0..1]>/
                    └── <digest[2..3]>/
                        └── <digest>.json.zst
```

### 2.2 Local Cache Layout

```
<cache_root>/
├── manifests/          # Manifest JSON files
│   └── <tenant>/...
├── cache/              # Hot artefacts
│   └── <namespace>/
│       └── <tenant>/
│           └── <digest>
└── temp/               # In-progress writes
```

### 2.3 Manifest URI Format

```
cas://<bucket>/<prefix>/<tenant>/<digest[0..1]>/<digest[2..3]>/<digest>.json
```

Example:
```
cas://surface-cache/manifests/acme/ab/cd/abcdef0123456789...json
```

## Stage 3: Consumption

### 3.1 WebService API

```http
GET /api/v1/scans/{id}
```

Response includes Surface manifest pointer:
```json
{
  "id": "scan-1234",
  "status": "completed",
  "surface": {
    "manifestUri": "cas://surface-cache/manifests/acme/ab/cd/...",
    "manifestDigest": "sha256:abcdef...",
    "artifacts": [
      {
        "kind": "layer.fragments",
        "uri": "cas://surface-cache/payloads/acme/layer.fragments/...",
        "digest": "sha256:123456...",
        "mediaType": "application/vnd.stellaops.layer-fragments+json"
      }
    ]
  }
}
```

### 3.2 Zastava Drift Detection

```csharp
public async Task<DriftResult> DetectDriftAsync(
    string imageDigest,
    CancellationToken ct)
{
    // 1. Fetch baseline manifest
    var manifestUri = await _surfacePointerService.GetManifestUriAsync(imageDigest, ct);
    var manifest = await _manifestReader.TryGetByUriAsync(manifestUri, ct);

    if (manifest is null)
    {
        return DriftResult.NoBaseline();
    }

    // 2. Get EntryTrace artefact
    var entryTraceArtifact = manifest.Artifacts
        .FirstOrDefault(a => a.Kind == "entrytrace.graph");

    if (entryTraceArtifact is null)
    {
        return DriftResult.NoEntryTrace();
    }

    // 3. Compare with runtime
    var baseline = await _payloadStore.GetAsync<EntryTraceGraph>(
        entryTraceArtifact.Uri, ct);

    var runtime = await _runtimeCollector.CollectAsync(ct);

    return CompareGraphs(baseline, runtime);
}
```

### 3.3 Scheduler Rescan Planning

```csharp
public async Task<RescanPlan> CreateRescanPlanAsync(
    string imageDigest,
    CancellationToken ct)
{
    // 1. Read manifest to understand what was scanned
    var manifest = await _manifestReader.TryGetByDigestAsync(imageDigest, ct);

    if (manifest is null || IsExpired(manifest))
    {
        return RescanPlan.FullRescan();
    }

    // 2. Check for layer changes
    var layerArtifact = manifest.Artifacts
        .FirstOrDefault(a => a.Kind == "layer.fragments");

    if (layerArtifact is not null)
    {
        var layers = await _payloadStore.GetAsync<LayerFragments>(
            layerArtifact.Uri, ct);

        var changedLayers = await DetectChangedLayersAsync(layers, ct);

        if (changedLayers.Any())
        {
            return RescanPlan.IncrementalRescan(changedLayers);
        }
    }

    return RescanPlan.NoRescanNeeded();
}
```

## Offline Kit Workflow

### Export (Online Environment)

```bash
# 1. Build offline kit with Surface manifests
python ops/offline-kit/build_offline_kit.py \
  --version 2025.10.0 \
  --include-surface-manifests \
  --output-dir out/offline-kit

# 2. Kit structure includes:
# offline/
#   surface/
#     manifests/
#       <tenant>/<digest[0..1]>/<digest[2..3]>/<digest>.json
#     payloads/
#       <tenant>/<kind>/sha256/<digest[0..1]>/<digest[2..3]>/<digest>.json.zst
#     manifest-index.json
```

### Import (Air-Gapped Environment)

```csharp
public async Task ImportOfflineKitAsync(
    string kitPath,
    CancellationToken ct)
{
    var surfacePath = Path.Combine(kitPath, "surface");
    var indexPath = Path.Combine(surfacePath, "manifest-index.json");

    var index = await LoadIndexAsync(indexPath, ct);

    foreach (var entry in index.Manifests)
    {
        // 1. Load and verify manifest
        var manifestPath = Path.Combine(surfacePath, entry.RelativePath);
        var manifest = await LoadManifestAsync(manifestPath, ct);

        // 2. Verify digest
        var computedDigest = ComputeDigest(manifest);
        if (computedDigest != entry.Digest)
        {
            throw new InvalidOperationException(
                $"Manifest digest mismatch: expected {entry.Digest}, got {computedDigest}");
        }

        // 3. Import via Surface.FS API
        await _manifestWriter.PublishAsync(manifest, ct);

        _logger.LogInformation(
            "Imported manifest {Digest} for image {Image}",
            entry.Digest,
            manifest.ImageDigest);
    }

    // 4. Import payloads
    foreach (var payload in index.Payloads)
    {
        var payloadPath = Path.Combine(surfacePath, payload.RelativePath);
        await _payloadStore.ImportAsync(payloadPath, payload.Uri, ct);
    }
}
```

### Offline Operation

Once imported, Surface.FS consumers operate normally:

```csharp
// Same code works online and offline
var manifest = await _manifestReader.TryGetByUriAsync(manifestUri, ct);
var payload = await _payloadStore.GetAsync(artifact.Uri, ct);
```

## Configuration Reference

### SurfaceManifestStoreOptions

| Option | Default | Description |
|--------|---------|-------------|
| `Bucket` | `surface-cache` | Object store bucket |
| `ManifestPrefix` | `manifests` | Prefix for manifest objects |
| `PayloadPrefix` | `payloads` | Prefix for payload objects |
| `LocalManifestRoot` | `<cache>/manifests` | Local manifest directory |

### SurfaceCacheOptions

| Option | Default | Description |
|--------|---------|-------------|
| `Root` | `<temp>/stellaops/surface` | Cache root directory |
| `QuotaMegabytes` | `4096` | Cache size limit |
| `EvictionThreshold` | `0.9` | Trigger eviction at 90% quota |

## Metrics

| Metric | Labels | Description |
|--------|--------|-------------|
| `surface_manifest_published_total` | `tenant`, `kind` | Manifests published |
| `surface_manifest_cache_hit_total` | `namespace`, `tenant` | Cache hits |
| `surface_manifest_publish_duration_ms` | `tenant` | Publish latency |
| `surface_payload_persisted_total` | `kind` | Payloads stored |

## Troubleshooting

### Manifest Not Found

1. Check tenant matches between writer and reader
2. Verify Surface.FS endpoint is reachable
3. Check bucket permissions
4. Review `surface_manifest_published_total` metric

### Cache Miss Despite Expected Hit

1. Verify cache key components match (namespace, tenant, digest)
2. Check cache quota - eviction may have occurred
3. Review `surface_manifest_cache_hit_total` metric

### Offline Import Failures

1. Verify manifest digest matches index
2. Check file permissions on import path
3. Ensure Surface.FS endpoint is writable
4. Review import logs for specific errors

## References

- [Surface.FS Design](../design/surface-fs.md)
- [Surface.Env Design](../design/surface-env.md)
- [Surface.Validation Guide](./surface-validation-extensibility.md)
- [Offline Kit Documentation](../../../../24_OFFLINE_KIT.md)