Files

Docs CI / lint-and-preview (push) Has been cancelled

Details

Implement Advisory Canonicalization and Backfill Migration

- Added AdvisoryCanonicalizer for canonicalizing advisory identifiers.
- Created EnsureAdvisoryCanonicalKeyBackfillMigration to populate advisory_key and links in advisory_raw documents.
- Introduced FileSurfaceManifestStore for managing surface manifests with file system backing.
- Developed ISurfaceManifestReader and ISurfaceManifestWriter interfaces for reading and writing manifests.
- Implemented SurfaceManifestPathBuilder for constructing paths and URIs for surface manifests.
- Added tests for FileSurfaceManifestStore to ensure correct functionality and deterministic behavior.
- Updated documentation for new features and migration steps.

2025-11-07 19:54:02 +02:00

2.8 KiB

Raw Blame History

Surface.FS Consumer Integration Guide (Scheduler & Zastava)

Updated: 2025-11-07
Audience: Scheduler Worker Guild • Zastava Observer Guild • Surface FS Guild
Depends on: SURFACE-FS-02 (FileSurfaceManifestStore), Surface.Env/Surface.Secrets libraries.

This note captures the minimum wiring required for downstream services now that FileSurfaceManifestStore and the manifest reader/writer abstractions have landed.

1. Shared prerequisites

Reference StellaOps.Scanner.Surface.FS (net10.0) and call:
```
services
    .AddSurfaceFileCache()
    .AddSurfaceManifestStore();
```
This binds Surface:Cache and Surface:Manifest (or SCANNER_SURFACE_* overrides).
Pull runtime settings via ISurfaceEnvironment to ensure tenants/endpoints line up with Scanner.
Cache root (Surface:Cache:Root) must be writable; manifests fall back to <Root>/manifests unless explicitly overridden with Surface:Manifest:RootDirectory.

2. Manifest reader usage

var reader = serviceProvider.GetRequiredService<ISurfaceManifestReader>();
var manifest = await reader.TryGetByUriAsync(surfaceUri, cancellationToken);

Accept cas://{bucket}/{prefix}/{tenant}/{hh}/{tt}/{digest}.json pointers.
On cache miss, return null—callers should fall back to existing recompute paths.
All timestamps are stored in canonical UTC, and metadata dictionaries are alphabetically sorted to keep digests deterministic.

3. Scheduler worker checklist (`SCHED-SURFACE-02`)

Prefetch manifests during planning so reruns can skip redundant layers.
Persist {manifestUri, manifestDigest} alongside run plans for traceability.
Emit telemetry counters: scheduler_surface_manifest_prefetch_total{result=hit|miss}.
Update docs/SCHED-WORKER-16-201-PLANNER.md with the new prefetch flow.

4. Zastava observer checklist (`ZASTAVA-SURFACE-02`)

Resolve manifest pointer from runtime drift events (entrytrace.graph, layer.fragments kinds).
Enrich drift diagnostics with manifestDigest and Artifacts[n].metadata.
Add failure metric zastava_surface_manifest_failures_total{reason=not_found|fetch_error}.
Expand observer runbook (docs/modules/zastava/operations/drift.md) with Surface manifest troubleshooting.

5. Testing guidance

Unit-test manifest prefetch/adoption with local FileSurfaceManifestStore; use temp directories for isolation.
For integration environments, smoke-test by pointing to the same Surface:Manifest:RootDirectory used by Scanner Worker and verifying pointer fetch before scan jobs execute.

Coordinate status updates in the relevant TASKS.md entries and docs/implplan/SPRINT_130_scanner_surface.md once each guild completes its part. If you discover additional shared requirements, extend this guide so future consumers (CLI, Orchestrator) can reuse the flow.

2.8 KiB Raw Blame History