master
66cb6c4b8a
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Introduced guild charters for Scanner Deno, PHP, Ruby, Native, WebService, Java, Surface.Env, Surface.FS, Surface.Secrets, Surface.Validation, UI, Zastava Observer, Zastava Webhook, Zastava Core, and Plugin Platform. - Each charter outlines the mission, scope, required reading, and working agreements for the respective guilds. - Created task boards for Surface.Env, Surface.FS, Surface.Secrets, Surface.Validation, and Zastava components to track progress and dependencies. - Ensured all documents emphasize determinism, offline readiness, security, and integration with shared Surface libraries.
148 lines
7.9 KiB
Markdown
148 lines
7.9 KiB
Markdown
# Surface.Env Design (Epic: SURFACE-SHARING)
|
||
|
||
> **Status:** Draft v1.0 — aligns with tasks `SURFACE-ENV-01..05`, `SCANNER-ENV-01..03`, `ZASTAVA-ENV-01..02`, `OPS-ENV-01`.
|
||
>
|
||
> **Audience:** Scanner Worker/WebService engineers, Zastava engineers, DevOps/Ops teams.
|
||
|
||
## 1. Goals
|
||
|
||
Surface.Env centralises configuration discovery for every component that touches the shared Scanner “surface” (cache, manifests, secrets). The library replaces ad-hoc environment lookups with a deterministic, validated contract that:
|
||
|
||
1. Works identically across Scanner Worker, Scanner WebService, BuildX plug-ins, Zastava Observer/Webhook, and future consumers (Scheduler planners, CLI runners).
|
||
2. Supports both connected and air-gapped deployments with clear defaults.
|
||
3. Records configuration intent (tenant isolation, cache limits, TLS, feature flags) so Surface.Validation can enforce preconditions before any work executes.
|
||
|
||
## 2. Architecture Overview
|
||
|
||
```
|
||
+-----------------------+
|
||
| Host (Worker/WebSvc) |
|
||
| - IConfiguration |
|
||
| - ILogger |
|
||
| |
|
||
| +-----------------+ |
|
||
| | SurfaceEnv | | loads env vars / config file
|
||
| | - Provider |--+------------------------------+
|
||
| | - Validators | |
|
||
| +-----------------+ |
|
||
| | |
|
||
| | IResolvedSurfaceConfiguration |
|
||
| v v
|
||
| Surface.FS / Surface.Secrets / Surface.Validation consumers
|
||
+-------------------------------------------------------------
|
||
```
|
||
|
||
Surface.Env exposes `ISurfaceEnvironment` which returns an immutable `SurfaceEnvironmentSettings` record. Hosts call `SurfaceEnvBuilder.Build()` during startup, passing optional configuration overrides (for example, Helm chart values). The builder resolves environment variables, applies defaults, and executes Surface.Validation rules before handing settings to downstream services.
|
||
|
||
## 3. Configuration Schema
|
||
|
||
### 3.1 Common keys
|
||
|
||
| Variable | Description | Default | Notes |
|
||
|----------|-------------|---------|-------|
|
||
| `SCANNER_SURFACE_FS_ENDPOINT` | Base URI for Surface.FS service (RustFS, S3-compatible). | _required_ | e.g. `https://surface-cache.svc.cluster.local`. Zastava uses `ZASTAVA_SURFACE_FS_ENDPOINT`; when absent, falls back to scanner value. |
|
||
| `SCANNER_SURFACE_FS_BUCKET` | Bucket/container name used for manifests and artefacts. | `surface-cache` | Must be unique per tenant. |
|
||
| `SCANNER_SURFACE_FS_REGION` | Optional region (S3-style). | `null` | Required for AWS S3. |
|
||
| `SCANNER_SURFACE_CACHE_ROOT` | Local filesystem directory for warm caches. | `/var/lib/stellaops/surface` | Should reside on fast SSD. |
|
||
| `SCANNER_SURFACE_CACHE_QUOTA_MB` | Soft limit for local cache usage. | `4096` | Enforced by Surface.FS eviction policy. |
|
||
| `SCANNER_SURFACE_TLS_CERT_PATH` | Path to PEM bundle for mutual TLS with Surface.FS. | `null` | If provided, library loads cert/key pair. |
|
||
| `SCANNER_SURFACE_TENANT` | Tenant identifier used for cache namespaces. | derived from Authority token | Can be overridden for multi-tenant workers. |
|
||
| `SCANNER_SURFACE_PREFETCH_ENABLED` | Toggle surface prefetch threads. | `false` | If `true`, Worker prefetches manifests before analyzer stage. |
|
||
| `SCANNER_SURFACE_FEATURES` | Comma-separated feature switches. | `""` | e.g. `validation,prewarm,runtime-diff`. |
|
||
|
||
### 3.2 Secrets provider keys
|
||
|
||
| Variable | Description | Notes |
|
||
|----------|-------------|-------|
|
||
| `SCANNER_SURFACE_SECRETS_PROVIDER` | Provider ID (`kubernetes`, `file`, `inline`). | Controls Surface.Secrets back-end. |
|
||
| `SCANNER_SURFACE_SECRETS_ROOT` | Path or secret namespace. | Example: `/etc/stellaops/secrets` for file provider. |
|
||
| `SCANNER_SURFACE_SECRETS_TENANT` | Tenant override for secret lookup. | Defaults to `SCANNER_SURFACE_TENANT`. |
|
||
|
||
### 3.3 Zastava-specific keys
|
||
|
||
Zastava containers read the same primary variables but may override names under the `ZASTAVA_` prefix (e.g., `ZASTAVA_SURFACE_CACHE_ROOT`, `ZASTAVA_SURFACE_FEATURES`). Surface.Env automatically checks component-specific prefixes before falling back to the scanner defaults.
|
||
|
||
### 3.4 Configuration precedence
|
||
|
||
1. Explicit overrides passed to `SurfaceEnvBuilder` (e.g., from appsettings).
|
||
2. Component-specific env (e.g., `ZASTAVA_SURFACE_FS_ENDPOINT`).
|
||
3. Scanner global env (e.g., `SCANNER_SURFACE_FS_ENDPOINT`).
|
||
4. `SurfaceEnvDefaults.json` (shipped with library for sensible defaults).
|
||
5. Emergency fallback values defined in code (only for development scenarios).
|
||
|
||
## 4. API Surface
|
||
|
||
```csharp
|
||
public interface ISurfaceEnvironment
|
||
{
|
||
SurfaceEnvironmentSettings Settings { get; }
|
||
IReadOnlyDictionary<string, string> RawVariables { get; }
|
||
}
|
||
|
||
public sealed record SurfaceEnvironmentSettings
|
||
(
|
||
Uri SurfaceFsEndpoint,
|
||
string SurfaceFsBucket,
|
||
string? SurfaceFsRegion,
|
||
DirectoryInfo CacheRoot,
|
||
int CacheQuotaMegabytes,
|
||
X509Certificate2Collection? ClientCertificates,
|
||
string Tenant,
|
||
bool PrefetchEnabled,
|
||
IReadOnlyCollection<string> FeatureFlags,
|
||
SecretProviderConfiguration Secrets,
|
||
IDictionary<string,string> ComponentOverrides
|
||
);
|
||
```
|
||
|
||
Consumers access `ISurfaceEnvironment.Settings` and pass the record into Surface.FS / Surface.Secrets factories. The interface memoises results so repeated access is cheap.
|
||
|
||
## 5. Validation
|
||
|
||
Surface.Env invokes the following validators (implemented in Surface.Validation):
|
||
|
||
1. **EndpointValidator** – ensures endpoint URI is absolute HTTPS and not localhost in production.
|
||
2. **CacheQuotaValidator** – verifies quota > 0 and below host max.
|
||
3. **FilesystemValidator** – checks cache root exists/writable; attempts to create directory if missing.
|
||
4. **SecretsProviderValidator** – ensures provider-specific settings (e.g., Kubernetes namespace) are present.
|
||
5. **FeatureFlagValidator** – warns on unknown feature flag tokens.
|
||
|
||
Failures throw `SurfaceEnvironmentException` with error codes (`SURFACE_ENV_MISSING_ENDPOINT`, `SURFACE_ENV_CACHE_DIR_UNWRITABLE`, etc.). Hosts log the error and fail fast during startup.
|
||
|
||
## 6. Integration Guidance
|
||
|
||
- **Scanner Worker**: call `services.AddSurfaceEnvironment()` in `Program.cs` before registering analyzers. Pass `hostContext.Configuration.GetSection("Surface")` for overrides.
|
||
- **Scanner WebService**: build environment during startup, then expose selected values via diagnostics (`/internal/surface` when diagnostics enabled).
|
||
- **Zastava Observer/Webhook**: use the same builder; ensure Helm charts set `ZASTAVA_` variables.
|
||
- **Scheduler Planner (future)**: treat Surface.Env as read-only input; do not mutate settings.
|
||
|
||
## 7. Security & Observability
|
||
|
||
- Never log raw secrets; Surface.Env redacts values by default.
|
||
- Emit metric `surface_env_validation_total{status}` to observe validation outcomes.
|
||
- Provide `/metrics` gauge for cache quota/residual via Surface.FS integration.
|
||
|
||
## 8. Offline & Air-Gap Support
|
||
|
||
- Defaults assume no public network access; endpoints should point to internal RustFS or S3-compatible system.
|
||
- Offline kit templates supply env files under `offline/scanner/surface-env.env`.
|
||
- Document steps in `docs/modules/devops/runbooks/zastava-deployment.md` and `offline-kit` tasks for synchronising env values.
|
||
|
||
## 9. Testing Strategy
|
||
|
||
- Unit tests for each resolver/validator.
|
||
- Integration tests for Worker & Observer verifying that missing configuration causes deterministic failures.
|
||
- Golden tests for configuration precedence (component overrides, defaults).
|
||
|
||
## 10. Open Questions / Future Work
|
||
|
||
- Dynamic refresh of environment (watch ConfigMap) is out of scope for v1.
|
||
- Evaluate adding support for environment discovery via `IConfiguration` only (no env vars) for Windows service deployments.
|
||
|
||
## 11. References
|
||
|
||
- Surface.FS Design (`docs/modules/scanner/design/surface-fs.md`)
|
||
- Surface.Secrets Design (`docs/modules/scanner/design/surface-secrets.md`)
|
||
- Surface.Validation Design (`docs/modules/scanner/design/surface-validation.md`)
|
||
- AirGap mode overview (`docs/airgap/airgap-mode.md`)
|