feat: Add guild charters and task boards for various components
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Introduced guild charters for Scanner Deno, PHP, Ruby, Native, WebService, Java, Surface.Env, Surface.FS, Surface.Secrets, Surface.Validation, UI, Zastava Observer, Zastava Webhook, Zastava Core, and Plugin Platform. - Each charter outlines the mission, scope, required reading, and working agreements for the respective guilds. - Created task boards for Surface.Env, Surface.FS, Surface.Secrets, Surface.Validation, and Zastava components to track progress and dependencies. - Ensured all documents emphasize determinism, offline readiness, security, and integration with shared Surface libraries.
This commit is contained in:
147
docs/modules/scanner/design/surface-env.md
Normal file
147
docs/modules/scanner/design/surface-env.md
Normal file
@@ -0,0 +1,147 @@
|
||||
# Surface.Env Design (Epic: SURFACE-SHARING)
|
||||
|
||||
> **Status:** Draft v1.0 — aligns with tasks `SURFACE-ENV-01..05`, `SCANNER-ENV-01..03`, `ZASTAVA-ENV-01..02`, `OPS-ENV-01`.
|
||||
>
|
||||
> **Audience:** Scanner Worker/WebService engineers, Zastava engineers, DevOps/Ops teams.
|
||||
|
||||
## 1. Goals
|
||||
|
||||
Surface.Env centralises configuration discovery for every component that touches the shared Scanner “surface” (cache, manifests, secrets). The library replaces ad-hoc environment lookups with a deterministic, validated contract that:
|
||||
|
||||
1. Works identically across Scanner Worker, Scanner WebService, BuildX plug-ins, Zastava Observer/Webhook, and future consumers (Scheduler planners, CLI runners).
|
||||
2. Supports both connected and air-gapped deployments with clear defaults.
|
||||
3. Records configuration intent (tenant isolation, cache limits, TLS, feature flags) so Surface.Validation can enforce preconditions before any work executes.
|
||||
|
||||
## 2. Architecture Overview
|
||||
|
||||
```
|
||||
+-----------------------+
|
||||
| Host (Worker/WebSvc) |
|
||||
| - IConfiguration |
|
||||
| - ILogger |
|
||||
| |
|
||||
| +-----------------+ |
|
||||
| | SurfaceEnv | | loads env vars / config file
|
||||
| | - Provider |--+------------------------------+
|
||||
| | - Validators | |
|
||||
| +-----------------+ |
|
||||
| | |
|
||||
| | IResolvedSurfaceConfiguration |
|
||||
| v v
|
||||
| Surface.FS / Surface.Secrets / Surface.Validation consumers
|
||||
+-------------------------------------------------------------
|
||||
```
|
||||
|
||||
Surface.Env exposes `ISurfaceEnvironment` which returns an immutable `SurfaceEnvironmentSettings` record. Hosts call `SurfaceEnvBuilder.Build()` during startup, passing optional configuration overrides (for example, Helm chart values). The builder resolves environment variables, applies defaults, and executes Surface.Validation rules before handing settings to downstream services.
|
||||
|
||||
## 3. Configuration Schema
|
||||
|
||||
### 3.1 Common keys
|
||||
|
||||
| Variable | Description | Default | Notes |
|
||||
|----------|-------------|---------|-------|
|
||||
| `SCANNER_SURFACE_FS_ENDPOINT` | Base URI for Surface.FS service (RustFS, S3-compatible). | _required_ | e.g. `https://surface-cache.svc.cluster.local`. Zastava uses `ZASTAVA_SURFACE_FS_ENDPOINT`; when absent, falls back to scanner value. |
|
||||
| `SCANNER_SURFACE_FS_BUCKET` | Bucket/container name used for manifests and artefacts. | `surface-cache` | Must be unique per tenant. |
|
||||
| `SCANNER_SURFACE_FS_REGION` | Optional region (S3-style). | `null` | Required for AWS S3. |
|
||||
| `SCANNER_SURFACE_CACHE_ROOT` | Local filesystem directory for warm caches. | `/var/lib/stellaops/surface` | Should reside on fast SSD. |
|
||||
| `SCANNER_SURFACE_CACHE_QUOTA_MB` | Soft limit for local cache usage. | `4096` | Enforced by Surface.FS eviction policy. |
|
||||
| `SCANNER_SURFACE_TLS_CERT_PATH` | Path to PEM bundle for mutual TLS with Surface.FS. | `null` | If provided, library loads cert/key pair. |
|
||||
| `SCANNER_SURFACE_TENANT` | Tenant identifier used for cache namespaces. | derived from Authority token | Can be overridden for multi-tenant workers. |
|
||||
| `SCANNER_SURFACE_PREFETCH_ENABLED` | Toggle surface prefetch threads. | `false` | If `true`, Worker prefetches manifests before analyzer stage. |
|
||||
| `SCANNER_SURFACE_FEATURES` | Comma-separated feature switches. | `""` | e.g. `validation,prewarm,runtime-diff`. |
|
||||
|
||||
### 3.2 Secrets provider keys
|
||||
|
||||
| Variable | Description | Notes |
|
||||
|----------|-------------|-------|
|
||||
| `SCANNER_SURFACE_SECRETS_PROVIDER` | Provider ID (`kubernetes`, `file`, `inline`). | Controls Surface.Secrets back-end. |
|
||||
| `SCANNER_SURFACE_SECRETS_ROOT` | Path or secret namespace. | Example: `/etc/stellaops/secrets` for file provider. |
|
||||
| `SCANNER_SURFACE_SECRETS_TENANT` | Tenant override for secret lookup. | Defaults to `SCANNER_SURFACE_TENANT`. |
|
||||
|
||||
### 3.3 Zastava-specific keys
|
||||
|
||||
Zastava containers read the same primary variables but may override names under the `ZASTAVA_` prefix (e.g., `ZASTAVA_SURFACE_CACHE_ROOT`, `ZASTAVA_SURFACE_FEATURES`). Surface.Env automatically checks component-specific prefixes before falling back to the scanner defaults.
|
||||
|
||||
### 3.4 Configuration precedence
|
||||
|
||||
1. Explicit overrides passed to `SurfaceEnvBuilder` (e.g., from appsettings).
|
||||
2. Component-specific env (e.g., `ZASTAVA_SURFACE_FS_ENDPOINT`).
|
||||
3. Scanner global env (e.g., `SCANNER_SURFACE_FS_ENDPOINT`).
|
||||
4. `SurfaceEnvDefaults.json` (shipped with library for sensible defaults).
|
||||
5. Emergency fallback values defined in code (only for development scenarios).
|
||||
|
||||
## 4. API Surface
|
||||
|
||||
```csharp
|
||||
public interface ISurfaceEnvironment
|
||||
{
|
||||
SurfaceEnvironmentSettings Settings { get; }
|
||||
IReadOnlyDictionary<string, string> RawVariables { get; }
|
||||
}
|
||||
|
||||
public sealed record SurfaceEnvironmentSettings
|
||||
(
|
||||
Uri SurfaceFsEndpoint,
|
||||
string SurfaceFsBucket,
|
||||
string? SurfaceFsRegion,
|
||||
DirectoryInfo CacheRoot,
|
||||
int CacheQuotaMegabytes,
|
||||
X509Certificate2Collection? ClientCertificates,
|
||||
string Tenant,
|
||||
bool PrefetchEnabled,
|
||||
IReadOnlyCollection<string> FeatureFlags,
|
||||
SecretProviderConfiguration Secrets,
|
||||
IDictionary<string,string> ComponentOverrides
|
||||
);
|
||||
```
|
||||
|
||||
Consumers access `ISurfaceEnvironment.Settings` and pass the record into Surface.FS / Surface.Secrets factories. The interface memoises results so repeated access is cheap.
|
||||
|
||||
## 5. Validation
|
||||
|
||||
Surface.Env invokes the following validators (implemented in Surface.Validation):
|
||||
|
||||
1. **EndpointValidator** – ensures endpoint URI is absolute HTTPS and not localhost in production.
|
||||
2. **CacheQuotaValidator** – verifies quota > 0 and below host max.
|
||||
3. **FilesystemValidator** – checks cache root exists/writable; attempts to create directory if missing.
|
||||
4. **SecretsProviderValidator** – ensures provider-specific settings (e.g., Kubernetes namespace) are present.
|
||||
5. **FeatureFlagValidator** – warns on unknown feature flag tokens.
|
||||
|
||||
Failures throw `SurfaceEnvironmentException` with error codes (`SURFACE_ENV_MISSING_ENDPOINT`, `SURFACE_ENV_CACHE_DIR_UNWRITABLE`, etc.). Hosts log the error and fail fast during startup.
|
||||
|
||||
## 6. Integration Guidance
|
||||
|
||||
- **Scanner Worker**: call `services.AddSurfaceEnvironment()` in `Program.cs` before registering analyzers. Pass `hostContext.Configuration.GetSection("Surface")` for overrides.
|
||||
- **Scanner WebService**: build environment during startup, then expose selected values via diagnostics (`/internal/surface` when diagnostics enabled).
|
||||
- **Zastava Observer/Webhook**: use the same builder; ensure Helm charts set `ZASTAVA_` variables.
|
||||
- **Scheduler Planner (future)**: treat Surface.Env as read-only input; do not mutate settings.
|
||||
|
||||
## 7. Security & Observability
|
||||
|
||||
- Never log raw secrets; Surface.Env redacts values by default.
|
||||
- Emit metric `surface_env_validation_total{status}` to observe validation outcomes.
|
||||
- Provide `/metrics` gauge for cache quota/residual via Surface.FS integration.
|
||||
|
||||
## 8. Offline & Air-Gap Support
|
||||
|
||||
- Defaults assume no public network access; endpoints should point to internal RustFS or S3-compatible system.
|
||||
- Offline kit templates supply env files under `offline/scanner/surface-env.env`.
|
||||
- Document steps in `docs/modules/devops/runbooks/zastava-deployment.md` and `offline-kit` tasks for synchronising env values.
|
||||
|
||||
## 9. Testing Strategy
|
||||
|
||||
- Unit tests for each resolver/validator.
|
||||
- Integration tests for Worker & Observer verifying that missing configuration causes deterministic failures.
|
||||
- Golden tests for configuration precedence (component overrides, defaults).
|
||||
|
||||
## 10. Open Questions / Future Work
|
||||
|
||||
- Dynamic refresh of environment (watch ConfigMap) is out of scope for v1.
|
||||
- Evaluate adding support for environment discovery via `IConfiguration` only (no env vars) for Windows service deployments.
|
||||
|
||||
## 11. References
|
||||
|
||||
- Surface.FS Design (`docs/modules/scanner/design/surface-fs.md`)
|
||||
- Surface.Secrets Design (`docs/modules/scanner/design/surface-secrets.md`)
|
||||
- Surface.Validation Design (`docs/modules/scanner/design/surface-validation.md`)
|
||||
- AirGap mode overview (`docs/airgap/airgap-mode.md`)
|
||||
Reference in New Issue
Block a user