docs consolidation work

2025-12-24 16:26:06 +02:00
parent 2c2bbf1005
commit 8197588e74
20 changed files with 403 additions and 37 deletions
--- a/docs/modules/concelier/operations/connectors/msrc.md
+++ b/docs/modules/concelier/operations/connectors/msrc.md
@@ -41,7 +41,7 @@ concelier:

 ### 4.1 State seeding helper

-Use `src/Tools/SourceStateSeeder` to queue historical advisories (detail JSON + optional CVRF artefacts) for replay without manual Mongo edits. Example seed file:
+Use `src/Tools/SourceStateSeeder` to queue historical advisories (detail JSON + optional CVRF artefacts) for replay without manual database edits. Example seed file:

 ```json
 {
--- a/docs/modules/excititor/architecture.md
+++ b/docs/modules/excititor/architecture.md
@@ -124,9 +124,9 @@ Excititor workers now hydrate signature metadata with issuer trust data retrieve

 `GET /v1/vex/statements/{advisory_key}` produces sorted JSON responses containing raw statement metadata (`issuer`, `content_hash`, `signature`), normalised tuples, and provenance pointers. Advisory AI consumes this endpoint to build retrieval contexts with explicit citations.

-### 1.5 Postgres raw store (replaces Mongo/GridFS)
+### 1.5 PostgreSQL raw store

-> Mongo/BSON/GridFS are being removed. This is the canonical design for the Postgres-backed raw store that powers `/vex/raw` and ingestion.
+> This is the canonical design for the PostgreSQL-backed raw store that powers `/vex/raw` and ingestion.

 Schema: `vex`

@@ -167,7 +167,7 @@ Schema: `vex`
 3. If `size <= inline_threshold_bytes` (default 256 KiB) set `inline_payload=true` and store in `content_json`; otherwise store bytes in `vex_raw_blobs` and set `inline_payload=false`.
 4. Persist `content_size_bytes` (pre-canonical length) and `payload_hash` for integrity.

-**API mapping (replaces Mongo/BSON)**  
+**API mapping**
 List/query `/vex/raw` via `SELECT ... FROM vex.vex_raw_documents WHERE tenant=@t ORDER BY retrieved_at DESC, digest LIMIT @n OFFSET @offset`; cursor uses `(retrieved_at, digest)`. `GET /vex/raw/{digest}` loads the row and optional blob; `GET /vex/raw/{digest}/provenance` projects `provenance_json` + `metadata_json`. Filters (`providerId`, `format`, `since`, `until`, `supersedes`, `hasAttachments`) map to indexed predicates; JSON subfields use `metadata_json ->> 'field'`.

 **Write semantics**
@@ -178,8 +178,8 @@ List/query `/vex/raw` via `SELECT ... FROM vex.vex_raw_documents WHERE tenant=@t
 **Rollout**

 1. Add migration under `src/Excititor/__Libraries/StellaOps.Excititor.Storage.Postgres/Migrations` creating the tables/indexes above.
-2. Implement `PostgresVexRawStore` and switch WebService/Worker DI to `AddExcititorPostgresStorage`; remove `VexMongoStorageOptions`, `IMongoDatabase`, and GridFS paths.
-3. Update `/vex/raw` endpoints/tests to the Postgres store; delete Mongo fixtures once parity is green. Mark Mongo storage paths as deprecated and remove them in the next release.
+2. Implement `PostgresVexRawStore` and switch WebService/Worker DI to `AddExcititorPostgresStorage`.
+3. Update `/vex/raw` endpoints/tests to the PostgreSQL store.

 ---

--- a/docs/modules/graph/README.md
+++ b/docs/modules/graph/README.md
@@ -5,7 +5,7 @@ Graph Indexer + Graph API build the tenant-scoped knowledge graph that powers bl
 ## Scope & responsibilities
 - Ingest SBOM snapshots, advisory/VEX events, policy overlays, and runtime signals to maintain a first-party graph representation with deterministic node/edge identities.
 - Serve APIs and saved-query tooling for impact analysis, dependency traversal, diffing, and policy/VEX overlays with explainable provenance.
- Supply Graph Explorer UI/CLI experiences, plus Offline Kit exports (`nodes.jsonl`, `edges.jsonl`, `overlays/`) with DSSE manifests for air-gapped replay. Analytics overlays are emitted as NDJSON (`overlays/clusters.ndjson`, `overlays/centrality.ndjson`) with deterministic ordering; Mongo-backed providers support production wiring.
+- Supply Graph Explorer UI/CLI experiences, plus Offline Kit exports (`nodes.jsonl`, `edges.jsonl`, `overlays/`) with DSSE manifests for air-gapped replay. Analytics overlays are emitted as NDJSON (`overlays/clusters.ndjson`, `overlays/centrality.ndjson`) with deterministic ordering; PostgreSQL-backed providers support production wiring.
 - Maintain the [Graph Index Canonical Schema](schema.md) and coordinate query/overlay lifecycle with Scheduler, Policy Engine, Vulnerability Explorer, and Export Center.

 ## Architecture snapshot (Sprint 30 groundwork)
@@ -13,7 +13,7 @@ Graph Indexer + Graph API build the tenant-scoped knowledge graph that powers bl
 - **Graph API service** — exposes `GET /graph/nodes`, `/graph/impact/{advisory}`, `/graph/query/saved`, `/graph/diff`, and overlay endpoints with RBAC scopes defined in Authority (`docs/updates/2025-10-26-authority-graph-scopes.md`).
 - **Overlay & diff workers** — materialise impact lists, saved-query caches, and signed diff manifests; feed Scheduler `GraphBuildJob`/`GraphOverlayJob` contracts (`docs/updates/2025-10-26-scheduler-graph-jobs.md`).
 - **Console & CLI integrations** — planned modules deliver WebGL explorer, timeline viz, and CLI `stella sbom graph ...` commands aligned with implementation plan phases.
- **Storage abstraction** — supports document + adjacency (Mongo) or pluggable graph engine; both paths enforce deterministic ordering and export manifests.
+- **Storage abstraction** — supports document + adjacency (PostgreSQL) or pluggable graph engine; both paths enforce deterministic ordering and export manifests.

 ## Current workstreams (Q4 2025)
 - `GRAPH-SVC-30-00x` (see `src/Graph/StellaOps.Graph.Indexer/TASKS.md`) — stand up Graph Indexer pipeline, identity registry, snapshot exports.
--- a/docs/modules/graph/packaging.md
+++ b/docs/modules/graph/packaging.md
@@ -3,7 +3,7 @@
 ## Deployment overlays
 - Helm/Compose should expose two timers for analytics: `GRAPH_ANALYTICS_CLUSTER_INTERVAL` and `GRAPH_ANALYTICS_CENTRALITY_INTERVAL` (ISO-8601 duration, default 5m). Map to `GraphAnalyticsOptions`.
 - Change-stream/backfill worker toggles via `GRAPH_CHANGE_POLL_INTERVAL`, `GRAPH_BACKFILL_INTERVAL`, `GRAPH_CHANGE_MAX_RETRIES`, `GRAPH_CHANGE_RETRY_BACKOFF`.
- Storage: current build uses in-memory graph storage (Mongo dependency removed). Reserve `GRAPH_STORAGE_CONNECTION` and `GRAPH_STORAGE_DB` for the upcoming Postgres-backed provider.
+- Storage: current build uses in-memory graph storage. `GRAPH_STORAGE_CONNECTION` configures the PostgreSQL-backed provider.

 ## Offline kit alignment
 - Cluster/centrality overlays are exportable alongside `nodes.jsonl`/`edges.jsonl`; keep under `artifacts/graph-snapshots/{snapshotId}/overlays/` for air-gapped imports.
--- a/docs/modules/router/aspnet-endpoint-bridge.md
+++ b/docs/modules/router/aspnet-endpoint-bridge.md
@@ -781,6 +781,189 @@ endpoints:

 ---

+## Troubleshooting
+
+### Common Issues
+
+#### 1. Endpoints Not Discovered
+
+**Symptom:** Gateway shows 0 endpoints for service, or specific endpoints missing.
+
+**Causes & Solutions:**
+
+| Cause | Solution |
+|-------|----------|
+| `UseStellaRouterBridge()` called before `MapControllers()` | Call `UseStellaRouterBridge()` after all endpoint registration |
+| Route filtered by `EndpointFilter` | Check filter logic, ensure endpoint matches |
+| Route filtered by `IncludePathPatterns` | Verify path pattern includes the endpoint |
+| Route filtered by `ExcludePathPatterns` | Verify endpoint isn't accidentally excluded |
+| Missing `[Authorize]` with `RequireExplicit` | Add authorization or change `MissingAuthorizationBehavior` |
+
+**Debug:**
+```csharp
+// Enable discovery logging
+builder.Logging.AddFilter("StellaOps.Microservice.AspNetCore", LogLevel.Debug);
+```
+
+#### 2. Authorization Claims Not Extracted
+
+**Symptom:** Endpoints registered but `RequiringClaims` is empty when it shouldn't be.
+
+**Causes & Solutions:**
+
+| Cause | Solution |
+|-------|----------|
+| `[Authorize]` without policy/roles | Add explicit policy or roles |
+| Policy not registered | Register policy with `AddAuthorization()` |
+| `AuthorizationMappingStrategy.YamlOnly` set | Use `Hybrid` or `AspNetMetadataOnly` |
+| Custom policy doesn't have claim requirements | Use claims-based policies |
+
+**Debug:**
+```csharp
+// Log authorization mapping
+var mapper = app.Services.GetRequiredService<IAuthorizationClaimMapper>();
+var result = await mapper.MapAsync(endpoint);
+Console.WriteLine($"Claims: {string.Join(", ", result.Claims)}");
+```
+
+#### 3. Request Dispatch Fails with 404
+
+**Symptom:** Gateway routes request but microservice returns 404.
+
+**Causes & Solutions:**
+
+| Cause | Solution |
+|-------|----------|
+| Path parameters not matched | Verify parameter names in route pattern |
+| Method mismatch | Verify HTTP method matches endpoint |
+| Route constraint rejected value | Use standard constraints that bridge supports |
+| Catch-all route not handled | Ensure `{**path}` is normalized correctly |
+
+**Debug:**
+```bash
+# Check registered endpoints
+curl http://localhost:5000/.well-known/stella-endpoints
+```
+
+#### 4. Model Binding Errors
+
+**Symptom:** Requests return 400 Bad Request with binding errors.
+
+**Causes & Solutions:**
+
+| Cause | Solution |
+|-------|----------|
+| `[FromBody]` type mismatch | Verify request body matches expected type |
+| Required parameter missing | Include all `[FromRoute]`/`[FromQuery]` parameters |
+| `[FromHeader]` not populated | Headers forwarded via `X-StellaOps-Header-*` |
+| Complex type not deserialized | Verify JSON serialization settings |
+
+**Debug:**
+```csharp
+// Enable model binding logging
+builder.Logging.AddFilter("Microsoft.AspNetCore.Mvc.ModelBinding", LogLevel.Debug);
+```
+
+#### 5. Identity Not Populated
+
+**Symptom:** `User.Identity` is null or claims missing in endpoint handler.
+
+**Causes & Solutions:**
+
+| Cause | Solution |
+|-------|----------|
+| Gateway not forwarding identity | Verify Gateway `identity-header-policy` configured |
+| Missing `X-StellaOps-UserId` header | Gateway must send identity headers |
+| `UseAuthentication()` not called | Call before `UseStellaRouterBridge()` |
+| Custom claims not mapped | Use `ClaimsPrincipalBuilder` for custom claims |
+
+**Debug:**
+```csharp
+app.MapGet("/debug/identity", (HttpContext ctx) =>
+    new {
+        IsAuthenticated = ctx.User.Identity?.IsAuthenticated,
+        Name = ctx.User.Identity?.Name,
+        Claims = ctx.User.Claims.Select(c => new { c.Type, c.Value })
+    });
+```
+
+#### 6. Performance Issues
+
+**Symptom:** High latency or memory usage.
+
+**Causes & Solutions:**
+
+| Cause | Solution |
+|-------|----------|
+| HttpContext allocation per request | Pool contexts (internal implementation) |
+| Large request bodies buffered | Use streaming endpoints for large payloads |
+| Discovery runs too frequently | Discovery runs once at startup; cache is stable |
+| Many endpoints slow startup | Discovery is O(n) but runs once |
+
+**Metrics to monitor:**
+- `stellaops_router_bridge_requests_total`
+- `stellaops_router_bridge_request_duration_seconds`
+- `stellaops_router_bridge_dispatch_errors_total`
+
+#### 7. YAML Override Not Applied
+
+**Symptom:** Claims from YAML file not appearing in endpoint registration.
+
+**Causes & Solutions:**
+
+| Cause | Solution |
+|-------|----------|
+| `YamlConfigPath` not set | Set `options.YamlConfigPath = "router.yaml"` |
+| File not found | Use absolute path or verify relative path |
+| Path pattern doesn't match | YAML paths are case-insensitive, verify pattern |
+| `AuthorizationMappingStrategy.AspNetMetadataOnly` | Use `YamlOnly` or `Hybrid` |
+
+**Example YAML:**
+```yaml
+# router.yaml
+endpoints:
+  - path: "/api/admin/**"
+    requiringClaims:
+      - type: "Role"
+        value: "admin"
+```
+
+### Diagnostic Endpoints
+
+The bridge adds optional diagnostic endpoints (development only):
+
+| Endpoint | Description |
+|----------|-------------|
+| `/.well-known/stella-endpoints` | Lists all discovered endpoints |
+| `/.well-known/stella-bridge-status` | Shows bridge configuration and health |
+
+Enable in development:
+```csharp
+builder.Services.AddStellaRouterBridge(options =>
+{
+    options.EnableDiagnosticEndpoints = builder.Environment.IsDevelopment();
+    // ...
+});
+```
+
+### Logging Categories
+
+Configure logging for troubleshooting:
+
+```json
+{
+  "Logging": {
+    "LogLevel": {
+      "StellaOps.Microservice.AspNetCore.Discovery": "Debug",
+      "StellaOps.Microservice.AspNetCore.Authorization": "Debug",
+      "StellaOps.Microservice.AspNetCore.Dispatch": "Information"
+    }
+  }
+}
+```
+
+---
+
 ## Testing Strategy

 ### Unit Tests
--- a/docs/modules/router/migration-guide.md
+++ b/docs/modules/router/migration-guide.md
@@ -96,6 +96,169 @@ Best for major refactoring or when HTTP compatibility is not needed.
 - More upfront work
 - Requires domain extraction

+### Strategy C: ASP.NET Endpoint Bridge (Recommended)
+
+Best for services that want automatic Router registration from existing ASP.NET endpoints without code changes.
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│  StellaOps.*.WebService                                         │
+│  ┌─────────────────────────────────────────────────────────────┐│
+│  │  ASP.NET Endpoints (unchanged)                              ││
+│  │  • Minimal APIs: app.MapGet("/api/...", handler)            ││
+│  │  • Controllers: [ApiController] with [HttpGet], etc.        ││
+│  │  • Route groups: app.MapGroup("/api").MapEndpoints()        ││
+│  └─────────────────────────────────────────────────────────────┘│
+│                            │                                    │
+│                            ▼ Auto-discovery                     │
+│  ┌─────────────────────────────────────────────────────────────┐│
+│  │  StellaOps.Microservice.AspNetCore                          ││
+│  │  • Discovers all endpoints from EndpointDataSource          ││
+│  │  • Extracts authorization metadata automatically            ││
+│  │  • Registers with Router via HELLO                          ││
+│  │  • Dispatches Router requests through ASP.NET pipeline      ││
+│  └─────────────────────────────────────────────────────────────┘│
+│                            │                                    │
+│                            ▼ Binary transport                   │
+│  ┌─────────────────────────────────────────────────────────────┐│
+│  │  Router Gateway                                             ││
+│  │  • Routes external HTTP → internal microservice             ││
+│  │  • Aggregates OpenAPI                                       ││
+│  └─────────────────────────────────────────────────────────────┘│
+└─────────────────────────────────────────────────────────────────┘
+```
+
+**Steps:**
+
+1. Add `StellaOps.Microservice.AspNetCore` package reference
+2. Configure bridge in `Program.cs`:
+
+```csharp
+using StellaOps.Microservice.AspNetCore;
+
+var builder = WebApplication.CreateBuilder(args);
+
+// Add authorization (required for claim mapping)
+builder.Services.AddAuthorization();
+
+// Add the ASP.NET Endpoint Bridge
+builder.Services.AddStellaRouterBridge(options =>
+{
+    options.ServiceName = "my-service";
+    options.Version = "1.0.0";
+    options.Region = "us-east-1";
+});
+
+var app = builder.Build();
+
+// Register ASP.NET endpoints as usual
+app.MapGet("/api/health", () => "healthy");
+app.MapGet("/api/items/{id}", (string id) => new { id, name = "Item " + id })
+    .RequireAuthorization("read-items");
+app.MapPost("/api/items", (CreateItemRequest req) => new { id = Guid.NewGuid() })
+    .RequireAuthorization("write-items");
+
+// Activate the bridge (discovers and registers endpoints)
+app.UseStellaRouterBridge();
+
+app.Run();
+```
+
+3. (Optional) Add YAML overrides for security hardening:
+
+```yaml
+# router.yaml
+endpoints:
+  - path: "/api/admin/**"
+    requiringClaims:
+      - type: "Role"
+        value: "admin"
+    timeoutMs: 60000
+```
+
+4. Test via Gateway
+5. Deploy to production
+
+**Pros:**
+- **Zero code changes** to existing endpoints
+- Automatic authorization metadata extraction
+- Full ASP.NET pipeline execution (filters, binding, validation)
+- Deterministic endpoint ordering
+- YAML overrides for security hardening without code changes
+- Supports both minimal APIs and controllers
+
+**Cons:**
+- Slightly more runtime overhead (HttpContext synthesis)
+- Some ASP.NET features not supported (see limitations below)
+
+#### Authorization Mapping
+
+The bridge automatically extracts authorization requirements:
+
+| ASP.NET Source | Router Mapping |
+|----------------|----------------|
+| `[Authorize(Policy = "policy-name")]` | Resolved via `IAuthorizationPolicyProvider` → `RequiringClaims` |
+| `[Authorize(Roles = "admin,user")]` | `ClaimRequirement(Role, admin)`, `ClaimRequirement(Role, user)` |
+| `.RequireAuthorization("policy")` | Resolved via policy provider |
+| `.RequireAuthorization(new AuthorizeAttribute { Roles = "..." })` | Direct role mapping |
+| `[AllowAnonymous]` | Empty `RequiringClaims` (public endpoint) |
+
+#### Authorization Mapping Strategies
+
+Configure how the bridge handles authorization:
+
+```csharp
+builder.Services.AddStellaRouterBridge(options =>
+{
+    options.ServiceName = "my-service";
+    options.Version = "1.0.0";
+    options.Region = "us-east-1";
+
+    // Strategy options:
+    // - YamlOnly: YAML claims replace code claims entirely
+    // - AspNetMetadataOnly: Code claims only, ignore YAML (default)
+    // - Hybrid: Merge code + YAML claims by type
+    options.AuthorizationMappingStrategy = AuthorizationMappingStrategy.Hybrid;
+
+    // Behavior when endpoint has no authorization:
+    // - RequireExplicit: Fail validation (secure default)
+    // - AllowAuthenticated: Require authentication but no specific claims
+    // - WarnAndAllow: Log warning, allow request (dev only)
+    options.MissingAuthorizationBehavior = MissingAuthorizationBehavior.RequireExplicit;
+});
+```
+
+#### Route Filtering
+
+Control which endpoints are bridged:
+
+```csharp
+builder.Services.AddStellaRouterBridge(options =>
+{
+    options.ServiceName = "my-service";
+    options.Version = "1.0.0";
+    options.Region = "us-east-1";
+
+    // Include only /api/* endpoints
+    options.IncludePathPatterns = ["/api/**"];
+
+    // Exclude internal endpoints
+    options.ExcludePathPatterns = ["/internal/**", "/metrics", "/health/**"];
+});
+```
+
+#### Limitations
+
+The ASP.NET Endpoint Bridge does **not** support:
+
+| Feature | Alternative |
+|---------|-------------|
+| SignalR/WebSocket | Use direct HTTP for real-time features |
+| gRPC endpoints | Use separate gRPC channel |
+| Streaming request bodies | Use `IRawStellaEndpoint` for streaming |
+| Custom route constraints (`{id:guid}`) | Constraints execute at dispatch, but not in discovery |
+| Header/query-based API versioning | Use path-based versioning (`/api/v1/...`) |
+
 ## Controller to Handler Mapping

 ### Before (ASP.NET Controller)