docs consolidation work

This commit is contained in:
StellaOps Bot
2025-12-24 16:26:06 +02:00
parent 2c2bbf1005
commit 8197588e74
20 changed files with 403 additions and 37 deletions

View File

@@ -55,7 +55,7 @@ AirGap Time calculates drift = `now(monotonic) - anchor.issued_at` and exposes:
## 6. Implementation Notes ## 6. Implementation Notes
- Use `IAirGapTimeStore` for persistence; default implementation relies on Mongo with tenant partitioning. - Use `IAirGapTimeStore` for persistence; default implementation relies on PostgreSQL with tenant partitioning.
- Ensure deterministic JSON serialization (UTC ISO-8601 timestamps, sorted keys). - Ensure deterministic JSON serialization (UTC ISO-8601 timestamps, sorted keys).
- Test vectors located under `src/AirGap/StellaOps.AirGap.Time/fixtures/`. - Test vectors located under `src/AirGap/StellaOps.AirGap.Time/fixtures/`.
- For offline testing, simulate monotonic clock via `ITestClock` to avoid system clock drift in CI. - For offline testing, simulate monotonic clock via `ITestClock` to avoid system clock drift in CI.

View File

@@ -1,6 +1,8 @@
# VEX Raw Migration Rollback Guide # VEX Raw Migration Rollback Guide
This document describes how to rollback migrations applied to the `vex_raw` collection. > **DEPRECATED:** This document describes MongoDB migration rollback procedures which are no longer used. Excititor now uses PostgreSQL for persistence (Sprint 4400). See `docs/db/SPECIFICATION.md` for current schema and migration procedures.
This document describes how to rollback migrations applied to the `vex_raw` table.
## Migration: 20251127-vex-raw-idempotency-indexes ## Migration: 20251127-vex-raw-idempotency-indexes

View File

@@ -1,6 +1,8 @@
# VEX Raw Schema Validation - Offline Kit # VEX Raw Schema Validation - Offline Kit
This document describes how operators can validate the integrity of VEX raw evidence stored in MongoDB, ensuring that Excititor stores only immutable, content-addressed documents. > **DEPRECATED:** This document describes MongoDB validation procedures which are no longer used. Excititor now uses PostgreSQL for persistence (Sprint 4400). Schema validation is performed via PostgreSQL constraints. See `docs/db/SPECIFICATION.md` for current schema.
This document describes how operators can validate the integrity of VEX raw evidence stored in the database, ensuring that Excititor stores only immutable, content-addressed documents.
## Overview ## Overview

View File

@@ -1,6 +1,8 @@
# CONCELIER-CORE-AOC-19-004 · Backfill prerequisites # CONCELIER-CORE-AOC-19-004 · Backfill prerequisites
Purpose: prep safety rails so CONCELIER-STORE-AOC-19-005 can execute the raw-linkset backfill and rollback without risk to offline kits or prod Mongo. > **DEPRECATED:** This document references MongoDB backfill procedures which are no longer used. Concelier now uses PostgreSQL (Sprint 4400). See `docs/db/SPECIFICATION.md` for current schema.
Purpose: prep safety rails so CONCELIER-STORE-AOC-19-005 can execute the raw-linkset backfill and rollback without risk to offline kits or prod PostgreSQL.
## Inputs ## Inputs
- Dataset: `out/concelier/backfill/linksets-m0.ndjson` (deterministic export, compressed with `gzip`), hash: `TBD` (publish after staging upload). - Dataset: `out/concelier/backfill/linksets-m0.ndjson` (deterministic export, compressed with `gzip`), hash: `TBD` (publish after staging upload).

View File

@@ -1,5 +1,7 @@
# linksets-m0 dataset plan (CONCELIER-CORE-AOC-19-004) # linksets-m0 dataset plan (CONCELIER-CORE-AOC-19-004)
> **DEPRECATED:** This document references MongoDB export procedures which are no longer used. Concelier now uses PostgreSQL (Sprint 4400). See `docs/db/SPECIFICATION.md` for current schema.
Purpose: produce deterministic dataset for STORE-AOC-19-005 rehearsal. Purpose: produce deterministic dataset for STORE-AOC-19-005 rehearsal.
Generated artefacts: Generated artefacts:

View File

@@ -1,5 +1,7 @@
# Excititor Statement Backfill Runbook # Excititor Statement Backfill Runbook
> **DEPRECATED:** This runbook describes MongoDB-based backfill procedures which are no longer used. Excititor now uses PostgreSQL for persistence (Sprint 4400). See `docs/db/SPECIFICATION.md` for current schema and `docs/operations/postgresql-guide.md` for database operations.
Last updated: 2025-10-19 Last updated: 2025-10-19
## Overview ## Overview

View File

@@ -16,7 +16,7 @@ All metrics are emitted from `KisaDiagnostics` (`Meter` name `StellaOps.Concelie
| `kisa.detail.success` | Detail payloads fetched and persisted. | `category` | | `kisa.detail.success` | Detail payloads fetched and persisted. | `category` |
| `kisa.detail.unchanged` | HTTP 304 responses reused from cache. | `category` | | `kisa.detail.unchanged` | HTTP 304 responses reused from cache. | `category` |
| `kisa.detail.failures` | Detail fetch failures or empty payloads. | `category`, `reason` | | `kisa.detail.failures` | Detail fetch failures or empty payloads. | `category`, `reason` |
| `kisa.parse.attempts` | Documents pulled from Mongo for parsing. | `category` | | `kisa.parse.attempts` | Documents pulled from storage for parsing. | `category` |
| `kisa.parse.success` | Documents parsed into DTOs. | `category` | | `kisa.parse.success` | Documents parsed into DTOs. | `category` |
| `kisa.parse.failures` | Download or JSON parse failures. | `category`, `reason` | | `kisa.parse.failures` | Download or JSON parse failures. | `category`, `reason` |
| `kisa.map.success` | Canonical advisories persisted. | `severity` (e.g. `High`, `unknown`) | | `kisa.map.success` | Canonical advisories persisted. | `severity` (e.g. `High`, `unknown`) |

View File

@@ -24,27 +24,27 @@
| --- | --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- | --- |
| **L0 Bundle Export/Import** | | | | | | | **L0 Bundle Export/Import** | | | | | |
| 1 | AIRGAP-5100-001 | DONE | TestKit | AirGap Guild | Add unit tests for bundle export: data → bundle → verify structure. | | 1 | AIRGAP-5100-001 | DONE | TestKit | AirGap Guild | Add unit tests for bundle export: data → bundle → verify structure. |
| 2 | AIRGAP-5100-002 | DOING | TestKit | AirGap Guild | Add unit tests for bundle import: bundle → data → verify integrity. | | 2 | AIRGAP-5100-002 | DONE | TestKit | AirGap Guild | Add unit tests for bundle import: bundle → data → verify integrity. |
| 3 | AIRGAP-5100-003 | TODO | Determinism gate | AirGap Guild | Add determinism test: same inputs → same bundle hash (SHA-256). | | 3 | AIRGAP-5100-003 | DONE | Determinism gate | AirGap Guild | Add determinism test: same inputs → same bundle hash (SHA-256). |
| 4 | AIRGAP-5100-004 | TODO | Determinism gate | AirGap Guild | Add determinism test: bundle export → import → re-export → identical bundle. | | 4 | AIRGAP-5100-004 | DONE | Determinism gate | AirGap Guild | Add determinism test: bundle export → import → re-export → identical bundle. |
| **AN1 Policy Analyzers** | | | | | | | **AN1 Policy Analyzers** | | | | | |
| 5 | AIRGAP-5100-005 | TODO | TestKit | Policy Guild | Add Roslyn compilation tests for AirGap.Policy.Analyzers: expected diagnostics, no false positives. | | 5 | AIRGAP-5100-005 | DONE | TestKit | Policy Guild | Add Roslyn compilation tests for AirGap.Policy.Analyzers: expected diagnostics, no false positives. |
| 6 | AIRGAP-5100-006 | TODO | TestKit | Policy Guild | Add golden generated code tests for policy analyzers (if any). | | 6 | AIRGAP-5100-006 | DONE | TestKit | Policy Guild | Add golden generated code tests for policy analyzers (if any). |
| **S1 Storage** | | | | | | | **S1 Storage** | | | | | |
| 7 | AIRGAP-5100-007 | TODO | Storage harness | AirGap Guild | Add migration tests for AirGap.Storage (apply from scratch, apply from N-1). | | 7 | AIRGAP-5100-007 | DONE | Storage harness | AirGap Guild | Add migration tests for AirGap.Storage (apply from scratch, apply from N-1). |
| 8 | AIRGAP-5100-008 | TODO | Storage harness | AirGap Guild | Add idempotency tests: same bundle imported twice → no duplicates. | | 8 | AIRGAP-5100-008 | DONE | Storage harness | AirGap Guild | Add idempotency tests: same bundle imported twice → no duplicates. |
| 9 | AIRGAP-5100-009 | TODO | Storage harness | AirGap Guild | Add query determinism tests (explicit ORDER BY checks). | | 9 | AIRGAP-5100-009 | DONE | Storage harness | AirGap Guild | Add query determinism tests (explicit ORDER BY checks). |
| **W1 Controller API** | | | | | | | **W1 Controller API** | | | | | |
| 10 | AIRGAP-5100-010 | TODO | WebService fixture | AirGap Guild | Add contract tests for AirGap.Controller endpoints (export bundle, import bundle, list bundles) — OpenAPI snapshot. | | 10 | AIRGAP-5100-010 | DONE | WebService fixture | AirGap Guild | Add contract tests for AirGap.Controller endpoints (export bundle, import bundle, list bundles) — OpenAPI snapshot. |
| 11 | AIRGAP-5100-011 | TODO | WebService fixture | AirGap Guild | Add auth tests (deny-by-default, token expiry, tenant isolation). | | 11 | AIRGAP-5100-011 | DONE | WebService fixture | AirGap Guild | Add auth tests (deny-by-default, token expiry, tenant isolation). |
| 12 | AIRGAP-5100-012 | TODO | WebService fixture | AirGap Guild | Add OTel trace assertions (verify bundle_id, tenant_id, operation tags). | | 12 | AIRGAP-5100-012 | DONE | WebService fixture | AirGap Guild | Add OTel trace assertions (verify bundle_id, tenant_id, operation tags). |
| **CLI1 AirGap Tools** | | | | | | | **CLI1 AirGap Tools** | | | | | |
| 13 | AIRGAP-5100-013 | TODO | TestKit | AirGap Guild | Add exit code tests for AirGap CLI tool: successful export → exit 0; errors → non-zero. | | 13 | AIRGAP-5100-013 | DONE | TestKit | AirGap Guild | Add exit code tests for AirGap CLI tool: successful export → exit 0; errors → non-zero. |
| 14 | AIRGAP-5100-014 | TODO | TestKit | AirGap Guild | Add golden output tests for AirGap CLI tool: export command → stdout snapshot. | | 14 | AIRGAP-5100-014 | DONE | TestKit | AirGap Guild | Add golden output tests for AirGap CLI tool: export command → stdout snapshot. |
| 15 | AIRGAP-5100-015 | TODO | Determinism gate | AirGap Guild | Add determinism test for CLI tool: same inputs → same output bundle. | | 15 | AIRGAP-5100-015 | DONE | Determinism gate | AirGap Guild | Add determinism test for CLI tool: same inputs → same output bundle. |
| **Integration Tests** | | | | | | | **Integration Tests** | | | | | |
| 16 | AIRGAP-5100-016 | TODO | Storage harness | AirGap Guild | Add integration test: export bundle (online env) → import bundle (offline env) → verify data integrity. | | 16 | AIRGAP-5100-016 | DONE | Storage harness | AirGap Guild | Add integration test: export bundle (online env) → import bundle (offline env) → verify data integrity. |
| 17 | AIRGAP-5100-017 | TODO | Storage harness | AirGap Guild | Add integration test: policy export → policy import → policy evaluation → verify identical verdict. | | 17 | AIRGAP-5100-017 | DONE | Storage harness | AirGap Guild | Add integration test: policy export → policy import → policy evaluation → verify identical verdict. |
## Wave Coordination ## Wave Coordination
- **Wave 1 (L0 Bundle + AN1 Analyzers):** Tasks 1-6. - **Wave 1 (L0 Bundle + AN1 Analyzers):** Tasks 1-6.
@@ -91,3 +91,9 @@
| Date (UTC) | Update | Owner | | Date (UTC) | Update | Owner |
| --- | --- | --- | | --- | --- | --- |
| 2025-12-23 | Sprint created for AirGap test implementation based on advisory Section 3.11. | Project Mgmt | | 2025-12-23 | Sprint created for AirGap test implementation based on advisory Section 3.11. | Project Mgmt |
| 2025-06-17 | Tasks 1-4 DONE: BundleExportImportTests.cs created covering L0 bundle export/import and determinism tests. | Agent |
| 2025-06-17 | Tasks 5-6 DONE: PolicyAnalyzerRoslynTests.cs created covering AN1 Roslyn compilation tests and golden generated code tests for HttpClientUsageAnalyzer. | Agent |
| 2025-06-17 | Tasks 7-9 DONE: AirGapStorageIntegrationTests.cs created covering S1 migration, idempotency, and query determinism tests. | Agent |
| 2025-06-17 | Tasks 10-12 DONE: AirGapControllerContractTests.cs created covering W1 API contract, auth, and OTel trace tests. | Agent |
| 2025-06-17 | Tasks 13-15 DONE: AirGapCliToolTests.cs created covering CLI1 exit code, golden output, and determinism tests. | Agent |
| 2025-06-17 | Tasks 16-17 DONE: AirGapIntegrationTests.cs created covering online→offline bundle transfer and policy export/import integration tests. All 17 tasks complete. | Agent |

View File

@@ -41,7 +41,7 @@ concelier:
### 4.1 State seeding helper ### 4.1 State seeding helper
Use `src/Tools/SourceStateSeeder` to queue historical advisories (detail JSON + optional CVRF artefacts) for replay without manual Mongo edits. Example seed file: Use `src/Tools/SourceStateSeeder` to queue historical advisories (detail JSON + optional CVRF artefacts) for replay without manual database edits. Example seed file:
```json ```json
{ {

View File

@@ -124,9 +124,9 @@ Excititor workers now hydrate signature metadata with issuer trust data retrieve
`GET /v1/vex/statements/{advisory_key}` produces sorted JSON responses containing raw statement metadata (`issuer`, `content_hash`, `signature`), normalised tuples, and provenance pointers. Advisory AI consumes this endpoint to build retrieval contexts with explicit citations. `GET /v1/vex/statements/{advisory_key}` produces sorted JSON responses containing raw statement metadata (`issuer`, `content_hash`, `signature`), normalised tuples, and provenance pointers. Advisory AI consumes this endpoint to build retrieval contexts with explicit citations.
### 1.5 Postgres raw store (replaces Mongo/GridFS) ### 1.5 PostgreSQL raw store
> Mongo/BSON/GridFS are being removed. This is the canonical design for the Postgres-backed raw store that powers `/vex/raw` and ingestion. > This is the canonical design for the PostgreSQL-backed raw store that powers `/vex/raw` and ingestion.
Schema: `vex` Schema: `vex`
@@ -167,7 +167,7 @@ Schema: `vex`
3. If `size <= inline_threshold_bytes` (default 256 KiB) set `inline_payload=true` and store in `content_json`; otherwise store bytes in `vex_raw_blobs` and set `inline_payload=false`. 3. If `size <= inline_threshold_bytes` (default 256 KiB) set `inline_payload=true` and store in `content_json`; otherwise store bytes in `vex_raw_blobs` and set `inline_payload=false`.
4. Persist `content_size_bytes` (pre-canonical length) and `payload_hash` for integrity. 4. Persist `content_size_bytes` (pre-canonical length) and `payload_hash` for integrity.
**API mapping (replaces Mongo/BSON)** **API mapping**
List/query `/vex/raw` via `SELECT ... FROM vex.vex_raw_documents WHERE tenant=@t ORDER BY retrieved_at DESC, digest LIMIT @n OFFSET @offset`; cursor uses `(retrieved_at, digest)`. `GET /vex/raw/{digest}` loads the row and optional blob; `GET /vex/raw/{digest}/provenance` projects `provenance_json` + `metadata_json`. Filters (`providerId`, `format`, `since`, `until`, `supersedes`, `hasAttachments`) map to indexed predicates; JSON subfields use `metadata_json ->> 'field'`. List/query `/vex/raw` via `SELECT ... FROM vex.vex_raw_documents WHERE tenant=@t ORDER BY retrieved_at DESC, digest LIMIT @n OFFSET @offset`; cursor uses `(retrieved_at, digest)`. `GET /vex/raw/{digest}` loads the row and optional blob; `GET /vex/raw/{digest}/provenance` projects `provenance_json` + `metadata_json`. Filters (`providerId`, `format`, `since`, `until`, `supersedes`, `hasAttachments`) map to indexed predicates; JSON subfields use `metadata_json ->> 'field'`.
**Write semantics** **Write semantics**
@@ -178,8 +178,8 @@ List/query `/vex/raw` via `SELECT ... FROM vex.vex_raw_documents WHERE tenant=@t
**Rollout** **Rollout**
1. Add migration under `src/Excititor/__Libraries/StellaOps.Excititor.Storage.Postgres/Migrations` creating the tables/indexes above. 1. Add migration under `src/Excititor/__Libraries/StellaOps.Excititor.Storage.Postgres/Migrations` creating the tables/indexes above.
2. Implement `PostgresVexRawStore` and switch WebService/Worker DI to `AddExcititorPostgresStorage`; remove `VexMongoStorageOptions`, `IMongoDatabase`, and GridFS paths. 2. Implement `PostgresVexRawStore` and switch WebService/Worker DI to `AddExcititorPostgresStorage`.
3. Update `/vex/raw` endpoints/tests to the Postgres store; delete Mongo fixtures once parity is green. Mark Mongo storage paths as deprecated and remove them in the next release. 3. Update `/vex/raw` endpoints/tests to the PostgreSQL store.
--- ---

View File

@@ -5,7 +5,7 @@ Graph Indexer + Graph API build the tenant-scoped knowledge graph that powers bl
## Scope & responsibilities ## Scope & responsibilities
- Ingest SBOM snapshots, advisory/VEX events, policy overlays, and runtime signals to maintain a first-party graph representation with deterministic node/edge identities. - Ingest SBOM snapshots, advisory/VEX events, policy overlays, and runtime signals to maintain a first-party graph representation with deterministic node/edge identities.
- Serve APIs and saved-query tooling for impact analysis, dependency traversal, diffing, and policy/VEX overlays with explainable provenance. - Serve APIs and saved-query tooling for impact analysis, dependency traversal, diffing, and policy/VEX overlays with explainable provenance.
- Supply Graph Explorer UI/CLI experiences, plus Offline Kit exports (`nodes.jsonl`, `edges.jsonl`, `overlays/`) with DSSE manifests for air-gapped replay. Analytics overlays are emitted as NDJSON (`overlays/clusters.ndjson`, `overlays/centrality.ndjson`) with deterministic ordering; Mongo-backed providers support production wiring. - Supply Graph Explorer UI/CLI experiences, plus Offline Kit exports (`nodes.jsonl`, `edges.jsonl`, `overlays/`) with DSSE manifests for air-gapped replay. Analytics overlays are emitted as NDJSON (`overlays/clusters.ndjson`, `overlays/centrality.ndjson`) with deterministic ordering; PostgreSQL-backed providers support production wiring.
- Maintain the [Graph Index Canonical Schema](schema.md) and coordinate query/overlay lifecycle with Scheduler, Policy Engine, Vulnerability Explorer, and Export Center. - Maintain the [Graph Index Canonical Schema](schema.md) and coordinate query/overlay lifecycle with Scheduler, Policy Engine, Vulnerability Explorer, and Export Center.
## Architecture snapshot (Sprint 30 groundwork) ## Architecture snapshot (Sprint 30 groundwork)
@@ -13,7 +13,7 @@ Graph Indexer + Graph API build the tenant-scoped knowledge graph that powers bl
- **Graph API service** — exposes `GET /graph/nodes`, `/graph/impact/{advisory}`, `/graph/query/saved`, `/graph/diff`, and overlay endpoints with RBAC scopes defined in Authority (`docs/updates/2025-10-26-authority-graph-scopes.md`). - **Graph API service** — exposes `GET /graph/nodes`, `/graph/impact/{advisory}`, `/graph/query/saved`, `/graph/diff`, and overlay endpoints with RBAC scopes defined in Authority (`docs/updates/2025-10-26-authority-graph-scopes.md`).
- **Overlay & diff workers** — materialise impact lists, saved-query caches, and signed diff manifests; feed Scheduler `GraphBuildJob`/`GraphOverlayJob` contracts (`docs/updates/2025-10-26-scheduler-graph-jobs.md`). - **Overlay & diff workers** — materialise impact lists, saved-query caches, and signed diff manifests; feed Scheduler `GraphBuildJob`/`GraphOverlayJob` contracts (`docs/updates/2025-10-26-scheduler-graph-jobs.md`).
- **Console & CLI integrations** — planned modules deliver WebGL explorer, timeline viz, and CLI `stella sbom graph ...` commands aligned with implementation plan phases. - **Console & CLI integrations** — planned modules deliver WebGL explorer, timeline viz, and CLI `stella sbom graph ...` commands aligned with implementation plan phases.
- **Storage abstraction** — supports document + adjacency (Mongo) or pluggable graph engine; both paths enforce deterministic ordering and export manifests. - **Storage abstraction** — supports document + adjacency (PostgreSQL) or pluggable graph engine; both paths enforce deterministic ordering and export manifests.
## Current workstreams (Q42025) ## Current workstreams (Q42025)
- `GRAPH-SVC-30-00x` (see `src/Graph/StellaOps.Graph.Indexer/TASKS.md`) — stand up Graph Indexer pipeline, identity registry, snapshot exports. - `GRAPH-SVC-30-00x` (see `src/Graph/StellaOps.Graph.Indexer/TASKS.md`) — stand up Graph Indexer pipeline, identity registry, snapshot exports.

View File

@@ -3,7 +3,7 @@
## Deployment overlays ## Deployment overlays
- Helm/Compose should expose two timers for analytics: `GRAPH_ANALYTICS_CLUSTER_INTERVAL` and `GRAPH_ANALYTICS_CENTRALITY_INTERVAL` (ISO-8601 duration, default 5m). Map to `GraphAnalyticsOptions`. - Helm/Compose should expose two timers for analytics: `GRAPH_ANALYTICS_CLUSTER_INTERVAL` and `GRAPH_ANALYTICS_CENTRALITY_INTERVAL` (ISO-8601 duration, default 5m). Map to `GraphAnalyticsOptions`.
- Change-stream/backfill worker toggles via `GRAPH_CHANGE_POLL_INTERVAL`, `GRAPH_BACKFILL_INTERVAL`, `GRAPH_CHANGE_MAX_RETRIES`, `GRAPH_CHANGE_RETRY_BACKOFF`. - Change-stream/backfill worker toggles via `GRAPH_CHANGE_POLL_INTERVAL`, `GRAPH_BACKFILL_INTERVAL`, `GRAPH_CHANGE_MAX_RETRIES`, `GRAPH_CHANGE_RETRY_BACKOFF`.
- Storage: current build uses in-memory graph storage (Mongo dependency removed). Reserve `GRAPH_STORAGE_CONNECTION` and `GRAPH_STORAGE_DB` for the upcoming Postgres-backed provider. - Storage: current build uses in-memory graph storage. `GRAPH_STORAGE_CONNECTION` configures the PostgreSQL-backed provider.
## Offline kit alignment ## Offline kit alignment
- Cluster/centrality overlays are exportable alongside `nodes.jsonl`/`edges.jsonl`; keep under `artifacts/graph-snapshots/{snapshotId}/overlays/` for air-gapped imports. - Cluster/centrality overlays are exportable alongside `nodes.jsonl`/`edges.jsonl`; keep under `artifacts/graph-snapshots/{snapshotId}/overlays/` for air-gapped imports.

View File

@@ -781,6 +781,189 @@ endpoints:
--- ---
## Troubleshooting
### Common Issues
#### 1. Endpoints Not Discovered
**Symptom:** Gateway shows 0 endpoints for service, or specific endpoints missing.
**Causes & Solutions:**
| Cause | Solution |
|-------|----------|
| `UseStellaRouterBridge()` called before `MapControllers()` | Call `UseStellaRouterBridge()` after all endpoint registration |
| Route filtered by `EndpointFilter` | Check filter logic, ensure endpoint matches |
| Route filtered by `IncludePathPatterns` | Verify path pattern includes the endpoint |
| Route filtered by `ExcludePathPatterns` | Verify endpoint isn't accidentally excluded |
| Missing `[Authorize]` with `RequireExplicit` | Add authorization or change `MissingAuthorizationBehavior` |
**Debug:**
```csharp
// Enable discovery logging
builder.Logging.AddFilter("StellaOps.Microservice.AspNetCore", LogLevel.Debug);
```
#### 2. Authorization Claims Not Extracted
**Symptom:** Endpoints registered but `RequiringClaims` is empty when it shouldn't be.
**Causes & Solutions:**
| Cause | Solution |
|-------|----------|
| `[Authorize]` without policy/roles | Add explicit policy or roles |
| Policy not registered | Register policy with `AddAuthorization()` |
| `AuthorizationMappingStrategy.YamlOnly` set | Use `Hybrid` or `AspNetMetadataOnly` |
| Custom policy doesn't have claim requirements | Use claims-based policies |
**Debug:**
```csharp
// Log authorization mapping
var mapper = app.Services.GetRequiredService<IAuthorizationClaimMapper>();
var result = await mapper.MapAsync(endpoint);
Console.WriteLine($"Claims: {string.Join(", ", result.Claims)}");
```
#### 3. Request Dispatch Fails with 404
**Symptom:** Gateway routes request but microservice returns 404.
**Causes & Solutions:**
| Cause | Solution |
|-------|----------|
| Path parameters not matched | Verify parameter names in route pattern |
| Method mismatch | Verify HTTP method matches endpoint |
| Route constraint rejected value | Use standard constraints that bridge supports |
| Catch-all route not handled | Ensure `{**path}` is normalized correctly |
**Debug:**
```bash
# Check registered endpoints
curl http://localhost:5000/.well-known/stella-endpoints
```
#### 4. Model Binding Errors
**Symptom:** Requests return 400 Bad Request with binding errors.
**Causes & Solutions:**
| Cause | Solution |
|-------|----------|
| `[FromBody]` type mismatch | Verify request body matches expected type |
| Required parameter missing | Include all `[FromRoute]`/`[FromQuery]` parameters |
| `[FromHeader]` not populated | Headers forwarded via `X-StellaOps-Header-*` |
| Complex type not deserialized | Verify JSON serialization settings |
**Debug:**
```csharp
// Enable model binding logging
builder.Logging.AddFilter("Microsoft.AspNetCore.Mvc.ModelBinding", LogLevel.Debug);
```
#### 5. Identity Not Populated
**Symptom:** `User.Identity` is null or claims missing in endpoint handler.
**Causes & Solutions:**
| Cause | Solution |
|-------|----------|
| Gateway not forwarding identity | Verify Gateway `identity-header-policy` configured |
| Missing `X-StellaOps-UserId` header | Gateway must send identity headers |
| `UseAuthentication()` not called | Call before `UseStellaRouterBridge()` |
| Custom claims not mapped | Use `ClaimsPrincipalBuilder` for custom claims |
**Debug:**
```csharp
app.MapGet("/debug/identity", (HttpContext ctx) =>
new {
IsAuthenticated = ctx.User.Identity?.IsAuthenticated,
Name = ctx.User.Identity?.Name,
Claims = ctx.User.Claims.Select(c => new { c.Type, c.Value })
});
```
#### 6. Performance Issues
**Symptom:** High latency or memory usage.
**Causes & Solutions:**
| Cause | Solution |
|-------|----------|
| HttpContext allocation per request | Pool contexts (internal implementation) |
| Large request bodies buffered | Use streaming endpoints for large payloads |
| Discovery runs too frequently | Discovery runs once at startup; cache is stable |
| Many endpoints slow startup | Discovery is O(n) but runs once |
**Metrics to monitor:**
- `stellaops_router_bridge_requests_total`
- `stellaops_router_bridge_request_duration_seconds`
- `stellaops_router_bridge_dispatch_errors_total`
#### 7. YAML Override Not Applied
**Symptom:** Claims from YAML file not appearing in endpoint registration.
**Causes & Solutions:**
| Cause | Solution |
|-------|----------|
| `YamlConfigPath` not set | Set `options.YamlConfigPath = "router.yaml"` |
| File not found | Use absolute path or verify relative path |
| Path pattern doesn't match | YAML paths are case-insensitive, verify pattern |
| `AuthorizationMappingStrategy.AspNetMetadataOnly` | Use `YamlOnly` or `Hybrid` |
**Example YAML:**
```yaml
# router.yaml
endpoints:
- path: "/api/admin/**"
requiringClaims:
- type: "Role"
value: "admin"
```
### Diagnostic Endpoints
The bridge adds optional diagnostic endpoints (development only):
| Endpoint | Description |
|----------|-------------|
| `/.well-known/stella-endpoints` | Lists all discovered endpoints |
| `/.well-known/stella-bridge-status` | Shows bridge configuration and health |
Enable in development:
```csharp
builder.Services.AddStellaRouterBridge(options =>
{
options.EnableDiagnosticEndpoints = builder.Environment.IsDevelopment();
// ...
});
```
### Logging Categories
Configure logging for troubleshooting:
```json
{
"Logging": {
"LogLevel": {
"StellaOps.Microservice.AspNetCore.Discovery": "Debug",
"StellaOps.Microservice.AspNetCore.Authorization": "Debug",
"StellaOps.Microservice.AspNetCore.Dispatch": "Information"
}
}
}
```
---
## Testing Strategy ## Testing Strategy
### Unit Tests ### Unit Tests

View File

@@ -96,6 +96,169 @@ Best for major refactoring or when HTTP compatibility is not needed.
- More upfront work - More upfront work
- Requires domain extraction - Requires domain extraction
### Strategy C: ASP.NET Endpoint Bridge (Recommended)
Best for services that want automatic Router registration from existing ASP.NET endpoints without code changes.
```
┌─────────────────────────────────────────────────────────────────┐
│ StellaOps.*.WebService │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ ASP.NET Endpoints (unchanged) ││
│ │ • Minimal APIs: app.MapGet("/api/...", handler) ││
│ │ • Controllers: [ApiController] with [HttpGet], etc. ││
│ │ • Route groups: app.MapGroup("/api").MapEndpoints() ││
│ └─────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ Auto-discovery │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ StellaOps.Microservice.AspNetCore ││
│ │ • Discovers all endpoints from EndpointDataSource ││
│ │ • Extracts authorization metadata automatically ││
│ │ • Registers with Router via HELLO ││
│ │ • Dispatches Router requests through ASP.NET pipeline ││
│ └─────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ Binary transport │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ Router Gateway ││
│ │ • Routes external HTTP → internal microservice ││
│ │ • Aggregates OpenAPI ││
│ └─────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────┘
```
**Steps:**
1. Add `StellaOps.Microservice.AspNetCore` package reference
2. Configure bridge in `Program.cs`:
```csharp
using StellaOps.Microservice.AspNetCore;
var builder = WebApplication.CreateBuilder(args);
// Add authorization (required for claim mapping)
builder.Services.AddAuthorization();
// Add the ASP.NET Endpoint Bridge
builder.Services.AddStellaRouterBridge(options =>
{
options.ServiceName = "my-service";
options.Version = "1.0.0";
options.Region = "us-east-1";
});
var app = builder.Build();
// Register ASP.NET endpoints as usual
app.MapGet("/api/health", () => "healthy");
app.MapGet("/api/items/{id}", (string id) => new { id, name = "Item " + id })
.RequireAuthorization("read-items");
app.MapPost("/api/items", (CreateItemRequest req) => new { id = Guid.NewGuid() })
.RequireAuthorization("write-items");
// Activate the bridge (discovers and registers endpoints)
app.UseStellaRouterBridge();
app.Run();
```
3. (Optional) Add YAML overrides for security hardening:
```yaml
# router.yaml
endpoints:
- path: "/api/admin/**"
requiringClaims:
- type: "Role"
value: "admin"
timeoutMs: 60000
```
4. Test via Gateway
5. Deploy to production
**Pros:**
- **Zero code changes** to existing endpoints
- Automatic authorization metadata extraction
- Full ASP.NET pipeline execution (filters, binding, validation)
- Deterministic endpoint ordering
- YAML overrides for security hardening without code changes
- Supports both minimal APIs and controllers
**Cons:**
- Slightly more runtime overhead (HttpContext synthesis)
- Some ASP.NET features not supported (see limitations below)
#### Authorization Mapping
The bridge automatically extracts authorization requirements:
| ASP.NET Source | Router Mapping |
|----------------|----------------|
| `[Authorize(Policy = "policy-name")]` | Resolved via `IAuthorizationPolicyProvider``RequiringClaims` |
| `[Authorize(Roles = "admin,user")]` | `ClaimRequirement(Role, admin)`, `ClaimRequirement(Role, user)` |
| `.RequireAuthorization("policy")` | Resolved via policy provider |
| `.RequireAuthorization(new AuthorizeAttribute { Roles = "..." })` | Direct role mapping |
| `[AllowAnonymous]` | Empty `RequiringClaims` (public endpoint) |
#### Authorization Mapping Strategies
Configure how the bridge handles authorization:
```csharp
builder.Services.AddStellaRouterBridge(options =>
{
options.ServiceName = "my-service";
options.Version = "1.0.0";
options.Region = "us-east-1";
// Strategy options:
// - YamlOnly: YAML claims replace code claims entirely
// - AspNetMetadataOnly: Code claims only, ignore YAML (default)
// - Hybrid: Merge code + YAML claims by type
options.AuthorizationMappingStrategy = AuthorizationMappingStrategy.Hybrid;
// Behavior when endpoint has no authorization:
// - RequireExplicit: Fail validation (secure default)
// - AllowAuthenticated: Require authentication but no specific claims
// - WarnAndAllow: Log warning, allow request (dev only)
options.MissingAuthorizationBehavior = MissingAuthorizationBehavior.RequireExplicit;
});
```
#### Route Filtering
Control which endpoints are bridged:
```csharp
builder.Services.AddStellaRouterBridge(options =>
{
options.ServiceName = "my-service";
options.Version = "1.0.0";
options.Region = "us-east-1";
// Include only /api/* endpoints
options.IncludePathPatterns = ["/api/**"];
// Exclude internal endpoints
options.ExcludePathPatterns = ["/internal/**", "/metrics", "/health/**"];
});
```
#### Limitations
The ASP.NET Endpoint Bridge does **not** support:
| Feature | Alternative |
|---------|-------------|
| SignalR/WebSocket | Use direct HTTP for real-time features |
| gRPC endpoints | Use separate gRPC channel |
| Streaming request bodies | Use `IRawStellaEndpoint` for streaming |
| Custom route constraints (`{id:guid}`) | Constraints execute at dispatch, but not in discovery |
| Header/query-based API versioning | Use path-based versioning (`/api/v1/...`) |
## Controller to Handler Mapping ## Controller to Handler Mapping
### Before (ASP.NET Controller) ### Before (ASP.NET Controller)

View File

@@ -54,12 +54,12 @@ sequenceDiagram
autonumber autonumber
participant Trigger as Trigger (CLI / Console / Change Stream) participant Trigger as Trigger (CLI / Console / Change Stream)
participant Orchestrator as Policy Orchestrator participant Orchestrator as Policy Orchestrator
participant Queue as Scheduler Queue (Mongo/NATS) participant Queue as Scheduler Queue (PostgreSQL/NATS)
participant Engine as Policy Engine Workers participant Engine as Policy Engine Workers
participant Concelier as Concelier Service participant Concelier as Concelier Service
participant Excititor as Excititor Service participant Excititor as Excititor Service
participant SBOM as SBOM Service participant SBOM as SBOM Service
participant Store as Mongo (policy_runs & effective_finding_*) participant Store as PostgreSQL (policy_runs & effective_finding_*)
participant Observability as Metrics/Events participant Observability as Metrics/Events
Trigger->>Orchestrator: Run request (mode, scope, env) Trigger->>Orchestrator: Run request (mode, scope, env)
@@ -78,9 +78,9 @@ sequenceDiagram
- **Trigger** CLI, Console, or automated change stream publishes a `PolicyRunRequest`. - **Trigger** CLI, Console, or automated change stream publishes a `PolicyRunRequest`.
- **Orchestrator** Runs inside `StellaOps.Policy.Engine` worker host; applies fairness (tenant + policy quotas) and idempotency using run keys. - **Orchestrator** Runs inside `StellaOps.Policy.Engine` worker host; applies fairness (tenant + policy quotas) and idempotency using run keys.
- **Queue** Backed by Mongo + optional NATS for fan-out; supports leases and replay on crash. - **Queue** Backed by PostgreSQL + optional NATS for fan-out; supports leases and replay on crash.
- **Engine** Stateless worker executing the deterministic evaluator. - **Engine** Stateless worker executing the deterministic evaluator.
- **Store** Mongo collections: `policy_runs`, `effective_finding_{policyId}`, `policy_run_events` (append-only history), optional object storage for explain traces. - **Store** PostgreSQL tables: `policy_runs`, `effective_finding_{policyId}`, `policy_run_events` (append-only history), optional object storage for explain traces.
- **Observability** Prometheus metrics (`policy_run_seconds`, `policy_simulation_queue_depth`, `policy_simulation_latency_seconds`), OTLP traces, structured logs. - **Observability** Prometheus metrics (`policy_run_seconds`, `policy_simulation_queue_depth`, `policy_simulation_latency_seconds`), OTLP traces, structured logs.
--- ---

View File

@@ -1,5 +1,7 @@
# Evidence & Suppression Patterns (Gaps Stub) # Evidence & Suppression Patterns (Gaps Stub)
> **Development Placeholder:** This document tracks implementation gaps for sprint planning. For evidence and suppression documentation, see `docs/policy/` and `docs/modules/policy/README.md`.
Use with sprint task 9 (EVIDENCE-PATTERNS-GAPS-300-016) and advisory `30-Nov-2025 - Comparative Evidence Patterns for Stella Ops.md`. Use with sprint task 9 (EVIDENCE-PATTERNS-GAPS-300-016) and advisory `30-Nov-2025 - Comparative Evidence Patterns for Stella Ops.md`.
- TODO: Canonical schema for evidence, suppression, export; align across modules. - TODO: Canonical schema for evidence, suppression, export; align across modules.

View File

@@ -1,5 +1,7 @@
# Plugin Architecture Gaps (Stub) # Plugin Architecture Gaps (Stub)
> **Development Placeholder:** This document tracks implementation gaps for sprint planning. For plugin architecture documentation, see `docs/modules/*/AGENTS.md` and `docs/dev/30_PLUGIN_DEV_GUIDE.md`.
Use with sprint task 14 (Plugin architecture gaps remediation). Use with sprint task 14 (Plugin architecture gaps remediation).
- TODO: Signed schemas/capability catalog for plugins. - TODO: Signed schemas/capability catalog for plugins.

View File

@@ -12,7 +12,7 @@ Scope: deploy sealed-mode Concelier evidence bundles using deterministic NDJSON
## Preconditions ## Preconditions
- Concelier WebService running with `concelier:features:airgap` enabled. - Concelier WebService running with `concelier:features:airgap` enabled.
- No external egress; only local file system allowed for bundle path. - No external egress; only local file system allowed for bundle path.
- Mongo indexes applied (`advisory_observations`, `advisory_linksets`). - PostgreSQL indexes applied (`advisory_observations`, `advisory_linksets` tables).
## Steps ## Steps
1) Transfer bundle directory to offline controller host. 1) Transfer bundle directory to offline controller host.

View File

@@ -9,7 +9,7 @@ Status: DRAFT (2025-12-06 UTC). Safe for dev/mock exercises; production rollouts
2) Render plan 2) Render plan
- Helm (mock overlay): `helm template vex-mock ./deploy/helm/stellaops -f deploy/helm/stellaops/values-mock.yaml --debug --validate > /tmp/vex-mock.yaml` - Helm (mock overlay): `helm template vex-mock ./deploy/helm/stellaops -f deploy/helm/stellaops/values-mock.yaml --debug --validate > /tmp/vex-mock.yaml`
- Compose (dev with overlay): `USE_MOCK=1 deploy/compose/scripts/quickstart.sh env/dev.env.example && docker compose --env-file env/dev.env.example -f deploy/compose/docker-compose.dev.yaml -f deploy/compose/docker-compose.mock.yaml config > /tmp/vex-compose.yaml` - Compose (dev with overlay): `USE_MOCK=1 deploy/compose/scripts/quickstart.sh env/dev.env.example && docker compose --env-file env/dev.env.example -f deploy/compose/docker-compose.dev.yaml -f deploy/compose/docker-compose.mock.yaml config > /tmp/vex-compose.yaml`
3) Backups (when touching prod data) — not required for mock, but in prod take Mongo snapshots for issuer-directory and VEX state before rollout. 3) Backups (when touching prod data) — not required for mock, but in prod take PostgreSQL snapshots for issuer-directory and VEX state before rollout.
## Deploy (mock path) ## Deploy (mock path)
- Helm dry-run already covers structural checks. To apply in a dev cluster: `helm upgrade --install stellaops ./deploy/helm/stellaops -f deploy/helm/stellaops/values-mock.yaml --atomic --timeout 10m`. - Helm dry-run already covers structural checks. To apply in a dev cluster: `helm upgrade --install stellaops ./deploy/helm/stellaops -f deploy/helm/stellaops/values-mock.yaml --atomic --timeout 10m`.

View File

@@ -17,7 +17,7 @@ Specifications covering APIs, data contracts, event envelopes, and enforcement m
- [../60_POLICY_TEMPLATES.md](../../60_POLICY_TEMPLATES.md) YAML/Rego samples. - [../60_POLICY_TEMPLATES.md](../../60_POLICY_TEMPLATES.md) YAML/Rego samples.
## Data Schemas & Storage Contracts ## Data Schemas & Storage Contracts
- [../11_DATA_SCHEMAS.md](../../11_DATA_SCHEMAS.md) MongoDB/Redis/document shapes. - [../11_DATA_SCHEMAS.md](../../11_DATA_SCHEMAS.md) PostgreSQL/Valkey/document shapes.
- JSON schemas under [../schemas/](../../schemas/) policy diff, explain trace, run request, run status, preview sample, report sample. - JSON schemas under [../schemas/](../../schemas/) policy diff, explain trace, run request, run status, preview sample, report sample.
- [../../modules/scanner/architecture.md](../../modules/scanner/architecture.md) SBOM cache and scan job contracts. - [../../modules/scanner/architecture.md](../../modules/scanner/architecture.md) SBOM cache and scan job contracts.
- [../../scanner-core-contracts.md](../../scanner-core-contracts.md) shared scanner DTOs. - [../../scanner-core-contracts.md](../../scanner-core-contracts.md) shared scanner DTOs.