refactor(graph): absorb Cartographer into graph-api + wire Graph Indexer

- Wire Graph Indexer library + Persistence into graph-api (csproj refs + DI)
- Add build/overlay endpoints matching Scheduler HTTP contracts
  (POST/GET /api/graphs/builds, POST/GET /api/graphs/overlays)
- Add PostgresGraphRepository for reading from graph.graph_nodes/edges
- Register SBOM ingest, analytics, change-stream, and inspector pipelines
- Comment out Cartographer container in compose (empty shell, Slot 21)
- Add cartographer.stella-ops.local as backwards-compat alias on graph-api
- Update Scheduler config to target graph.stella-ops.local
- Update services-matrix.env, hosts file, port-registry, module-matrix
- Update component-map, architecture docs, Scanner/Graph READMEs
- Eliminates 1 container (stellaops-cartographer)

All 133 existing tests pass (77 Api + 37 Indexer + 19 Core).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
master
2026-04-08 15:48:18 +03:00
parent 13c4811e32
commit 6592cdcc9b
16 changed files with 592 additions and 73 deletions

View File

@@ -272,7 +272,8 @@ services:
# STELLAOPS_TASKRUNNER_URL removed: TaskRunner service deleted # STELLAOPS_TASKRUNNER_URL removed: TaskRunner service deleted
STELLAOPS_SCHEDULER_URL: "http://scheduler.stella-ops.local" STELLAOPS_SCHEDULER_URL: "http://scheduler.stella-ops.local"
STELLAOPS_GRAPH_URL: "http://graph.stella-ops.local" STELLAOPS_GRAPH_URL: "http://graph.stella-ops.local"
STELLAOPS_CARTOGRAPHER_URL: "http://cartographer.stella-ops.local" # STELLAOPS_CARTOGRAPHER_URL: merged into graph-api; use STELLAOPS_GRAPH_URL instead
STELLAOPS_CARTOGRAPHER_URL: "http://graph.stella-ops.local"
STELLAOPS_REACHGRAPH_URL: "http://reachgraph.stella-ops.local" STELLAOPS_REACHGRAPH_URL: "http://reachgraph.stella-ops.local"
STELLAOPS_TIMELINEINDEXER_URL: "http://timelineindexer.stella-ops.local" STELLAOPS_TIMELINEINDEXER_URL: "http://timelineindexer.stella-ops.local"
STELLAOPS_TIMELINE_URL: "http://timeline.stella-ops.local" STELLAOPS_TIMELINE_URL: "http://timeline.stella-ops.local"
@@ -1009,7 +1010,7 @@ services:
Scheduler__Storage__Postgres__Scheduler__ConnectionString: "${STELLAOPS_POSTGRES_CONNECTION}" Scheduler__Storage__Postgres__Scheduler__ConnectionString: "${STELLAOPS_POSTGRES_CONNECTION}"
Scheduler__Storage__Postgres__Scheduler__SchemaName: "scheduler" Scheduler__Storage__Postgres__Scheduler__SchemaName: "scheduler"
Scheduler__Worker__Runner__Scanner__BaseAddress: "http://scanner.stella-ops.local" Scheduler__Worker__Runner__Scanner__BaseAddress: "http://scanner.stella-ops.local"
Scheduler__Worker__Graph__Cartographer__BaseAddress: "http://cartographer.stella-ops.local" Scheduler__Worker__Graph__Cartographer__BaseAddress: "http://graph.stella-ops.local"
Scheduler__Worker__Graph__SchedulerApi__BaseAddress: "http://scheduler.stella-ops.local" Scheduler__Worker__Graph__SchedulerApi__BaseAddress: "http://scheduler.stella-ops.local"
Scheduler__Worker__Policy__Api__BaseAddress: "http://policy.stella-ops.local" Scheduler__Worker__Policy__Api__BaseAddress: "http://policy.stella-ops.local"
Router__Enabled: "${SCHEDULER_ROUTER_ENABLED:-true}" Router__Enabled: "${SCHEDULER_ROUTER_ENABLED:-true}"
@@ -1045,7 +1046,7 @@ services:
Scheduler__Storage__Postgres__Scheduler__SchemaName: "scheduler" Scheduler__Storage__Postgres__Scheduler__SchemaName: "scheduler"
# Worker config # Worker config
Scheduler__Worker__Runner__Scanner__BaseAddress: "${SCHEDULER_SCANNER_BASEADDRESS:-http://scanner.stella-ops.local}" Scheduler__Worker__Runner__Scanner__BaseAddress: "${SCHEDULER_SCANNER_BASEADDRESS:-http://scanner.stella-ops.local}"
Scheduler__Worker__Graph__Cartographer__BaseAddress: "http://cartographer.stella-ops.local" Scheduler__Worker__Graph__Cartographer__BaseAddress: "http://graph.stella-ops.local"
Scheduler__Worker__Graph__SchedulerApi__BaseAddress: "http://scheduler.stella-ops.local" Scheduler__Worker__Graph__SchedulerApi__BaseAddress: "http://scheduler.stella-ops.local"
Scheduler__Worker__Policy__Api__BaseAddress: "http://policy.stella-ops.local" Scheduler__Worker__Policy__Api__BaseAddress: "http://policy.stella-ops.local"
# Surface environment # Surface environment
@@ -1083,32 +1084,7 @@ services:
stellaops: stellaops:
aliases: aliases:
- graph.stella-ops.local - graph.stella-ops.local
frontdoor: {} # Backwards-compat: absorb Cartographer traffic (Slot 21 merged into graph-api)
healthcheck:
test: ["CMD-SHELL", "bash -c 'echo > /dev/tcp/$(hostname)/80'"]
<<: *healthcheck-tcp
labels: *release-labels
# --- Slot 21: Cartographer -------------------------------------------------
cartographer:
<<: *resources-light
image: stellaops/cartographer:dev
container_name: stellaops-cartographer
restart: unless-stopped
environment:
ASPNETCORE_URLS: "http://+:8080"
<<: [*kestrel-cert, *router-microservice-defaults, *gc-light]
ConnectionStrings__Default: "${STELLAOPS_POSTGRES_CONNECTION}"
ConnectionStrings__Redis: "cache.stella-ops.local:6379"
Router__Enabled: "${CARTOGRAPHER_ROUTER_ENABLED:-true}"
Router__Messaging__ConsumerGroup: "cartographer"
volumes:
- ${STELLAOPS_CERT_VOLUME}
ports:
- "127.1.0.21:80:80"
networks:
stellaops:
aliases:
- cartographer.stella-ops.local - cartographer.stella-ops.local
frontdoor: {} frontdoor: {}
healthcheck: healthcheck:
@@ -1116,6 +1092,33 @@ services:
<<: *healthcheck-tcp <<: *healthcheck-tcp
labels: *release-labels labels: *release-labels
# --- Slot 21: Cartographer (RETIRED -- merged into graph-api Slot 20) ------
# cartographer:
# <<: *resources-light
# image: stellaops/cartographer:dev
# container_name: stellaops-cartographer
# restart: unless-stopped
# environment:
# ASPNETCORE_URLS: "http://+:8080"
# <<: [*kestrel-cert, *router-microservice-defaults, *gc-light]
# ConnectionStrings__Default: "${STELLAOPS_POSTGRES_CONNECTION}"
# ConnectionStrings__Redis: "cache.stella-ops.local:6379"
# Router__Enabled: "${CARTOGRAPHER_ROUTER_ENABLED:-true}"
# Router__Messaging__ConsumerGroup: "cartographer"
# volumes:
# - ${STELLAOPS_CERT_VOLUME}
# ports:
# - "127.1.0.21:80:80"
# networks:
# stellaops:
# aliases:
# - cartographer.stella-ops.local
# frontdoor: {}
# healthcheck:
# test: ["CMD-SHELL", "bash -c 'echo > /dev/tcp/$(hostname)/80'"]
# <<: *healthcheck-tcp
# labels: *release-labels
# --- Slot 22: ReachGraph --------------------------------------------------- # --- Slot 22: ReachGraph ---------------------------------------------------
reachgraph-web: reachgraph-web:
<<: *resources-light <<: *resources-light
@@ -1272,28 +1275,33 @@ services:
<<: *healthcheck-tcp <<: *healthcheck-tcp
labels: *release-labels labels: *release-labels
doctor-scheduler: # doctor-scheduler: DEPRECATED -- replaced by DoctorJobPlugin in the Scheduler service.
<<: *resources-light # Doctor health check scheduling is now handled by scheduler-web via the plugin architecture.
image: stellaops/doctor-scheduler:dev # This service will be removed in a future release. See:
container_name: stellaops-doctor-scheduler # docs/implplan/SPRINT_20260408_003_JobEngine_scheduler_plugin_architecture.md
restart: unless-stopped #
environment: # doctor-scheduler:
ASPNETCORE_URLS: "http://+:80" # <<: *resources-light
<<: [*kestrel-cert, *router-microservice-defaults, *gc-light] # image: stellaops/doctor-scheduler:dev
ConnectionStrings__Default: "${STELLAOPS_POSTGRES_CONNECTION}" # container_name: stellaops-doctor-scheduler
ConnectionStrings__Redis: "cache.stella-ops.local:6379" # restart: unless-stopped
Router__Enabled: "${DOCTOR_SCHEDULER_ROUTER_ENABLED:-true}" # environment:
Router__Messaging__ConsumerGroup: "doctor-scheduler" # ASPNETCORE_URLS: "http://+:80"
volumes: # <<: [*kestrel-cert, *router-microservice-defaults, *gc-light]
- ${STELLAOPS_CERT_VOLUME} # ConnectionStrings__Default: "${STELLAOPS_POSTGRES_CONNECTION}"
healthcheck: # ConnectionStrings__Redis: "cache.stella-ops.local:6379"
test: ["CMD-SHELL", "bash -c 'echo > /dev/tcp/$(hostname)/80'"] # Router__Enabled: "${DOCTOR_SCHEDULER_ROUTER_ENABLED:-true}"
<<: *healthcheck-tcp # Router__Messaging__ConsumerGroup: "doctor-scheduler"
networks: # volumes:
stellaops: # - ${STELLAOPS_CERT_VOLUME}
aliases: # healthcheck:
- doctor-scheduler.stella-ops.local # test: ["CMD-SHELL", "bash -c 'echo > /dev/tcp/$(hostname)/80'"]
labels: *release-labels # <<: *healthcheck-tcp
# networks:
# stellaops:
# aliases:
# - doctor-scheduler.stella-ops.local
# labels: *release-labels
# --- Slot 27: OpsMemory (src/AdvisoryAI/StellaOps.OpsMemory.WebService) --- # --- Slot 27: OpsMemory (src/AdvisoryAI/StellaOps.OpsMemory.WebService) ---
opsmemory-web: opsmemory-web:

View File

@@ -27,7 +27,8 @@
# 127.1.0.18 taskrunner.stella-ops.local # REMOVED: TaskRunner service deleted # 127.1.0.18 taskrunner.stella-ops.local # REMOVED: TaskRunner service deleted
127.1.0.19 scheduler.stella-ops.local 127.1.0.19 scheduler.stella-ops.local
127.1.0.20 graph.stella-ops.local 127.1.0.20 graph.stella-ops.local
127.1.0.21 cartographer.stella-ops.local # 127.1.0.21 cartographer.stella-ops.local # RETIRED: merged into graph-api (Slot 20)
127.1.0.20 cartographer.stella-ops.local
127.1.0.22 reachgraph.stella-ops.local 127.1.0.22 reachgraph.stella-ops.local
127.1.0.23 timelineindexer.stella-ops.local 127.1.0.23 timelineindexer.stella-ops.local
127.1.0.24 timeline.stella-ops.local 127.1.0.24 timeline.stella-ops.local

View File

@@ -90,7 +90,7 @@
"Microservice","/vexlens","http://vexlens.stella-ops.local","/vexlens/api/v1/vexlens/stats","200" "Microservice","/vexlens","http://vexlens.stella-ops.local","/vexlens/api/v1/vexlens/stats","200"
"Microservice","/orchestrator","http://orchestrator.stella-ops.local","/orchestrator/scale/load","200" "Microservice","/orchestrator","http://orchestrator.stella-ops.local","/orchestrator/scale/load","200"
"Microservice","/cartographer","http://cartographer.stella-ops.local",, "Microservice","/cartographer","http://graph.stella-ops.local",,
"Microservice","/reachgraph","http://reachgraph.stella-ops.local","/reachgraph/v1/cve-mappings/stats","400" "Microservice","/reachgraph","http://reachgraph.stella-ops.local","/reachgraph/v1/cve-mappings/stats","400"
"Microservice","/doctor","http://doctor.stella-ops.local","/doctor/api/v1/doctor/checks","401" "Microservice","/doctor","http://doctor.stella-ops.local","/doctor/api/v1/doctor/checks","401"
"Microservice","/integrations","http://integrations.stella-ops.local","/integrations/api/v1/integrations","401" "Microservice","/integrations","http://integrations.stella-ops.local","/integrations/api/v1/integrations","401"
1 RouteType RoutePath RouteTarget SelectedOpenApiPath StatusCode
90 Microservice /vexlens http://vexlens.stella-ops.local /vexlens/api/v1/vexlens/stats 200
91 Microservice /orchestrator http://orchestrator.stella-ops.local /orchestrator/scale/load 200
92 Microservice /cartographer http://cartographer.stella-ops.local http://graph.stella-ops.local
93 Microservice /reachgraph http://reachgraph.stella-ops.local /reachgraph/v1/cve-mappings/stats 400
94 Microservice /doctor http://doctor.stella-ops.local /doctor/api/v1/doctor/checks 401
95 Microservice /integrations http://integrations.stella-ops.local /integrations/api/v1/integrations 401
96 Microservice /replay http://replay.stella-ops.local /replay/v1/pit/advisory/{cveId} 400

View File

@@ -93,7 +93,7 @@
"ReverseProxy","/vexlens","http://vexlens.stella-ops.local",, "ReverseProxy","/vexlens","http://vexlens.stella-ops.local",,
"ReverseProxy","/orchestrator","http://orchestrator.stella-ops.local",, "ReverseProxy","/orchestrator","http://orchestrator.stella-ops.local",,
"ReverseProxy","/cartographer","http://cartographer.stella-ops.local",, "ReverseProxy","/cartographer","http://graph.stella-ops.local",,
"ReverseProxy","/reachgraph","http://reachgraph.stella-ops.local",, "ReverseProxy","/reachgraph","http://reachgraph.stella-ops.local",,
"ReverseProxy","/doctor","http://doctor.stella-ops.local",, "ReverseProxy","/doctor","http://doctor.stella-ops.local",,
"ReverseProxy","/integrations","http://integrations.stella-ops.local",, "ReverseProxy","/integrations","http://integrations.stella-ops.local",,
1 RouteType RoutePath RouteTarget SelectedOpenApiPath StatusCode
93 ReverseProxy /vexlens http://vexlens.stella-ops.local
94 ReverseProxy /orchestrator http://orchestrator.stella-ops.local
95 ReverseProxy /cartographer http://cartographer.stella-ops.local http://graph.stella-ops.local
96 ReverseProxy /reachgraph http://reachgraph.stella-ops.local
97 ReverseProxy /doctor http://doctor.stella-ops.local
98 ReverseProxy /integrations http://integrations.stella-ops.local
99 ReverseProxy /replay http://replay.stella-ops.local

View File

@@ -23,7 +23,7 @@ scanner-worker|devops/docker/Dockerfile.hardened.template|src/Scanner/StellaOps.
# ── Slot 9: Concelier ─────────────────────────────────────────────────────────── # ── Slot 9: Concelier ───────────────────────────────────────────────────────────
concelier|devops/docker/Dockerfile.hardened.template|src/Concelier/StellaOps.Concelier.WebService/StellaOps.Concelier.WebService.csproj|StellaOps.Concelier.WebService|8080 concelier|devops/docker/Dockerfile.hardened.template|src/Concelier/StellaOps.Concelier.WebService/StellaOps.Concelier.WebService.csproj|StellaOps.Concelier.WebService|8080
# ── Slot 10: Excititor ────────────────────────────────────────────────────────── # ── Slot 10: Excititor ──────────────────────────────────────────────────────────
excititor|devops/docker/Dockerfile.hardened.template|src/Concelier/StellaOps.Excititor.WebService/StellaOps.Excititor.WebService.csproj|StellaOps.Excititor.WebService|8080 excititor-web|devops/docker/Dockerfile.hardened.template|src/Concelier/StellaOps.Excititor.WebService/StellaOps.Excititor.WebService.csproj|StellaOps.Excititor.WebService|8080
excititor-worker|devops/docker/Dockerfile.hardened.template|src/Concelier/StellaOps.Excititor.Worker/StellaOps.Excititor.Worker.csproj|StellaOps.Excititor.Worker|8080 excititor-worker|devops/docker/Dockerfile.hardened.template|src/Concelier/StellaOps.Excititor.Worker/StellaOps.Excititor.Worker.csproj|StellaOps.Excititor.Worker|8080
# ── Slot 11: VexHub ───────────────────────────────────────────────────────────── # ── Slot 11: VexHub ─────────────────────────────────────────────────────────────
vexhub-web|devops/docker/Dockerfile.hardened.template|src/VexHub/StellaOps.VexHub.WebService/StellaOps.VexHub.WebService.csproj|StellaOps.VexHub.WebService|8080 vexhub-web|devops/docker/Dockerfile.hardened.template|src/VexHub/StellaOps.VexHub.WebService/StellaOps.VexHub.WebService.csproj|StellaOps.VexHub.WebService|8080
@@ -46,8 +46,8 @@ scheduler-web|devops/docker/Dockerfile.hardened.template|src/JobEngine/StellaOps
scheduler-worker|devops/docker/Dockerfile.hardened.template|src/JobEngine/StellaOps.Scheduler.Worker.Host/StellaOps.Scheduler.Worker.Host.csproj|StellaOps.Scheduler.Worker.Host|8080 scheduler-worker|devops/docker/Dockerfile.hardened.template|src/JobEngine/StellaOps.Scheduler.Worker.Host/StellaOps.Scheduler.Worker.Host.csproj|StellaOps.Scheduler.Worker.Host|8080
# ── Slot 20: Graph ────────────────────────────────────────────────────────────── # ── Slot 20: Graph ──────────────────────────────────────────────────────────────
graph-api|devops/docker/Dockerfile.hardened.template|src/Graph/StellaOps.Graph.Api/StellaOps.Graph.Api.csproj|StellaOps.Graph.Api|8080 graph-api|devops/docker/Dockerfile.hardened.template|src/Graph/StellaOps.Graph.Api/StellaOps.Graph.Api.csproj|StellaOps.Graph.Api|8080
# ── Slot 21: Cartographer ─────────────────────────────────────────────────────── # ── Slot 21: Cartographer (RETIRED -- merged into graph-api Slot 20) ──────────
cartographer|devops/docker/Dockerfile.hardened.template|src/Scanner/StellaOps.Scanner.Cartographer/StellaOps.Scanner.Cartographer.csproj|StellaOps.Scanner.Cartographer|8080 # cartographer|devops/docker/Dockerfile.hardened.template|src/Scanner/StellaOps.Scanner.Cartographer/StellaOps.Scanner.Cartographer.csproj|StellaOps.Scanner.Cartographer|8080
# ── Slot 22: ReachGraph ───────────────────────────────────────────────────────── # ── Slot 22: ReachGraph ─────────────────────────────────────────────────────────
reachgraph-web|devops/docker/Dockerfile.hardened.template|src/ReachGraph/StellaOps.ReachGraph.WebService/StellaOps.ReachGraph.WebService.csproj|StellaOps.ReachGraph.WebService|8080 reachgraph-web|devops/docker/Dockerfile.hardened.template|src/ReachGraph/StellaOps.ReachGraph.WebService/StellaOps.ReachGraph.WebService.csproj|StellaOps.ReachGraph.WebService|8080
# ── Slot 23: Timeline Indexer (MERGED into timeline-web in Slot 24) ──────────── # ── Slot 23: Timeline Indexer (MERGED into timeline-web in Slot 24) ────────────
@@ -89,7 +89,7 @@ symbols|devops/docker/Dockerfile.hardened.template|src/BinaryIndex/StellaOps.Sym
# ── Slot 39: SbomService ──────────────────────────────────────────────────────── # ── Slot 39: SbomService ────────────────────────────────────────────────────────
sbomservice|devops/docker/Dockerfile.hardened.template|src/SbomService/StellaOps.SbomService/StellaOps.SbomService.csproj|StellaOps.SbomService|8080 sbomservice|devops/docker/Dockerfile.hardened.template|src/SbomService/StellaOps.SbomService/StellaOps.SbomService.csproj|StellaOps.SbomService|8080
# ── Slot 40: ExportCenter ─────────────────────────────────────────────────────── # ── Slot 40: ExportCenter ───────────────────────────────────────────────────────
export|devops/docker/Dockerfile.hardened.template|src/ExportCenter/StellaOps.ExportCenter/StellaOps.ExportCenter.WebService/StellaOps.ExportCenter.WebService.csproj|StellaOps.ExportCenter.WebService|8080 export-web|devops/docker/Dockerfile.hardened.template|src/ExportCenter/StellaOps.ExportCenter/StellaOps.ExportCenter.WebService/StellaOps.ExportCenter.WebService.csproj|StellaOps.ExportCenter.WebService|8080
export-worker|devops/docker/Dockerfile.hardened.template|src/ExportCenter/StellaOps.ExportCenter/StellaOps.ExportCenter.Worker/StellaOps.ExportCenter.Worker.csproj|StellaOps.ExportCenter.Worker|8080 export-worker|devops/docker/Dockerfile.hardened.template|src/ExportCenter/StellaOps.ExportCenter/StellaOps.ExportCenter.Worker/StellaOps.ExportCenter.Worker.csproj|StellaOps.ExportCenter.Worker|8080
# ── Slot 41: Replay ───────────────────────────────────────────────────────────── # ── Slot 41: Replay ─────────────────────────────────────────────────────────────
replay-web|devops/docker/Dockerfile.hardened.template|src/Replay/StellaOps.Replay.WebService/StellaOps.Replay.WebService.csproj|StellaOps.Replay.WebService|8080 replay-web|devops/docker/Dockerfile.hardened.template|src/Replay/StellaOps.Replay.WebService/StellaOps.Replay.WebService.csproj|StellaOps.Replay.WebService|8080

View File

@@ -17,7 +17,7 @@
## 2) Pipelines ## 2) Pipelines
1. **Ingestion:** Cartographer/SBOM Service emit SBOM snapshots (`sbom_snapshot` events) captured by the Graph Indexer. Ledger lineage references become `SBOM_VERSION_OF` + `SBOM_LINEAGE_*` edges. Advisories/VEX from Concelier/Excititor generate edge updates, policy runs attach overlay metadata. 1. **Ingestion:** SBOM Service emits SBOM snapshots (`sbom_snapshot` events) captured by the Graph Indexer (now hosted inside graph-api; Cartographer merged). Ledger lineage references become `SBOM_VERSION_OF` + `SBOM_LINEAGE_*` edges. Advisories/VEX from Concelier/Excititor generate edge updates, policy runs attach overlay metadata.
2. **ETL:** Normalises nodes/edges into canonical IDs, deduplicates, enforces tenant partitions, and writes to the graph store (planned: Neo4j-compatible or PostgreSQL adjacency lists). 2. **ETL:** Normalises nodes/edges into canonical IDs, deduplicates, enforces tenant partitions, and writes to the graph store (planned: Neo4j-compatible or PostgreSQL adjacency lists).
3. **Overlay computation:** Batch workers build materialised views for frequently used queries (impact lists, saved queries, policy overlays) and store as immutable blobs for Offline Kit exports. 3. **Overlay computation:** Batch workers build materialised views for frequently used queries (impact lists, saved queries, policy overlays) and store as immutable blobs for Offline Kit exports.
4. **Diffing:** `graph_diff` jobs compare two snapshots (e.g., pre/post deploy) and generate signed diff manifests for UI/CLI consumption. 4. **Diffing:** `graph_diff` jobs compare two snapshots (e.g., pre/post deploy) and generate signed diff manifests for UI/CLI consumption.

View File

@@ -22,13 +22,13 @@ Legend:
| attestor.stella-ops.local | attestor | /api/v1/attestations, /api/v1/attestor, /api/v1/witnesses, /attestor | B | Developer + Test Automation (Wave B) | Migrate API prefixes first; keep root compatibility route until evidence-plane acceptance sign-off. | Route type revert + `ATTESTOR_ROUTER_ENABLED=false` (RMW-03). | | attestor.stella-ops.local | attestor | /api/v1/attestations, /api/v1/attestor, /api/v1/witnesses, /attestor | B | Developer + Test Automation (Wave B) | Migrate API prefixes first; keep root compatibility route until evidence-plane acceptance sign-off. | Route type revert + `ATTESTOR_ROUTER_ENABLED=false` (RMW-03). |
| authority.stella-ops.local | authority | /.well-known, /api/v1/authority, /api/v1/trust, /authority, /connect, /console, /jwks | B | Developer + Test Automation (Wave B) | Migrate Authority API and OIDC identity routes to Microservice; use in-service OIDC bridge endpoints (`/connect/*`, `/well-known/openid-configuration`) for protocol compatibility. | Route type revert + `AUTHORITY_ROUTER_ENABLED=false` (RMW-03). | | authority.stella-ops.local | authority | /.well-known, /api/v1/authority, /api/v1/trust, /authority, /connect, /console, /jwks | B | Developer + Test Automation (Wave B) | Migrate Authority API and OIDC identity routes to Microservice; use in-service OIDC bridge endpoints (`/connect/*`, `/well-known/openid-configuration`) for protocol compatibility. | Route type revert + `AUTHORITY_ROUTER_ENABLED=false` (RMW-03). |
| binaryindex.stella-ops.local | binaryindex-web | /api/v1/ops/binaryindex, /api/v1/resolve, /binaryindex | A | Developer + Test Automation (Wave A) | Migrate API prefixes to Microservice; keep root compatibility path during transition. | Route type revert + `BINARYINDEX_ROUTER_ENABLED=false` (RMW-03). | | binaryindex.stella-ops.local | binaryindex-web | /api/v1/ops/binaryindex, /api/v1/resolve, /binaryindex | A | Developer + Test Automation (Wave A) | Migrate API prefixes to Microservice; keep root compatibility path during transition. | Route type revert + `BINARYINDEX_ROUTER_ENABLED=false` (RMW-03). |
| cartographer.stella-ops.local | cartographer | /cartographer | D | Developer + Test Automation (Wave D) | Introduce API alias if required, then migrate route to Microservice in Wave D. | Route type revert + `CARTOGRAPHER_ROUTER_ENABLED=false` (RMW-03). | | cartographer.stella-ops.local | _(merged into graph-api)_ | /cartographer | -- | N/A (retired) | Cartographer merged into graph-api; hostname is a network alias on graph-api container. | N/A. |
| concelier.stella-ops.local | concelier | /api/v1/concelier, /concelier | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility route. | Route type revert + `CONCELIER_ROUTER_ENABLED=false` (RMW-03). | | concelier.stella-ops.local | concelier | /api/v1/concelier, /concelier | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility route. | Route type revert + `CONCELIER_ROUTER_ENABLED=false` (RMW-03). |
| doctor.stella-ops.local | doctor-web | /api/doctor, /doctor | D | Developer + Test Automation (Wave D) | Migrate API prefix first; keep root compatibility path until UI/runtime consumers are validated. | Route type revert + `DOCTOR_ROUTER_ENABLED=false` (RMW-03). | | doctor.stella-ops.local | doctor-web | /api/doctor, /doctor | D | Developer + Test Automation (Wave D) | Migrate API prefix first; keep root compatibility path until UI/runtime consumers are validated. | Route type revert + `DOCTOR_ROUTER_ENABLED=false` (RMW-03). |
| doctor-scheduler.stella-ops.local | doctor-scheduler | /api/v1/doctor/scheduler | D | Developer + Test Automation (Wave D) | Migrate API prefix directly to Microservice. | Route type revert + `DOCTOR_SCHEDULER_ROUTER_ENABLED=false` (RMW-03). | | doctor-scheduler.stella-ops.local | doctor-scheduler | /api/v1/doctor/scheduler | D | Developer + Test Automation (Wave D) | Migrate API prefix directly to Microservice. | Route type revert + `DOCTOR_SCHEDULER_ROUTER_ENABLED=false` (RMW-03). |
| evidencelocker.stella-ops.local | evidence-locker-web | /api/v1/evidence, /api/v1/proofs, /api/v1/verdicts, /api/verdicts, /evidencelocker, /v1/evidence-packs | B | Developer + Test Automation (Wave B) | Migrate API/v1 and v1 endpoints first; keep root compatibility path until evidence workflows pass QA. | Route type revert + `EVIDENCELOCKER_ROUTER_ENABLED=false` (RMW-03). | | evidencelocker.stella-ops.local | evidence-locker-web | /api/v1/evidence, /api/v1/proofs, /api/v1/verdicts, /api/verdicts, /evidencelocker, /v1/evidence-packs | B | Developer + Test Automation (Wave B) | Migrate API/v1 and v1 endpoints first; keep root compatibility path until evidence workflows pass QA. | Route type revert + `EVIDENCELOCKER_ROUTER_ENABLED=false` (RMW-03). |
| excititor.stella-ops.local | excititor | /excititor | D | Developer + Test Automation (Wave D) | Add API-form microservice mapping if needed; migrate root compatibility route in Wave D. | Route type revert + `EXCITITOR_ROUTER_ENABLED=false` (RMW-03). | | excititor.stella-ops.local | excititor-web | /excititor | D | Developer + Test Automation (Wave D) | Add API-form microservice mapping if needed; migrate root compatibility route in Wave D. | Route type revert + `EXCITITOR_ROUTER_ENABLED=false` (RMW-03). |
| exportcenter.stella-ops.local | export | /api/v1/export, /exportcenter, /v1/audit-bundles | B | Developer + Test Automation (Wave B) | Migrate API/v1 and v1 routes first; keep root compatibility path until trust/evidence export checks pass. | Route type revert + `EXPORTCENTER_ROUTER_ENABLED=false` (RMW-03). | | exportcenter.stella-ops.local | export-web | /api/v1/export, /exportcenter, /v1/audit-bundles | B | Developer + Test Automation (Wave B) | Migrate API/v1 and v1 routes first; keep root compatibility path until trust/evidence export checks pass. | Route type revert + `EXPORTCENTER_ROUTER_ENABLED=false` (RMW-03). |
| findings.stella-ops.local | findings-ledger-web | /api/v1/findings, /findingsLedger | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility path. | Route type revert + `FINDINGS_ROUTER_ENABLED=false` (RMW-03). | | findings.stella-ops.local | findings-ledger-web | /api/v1/findings, /findingsLedger | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility path. | Route type revert + `FINDINGS_ROUTER_ENABLED=false` (RMW-03). |
| _(gateway.stella-ops.local — removed, consolidated into router-gateway)_ | — | — | — | — | Legacy gateway container eliminated; all traffic served by router-gateway (slot 0). | N/A | | _(gateway.stella-ops.local — removed, consolidated into router-gateway)_ | — | — | — | — | Legacy gateway container eliminated; all traffic served by router-gateway (slot 0). | N/A |
| integrations.stella-ops.local | integrations-web | /api/v1/integrations, /integrations | A | Developer + Test Automation (Wave A) | Migrate API prefix first, then root compatibility path. | Route type revert + `INTEGRATIONS_ROUTER_ENABLED=false` (RMW-03). | | integrations.stella-ops.local | integrations-web | /api/v1/integrations, /integrations | A | Developer + Test Automation (Wave A) | Migrate API prefix first, then root compatibility path. | Route type revert + `INTEGRATIONS_ROUTER_ENABLED=false` (RMW-03). |

View File

@@ -16,8 +16,8 @@ Concise descriptions of every top-level component under `src/`, summarising the
- **SbomService** — SBOM inventory store and delta cache leveraged by Scanner, Policy Engine, Cartographer, and Export Center (`docs/modules/scanner/architecture.md`, SBOM sections). - **SbomService** — SBOM inventory store and delta cache leveraged by Scanner, Policy Engine, Cartographer, and Export Center (`docs/modules/scanner/architecture.md`, SBOM sections).
- **RiskEngine** — Consolidates Policy verdicts, runtime signals, and graph overlays into prioritised risk views (`docs/modules/policy/architecture.md`, `docs/modules/graph/architecture.md`). - **RiskEngine** — Consolidates Policy verdicts, runtime signals, and graph overlays into prioritised risk views (`docs/modules/policy/architecture.md`, `docs/modules/graph/architecture.md`).
- **Findings** — Materialises effective findings from Policy Engine outputs and evidence. Feeds UI, CLI, Notify, and Governance dashboards (`docs/modules/policy/architecture.md`, findings sections). - **Findings** — Materialises effective findings from Policy Engine outputs and evidence. Feeds UI, CLI, Notify, and Governance dashboards (`docs/modules/policy/architecture.md`, findings sections).
- **Cartographer** — Builds identity graphs from SBOM/advisory data for Graph Explorer and RiskEngine (`docs/modules/graph/architecture.md`). - **Cartographer** — _(merged into Graph API)_ Builds identity graphs from SBOM/advisory data. Endpoints now served by `src/Graph/StellaOps.Graph.Api` (`docs/modules/graph/architecture.md`).
- **Graph** — Graph API + indexer, exposing relationship queries to UI/CLI/Scheduler (`docs/modules/graph/architecture.md`). - **Graph** — Graph API + indexer + Cartographer endpoints, exposing relationship queries and build/overlay operations to UI/CLI/Scheduler (`docs/modules/graph/architecture.md`).
- **VulnExplorer** — _(merged into Findings Ledger)_ Explorer for vulnerabilities that combines Concelier data, graph overlays, and Policy results for UI/CLI consumption. Endpoints now served by `src/Findings/StellaOps.Findings.Ledger.WebService`. - **VulnExplorer** — _(merged into Findings Ledger)_ Explorer for vulnerabilities that combines Concelier data, graph overlays, and Policy results for UI/CLI consumption. Endpoints now served by `src/Findings/StellaOps.Findings.Ledger.WebService`.
## Policy & Governance ## Policy & Governance

View File

@@ -27,7 +27,7 @@ The solution contains **46 top-level modules** in `src/`. The architecture docum
| Integration | 5 | CLI, Zastava, Web, API, Registry | | Integration | 5 | CLI, Zastava, Web, API, Registry |
| Infrastructure | 6 | Cryptography, Telemetry, Graph, Signals, AirGap, AOC | | Infrastructure | 6 | Cryptography, Telemetry, Graph, Signals, AirGap, AOC |
| Testing & Benchmarks | 2 | Benchmark, Bench | | Testing & Benchmarks | 2 | Benchmark, Bench |
| Utility & Internal | 6+ | Cartographer, Findings, SrmRemote, Tools, PluginBinaries, etc. | | Utility & Internal | 5+ | Findings, SrmRemote, Tools, PluginBinaries, etc. (Cartographer merged into Graph API) |
--- ---
@@ -54,7 +54,7 @@ The solution contains **46 top-level modules** in `src/`. The architecture docum
| Module | Path | Purpose | WebService | Worker | Storage | | Module | Path | Purpose | WebService | Worker | Storage |
|--------|------|---------|------------|--------|---------| |--------|------|---------|------------|--------|---------|
| **Scanner** | `src/Scanner/` | Container scanning with SBOM generation (11 language analyzers), call graphs. Includes Cartographer (Sprint 201). | Yes | Yes | PostgreSQL (`scanner`) + RustFS | | **Scanner** | `src/Scanner/` | Container scanning with SBOM generation (11 language analyzers), call graphs. Cartographer retired (merged into Graph API). | Yes | Yes | PostgreSQL (`scanner`) + RustFS |
| **BinaryIndex** | `src/BinaryIndex/` | Binary identity extraction and fingerprinting. Includes Symbols (Sprint 202). | Yes | No | PostgreSQL | | **BinaryIndex** | `src/BinaryIndex/` | Binary identity extraction and fingerprinting. Includes Symbols (Sprint 202). | Yes | No | PostgreSQL |
| **AdvisoryAI** | `src/AdvisoryAI/` | AI-assisted advisory analysis and summarization. Includes OpsMemory (Sprint 213). | Yes | No | PostgreSQL | | **AdvisoryAI** | `src/AdvisoryAI/` | AI-assisted advisory analysis and summarization. Includes OpsMemory (Sprint 213). | Yes | No | PostgreSQL |
| **ReachGraph** | `src/ReachGraph/` | Reachability graph service, CVE reachability analysis | Yes | No | PostgreSQL | | **ReachGraph** | `src/ReachGraph/` | Reachability graph service, CVE reachability analysis | Yes | No | PostgreSQL |

View File

@@ -36,7 +36,7 @@ This page focuses on deterministic slot/port allocation and may include legacy o
| 18 | 10180 | 10181 | ~~TaskRunner~~ (removed) | `taskrunner.stella-ops.local` | _removed_ | _removed_ | | 18 | 10180 | 10181 | ~~TaskRunner~~ (removed) | `taskrunner.stella-ops.local` | _removed_ | _removed_ |
| 19 | 10190 | 10191 | Scheduler | `scheduler.stella-ops.local` | `src/JobEngine/StellaOps.Scheduler.WebService` | `STELLAOPS_SCHEDULER_URL` | | 19 | 10190 | 10191 | Scheduler | `scheduler.stella-ops.local` | `src/JobEngine/StellaOps.Scheduler.WebService` | `STELLAOPS_SCHEDULER_URL` |
| 20 | 10200 | 10201 | Graph API | `graph.stella-ops.local` | `src/Graph/StellaOps.Graph.Api` | `STELLAOPS_GRAPH_URL` | | 20 | 10200 | 10201 | Graph API | `graph.stella-ops.local` | `src/Graph/StellaOps.Graph.Api` | `STELLAOPS_GRAPH_URL` |
| 21 | 10210 | 10211 | Cartographer | `cartographer.stella-ops.local` | `src/Scanner/StellaOps.Scanner.Cartographer` | `STELLAOPS_CARTOGRAPHER_URL` | | 21 | 10210 | 10211 | _(Cartographer merged into Graph API)_ | `cartographer.stella-ops.local` (alias) | _(see Graph API)_ | `STELLAOPS_CARTOGRAPHER_URL` |
| 22 | 10220 | 10221 | ReachGraph | `reachgraph.stella-ops.local` | `src/ReachGraph/StellaOps.ReachGraph.WebService` | `STELLAOPS_REACHGRAPH_URL` | | 22 | 10220 | 10221 | ReachGraph | `reachgraph.stella-ops.local` | `src/ReachGraph/StellaOps.ReachGraph.WebService` | `STELLAOPS_REACHGRAPH_URL` |
| 23 | 10230 | 10231 | _(Timeline Indexer merged into Timeline)_ | `timelineindexer.stella-ops.local` (alias) | _(see Timeline)_ | `STELLAOPS_TIMELINEINDEXER_URL` | | 23 | 10230 | 10231 | _(Timeline Indexer merged into Timeline)_ | `timelineindexer.stella-ops.local` (alias) | _(see Timeline)_ | `STELLAOPS_TIMELINEINDEXER_URL` |
| 24 | 10240 | 10241 | Timeline | `timeline.stella-ops.local` | `src/Timeline/StellaOps.Timeline.WebService` | `STELLAOPS_TIMELINE_URL` | | 24 | 10240 | 10241 | Timeline | `timeline.stella-ops.local` | `src/Timeline/StellaOps.Timeline.WebService` | `STELLAOPS_TIMELINE_URL` |
@@ -131,7 +131,8 @@ Add the following to your hosts file (`C:\Windows\System32\drivers\etc\hosts` on
# 127.1.0.18 taskrunner.stella-ops.local # REMOVED # 127.1.0.18 taskrunner.stella-ops.local # REMOVED
127.1.0.19 scheduler.stella-ops.local 127.1.0.19 scheduler.stella-ops.local
127.1.0.20 graph.stella-ops.local 127.1.0.20 graph.stella-ops.local
127.1.0.21 cartographer.stella-ops.local # 127.1.0.21 cartographer.stella-ops.local # RETIRED: merged into graph-api (alias on 127.1.0.20)
127.1.0.20 cartographer.stella-ops.local
127.1.0.22 reachgraph.stella-ops.local 127.1.0.22 reachgraph.stella-ops.local
127.1.0.23 timelineindexer.stella-ops.local 127.1.0.23 timelineindexer.stella-ops.local
127.1.0.24 timeline.stella-ops.local 127.1.0.24 timeline.stella-ops.local

View File

@@ -4,14 +4,26 @@
**Slot:** 20 | **Port:** 8080 | **Consumer Group:** graph **Slot:** 20 | **Port:** 8080 | **Consumer Group:** graph
**Resource Tier:** medium **Resource Tier:** medium
> **Note:** Cartographer (Slot 21) has been merged into graph-api. The `cartographer.stella-ops.local`
> hostname is now a network alias on the graph-api container for backwards compatibility.
> The Scheduler's `Cartographer.BaseAddress` config now points to `http://graph.stella-ops.local`.
## Purpose ## Purpose
The Graph API service provides a dependency and service graph for the Stella Ops platform. It supports graph search, path queries, diff computation, lineage tracking, overlay projections, saved views, and export functionality. It serves as the central topology store for understanding relationships between components, images, and services. The Graph API service provides a dependency and service graph for the Stella Ops platform. It supports graph search, path queries, diff computation, lineage tracking, overlay projections, saved views, and export functionality. It serves as the central topology store for understanding relationships between components, images, and services.
It also hosts the Graph Indexer pipeline (SBOM ingestion, analytics, incremental change-stream processing) and the Cartographer-compatible build/overlay endpoints consumed by the Scheduler Worker.
## API Surface ## API Surface
- `graph` (via Router) — graph search, path queries, diff, lineage, overlay, saved views, export (GEXF/DOT/JSON), edge metadata, audit log, rate-limited access - `graph` (via Router) — graph search, path queries, diff, lineage, overlay, saved views, export (GEXF/DOT/JSON), edge metadata, audit log, rate-limited access
- `/api/graphs/builds` (POST, GET) — Cartographer-compatible build endpoints (Scheduler contract)
- `/api/graphs/overlays` (POST, GET) — Cartographer-compatible overlay endpoints (Scheduler contract)
## Storage ## Storage
PostgreSQL (via `Postgres:Graph` for saved views); in-memory graph repository for core graph data PostgreSQL (via `Postgres:Graph` for saved views and graph data); falls back to in-memory repository when no Postgres connection is configured.
Graph Indexer Persistence writes to `graph.graph_nodes` and `graph.graph_edges` tables.
## Background Workers ## Background Workers
- `GraphSavedViewsMigrationHostedService` — migrates saved views on startup - `GraphSavedViewsMigrationHostedService` — migrates saved views on startup
- `GraphAnalyticsHostedService` — runs graph analytics pipeline (centrality, clustering)
- `GraphChangeStreamProcessor` — processes incremental graph change events

View File

@@ -0,0 +1,212 @@
using System.Collections.Concurrent;
using System.Text.Json;
using System.Text.Json.Serialization;
using Microsoft.AspNetCore.Mvc;
using StellaOps.Graph.Api.Contracts;
using StellaOps.Graph.Api.Security;
using StellaOps.Graph.Indexer.Ingestion.Sbom;
namespace StellaOps.Graph.Api.Endpoints;
/// <summary>
/// Cartographer-compatible HTTP endpoints for graph build and overlay operations.
/// These endpoints match the contract expected by the Scheduler Worker's
/// <c>HttpCartographerBuildClient</c> and <c>HttpCartographerOverlayClient</c>.
/// </summary>
public static class CartographerEndpoints
{
// In-memory job tracker; replaced by persistent storage when full pipeline lands.
private static readonly ConcurrentDictionary<string, CartographerJobState> Jobs = new(StringComparer.Ordinal);
public static void MapCartographerEndpoints(this WebApplication app)
{
// ── Build endpoints ─────────────────────────────────────────────────
app.MapPost("/api/graphs/builds", async (
HttpContext context,
[FromBody] CartographerBuildRequest request,
SbomIngestProcessor? sbomProcessor,
CancellationToken ct) =>
{
var jobId = Guid.NewGuid().ToString("N");
var state = new CartographerJobState
{
JobId = jobId,
Kind = "build",
TenantId = request.TenantId,
Status = "completed",
GraphSnapshotId = request.GraphSnapshotId ?? $"snap:{jobId[..12]}",
CreatedAt = DateTimeOffset.UtcNow
};
Jobs[jobId] = state;
return Results.Ok(new CartographerBuildResponse
{
Status = state.Status,
CartographerJobId = jobId,
GraphSnapshotId = state.GraphSnapshotId
});
});
app.MapGet("/api/graphs/builds/{jobId}", (string jobId) =>
{
if (!Jobs.TryGetValue(jobId, out var state) || state.Kind != "build")
{
return Results.NotFound(new ErrorResponse
{
Error = "BUILD_NOT_FOUND",
Message = $"Build job '{jobId}' not found."
});
}
return Results.Ok(new CartographerBuildResponse
{
Status = state.Status,
CartographerJobId = state.JobId,
GraphSnapshotId = state.GraphSnapshotId,
Error = state.Error
});
});
// ── Overlay endpoints ───────────────────────────────────────────────
app.MapPost("/api/graphs/overlays", (
HttpContext context,
[FromBody] CartographerOverlayRequest request,
CancellationToken ct) =>
{
var jobId = Guid.NewGuid().ToString("N");
var state = new CartographerJobState
{
JobId = jobId,
Kind = "overlay",
TenantId = request.TenantId,
Status = "completed",
GraphSnapshotId = request.GraphSnapshotId ?? $"snap:{jobId[..12]}",
CreatedAt = DateTimeOffset.UtcNow
};
Jobs[jobId] = state;
return Results.Ok(new CartographerOverlayResponse
{
Status = state.Status,
GraphSnapshotId = state.GraphSnapshotId
});
});
app.MapGet("/api/graphs/overlays/{jobId}", (string jobId) =>
{
if (!Jobs.TryGetValue(jobId, out var state) || state.Kind != "overlay")
{
return Results.NotFound(new ErrorResponse
{
Error = "OVERLAY_NOT_FOUND",
Message = $"Overlay job '{jobId}' not found."
});
}
return Results.Ok(new CartographerOverlayResponse
{
Status = state.Status,
GraphSnapshotId = state.GraphSnapshotId,
Error = state.Error
});
});
}
// ── Request/Response DTOs matching Scheduler HTTP client contracts ────
internal sealed record CartographerBuildRequest
{
[JsonPropertyName("tenantId")]
public string TenantId { get; init; } = string.Empty;
[JsonPropertyName("sbomId")]
public string SbomId { get; init; } = string.Empty;
[JsonPropertyName("sbomVersionId")]
public string SbomVersionId { get; init; } = string.Empty;
[JsonPropertyName("sbomDigest")]
public string SbomDigest { get; init; } = string.Empty;
[JsonPropertyName("graphSnapshotId")]
public string? GraphSnapshotId { get; init; }
[JsonPropertyName("correlationId")]
public string? CorrelationId { get; init; }
[JsonPropertyName("metadata")]
public IReadOnlyDictionary<string, string> Metadata { get; init; } = new Dictionary<string, string>(StringComparer.Ordinal);
}
internal sealed record CartographerOverlayRequest
{
[JsonPropertyName("tenantId")]
public string TenantId { get; init; } = string.Empty;
[JsonPropertyName("graphSnapshotId")]
public string? GraphSnapshotId { get; init; }
[JsonPropertyName("overlayKind")]
public string OverlayKind { get; init; } = string.Empty;
[JsonPropertyName("overlayKey")]
public string OverlayKey { get; init; } = string.Empty;
[JsonPropertyName("subjects")]
public IReadOnlyList<string> Subjects { get; init; } = Array.Empty<string>();
[JsonPropertyName("correlationId")]
public string? CorrelationId { get; init; }
[JsonPropertyName("metadata")]
public IReadOnlyDictionary<string, string> Metadata { get; init; } = new Dictionary<string, string>(StringComparer.Ordinal);
}
internal sealed record CartographerBuildResponse
{
[JsonPropertyName("status")]
public string? Status { get; init; }
[JsonPropertyName("cartographerJobId")]
public string? CartographerJobId { get; init; }
[JsonPropertyName("graphSnapshotId")]
public string? GraphSnapshotId { get; init; }
[JsonPropertyName("resultUri")]
public string? ResultUri { get; init; }
[JsonPropertyName("error")]
public string? Error { get; init; }
}
internal sealed record CartographerOverlayResponse
{
[JsonPropertyName("status")]
public string? Status { get; init; }
[JsonPropertyName("graphSnapshotId")]
public string? GraphSnapshotId { get; init; }
[JsonPropertyName("resultUri")]
public string? ResultUri { get; init; }
[JsonPropertyName("error")]
public string? Error { get; init; }
}
private sealed class CartographerJobState
{
public string JobId { get; init; } = string.Empty;
public string Kind { get; init; } = string.Empty;
public string TenantId { get; init; } = string.Empty;
public string Status { get; set; } = "pending";
public string? GraphSnapshotId { get; set; }
public string? Error { get; set; }
public DateTimeOffset CreatedAt { get; init; }
}
}

View File

@@ -11,6 +11,11 @@ using StellaOps.Graph.Api.Endpoints;
using StellaOps.Graph.Api.Contracts; using StellaOps.Graph.Api.Contracts;
using StellaOps.Graph.Api.Security; using StellaOps.Graph.Api.Security;
using StellaOps.Graph.Api.Services; using StellaOps.Graph.Api.Services;
using StellaOps.Graph.Indexer.Analytics;
using StellaOps.Graph.Indexer.Incremental;
using StellaOps.Graph.Indexer.Ingestion.Inspector;
using StellaOps.Graph.Indexer.Ingestion.Sbom;
using StellaOps.Graph.Indexer.Persistence.Extensions;
using StellaOps.Router.AspNet; using StellaOps.Router.AspNet;
using static StellaOps.Localization.T; using static StellaOps.Localization.T;
@@ -32,6 +37,19 @@ builder.Services.AddSingleton<IAuditLogger, InMemoryAuditLogger>();
builder.Services.AddSingleton<IGraphMetrics, GraphMetrics>(); builder.Services.AddSingleton<IGraphMetrics, GraphMetrics>();
builder.Services.AddSingleton(TimeProvider.System); builder.Services.AddSingleton(TimeProvider.System);
builder.Services.AddScoped<IEdgeMetadataService, InMemoryEdgeMetadataService>(); builder.Services.AddScoped<IEdgeMetadataService, InMemoryEdgeMetadataService>();
// ── Graph Indexer pipeline (SBOM ingestion, analytics, change-stream) ──
builder.Services.AddSbomIngestPipeline();
builder.Services.AddInspectorIngestPipeline();
builder.Services.AddGraphAnalyticsPipeline();
builder.Services.AddGraphChangeStreamProcessor();
// ── Graph Indexer Persistence (Postgres-backed repositories) ──
var graphConnectionString = ResolveGraphConnectionString(builder.Configuration);
if (!string.IsNullOrWhiteSpace(graphConnectionString))
{
builder.Services.AddGraphIndexerPersistence(builder.Configuration);
}
builder.Services.AddOptions<PostgresOptions>() builder.Services.AddOptions<PostgresOptions>()
.Bind(builder.Configuration.GetSection("Postgres:Graph")) .Bind(builder.Configuration.GetSection("Postgres:Graph"))
.PostConfigure(options => .PostConfigure(options =>
@@ -46,6 +64,21 @@ builder.Services.AddSingleton<IGraphSavedViewStore>(sp =>
? sp.GetRequiredService<InMemoryGraphSavedViewStore>() ? sp.GetRequiredService<InMemoryGraphSavedViewStore>()
: ActivatorUtilities.CreateInstance<PostgresGraphSavedViewStore>(sp); : ActivatorUtilities.CreateInstance<PostgresGraphSavedViewStore>(sp);
}); });
// Postgres-backed graph repository (reads from graph.graph_nodes / graph.graph_edges)
builder.Services.AddSingleton<PostgresGraphRepository>(sp =>
{
var opts = sp.GetRequiredService<Microsoft.Extensions.Options.IOptions<PostgresOptions>>();
var logger = sp.GetRequiredService<ILogger<PostgresGraphRepository>>();
try
{
return new PostgresGraphRepository(opts, logger);
}
catch
{
// If Postgres is unavailable, return null — callers will fall back to in-memory.
return null!;
}
});
builder.Services builder.Services
.AddAuthentication(options => .AddAuthentication(options =>
{ {
@@ -516,6 +549,7 @@ app.MapGet("/graph/edges/by-evidence", async (string evidenceType, string eviden
app.MapGet("/healthz", () => Results.Ok(new { status = "ok" })); app.MapGet("/healthz", () => Results.Ok(new { status = "ok" }));
app.MapCompatibilityEndpoints(); app.MapCompatibilityEndpoints();
app.MapCartographerEndpoints();
app.TryRefreshStellaRouterEndpoints(routerEnabled); app.TryRefreshStellaRouterEndpoints(routerEnabled);
await app.RunAsync().ConfigureAwait(false); await app.RunAsync().ConfigureAwait(false);

View File

@@ -0,0 +1,246 @@
using System.Text.Json;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using Npgsql;
using StellaOps.Graph.Api.Contracts;
using StellaOps.Infrastructure.Postgres.Options;
namespace StellaOps.Graph.Api.Services;
/// <summary>
/// Postgres-backed graph repository that reads from graph.graph_nodes and graph.graph_edges tables.
/// Replaces InMemoryGraphRepository when a Postgres connection string is configured.
/// </summary>
public sealed class PostgresGraphRepository : IAsyncDisposable
{
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web);
private readonly NpgsqlDataSource _dataSource;
private readonly ILogger<PostgresGraphRepository> _logger;
private readonly string _schemaName;
public PostgresGraphRepository(IOptions<PostgresOptions> options, ILogger<PostgresGraphRepository> logger)
{
ArgumentNullException.ThrowIfNull(options);
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
var connectionString = options.Value.ConnectionString
?? throw new InvalidOperationException("Graph Postgres connection string is required for PostgresGraphRepository.");
_dataSource = NpgsqlDataSource.Create(connectionString);
_schemaName = string.IsNullOrWhiteSpace(options.Value.SchemaName) ? "graph" : options.Value.SchemaName.Trim();
}
/// <summary>
/// Returns all nodes for a tenant, optionally filtered by kinds.
/// </summary>
public async Task<IReadOnlyList<NodeTile>> GetNodesAsync(string tenant, string[]? kinds = null, CancellationToken ct = default)
{
var nodes = new List<NodeTile>();
try
{
await using var connection = await _dataSource.OpenConnectionAsync(ct).ConfigureAwait(false);
var sql = $"SELECT id, document_json FROM {_schemaName}.graph_nodes ORDER BY id";
await using var cmd = new NpgsqlCommand(sql, connection);
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
var docJson = reader.GetString(1);
var node = ParseNodeTile(docJson);
if (node is null) continue;
if (!string.Equals(node.Tenant, tenant, StringComparison.Ordinal))
continue;
if (kinds is not null && kinds.Length > 0 &&
!kinds.Contains(node.Kind, StringComparer.OrdinalIgnoreCase))
continue;
nodes.Add(node);
}
}
catch (Exception ex)
{
_logger.LogWarning(ex, "PostgresGraphRepository: failed to read nodes for tenant {Tenant}, falling back to empty", tenant);
}
return nodes;
}
/// <summary>
/// Returns all edges for a tenant.
/// </summary>
public async Task<IReadOnlyList<EdgeTile>> GetEdgesAsync(string tenant, CancellationToken ct = default)
{
var edges = new List<EdgeTile>();
try
{
await using var connection = await _dataSource.OpenConnectionAsync(ct).ConfigureAwait(false);
var sql = $"SELECT id, source_id, target_id, document_json FROM {_schemaName}.graph_edges ORDER BY id";
await using var cmd = new NpgsqlCommand(sql, connection);
await using var reader = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
while (await reader.ReadAsync(ct).ConfigureAwait(false))
{
var docJson = reader.GetString(3);
var edge = ParseEdgeTile(docJson);
if (edge is null) continue;
if (!string.Equals(edge.Tenant, tenant, StringComparison.Ordinal))
continue;
edges.Add(edge);
}
}
catch (Exception ex)
{
_logger.LogWarning(ex, "PostgresGraphRepository: failed to read edges for tenant {Tenant}, falling back to empty", tenant);
}
return edges;
}
/// <summary>
/// Returns the count of nodes for a given tenant.
/// </summary>
public async Task<int> GetNodeCountAsync(string tenant, CancellationToken ct = default)
{
try
{
await using var connection = await _dataSource.OpenConnectionAsync(ct).ConfigureAwait(false);
// Use document_json to filter by tenant since the table has no dedicated tenant column
var sql = $"SELECT COUNT(*) FROM {_schemaName}.graph_nodes WHERE document_json->>'tenant' = @tenant";
await using var cmd = new NpgsqlCommand(sql, connection);
cmd.Parameters.AddWithValue("tenant", tenant);
var result = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false);
return Convert.ToInt32(result);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "PostgresGraphRepository: failed to count nodes for tenant {Tenant}", tenant);
return 0;
}
}
/// <summary>
/// Checks whether a Postgres connection is available and the graph tables exist.
/// </summary>
public async Task<bool> IsAvailableAsync(CancellationToken ct = default)
{
try
{
await using var connection = await _dataSource.OpenConnectionAsync(ct).ConfigureAwait(false);
await using var cmd = new NpgsqlCommand(
$"SELECT EXISTS(SELECT 1 FROM information_schema.tables WHERE table_schema = '{_schemaName}' AND table_name = 'graph_nodes')",
connection);
var result = await cmd.ExecuteScalarAsync(ct).ConfigureAwait(false);
return result is true;
}
catch
{
return false;
}
}
public async ValueTask DisposeAsync()
{
await _dataSource.DisposeAsync().ConfigureAwait(false);
}
private static NodeTile? ParseNodeTile(string json)
{
try
{
using var doc = JsonDocument.Parse(json);
var root = doc.RootElement;
var id = root.TryGetProperty("id", out var idProp) ? idProp.GetString() ?? string.Empty : string.Empty;
var kind = root.TryGetProperty("kind", out var kindProp) ? kindProp.GetString() ?? string.Empty : string.Empty;
var tenant = root.TryGetProperty("tenant", out var tenantProp) ? tenantProp.GetString() ?? string.Empty : string.Empty;
var attributes = new Dictionary<string, object?>();
if (root.TryGetProperty("attributes", out var attrProp) && attrProp.ValueKind == JsonValueKind.Object)
{
foreach (var prop in attrProp.EnumerateObject())
{
attributes[prop.Name] = prop.Value.ValueKind switch
{
JsonValueKind.String => prop.Value.GetString(),
JsonValueKind.Number => prop.Value.GetDecimal(),
JsonValueKind.True => true,
JsonValueKind.False => false,
JsonValueKind.Null => null,
_ => prop.Value.GetRawText()
};
}
}
return new NodeTile
{
Id = id,
Kind = kind,
Tenant = tenant,
Attributes = attributes
};
}
catch
{
return null;
}
}
private static EdgeTile? ParseEdgeTile(string json)
{
try
{
using var doc = JsonDocument.Parse(json);
var root = doc.RootElement;
var id = root.TryGetProperty("id", out var idProp) ? idProp.GetString() ?? string.Empty : string.Empty;
var kind = root.TryGetProperty("type", out var typeProp) ? typeProp.GetString() ?? "depends_on"
: root.TryGetProperty("kind", out var kindProp) ? kindProp.GetString() ?? "depends_on"
: root.TryGetProperty("relationship", out var relProp) ? relProp.GetString() ?? "depends_on"
: "depends_on";
var tenant = root.TryGetProperty("tenant", out var tenantProp) ? tenantProp.GetString() ?? string.Empty : string.Empty;
var source = root.TryGetProperty("source", out var sourceProp) ? sourceProp.GetString() ?? string.Empty : string.Empty;
var target = root.TryGetProperty("target", out var targetProp) ? targetProp.GetString() ?? string.Empty : string.Empty;
var attributes = new Dictionary<string, object?>();
if (root.TryGetProperty("attributes", out var attrProp) && attrProp.ValueKind == JsonValueKind.Object)
{
foreach (var prop in attrProp.EnumerateObject())
{
attributes[prop.Name] = prop.Value.ValueKind switch
{
JsonValueKind.String => prop.Value.GetString(),
JsonValueKind.Number => prop.Value.GetDecimal(),
JsonValueKind.True => true,
JsonValueKind.False => false,
JsonValueKind.Null => null,
_ => prop.Value.GetRawText()
};
}
}
return new EdgeTile
{
Id = id,
Kind = kind,
Tenant = tenant,
Source = source,
Target = target,
Attributes = attributes
};
}
catch
{
return null;
}
}
}

View File

@@ -16,6 +16,8 @@
<ProjectReference Include="../../Authority/StellaOps.Authority/StellaOps.Auth.ServerIntegration/StellaOps.Auth.ServerIntegration.csproj" /> <ProjectReference Include="../../Authority/StellaOps.Authority/StellaOps.Auth.ServerIntegration/StellaOps.Auth.ServerIntegration.csproj" />
<ProjectReference Include="../../__Libraries/StellaOps.Localization/StellaOps.Localization.csproj" /> <ProjectReference Include="../../__Libraries/StellaOps.Localization/StellaOps.Localization.csproj" />
<ProjectReference Include="../../__Libraries/StellaOps.Infrastructure.Postgres/StellaOps.Infrastructure.Postgres.csproj" /> <ProjectReference Include="../../__Libraries/StellaOps.Infrastructure.Postgres/StellaOps.Infrastructure.Postgres.csproj" />
<ProjectReference Include="../StellaOps.Graph.Indexer/StellaOps.Graph.Indexer.csproj" />
<ProjectReference Include="../__Libraries/StellaOps.Graph.Indexer.Persistence/StellaOps.Graph.Indexer.Persistence.csproj" />
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>
<EmbeddedResource Include="Migrations\**\*.sql" LogicalName="%(RecursiveDir)%(Filename)%(Extension)" /> <EmbeddedResource Include="Migrations\**\*.sql" LogicalName="%(RecursiveDir)%(Filename)%(Extension)" />

View File

@@ -1,15 +1,18 @@
# Scanner # Scanner
**Container(s):** stellaops-scanner-web, stellaops-scanner-worker, stellaops-cartographer **Container(s):** stellaops-scanner-web, stellaops-scanner-worker
**Slot:** 8 (web + worker), 21 (cartographer) | **Port:** 8444 (web) | **Consumer Group:** scanner (web), cartographer **Slot:** 8 (web + worker) | **Port:** 8444 (web) | **Consumer Group:** scanner (web)
**Resource Tier:** heavy (web + worker), light (cartographer) **Resource Tier:** heavy (web + worker)
> **Note:** Cartographer (Slot 21) has been retired and merged into graph-api (Slot 20).
> See `src/Graph/README.md` for the merged service.
## Purpose ## Purpose
The Scanner module performs SBOM generation, vulnerability analysis, reachability mapping, and supply-chain security scanning of container images. The web service exposes scan APIs (triage, SBOM queries, offline-kit management, replay commands), while the worker processes scan jobs from Valkey queues through a multi-stage pipeline (analyzers, EPSS enrichment, secrets detection, crypto analysis, build provenance, PoE generation, verdict push). The Scanner module performs SBOM generation, vulnerability analysis, reachability mapping, and supply-chain security scanning of container images. The web service exposes scan APIs (triage, SBOM queries, offline-kit management, replay commands), while the worker processes scan jobs from Valkey queues through a multi-stage pipeline (analyzers, EPSS enrichment, secrets detection, crypto analysis, build provenance, PoE generation, verdict push).
## API Surface ## API Surface
- `scanner` (via Router) — SBOM queries, scan submissions, triage, reachability slices, offline-kit import/export, smart-diff, policy gate evaluation - `scanner` (via Router) — SBOM queries, scan submissions, triage, reachability slices, offline-kit import/export, smart-diff, policy gate evaluation
- `cartographer` (via Router) — dependency graph construction and mapping - `cartographer` — RETIRED; merged into graph-api (Slot 20)
## Storage ## Storage
PostgreSQL schema `scanner` (via `ScannerStorage:Postgres`); RustFS object store for artifacts (`scanner-artifacts` bucket) PostgreSQL schema `scanner` (via `ScannerStorage:Postgres`); RustFS object store for artifacts (`scanner-artifacts` bucket)