stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
5593212b414864ab213082c8945a267451b99901
git.stella-ops.org/docs/features/checked/graph/graph-indexer-incremental-update-pipeline.md

2.6 KiB
Raw Blame History

Graph Indexer Incremental Update Pipeline

Module

Graph

Status

VERIFIED

Description

Change-stream processor for incremental graph updates, consuming SBOM/scan events and applying delta mutations to the indexed graph with idempotency tracking and backfill metrics.

Implementation Details

  • Change stream processor: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeStreamProcessor.cs -- consumes SBOM/scan events and applies delta mutations to indexed graph
  • Change event model: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeEvent.cs -- event model for graph mutations
  • Stream options: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeStreamOptions.cs -- configurable stream processing parameters
  • Idempotency store: src/Graph/StellaOps.Graph.Indexer/Incremental/InMemoryIdempotencyStore.cs -- in-memory deduplication for event processing
  • Postgres idempotency: src/Graph/__Libraries/StellaOps.Graph.Indexer.Persistence/Postgres/Repositories/PostgresIdempotencyStore.cs -- PostgreSQL-backed idempotency tracking
  • No-op event source: src/Graph/StellaOps.Graph.Indexer/Incremental/NoOpGraphChangeEventSource.cs -- stub event source for testing
  • Backfill metrics: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphBackfillMetrics.cs -- metrics for backfill operations
  • DI extensions: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeStreamServiceCollectionExtensions.cs -- DI registration
  • SBOM ingestion: src/Graph/StellaOps.Graph.Indexer/Ingestion/Sbom/SbomIngestProcessor.cs, SbomIngestTransformer.cs -- SBOM event processing and graph transformation
  • Tests: src/Graph/__Tests/StellaOps.Graph.Indexer.Tests/GraphChangeStreamProcessorTests.cs, GraphIndexerEndToEndTests.cs
  • Persistence tests: src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests/PostgresIdempotencyStoreTests.cs
  • Source: SPRINT_0141_0001_0001_graph_indexer.md

E2E Test Plan

  • Verify change stream processor applies delta mutations correctly
  • Test idempotency ensures duplicate events are not processed
  • Verify backfill metrics track progress accurately
  • Test SBOM ingestion transforms events into graph updates
  • Verify PostgreSQL idempotency store persists across restarts (skipped: Docker unavailable)

Verification

  • Run ID: run-001
  • Date: 2026-02-09T16:00:00Z
  • Tier: 1 (Build + Test)
  • Result: PASS
  • Evidence: Graph.Indexer.Tests 37/37 pass. 4 PostgresIdempotencyStore tests skipped (Docker unavailable, env_issue). All source files verified (13/13).