stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
cf5b72974f923126944f4dcd47e85864ebb31a3b
git.stella-ops.org/docs/features/checked/graph/graph-indexer-incremental-update-pipeline.md
master cf5b72974f save checkpoint
2026-02-11 01:32:14 +02:00

7.1 KiB
Raw Blame History

Graph Indexer Incremental Update Pipeline

Module

Graph

Status

VERIFIED

Description

Change-stream processor for incremental graph updates, consuming SBOM/scan events and applying delta mutations to the indexed graph with idempotency tracking and backfill metrics.

Implementation Details

  • Change stream processor: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeStreamProcessor.cs -- consumes SBOM/scan events and applies delta mutations to indexed graph
  • Change event model: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeEvent.cs -- event model for graph mutations
  • Stream options: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeStreamOptions.cs -- configurable stream processing parameters
  • Idempotency store: src/Graph/StellaOps.Graph.Indexer/Incremental/InMemoryIdempotencyStore.cs -- in-memory deduplication for event processing
  • Postgres idempotency: src/Graph/__Libraries/StellaOps.Graph.Indexer.Persistence/Postgres/Repositories/PostgresIdempotencyStore.cs -- PostgreSQL-backed idempotency tracking
  • No-op event source: src/Graph/StellaOps.Graph.Indexer/Incremental/NoOpGraphChangeEventSource.cs -- stub event source for testing
  • Backfill metrics: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphBackfillMetrics.cs -- metrics for backfill operations
  • DI extensions: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeStreamServiceCollectionExtensions.cs -- DI registration
  • SBOM ingestion: src/Graph/StellaOps.Graph.Indexer/Ingestion/Sbom/SbomIngestProcessor.cs, SbomIngestTransformer.cs -- SBOM event processing and graph transformation
  • Tests: src/Graph/__Tests/StellaOps.Graph.Indexer.Tests/GraphChangeStreamProcessorTests.cs, GraphIndexerEndToEndTests.cs
  • Persistence tests: src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests/PostgresIdempotencyStoreTests.cs
  • Source: SPRINT_0141_0001_0001_graph_indexer.md

E2E Test Plan

  • Verify change stream processor applies delta mutations correctly
  • Test idempotency ensures duplicate events are not processed
  • Verify backfill metrics track progress accurately
  • Test SBOM ingestion transforms events into graph updates
  • Verify PostgreSQL idempotency store persists across restarts

Verification

  • Run ID: run-001
  • Date: 2026-02-09T16:00:00Z
  • Tier: 1 (Build + Test)
  • Result: PASS
  • Evidence: Graph.Indexer.Tests 37/37 pass. 4 PostgresIdempotencyStore tests skipped (Docker unavailable, env_issue). All source files verified (13/13).

Tier 2 Recheck (Behavioral Integration)

  • Run ID: run-002
  • Date: 2026-02-10T11:41:00Z
  • Tier: 2 (Behavioral integration verification)
  • Result: PASS
  • Evidence: Revalidated incremental update and idempotency behavior across indexer and persistence suites (Graph.Indexer.Tests 37/37 and Graph.Indexer.Persistence.Tests 17/17). Artifact: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-002/tier2-integration-check.json.

Tier 2 Recheck (Docker-Restored Persistence Replay)

  • Run ID: run-003
  • Date: 2026-02-10T16:37:52Z
  • Tier: 2 (Behavioral integration verification)
  • Result: PASS
  • Evidence: Replayed indexer + persistence suites with Docker available (Graph.Indexer.Tests 37/37, Graph.Indexer.Persistence.Tests 17/17). Artifact: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-003/tier2-integration-check.json.

Recheck (Run-005)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay.
  • Tests: PASS (src/Graph/__Tests/StellaOps.Graph.Indexer.Tests: 37/37; src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests: 17/17).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-005/tier2-integration-check.json
  • Outcome: Incremental update and idempotency behavior remains healthy.

Recheck (Run-006)

  • Verified: 2026-02-10
  • Method: Tier 2 replay validated via Graph API/indexer suites.
  • Tests: PASS (src/Graph/__Tests/StellaOps.Graph.Api.Tests: 66/66; src/Graph/__Tests/StellaOps.Graph.Indexer.Tests: 37/37; src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests: 17/17).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-006/tier2-integration-check.json
  • Outcome: Incremental indexing and persistence idempotency behavior remain healthy.

Recheck (Run-007)

  • Verified: 2026-02-10
  • Method: Tier 2 replay validated via Graph API/indexer suites.
  • Tests: PASS (src/Graph/__Tests/StellaOps.Graph.Api.Tests: 66/66; src/Graph/__Tests/StellaOps.Graph.Indexer.Tests: 37/37; src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests: 17/17).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-007/tier2-integration-check.json
  • Outcome: Incremental indexing and persistence idempotency behavior remain healthy.

Recheck (Run-008)

  • Verified: 2026-02-10
  • Method: Tier 2 replay validated via Graph API/indexer suites.
  • Tests: PASS (src/Graph/__Tests/StellaOps.Graph.Api.Tests: 66/66; src/Graph/__Tests/StellaOps.Graph.Indexer.Tests: 37/37; src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests: 17/17).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-008/tier2-integration-check.json
  • Outcome: Incremental indexing and persistence idempotency behavior remain healthy.

Recheck (Run-009)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay.
  • Tests: PASS (Graph.Api.Tests 66/66, Graph.Indexer.Tests 37/37, Graph.Indexer.Persistence.Tests 17/17).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-009/tier2-integration-check.json
  • Outcome: Checked Graph behavior remains healthy in continued replay.

Recheck (Run-010)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay.
  • Tests: PASS (Graph Api 66/66, Indexer 37/37, Indexer.Persistence 17/17; total 120/120).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-010/tier2-integration-check.json
  • Outcome: Checked graph behavior remains healthy in continued replay.

Recheck (Run-011)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay.
  • Tests: PASS (Graph Api 66/66, Indexer 37/37, Indexer.Persistence 17/17; total 120/120).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-011/tier2-integration-check.json
  • Outcome: Checked graph behavior remains healthy in continued replay.

Recheck (Run-012)

  • Verified: 2026-02-10
  • Method: Tier 2 replay + deterministic Graph suite replay.
  • Tests: PASS (Graph.Api 66/66, Graph.Indexer 37/37, Graph.Indexer.Persistence 17/17; total 120/120).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-012/tier2-integration-check.json
  • Outcome: Checked graph behavior remains healthy in continued replay.