stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
ab794e167c2759162758096872c5941d308e57b8
git.stella-ops.org/docs/features/checked/graph/graph-indexer-incremental-update-pipeline.md
master 5bca406787 save checkpoint: save features
2026-02-12 10:27:23 +02:00

8.0 KiB
Raw Blame History

Graph Indexer Incremental Update Pipeline

Module

Graph

Status

VERIFIED

Description

Change-stream processor for incremental graph updates, consuming SBOM/scan events and applying delta mutations to the indexed graph with idempotency tracking and backfill metrics.

Implementation Details

  • Change stream processor: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeStreamProcessor.cs -- consumes SBOM/scan events and applies delta mutations to indexed graph
  • Change event model: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeEvent.cs -- event model for graph mutations
  • Stream options: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeStreamOptions.cs -- configurable stream processing parameters
  • Idempotency store: src/Graph/StellaOps.Graph.Indexer/Incremental/InMemoryIdempotencyStore.cs -- in-memory deduplication for event processing
  • Postgres idempotency: src/Graph/__Libraries/StellaOps.Graph.Indexer.Persistence/Postgres/Repositories/PostgresIdempotencyStore.cs -- PostgreSQL-backed idempotency tracking
  • No-op event source: src/Graph/StellaOps.Graph.Indexer/Incremental/NoOpGraphChangeEventSource.cs -- stub event source for testing
  • Backfill metrics: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphBackfillMetrics.cs -- metrics for backfill operations
  • DI extensions: src/Graph/StellaOps.Graph.Indexer/Incremental/GraphChangeStreamServiceCollectionExtensions.cs -- DI registration
  • SBOM ingestion: src/Graph/StellaOps.Graph.Indexer/Ingestion/Sbom/SbomIngestProcessor.cs, SbomIngestTransformer.cs -- SBOM event processing and graph transformation
  • Tests: src/Graph/__Tests/StellaOps.Graph.Indexer.Tests/GraphChangeStreamProcessorTests.cs, GraphIndexerEndToEndTests.cs
  • Persistence tests: src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests/PostgresIdempotencyStoreTests.cs
  • Source: SPRINT_0141_0001_0001_graph_indexer.md

E2E Test Plan

  • Verify change stream processor applies delta mutations correctly
  • Test idempotency ensures duplicate events are not processed
  • Verify backfill metrics track progress accurately
  • Test SBOM ingestion transforms events into graph updates
  • Verify PostgreSQL idempotency store persists across restarts

Verification

  • Run ID: run-001
  • Date: 2026-02-09T16:00:00Z
  • Tier: 1 (Build + Test)
  • Result: PASS
  • Evidence: Graph.Indexer.Tests 37/37 pass. 4 PostgresIdempotencyStore tests skipped (Docker unavailable, env_issue). All source files verified (13/13).

Tier 2 Recheck (Behavioral Integration)

  • Run ID: run-002
  • Date: 2026-02-10T11:41:00Z
  • Tier: 2 (Behavioral integration verification)
  • Result: PASS
  • Evidence: Revalidated incremental update and idempotency behavior across indexer and persistence suites (Graph.Indexer.Tests 37/37 and Graph.Indexer.Persistence.Tests 17/17). Artifact: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-002/tier2-integration-check.json.

Tier 2 Recheck (Docker-Restored Persistence Replay)

  • Run ID: run-003
  • Date: 2026-02-10T16:37:52Z
  • Tier: 2 (Behavioral integration verification)
  • Result: PASS
  • Evidence: Replayed indexer + persistence suites with Docker available (Graph.Indexer.Tests 37/37, Graph.Indexer.Persistence.Tests 17/17). Artifact: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-003/tier2-integration-check.json.

Recheck (Run-005)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay.
  • Tests: PASS (src/Graph/__Tests/StellaOps.Graph.Indexer.Tests: 37/37; src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests: 17/17).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-005/tier2-integration-check.json
  • Outcome: Incremental update and idempotency behavior remains healthy.

Recheck (Run-006)

  • Verified: 2026-02-10
  • Method: Tier 2 replay validated via Graph API/indexer suites.
  • Tests: PASS (src/Graph/__Tests/StellaOps.Graph.Api.Tests: 66/66; src/Graph/__Tests/StellaOps.Graph.Indexer.Tests: 37/37; src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests: 17/17).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-006/tier2-integration-check.json
  • Outcome: Incremental indexing and persistence idempotency behavior remain healthy.

Recheck (Run-007)

  • Verified: 2026-02-10
  • Method: Tier 2 replay validated via Graph API/indexer suites.
  • Tests: PASS (src/Graph/__Tests/StellaOps.Graph.Api.Tests: 66/66; src/Graph/__Tests/StellaOps.Graph.Indexer.Tests: 37/37; src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests: 17/17).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-007/tier2-integration-check.json
  • Outcome: Incremental indexing and persistence idempotency behavior remain healthy.

Recheck (Run-008)

  • Verified: 2026-02-10
  • Method: Tier 2 replay validated via Graph API/indexer suites.
  • Tests: PASS (src/Graph/__Tests/StellaOps.Graph.Api.Tests: 66/66; src/Graph/__Tests/StellaOps.Graph.Indexer.Tests: 37/37; src/Graph/__Tests/StellaOps.Graph.Indexer.Persistence.Tests: 17/17).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-008/tier2-integration-check.json
  • Outcome: Incremental indexing and persistence idempotency behavior remain healthy.

Recheck (Run-009)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay.
  • Tests: PASS (Graph.Api.Tests 66/66, Graph.Indexer.Tests 37/37, Graph.Indexer.Persistence.Tests 17/17).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-009/tier2-integration-check.json
  • Outcome: Checked Graph behavior remains healthy in continued replay.

Recheck (Run-010)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay.
  • Tests: PASS (Graph Api 66/66, Indexer 37/37, Indexer.Persistence 17/17; total 120/120).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-010/tier2-integration-check.json
  • Outcome: Checked graph behavior remains healthy in continued replay.

Recheck (Run-011)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay.
  • Tests: PASS (Graph Api 66/66, Indexer 37/37, Indexer.Persistence 17/17; total 120/120).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-011/tier2-integration-check.json
  • Outcome: Checked graph behavior remains healthy in continued replay.

Recheck (Run-012)

  • Verified: 2026-02-10
  • Method: Tier 2 replay + deterministic Graph suite replay.
  • Tests: PASS (Graph.Api 66/66, Graph.Indexer 37/37, Graph.Indexer.Persistence 17/17; total 120/120).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-012/tier2-integration-check.json
  • Outcome: Checked graph behavior remains healthy in continued replay.

Recheck (Run-013)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay with fresh command-output evidence.
  • Tests: PASS (Graph matrix 120/120: Api 66, Indexer 37, Indexer.Persistence 17.)
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-013/tier2-integration-check.json
  • Outcome: Checked Graph behavior remains healthy in continued replay.

Recheck (Run-016)

  • Verified: 2026-02-11
  • Method: Strict Tier 2 API replay for incremental-update surfaces (diff, lineage, path, export) with validation negatives.
  • Tests: PASS (src/Graph/StellaOps.Graph.sln: 120/120).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/graph/graph-indexer-incremental-update-pipeline/run-016/tier2-api-check.json
  • Outcome: Fresh end-user replay verifies incremental graph updates remain externally observable through diff/lineage/export APIs.