stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 14 Releases Wiki Activity
Files
dd0067ea0bbd01c470be219caeccb1de03a9374b
git.stella-ops.org/docs/implplan/SPRINT_0207_0001_0001_graph.md
master 75f6942769
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Details
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Details
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Details
AOC Guard CI / aoc-guard (push) Has been cancelled
Details
AOC Guard CI / aoc-verify (push) Has been cancelled
Details
Add integration tests for migration categories and execution 
- Implemented MigrationCategoryTests to validate migration categorization for startup, release, seed, and data migrations.
- Added tests for edge cases, including null, empty, and whitespace migration names.
- Created StartupMigrationHostTests to verify the behavior of the migration host with real PostgreSQL instances using Testcontainers.
- Included tests for migration execution, schema creation, and handling of pending release migrations.
- Added SQL migration files for testing: creating a test table, adding a column, a release migration, and seeding data.
2025-12-04 19:10:54 +02:00

14 KiB
Raw Blame History

Sprint 0207-0001-0001 · Graph (Experience & SDKs 180.C)

Topic & Scope

  • Deliver graph API surface (search/query/paths/diff/export) with overlays, RBAC, and deterministic streaming tiles for Experience & SDKs stream 180.C.
  • Keep indexer snapshots aligned so ingest emits graph artifacts consumable by the API layer; retain offline/export readiness.
  • Instrument metrics/logging, budget enforcement, and job exports to match policy/overlay contracts.
  • Working directory: src/Graph/StellaOps.Graph.Api, src/Graph/StellaOps.Graph.Indexer.
  • Active items only; completed/historic work moves to docs/implplan/archived/tasks.md.

Dependencies & Concurrency

  • Upstream sprints: 120.A (AirGap), 130.A (Scanner), 150.A (Orchestrator), 170.A (Notifier) for feeds, digests, and events.
  • GRAPH-API-28-001 → 011 are sequential; do not parallelize past their stated dependencies.
  • Overlay integration (GRAPH-API-28-006) depends on POLICY-ENGINE-30-001..003 contracts staying stable.

Documentation Prerequisites

  • docs/README.md
  • docs/07_HIGH_LEVEL_ARCHITECTURE.md
  • docs/modules/platform/architecture-overview.md
  • docs/modules/graph/architecture.md
  • docs/modules/graph/implementation_plan.md
  • src/Graph/AGENTS.md

BLOCKED Tasks: Before working on BLOCKED tasks, review BLOCKED_DEPENDENCY_TREE.md for root blockers and dependencies.

Delivery Tracker

# Task ID Status Key dependency / next step Owners Task Definition
1 GRAPH-API-28-001 DONE (2025-11-24) Draft spec v0.0.3-pre published; cost + tile schema aligned. Graph API Guild (src/Graph/StellaOps.Graph.Api) Define OpenAPI + JSON schema for graph search/query/paths/diff/export endpoints, including cost metadata and streaming tile schema.
2 GRAPH-API-28-002 DONE (2025-11-25) GRAPH-API-28-001 Graph API Guild (src/Graph/StellaOps.Graph.Api) Implement /graph/search with multi-type index lookup, prefix/exact match, RBAC enforcement, and result ranking + caching.
3 GRAPH-API-28-003 DONE (2025-11-26) GRAPH-API-28-002 Graph API Guild (src/Graph/StellaOps.Graph.Api) Build query planner + cost estimator for /graph/query, stream tiles (nodes/edges/stats) progressively, enforce budgets, provide cursor tokens.
4 GRAPH-API-28-004 DONE (2025-11-26) GRAPH-API-28-003 Graph API Guild (src/Graph/StellaOps.Graph.Api) Implement /graph/paths with depth ≤6, constraint filters, heuristic shortest path search, and optional policy overlay rendering.
5 GRAPH-API-28-005 DONE (2025-11-26) GRAPH-API-28-004 Graph API Guild (src/Graph/StellaOps.Graph.Api) Implement /graph/diff streaming added/removed/changed nodes/edges between SBOM snapshots; include overlay deltas and policy/VEX/advisory metadata.
6 GRAPH-API-28-006 DONE (2025-11-26) GRAPH-API-28-005; POLICY-ENGINE-30-001..003 contracts Graph API Guild (src/Graph/StellaOps.Graph.Api) Consume Policy Engine overlay contract and surface advisory/VEX/policy overlays with caching, partial materialization, and explain trace sampling for focused nodes.
7 GRAPH-API-28-007 DONE (2025-11-26) GRAPH-API-28-006 Graph API Guild (src/Graph/StellaOps.Graph.Api) Implement exports (graphml, csv, ndjson, png, svg) with async job management, checksum manifests, and streaming downloads.
8 GRAPH-API-28-008 DONE (2025-11-26) GRAPH-API-28-007 Graph API + Authority Guilds (src/Graph/StellaOps.Graph.Api) Integrate RBAC scopes (graph:read, graph:query, graph:export), tenant headers, audit logging, and rate limiting.
9 GRAPH-API-28-009 DONE (2025-11-26) GRAPH-API-28-008 Graph API + Observability Guilds (src/Graph/StellaOps.Graph.Api) Instrument metrics (graph_tile_latency_seconds, graph_query_budget_denied_total, graph_overlay_cache_hit_ratio), structured logs, and traces per query stage; publish dashboards.
10 GRAPH-API-28-010 DONE (2025-11-26) GRAPH-API-28-009 Graph API Guild · QA Guild (src/Graph/StellaOps.Graph.Api) Build unit/integration/load tests with synthetic datasets (500k nodes/2M edges), fuzz query validation, verify determinism across runs.
11 GRAPH-API-28-011 DONE (2025-11-26) GRAPH-API-28-010 Graph API Guild (src/Graph/StellaOps.Graph.Api) Provide deployment manifests, offline kit support, API gateway integration docs, and smoke tests.
12 GRAPH-INDEX-28-011 DONE (2025-11-04) Downstream consumption by API once overlays ready Graph Indexer Guild (src/Graph/StellaOps.Graph.Indexer) Wire SBOM ingest runtime to emit graph snapshot artifacts, add DI factory helpers, and document Mongo/snapshot environment guidance.
13 GRAPH-ANALYTICS-GAPS-207-013 DONE (2025-12-02) None; informs tasks 1–12. Product Mgmt · Graph API Guild · Graph Indexer Guild Address graph analytics gaps GA1–GA10 from docs/product-advisories/31-Nov-2025 FINDINGS.md: versioned analytics schemas, deterministic seeds/rerun-hash CI, privacy/tenant redaction rules, baseline datasets/fixtures, performance budgets/quotas, explainability metadata (inputs/seeds/revision), checksum+DSSE for exports, algorithm versioning, offline analytics bundle schema, and SemVer/change-log governance.

Wave Coordination

  • Wave 1 · API surface and overlays: GRAPH-API-28-001..011 (sequential pipeline).
  • Wave 2 · Indexer readiness: GRAPH-INDEX-28-011 (completed; feeds Wave 1 runtime tests).

Wave Detail Snapshots

  • Wave 1: waiting on schema draft (GRAPH-API-28-001) to start downstream implementation; observe dependency chain.
  • Wave 2: snapshot emission ready; monitor for schema drift once Wave 1 schemas finalize.

Interlocks

  • Policy Engine overlays (POLICY-ENGINE-30-001..003) must stay in sync for GRAPH-API-28-006.
  • RBAC scopes and audit logging align with Authority module contracts; coordinate during GRAPH-API-28-008.
  • Observability dashboards to reuse shared metrics conventions from Observability Guild.

Upcoming Checkpoints

  • 2025-11-24 · Target date to circulate OpenAPI/JSON schema draft (GRAPH-API-28-001). Owner: Graph API Guild.
  • 2025-11-29 · Propose schema sign-off and budget model review before starting GRAPH-API-28-002/003; include Policy Engine overlays alignment.
  • 2025-12-03 · Overlay contract validation with Policy Engine Guild ahead of GRAPH-API-28-006.

Action Tracker

Action Owner Due (UTC) Status
Circulate initial schema/tiles draft for review (GRAPH-API-28-001). Evidence: docs/modules/graph/prep/2025-11-22-graph-api-schema-outline.md, docs/modules/graph/prep/2025-11-24-graph-api-schema-review.md, docs/api/graph-gateway-spec-draft.yaml. Graph API Guild 2025-11-24 Done
Hold joint OpenAPI review + budget model sign-off (Graph API + Policy Engine). Evidence: docs/api/graph-gateway-spec-draft.yaml review notes. Graph API Guild · Policy Engine Guild 2025-11-29 Open
Confirm POLICY-ENGINE-30-001..003 contract version for overlay consumption. Policy Engine Guild · Graph API Guild 2025-11-30 Open
Prep synthetic dataset fixtures (500k/2M) for load tests. QA Guild · Graph API Guild 2025-12-05 Open

Decisions & Risks

  • Schema and overlay contracts are prerequisites; any drift will stall downstream API tasks.
  • Export formats (GRAPH-API-28-007) require deterministic manifests to satisfy offline kit expectations.
  • Budget enforcement (GRAPH-API-28-003) risk: rejection without user-friendly explain traces could increase support load; mitigate by sampling explains early.
  • Analytics gaps closed via docs/modules/graph/analytics/GA1-GA10-analytics-plan.md + schemas; future analytics jobs must conform to analytics-result.schema.json and analytics-bundle.schema.json with rerun-hash and redaction metadata.
Risk Impact Mitigation Owner Status
Overlay contract drift vs POLICY-ENGINE-30-001..003 Blocks GRAPH-API-28-006 overlays; rework schemas; placeholder overlay payload fields in spec Freeze contract version before coding; joint review on 2025-12-03 checkpoint; update OverlayPayload.version once contract ratified Graph API Guild · Policy Engine Guild Open
Export manifest non-determinism Offline kit validation fails and retries Enforce checksum manifests + stable ordering in GRAPH-API-28-007 Graph API Guild Open
Budget enforcement lacks explain traces User confusion, support load, potential false negatives Implement sampled explain traces during GRAPH-API-28-003 and validate via QA fixtures Graph API Guild · QA Guild Open
Search stub vs real index Stubbed in-memory results may diverge from production relevance/caching Track follow-on wiring to real indexer before release; keep regression tests deterministic to catch scoring drift Graph API Guild Open

Execution Log

Date (UTC) Update Owner
2025-12-02 Completed GRAPH-ANALYTICS-GAPS-207-013: published analytics schemas/bundle sample + GA1–GA10 plan (docs/modules/graph/analytics/*), covering versioning, rerun-hash determinism, redaction, budgets, DSSE-ready manifests, and offline bundle governance. Product Mgmt
2025-11-26 GRAPH-API-28-003 completed: /graph/query NDJSON streaming covers nodes/edges/stats/cursor, budgets default to tiles=6000/nodes=5000/edges=10000, budget-exceeded tile implemented, and QueryServiceTests now pass locally. Graph API Guild
2025-12-01 Added GRAPH-ANALYTICS-GAPS-207-013 to capture GA1–GA10 remediation from 31-Nov-2025 FINDINGS.md. Product Mgmt
2025-11-26 GRAPH-API-28-004 completed: added /graph/paths NDJSON endpoint with tenant + graph:query scope guard, BFS heuristic (depth ≤6) producing node/edge/stats tiles, reuse budgets, and new PathService unit tests passing. Graph API Guild
2025-11-26 GRAPH-API-28-005 completed: /graph/diff NDJSON endpoint compares in-memory snapshots, streams node/edge added/removed/changed tiles, stats, budget enforcement, and unit tests for happy-path and missing snapshot cases now pass. Graph API Guild
2025-11-26 GRAPH-API-28-006 completed: overlay service now emits policy.overlay.v1 and openvex.v1 payloads with deterministic IDs, sampled explain trace, cache reuse, and query streaming includes overlays (QueryAsync_IncludesOverlaysAndSamplesExplainOnce test added). Graph API Guild
2025-11-26 GRAPH-API-28-007 completed: added /graph/export endpoint with in-memory job manifest, deterministic SHA256, download URL, and support for ndjson/csv/graphml/png/svg placeholders; export unit tests added. Graph API Guild
2025-11-26 GRAPH-API-28-008 completed: enforced scopes across endpoints, added fixed-window rate limiting per tenant/route, and in-memory audit logger with capped history; unit tests for rate limiter and audit logger passing. Graph API Guild
2025-11-26 GRAPH-API-28-009 completed: metrics instruments added (query latency histogram, budget-denied counter, overlay cache hit/miss counters, export latency); covered by unit tests listening via MeterListener. Graph API Guild
2025-11-26 GRAPH-API-28-010 completed: added synthetic graph load tests (deterministic builder, 1k/2k sample), deterministic ordering assertion, and fuzz validation for invalid budgets; keeps runs bounded for CI while scaffolding larger dataset path. Graph API Guild
2025-11-26 GRAPH-API-28-011 completed: added deployment manifests (Deploy/kubernetes.yaml, Deploy/docker-compose.yaml), health check doc, and /healthz endpoint; ready for offline kit packaging. Graph API Guild
2025-11-26 Test sweep: `dotnet test ... --filter "DiffServiceTests PathServiceTests
2025-11-25 GRAPH-API-28-002 completed: /graph/search NDJSON endpoint enforces tenant + scope headers, validation, cursor paging, relevance ranking, and in-memory cache; in-memory repo seeded. Tests pending due to long restore cycles. Graph API Guild
2025-11-26 GRAPH-API-28-003 in-progress snapshot: added budget caps (tiles/nodes/edges), cursor reservation, cache-key scoping, budget-exceeded error tile, and budget-focused unit tests; tests still pending at this point (see later entry for completion). Graph API Guild
2025-11-22 Normalized sprint to standard template and renamed file from SPRINT_207_graph.md to SPRINT_0207_0001_0001_graph.md; no task status changes. Project Mgmt
2025-11-22 Added module charter src/Graph/AGENTS.md to unblock implementers; no task status changes. Project Mgmt
2025-11-22 Drafted schema/tiles outline for GRAPH-API-28-001 at docs/modules/graph/prep/2025-11-22-graph-api-schema-outline.md; marked action as In progress. Project Mgmt
2025-11-22 Updated docs/api/graph-gateway-spec-draft.yaml to encode search/query/paths/diff/export endpoints, shared tile schemas, and examples; evidence for GRAPH-API-28-001; moved task to DOING. Project Mgmt
2025-11-22 Added joint OpenAPI + budget review action (due 2025-11-29) and updated checkpoints accordingly. Project Mgmt
2025-11-22 Created review notes shell at docs/modules/graph/prep/2025-11-24-graph-api-schema-review.md to capture schema sign-off outcomes. Project Mgmt
2025-11-24 GRAPH-API-28-001 completed: updated docs/api/graph-gateway-spec-draft.yaml to v0.0.3-pre with cursor/resume, overlays scaffold, rate-limit headers; action tracker item marked Done. Graph API Guild
2025-11-24 Started GRAPH-API-28-002: scaffolded StellaOps.Graph.Api host + /graph/search NDJSON endpoint with tenant/auth validation, cursor support, and in-memory index; added xUnit smoke test (SearchServiceTests). Graph API Guild
2025-11-24 Started GRAPH-API-28-002: scaffolded StellaOps.Graph.Api minimal host and /graph/search stub with NDJSON stream + tenant validation; added in-memory search service and xunit smoke test. Graph API Guild