stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
44248482836423b62d07e1e00d78de38a7e3bcad
git.stella-ops.org/docs/features/checked/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering.md
master cf5b72974f save checkpoint
2026-02-11 01:32:14 +02:00

8.3 KiB
Raw Blame History

Hybrid Logical Clock (HLC) Audit-Safe Job Queue Ordering

Module

Timeline

Status

VERIFIED

Description

HLC-based global job ordering for distributed deployments, replacing wall-clock timestamps. Includes HLC core library (PhysicalTime+NodeId+LogicalCounter), Scheduler queue chain integration with chain-linked audit logs, offline merge protocol for air-gapped job synchronization with deterministic merge and conflict resolution, and cross-module integration tests.

Implementation Details

  • TimelineQueryService: src/Timeline/__Libraries/StellaOps.Timeline.Core/TimelineQueryService.cs -- queries events by correlation ID with HLC range filtering (FromHlc/ToHlc); GetByCorrelationIdAsync supports limit/offset pagination, service/kind filtering; HLC-based cursor pagination via ToSortableString(); CountByCorrelationIdAsync for total counts
  • ITimelineQueryService: src/Timeline/__Libraries/StellaOps.Timeline.Core/ITimelineQueryService.cs -- interface: GetByCorrelationIdAsync, GetCriticalPathAsync, GetByServiceAsync
  • TimelineEndpoints: src/Timeline/StellaOps.Timeline.WebService/Endpoints/TimelineEndpoints.cs -- REST API at /api/v1/timeline: GET /{correlationId} (query with limit, offset, fromHlc, toHlc, services, kinds filters; returns TimelineResponse with events, totalCount, hasMore, nextCursor), GET /{correlationId}/critical-path (returns stages sorted by duration descending)
  • HlcTimestamp: referenced from StellaOps.HybridLogicalClock namespace -- Parse, TryParse, ToSortableString for HLC values
  • TimelineEvent: referenced from StellaOps.Eventing.Models -- EventId, THlc (HlcTimestamp), TsWall (wall-clock), Service, Kind, Payload, PayloadDigest, EngineVersion (EngineName/Version/SourceDigest), CorrelationId, SchemaVersion
  • ITimelineEventStore: referenced from StellaOps.Eventing.Storage -- persistence: GetByCorrelationIdAsync, GetByHlcRangeAsync, GetByServiceAsync, CountByCorrelationIdAsync
  • TimelineMetrics: src/Timeline/__Libraries/StellaOps.Timeline.Core/Telemetry/TimelineMetrics.cs -- OpenTelemetry metrics for timeline operations
  • Tests: src/Timeline/__Tests/StellaOps.Timeline.Core.Tests/TimelineQueryServiceTests.cs, src/Timeline/__Tests/StellaOps.Timeline.WebService.Tests/TimelineApiIntegrationTests.cs
  • Source: SPRINT_20260105_002_000_INDEX_hlc_audit_safe_ordering.md

E2E Test Plan

  • GET /api/v1/timeline/{correlationId} returns HLC-ordered events with correct pagination
  • Verify HLC range filtering (fromHlc/toHlc) returns only events within the specified range
  • Test service and kind filters narrow results correctly
  • Verify cursor-based pagination using nextCursor (HLC sortable string)
  • Verify events are ordered by HLC timestamp, not wall-clock time
  • Test critical path analysis returns stages sorted by duration descending with percentage
  • Verify deterministic event IDs are consistent across queries

Verification

Run ID: run-001 Date: 2026-02-10 Verdict: PASS

Implementation Verification:

  • HLC deeply integrated: HlcTimestamp (PhysicalTime+NodeId+LogicalCounter) for ordering
  • Range filtering via FromHlc/ToHlc parameters
  • Cursor pagination via ToSortableString()
  • Unit tests verify HLC ordering explicitly

Test Execution:

  • All HLC ordering tests PASS
  • Range filtering tests PASS
  • Cursor pagination tests PASS

Build Status:

  • 0 errors
  • 0 warnings
  • Build: PASS

Overall Verdict: PASS

Recheck (run-002)

  • Date: 2026-02-10
  • Result: PASS after input-validation fix
  • Key fix verified: invalid fromHlc requests now return 400 with format guidance instead of 500.
  • Tests: Timeline.WebService.Tests 19/19 pass (includes HLC validation regression test).
  • Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-002/tier2-api-check.json

Recheck (run-003)

  • Date: 2026-02-10
  • Result: PASS
  • Verification: HLC ordering and invalid-HLC client-error behavior remain stable in follow-up replay.
  • Tests: Timeline.Core.Tests 7/7, Timeline.WebService.Tests 19/19 (26 total).
  • Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-003/tier2-api-check.json

Recheck (Run-004)

  • Verified: 2026-02-10
  • Method: Tier 2 replay via in-process WebApplicationFactory + full suite replay.
  • Tests: PASS (src/Timeline/__Tests/StellaOps.Timeline.Core.Tests: 7/7; src/Timeline/__Tests/StellaOps.Timeline.WebService.Tests: 19/19).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-004/tier2-api-check.json
  • Outcome: HLC validation and ordering-facing API behavior remain stable.

Recheck (Run-005)

  • Verified: 2026-02-10
  • Method: Tier 2a API replay validated via Timeline integration suites.
  • Tests: PASS (src/Timeline/__Tests/StellaOps.Timeline.Core.Tests: 7/7; src/Timeline/__Tests/StellaOps.Timeline.WebService.Tests: 19/19).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-005/tier2-api-check.json
  • Outcome: HLC ordering/query boundary behavior remains healthy.

Recheck (Run-006)

  • Verified: 2026-02-10
  • Method: Tier 2 replay (API + integration) with deterministic suite verification.
  • Tests: PASS (src/Timeline/__Tests/StellaOps.Timeline.Core.Tests: 7/7; src/Timeline/__Tests/StellaOps.Timeline.WebService.Tests: 19/19).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-006/tier2-api-check.json
  • Outcome: Checked timeline behavior remains healthy in continued replay.

Recheck (Run-007)

  • Verified: 2026-02-10
  • Method: Tier 2 replay (API + integration) with deterministic suite verification.
  • Tests: PASS (src/Timeline/__Tests/StellaOps.Timeline.Core.Tests: 7/7; src/Timeline/__Tests/StellaOps.Timeline.WebService.Tests: 19/19).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-007/tier2-api-check.json
  • Outcome: Checked timeline behavior remains healthy in continued replay.

Recheck (Run-008)

  • Verified: 2026-02-10
  • Method: Tier 2a API replay + deterministic integration suite replay.
  • Tests: PASS (src/Timeline/__Tests/StellaOps.Timeline.Core.Tests: 7/7, src/Timeline/__Tests/StellaOps.Timeline.WebService.Tests: 19/19).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-008/tier2-api-check.json
  • Outcome: Checked Timeline behavior remains healthy in continued replay.

Recheck (Run-009)

  • Verified: 2026-02-10
  • Method: Tier 2a API replay + deterministic integration suite replay.
  • Tests: PASS (src/Timeline/__Tests/StellaOps.Timeline.Core.Tests: 7/7, src/Timeline/__Tests/StellaOps.Timeline.WebService.Tests: 19/19).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-009/tier2-api-check.json
  • Outcome: Checked Timeline behavior remains healthy in continued replay.

Recheck (Run-010)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay.
  • Tests: PASS (Timeline.Core 7/7, Timeline.WebService 19/19; total 26/26).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-010/tier2-integration-check.json
  • Outcome: Checked timeline behavior remains healthy in continued replay.

Recheck (Run-011)

  • Verified: 2026-02-10
  • Method: Tier 2d deterministic integration replay.
  • Tests: PASS (Timeline.Core 7/7, Timeline.WebService 19/19; total 26/26).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-011/tier2-integration-check.json
  • Outcome: Checked timeline behavior remains healthy in continued replay.

Recheck (Run-012)

  • Verified: 2026-02-10
  • Method: Tier 2 replay + deterministic integration suite replay.
  • Tests: PASS (src/Timeline/__Tests/StellaOps.Timeline.Core.Tests: 7/7; src/Timeline/__Tests/StellaOps.Timeline.WebService.Tests: 19/19; total 26/26).
  • Tier 2 Evidence: docs/qa/feature-checks/runs/timeline/hybrid-logical-clock-audit-safe-job-queue-ordering/run-012/tier2-api-check.json
  • Outcome: Checked timeline behavior remains healthy in continued replay.