stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
f709d519ec0db5d2a767ca4214ae6c33b0cd84b1
git.stella-ops.org/docs/features/checked/releaseorchestrator/a-b-testing-experiment-engine.md
master e9aeadc040 save checkpoint
2026-02-14 09:11:48 +02:00

1.9 KiB
Raw Blame History

A/B Testing Experiment Engine

Module

ReleaseOrchestrator

Status

VERIFIED

Description

A/B testing experiment engine with deterministic variant assignment, p-value statistical analysis, and experiment lifecycle management for controlled rollouts.

Implementation Details

  • Modules: src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.Progressive/, src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.ProgressiveDelivery/
  • Key Classes:
    • AbReleaseManager (src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.Progressive/AbRelease/AbReleaseManager.cs) - experiment lifecycle management with variant assignment
    • CanaryController (src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.ProgressiveDelivery/CanaryController.cs) - statistical analysis and auto-advance logic for experiments
    • TrafficManager (src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.ProgressiveDelivery/TrafficManager.cs) - manages traffic distribution between experiment variants
    • FeatureFlagBridge (src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.Progressive/FeatureFlags/FeatureFlagBridge.cs) - bridges feature flags with experiment variants
  • Source: SPRINT_20260117_035

E2E Test Plan

  • Create an experiment with control and treatment variants and verify deterministic variant assignment
  • Verify statistical analysis: collect metrics during experiment and verify p-value calculation
  • Verify experiment lifecycle: create, start, observe, conclude experiment through full lifecycle
  • Verify early stopping: trigger statistical significance threshold and verify auto-conclusion
  • Verify FeatureFlagBridge maps experiment variants to feature flag states

Verification

  • Verified: 2026-02-13T21:00:00Z
  • Method: Tier 2d integration tests
  • Result: PASS