stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
cf5b72974f923126944f4dcd47e85864ebb31a3b
git.stella-ops.org/docs/features/unchecked/releaseorchestrator/a-b-testing-experiment-engine.md

1.8 KiB
Raw Blame History

A/B Testing Experiment Engine

Module

ReleaseOrchestrator

Status

IMPLEMENTED

Description

A/B testing experiment engine with deterministic variant assignment, p-value statistical analysis, and experiment lifecycle management for controlled rollouts.

Implementation Details

  • Modules: src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.Progressive/, src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.ProgressiveDelivery/
  • Key Classes:
    • AbReleaseManager (src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.Progressive/AbRelease/AbReleaseManager.cs) - experiment lifecycle management with variant assignment
    • CanaryController (src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.ProgressiveDelivery/CanaryController.cs) - statistical analysis and auto-advance logic for experiments
    • TrafficManager (src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.ProgressiveDelivery/TrafficManager.cs) - manages traffic distribution between experiment variants
    • FeatureFlagBridge (src/ReleaseOrchestrator/__Libraries/StellaOps.ReleaseOrchestrator.Progressive/FeatureFlags/FeatureFlagBridge.cs) - bridges feature flags with experiment variants
  • Source: SPRINT_20260117_035

E2E Test Plan

  • Create an experiment with control and treatment variants and verify deterministic variant assignment
  • Verify statistical analysis: collect metrics during experiment and verify p-value calculation
  • Verify experiment lifecycle: create, start, observe, conclude experiment through full lifecycle
  • Verify early stopping: trigger statistical significance threshold and verify auto-conclusion
  • Verify FeatureFlagBridge maps experiment variants to feature flag states