Impact analyses

Build an impact analysis in Hypertune to see the impact of an experiment on multiple metrics in a single view.

Choose between:

• Frequentist analysis — shows you conversion rate uplift with confidence intervals

• Bayesian analysis — shows you the probability of each variant being the best

Both methods automatically apply adjustments for:

• Sequential testing — avoids the peeking problem, enabling you to view results at any time and ship a variant if it has a significant result

• Multiple comparisons — adjusts for the family-wise error rate, enabling you to compare more than two variants (in a multi-arm test), to determine an overall winner