Back to Glossary
Glossary
Statistical Significance
A measure of how likely it is that the difference in performance between variants is real and not due to random chance.
What is Statistical Significance?
Statistical significance is the "math" that tells you if your test results are reliable. It helps you distinguish between a real improvement and a lucky streak.
In A/B testing, we usually aim for 95% significance, meaning we are 95% confident that the results are real.
Why You Can't Ignore It
If you stop a test too early just because Variant B looks like it's winning, you might be falling victim to random noise. Statistical significance filters out that noise.