A/B Testing Dictionary
Everything you need to know about A/B testing, conversion optimization, and statistics, from A to Z.
A
B
C
Click-Through Rate (CTR)
The ratio of users who click on a specific link to the number of total users who view the page.
Confidence Interval
A range likely to contain the true difference between variants.
Confidence Level
The threshold used to determine statistical significance.
Control Variant
The original version of your page (Variant A), used as a baseline for comparison.
Conversion Rate
The percentage of users who take a desired action (like clicking a button or making a purchase).
Credible Interval
A range where the true value has a specified probability of falling.
E
F
False Negative (Type II Error)
When you fail to detect a real difference between variants.
False Positive (Type I Error)
When you incorrectly conclude that there is a difference between variants when there is none.
Frequentist A/B Testing
Traditional hypothesis testing using p-values and confidence intervals.
M
P
S
T
Ready to run your first A/B test?
runab makes A/B testing simple. Just add one line of code.
Get started free