Glossary

A/B Testing Dictionary

Everything you need to know about A/B testing, conversion optimization, and statistics, from A to Z.

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

A

A/B Testing

The process of comparing two versions of a webpage or app to determine which one performs better.

Average Order Value (AOV)

The average amount of money each customer spends per transaction.

B

Bayesian A/B Testing

A statistical approach that calculates the probability one variant is better than another.

Bounce Rate

The percentage of visitors who leave a website after viewing only one page.

C

Click-Through Rate (CTR)

The ratio of users who click on a specific link to the number of total users who view the page.

Confidence Interval

A range likely to contain the true difference between variants.

Confidence Level

The threshold used to determine statistical significance.

Control Variant

The original version of your page (Variant A), used as a baseline for comparison.

Conversion Rate

The percentage of users who take a desired action (like clicking a button or making a purchase).

Credible Interval

A range where the true value has a specified probability of falling.

E

Expected Lift

The most likely improvement of one variant over another.

Expected Loss

The expected cost of choosing the wrong variant.

Experimentation Culture

A data-driven mindset where decisions are made based on experimental evidence rather than intuition.

F

False Negative (Type II Error)

When you fail to detect a real difference between variants.

False Positive (Type I Error)

When you incorrectly conclude that there is a difference between variants when there is none.

Frequentist A/B Testing

Traditional hypothesis testing using p-values and confidence intervals.

L

Landing Page Optimization (LPO)

The process of improving elements on a landing page to increase conversions.

M

Minimum Detectable Effect (MDE)

The smallest change in conversion rate that you want to be able to detect.

Multivariate Testing (MVT)

Testing multiple variables simultaneously to see which combination performs best.

P

P-Value

The probability of seeing results as extreme by random chance.

Personalization

Tailoring content or experiences to individual users based on their unique data.

Probability to Win

The likelihood that a variant has a higher true conversion rate.

R

Relative Lift

The percentage improvement of one variant over another.

S

Sample Size

The number of visitors or participants included in your A/B test.

Split Testing

Another term for A/B testing, often used when testing completely different page URLs.

Statistical Significance

A measure of whether the difference in performance is likely real or just due to chance.

T

Test Variant

The modified version of your page (Variant B) that you are testing against the control.

Testing Hypothesis

A proposed explanation or prediction for how a change will affect user behavior.

U

User Segmentation

Dividing your audience into groups based on characteristics like device, location, or behavior.

Ready to run your first A/B test?

runab makes A/B testing simple. Just add one line of code.

Get started free