A/B Test Significance Calculator
Know if your test truly won — z-score, p-value and a plain-English verdict.
In
Variant A (control)
Variant B (challenge)
Out
Variant B wins
18.2%
Relative lift for B over A · 97.96% confidence achieved
How the A/B Test Significance calculation works
The calculator runs a two-proportion z-test: it pools both variants' rates to estimate shared variance, computes how many standard errors separate the two conversion rates, and converts that z-score into a p-value.
The p-value is the probability of seeing a gap this large if the two variants were actually identical. Below 5% (at 95% confidence), random chance is an unlikely explanation — most teams call the test there.
Relative lift puts the gap in business terms. A significant 1% lift can be worth running; a big but non-significant lift is still a coin flip.
Worked example
A: 340 of 10,000 (3.4%). B: 402 of 10,000 (4.02%). That is an 18.2% relative lift with a z-score near 2.32 and p ≈ 0.02 — significant at 95%, so variant B ships. At 99% confidence it would still pass, comfortably above the threshold.
The tool above opens with these numbers. Change them to run your own scenario, or press Reset to come back to this example.
Frequently asked questions
Related tools
The honest version of marketing ROI — profit after margin, not vanity revenue.
Paste one campaign row, get every metric that matters.
What a customer really costs — and how many months it takes to earn them back.
What a customer is worth over their whole life — and the 3:1 health check.
Your conversion rate — plus the sessions you'll need for any goal.
Work backwards from a revenue goal to the budget that actually funds it.
- a/b test
- significance
- p-value
- statistics
- split test