This A/B test significance calculator tells you whether the difference between two variants is real or likely to be chance. Enter the visitors and conversions for the control and the variant, pick a 90%, 95% or 99% confidence level and a one- or two-sided test, and it shows each conversion rate, the relative lift, the p-value, a confidence interval for the difference and a plain verdict. It also checks for a sample ratio mismatch — a visitor split far from 50/50 that usually signals a tracking bug — and, before you launch, how many visitors per variant you need to detect the lift you care about. Marketers and product teams can paste a CSV of many tests or segments and each is evaluated at once. It is free, runs in your browser, and your experiment data never leaves your device.
| Baseline rate | 5% lift | 10% lift | 20% lift |
|---|---|---|---|
| 1% | 637,010 | 163,095 | 42,693 |
| 2% | 315,206 | 80,682 | 21,109 |
| 5% | 122,124 | 31,234 | 8,158 |
| 10% | 57,763 | 14,751 | 3,841 |
That a difference this large would be unlikely if the variants truly performed the same. At 95% confidence, the p-value must be below 0.05.
The probability of seeing a difference at least this big if there were no real difference. Smaller means stronger evidence.
It depends on your baseline rate and the lift you want to detect. At a 5% baseline, detecting a 10% relative lift needs about 31,000 visitors per variant at 95% confidence and 80% power.
No. Checking repeatedly and stopping at the first significant result inflates false positives. Fix the sample size in advance.
No. Everything is calculated in your browser.
Conversion rates are conversions ÷ visitors for each variant and the lift is (B − A) ÷ A. Significance uses a two-proportion z-test: z = (pB − pA) ÷ √(p̂(1 − p̂)(1/nA + 1/nB)) with the pooled rate p̂, and the p-value from the standard normal distribution, two-sided or one-sided. The confidence interval for the absolute difference uses the unpooled standard error and the critical z for the chosen confidence. A sample ratio mismatch check runs a one-degree-of-freedom chi-square test of the visitor split against 50/50 and warns below p = 0.01. The sample size panel gives visitors per variant for a baseline rate and a relative minimum detectable effect at the chosen confidence and 80% power.