About the A/B Test Significance Calculator
This tool runs a two-proportion z-test in your browser so nothing is uploaded. You enter Version A visitors and conversions, Version B visitors and conversions, and choose a confidence level [90% / 95% / 99%]. The tool shows Version A and B conversion rates, relative uplift of B over A, the pooled z score, the two-sided p-value, and the confidence (1 - p-value). Results update instantly as you type; there is no calculate button. You can copy the result or share a link that reopens the page with your numbers. The page also warns you to set sample size before the test and not to stop the test the moment it turns significant. Very small samples, roughly under 10 conversions per version, make the normal approximation weak.
How to use it
- 1Enter Version A visitors and Version A conversions
- 2Enter Version B visitors and Version B conversions
- 3Choose the confidence level you need [90% / 95% / 99%]
- 4See the Result, conversion rates, uplift, z score, p-value and confidence update instantly
- 5Copy the result or share the link to reopen with your numbers
Worked example
Input: Version A 5,000 visitors and 200 conversions; Version B 5,000 visitors and 240 conversions; Confidence 95%. Output: Version A rate 4.00%, Version B rate 4.80%, uplift 20.0%, z = 1.95, p = 0.0512, confidence = 94.88%. Check: not significant at 95%.
The formula
Two-proportion z-test with a pooled rate: p = (conversions A + conversions B) / (visitors A + visitors B), standard error = square root of (p x (1 - p) x (1/visitors A + 1/visitors B)), z = (rate B - rate A) / standard errorTwo-sided p-value = 2 x (1 - normal CDF(|z|))Confidence = 1 - p-valueUplift = (rate B - rate A) / rate ASource: NIST e-Handbook of Statistical Methods - Two ProportionsCXL A/B Test Calculator (sample size and analysis). Last checked 14 November 2026.
What it is good for
- Runs fully in your browser, so no upload is required
- Updates results instantly as you type, with no calculate button
- Shows pooled z score, two-sided p-value, uplift percent and confidence
- Uses the fixed-horizon pooled two-proportion z-test only
- Normal approximation weak for very small counts, roughly under 10 conversions per arm
Questions people ask
Which test does this calculator use?
It uses the pooled two-proportion z-test. The pooled rate combines conversions across both versions to compute the standard error. See the NIST e-Handbook for the standard reference.
How do I interpret the p-value?
The p-value is the probability of data as extreme as yours if there were no true difference. A p-value below your alpha means you reject the null, but not necessarily that the effect is large.
Can I check results every day and stop when significant?
No. Repeated peeking inflates false positives for a fixed-horizon z-test. Decide sample size ahead or use a sequential testing method if you need interim looks.
What if my counts are small?
If you have very few conversions per arm, the normal approximation can be poor. Use Fisher's exact test or a Bayesian approach for small samples instead.
Can I test more than two variants here?
This calculator focuses on a single A versus B comparison. Multiple variants require multiple comparisons or a multi-arm method and a correction such as Bonferroni.
Does data leave my browser?
No. The calculation runs in your browser and nothing is uploaded. You can share a link that encodes your numbers if you want others to reopen the same inputs.
Help keep every tool free
FreeToolr stays free because people help it grow. Share a tool you like, tell us what to build next, or support us on Ko-fi.