Free tools that run in your browser. Use any tool without an account.

Favourites Cart
Sign in
Join free

A/B Test Significance Calculator

  • Free
  • Runs in your browser
  • No account needed

You can test whether B beats A using a two-proportion z-test. For example, 200 of 5,000 (4.00%) vs 240 of 5,000 (4.80%) gives z = 1.95, p = 0.0512, confidence 94.9%.

Result
Significant at 95%: version B converts better
Version A conversion rate
10%
Version B conversion rate
13%
Relative uplift of B over A
30%
Z score
2.1027
P-value (two-sided)
0.0355
Confidence
96.45%

Decide the sample size before you start and do not stop the test the moment it turns significant. Very small samples (under about 10 conversions per version) make the normal approximation weak.

Runs in your browser. Your files and values are not sent to us. Report a problem
Advertisement

About the A/B Test Significance Calculator

This tool runs a two-proportion z-test in your browser so nothing is uploaded. You enter Version A visitors and conversions, Version B visitors and conversions, and choose a confidence level [90% / 95% / 99%]. The tool shows Version A and B conversion rates, relative uplift of B over A, the pooled z score, the two-sided p-value, and the confidence (1 - p-value). Results update instantly as you type; there is no calculate button. You can copy the result or share a link that reopens the page with your numbers. The page also warns you to set sample size before the test and not to stop the test the moment it turns significant. Very small samples, roughly under 10 conversions per version, make the normal approximation weak.

How to use it

  1. 1Enter Version A visitors and Version A conversions
  2. 2Enter Version B visitors and Version B conversions
  3. 3Choose the confidence level you need [90% / 95% / 99%]
  4. 4See the Result, conversion rates, uplift, z score, p-value and confidence update instantly
  5. 5Copy the result or share the link to reopen with your numbers
Advertisement

Worked example

Input: Version A 5,000 visitors and 200 conversions; Version B 5,000 visitors and 240 conversions; Confidence 95%. Output: Version A rate 4.00%, Version B rate 4.80%, uplift 20.0%, z = 1.95, p = 0.0512, confidence = 94.88%. Check: not significant at 95%.

The formula

Two-proportion z-test with a pooled rate: p = (conversions A + conversions B) / (visitors A + visitors B), standard error = square root of (p x (1 - p) x (1/visitors A + 1/visitors B)), z = (rate B - rate A) / standard errorTwo-sided p-value = 2 x (1 - normal CDF(|z|))Confidence = 1 - p-valueUplift = (rate B - rate A) / rate A

Source: NIST e-Handbook of Statistical Methods - Two ProportionsCXL A/B Test Calculator (sample size and analysis). Last checked 14 November 2026.

What it is good for

  • Runs fully in your browser, so no upload is required
  • Updates results instantly as you type, with no calculate button
  • Shows pooled z score, two-sided p-value, uplift percent and confidence
  • Uses the fixed-horizon pooled two-proportion z-test only
  • Normal approximation weak for very small counts, roughly under 10 conversions per arm
Advertisement

Questions people ask

Which test does this calculator use?

It uses the pooled two-proportion z-test. The pooled rate combines conversions across both versions to compute the standard error. See the NIST e-Handbook for the standard reference.

How do I interpret the p-value?

The p-value is the probability of data as extreme as yours if there were no true difference. A p-value below your alpha means you reject the null, but not necessarily that the effect is large.

Can I check results every day and stop when significant?

No. Repeated peeking inflates false positives for a fixed-horizon z-test. Decide sample size ahead or use a sequential testing method if you need interim looks.

What if my counts are small?

If you have very few conversions per arm, the normal approximation can be poor. Use Fisher's exact test or a Bayesian approach for small samples instead.

Can I test more than two variants here?

This calculator focuses on a single A versus B comparison. Multiple variants require multiple comparisons or a multi-arm method and a correction such as Bonferroni.

Does data leave my browser?

No. The calculation runs in your browser and nothing is uploaded. You can share a link that encodes your numbers if you want others to reopen the same inputs.

Help keep every tool free

FreeToolr stays free because people help it grow. Share a tool you like, tell us what to build next, or support us on Ko-fi.

Advertisement
Advertisement