Plan an A/B test that can actually finish.

Sample size is the number of visitors (or users, or sessions) each variant of an A/B test needs to see before the test has a high enough statistical power to reliably detect the smallest lift you care about. Run an undersized test and you have two failure modes: either it finishes "inconclusive" forever, or worse, it lands on a chance result and you ship a change that does nothing. Run an oversized test and you waste traffic, lengthen the test cycle, and slow product velocity. Calculating sample size up front means you commit to a stopping rule before peeking at results, which is the single most important habit for trustworthy testing.

For a two-proportion z-test (the standard for conversion-rate A/B tests): n per variant = (z_alpha + z_beta)² × (p₁(1-p₁) + p₂(1-p₂)) ÷ (p₂ - p₁)². Where p₁ is your baseline conversion rate, p₂ is the treatment rate after applying the minimum detectable effect, z_alpha is the critical value from the standard normal for your significance level (1.96 for a two-sided 5% alpha), and z_beta is the critical value for your statistical power (0.84 for 80% power). The square in the denominator is why halving the MDE quadruples the sample size required.

Statistical power is the probability that the test correctly detects a real lift when one exists. 80% power is the textbook industry default: if you ran the same test 100 times against a true 10% lift, 80 of those tests would correctly land on "significant" at your chosen alpha. Higher power (90 or 95%) cuts your false-negative rate but inflates sample size sharply: going from 80% to 95% roughly doubles the required visitors. Use 80% for routine tests, 90% for important launches you cannot afford to call wrong, and 95% only for very high-stakes decisions where a missed positive is more costly than the larger test.

Alpha is the false-positive tolerance: the probability the test calls a win when nothing real changed. 5% alpha is the textbook default, used in 95%+ of A/B tests. It means: if you ran the same null-effect test 100 times, you would falsely call 5 of them significant by chance. Tighter alphas (1% or 0.1%) cut false positives but inflate sample size. Looser alphas (10%) sometimes appear in exploratory tests but produce lots of fake wins. Unless you have a specific reason to deviate, use 5% alpha. The test math in this calculator treats your alpha as the per-test rate, then applies a Bonferroni correction internally when you specify more than two variants.

MDE is the smallest lift the test is powered to reliably detect. It is a planning input, not a result. Setting MDE = 10% relative says: "I want the test to find a 10% relative lift if one exists." A smaller MDE asks the test to spot a smaller signal, which requires more samples. MDE is the single biggest lever on sample size: cutting MDE in half quadruples the sample required, because the formula puts MDE squared in the denominator. Realistic MDE values for indie SaaS landing tests range from 5% to 25% relative. Anything below 5% relative on a normal conversion-rate test demands an unrealistic visitor count for most indie products.

Relative MDE is what most testing platforms and PMs talk about: "I want to detect a 10% lift", meaning a 10% relative change to the baseline (a 3% baseline becomes 3.3%). Absolute MDE is measured in percentage points: "I want to detect a 0.5pp lift", meaning a 3% baseline becomes 3.5%. Both produce the same sample size given the same target, just framed differently. Relative is more intuitive when comparing tests across different baselines. Absolute is more intuitive when you have a fixed business target (e.g. "we need to lift checkout from 3% to 4%"). This calculator supports both modes and shows the equivalent in the other framing.

A two-sided test tests for any difference (win OR loss). A one-sided test only tests for one direction (win only, or loss only). One-sided tests need ~20% fewer samples than two-sided to achieve the same power, but they cannot tell you if your variant actively hurt the metric, only whether it failed to win. Use two-sided tests for almost everything: routine A/B tests, landing-page redesigns, pricing changes, onboarding flows. Use one-sided only when a result in the wrong direction is logically impossible (e.g. a new feature that is strictly additive). The vast majority of indie A/B tests should be two-sided.

When you compare more than two variants against a control, you run multiple pairwise comparisons and the family-wise Type-I error rate inflates. A naive 5% alpha with three variants gives you roughly a 14% chance of seeing at least one false-positive winner. The standard fix is a Bonferroni correction: divide your overall alpha by the number of comparisons. This calculator applies it automatically: with a control plus three variants, your per-comparison alpha drops from 5% to ~1.67%, increasing the sample requirement substantially. For more than four variants, consider sequential testing or false-discovery-rate methods rather than Bonferroni, which becomes overly conservative.

Run the test until you reach the calculated sample per variant, and ideally for at least one full business cycle (often one week, occasionally two) to absorb day-of-week effects. Never stop early because results "look significant". That is called peeking and inflates your false-positive rate dramatically. Never extend the test past the planned sample size hoping for significance. That is called fishing and is equally bad. Run the planned sample, stop, and report. If the sample requirement gives a duration over 8 weeks, redesign the test: chase a bigger MDE, accept lower power, or test a higher-traffic surface.

Sample size is a pre-test planning calculation: given your assumptions, how many visitors do you need? Statistical significance is a post-test result: given the data you actually observed, how unlikely was this result under the null hypothesis (no real effect)? They are flip sides of the same coin. The sample-size calculator answers "should I run this test?". The significance calculator answers "did this test prove something?". Run sample-size before launch, significance after the planned sample is hit. This page handles the pre-test side; pair it with the post-test significance calculator linked below.

Low-traffic indie products run into hard constraints: under roughly 200 conversions per variant, even a 30% relative lift can take months to detect at 80% power. You have three honest choices. First, increase your MDE: only test changes that would credibly produce a 25%+ relative lift (full redesigns, headline changes, pricing). Second, accept lower power (70% or even 60%). You will miss more real wins but you will at least finish the test. Third, switch to qualitative methods entirely: user interviews, session recordings, intent surveys. Do not run a 1,000-visitor A/B test and treat the result as decisive. It is not.

The sample-size formula has MDE squared in the denominator: n ∝ 1 / (p₂ - p₁)². Halving the absolute effect quarters the denominator, which quadruples n. This is the law of large numbers in action: detecting a smaller signal against the same noise floor needs disproportionately more data. It is also the single most underappreciated fact in A/B testing. If your team is debating "should we test for a 5% lift or a 10% lift?", choosing 5% costs 4x the visitors and 4x the test duration. Pick MDE based on what is realistically detectable in your traffic budget, then design test variants that aim for at least that effect.

This calculator uses frequentist methods, specifically the two-proportion z-test, which is what platforms like Optimizely Web and Google Optimize historically used. Bayesian A/B testing has different math (posterior probabilities, credible intervals) and different stopping rules. Both are valid. Frequentist is dominant because it is older, has clean stopping rules, and is what most platforms default to. Bayesian shines when you want continuous decision-making (early stopping is honest) and when you can specify informative priors. For indie SaaS doing 1-2 tests at a time with no analytics team, frequentist is the right starting point because the stopping rules are mechanical and hard to abuse.

No. This calculator computes sample size for proportion-type metrics: conversion rate, click-through rate, signup-to-paid rate, anything where each visitor is binary (converted or not). For continuous metrics like average revenue per user, average session duration, or NPS, you need a t-test sample-size formula, which uses the standard deviation of the metric instead of p(1-p) variance. The math is different and the result will be different. If your primary KPI is revenue per visitor, test on conversion rate instead and treat revenue as a secondary read, or use a continuous-metric calculator.

Peeking means checking the test result before the planned sample is reached and deciding whether to stop or continue based on what you see. It feels harmless ("the variant is clearly winning, let me ship") but it dramatically inflates the false-positive rate. A pre-planned 5% alpha test that gets peeked at daily can have an actual false-positive rate of 15-30%. The fix is mechanical: pick a sample size, run to that sample, stop. If you must look mid-test for operational reasons (e.g. checking the test is correctly bucketing users), do not make a stop/continue decision based on the lift number. Sequential testing methods exist for legitimate early-stopping but they require their own math, not the standard z-test.

Use the most recent stable estimate of your control conversion rate from production data, not a rough guess. Pull the last 30 days of conversion data for the surface you will test on. Confirm the rate is stable (no recent product launches or marketing campaigns skewing it). Then plug that rate as baseline. If the baseline is wrong by even a few percentage points, the sample-size calculation can be off by 2-3x because p(1-p) variance scales nonlinearly. For new pages with no historical data, segment to a similar existing page or use a known industry benchmark, and revisit the sample size after the first week of traffic.

For typical conversion-rate testing at indie SaaS scale: 5,000 to 50,000 visitors per variant per test, at 80% power, 5% alpha, two-sided, baseline rates between 1-10% and MDE between 10-25% relative. Larger surfaces (e.g. checkout funnels with millions of users) routinely run tests at 100,000+ per variant for precision on small MDEs. Smaller indie surfaces (new pricing pages, niche landing pages) sometimes operate at 1,000-3,000 per variant but accept lower power or larger MDE. The right answer is whatever this calculator says given your baseline, MDE, and traffic.