A/B test calculator

Test, AI Optimize or personalize?

• A test splits traffic between your current page, No Change, and one or more variations, and declares a winner once the result is statistically significant. Use it when you want one clear answer.[1][2]
• AI Optimize shows each visitor the variation most likely to lead to a conversion, sending more traffic to what performs better as it learns. It doesn’t declare a single winner; it keeps optimizing.[3][4]
• Personalize shows different content to an audience you define, like visitors from one ad campaign, from one country, or coming back for a second visit.[5]

Webflow’s own advice: use a classic test when you need clarity and control, and AI Optimize when speed, scale or flexibility matter most.[6]

What “smallest lift” means

It’s the smallest improvement the test is set up to detect reliably, measured relative to your current rate. At a 2.5% conversion rate, a 10% lift means telling 2.5% apart from 2.75%. Small lifts need far more traffic than big ones, and Webflow suggests aiming for at least a 5% lift in your goal.[17][18][7]

Run whole weeks, at least two

Shoppers behave differently on a Tuesday than on a Saturday. Webflow recommends running a test for at least one to two weeks, and notes that a 30-day test is typically more reliable than a 7-day one. Researchers at Microsoft recommend whole weeks too. That’s why the calculator never shows fewer than two.[7][8][19]

Why a real test can finish sooner or later

This calculator plans a fixed-length test. Webflow Optimize calculates significance with sequential hypothesis testing and declares a winner once significance is reached, so a real test can finish sooner or later than this estimate.[9][2]

If your tool uses a fixed sample instead, decide the length up front and don’t stop early: peek ten times, and what looks like 1% significance is really only 5%.[20]

How AI Optimize reports results

AI Optimize doesn’t use statistical significance, because its traffic isn’t split evenly. It reports Strength instead: every variation starts at 50%, and above 70% counts as a success while below 30% doesn’t.[9][10][11]

By default, AI Optimize runs for 80% of visitors, and the rest are held out to measure the difference. Webflow’s rule of thumb is that the holdout should see at least 100 sessions a day, on a goal that converts at 10% or more. That’s what the AI Optimize check above measures.[3][12]

Personalize with a light touch

Start simple, with a high-signal audience you already understand, like visitors from one paid campaign, and tie every personalization to a goal. Webflow also warns that a site that seems all-knowing comes across as off-putting.[13][14]

What to test first

• Pages with the most traffic and high bounce or exit rates.[11]
• Prominent copy first: headlines, subheadings and section headings often give the best return.[11]
• Write each idea as if, then, because: if we change this, then this goal moves, because of what we saw in the data.[11]
• Expect most tests to lose. Webflow puts the win rate for teams that use data at roughly 20 to 30%.[11]

Check the split before you read the result

If you set a 50/50 split and the session counts come out noticeably uneven, something in the setup is broken, and the result can’t be trusted until you find out what. Microsoft found this in about 6% of its own tests.[21]

When you don’t have the traffic

• Test a bigger change. A bold change is more likely to produce an effect large enough to detect with fewer visitors.[22]
• Measure an earlier step. Add to cart happens far more often than an order, so enter your add-to-cart rate instead and see how the numbers change.[22]
• Test two versions, not several. Every extra variation needs its own share of traffic.[22]
• Don’t treat AI Optimize as a shortcut. Webflow notes it needs enough traffic to learn.[1]
• Learn from research instead. An A/B test tells you which version won, not why. Session recordings, customer questions and an audit show you where shoppers struggle.[23]

Running tests on a Shopify store

Webflow Optimize runs natively on Webflow sites, and on Shopify stores through Webflow’s app for Shopify. Without Shopify Plus, it can’t run in checkout, so checkout-based goals aren’t available.[15]

Shopify’s own Rollouts can also split traffic between your current theme and a changed version, 50/50 by default. Experiments need the Grow plan or higher. Testing apps such as Intelligems and Shoplift add other test types; Shopify’s roundup lists Intelligems for testing prices, shipping and post-purchase offers.[24][25][26][28]

Count sessions and conversions the same way in every tool. Shopify and Webflow Optimize both report conversion rate per session.[27][16]

The math behind it

The calculator uses the standard fixed-sample formula for comparing two conversion rates: a two-sided test at 95% confidence with 80% power, and equal traffic to each version. Visitors per version equal (1.96 + 0.84)² times the sum of each version’s rate times one minus that rate, divided by the squared difference between the two rates.[29]

Other calculators use slightly different versions of the formula, so their answers can differ by several percent. Evan Miller’s popular calculator, for example, shows about 7,700 visitors per version for a 5% to 6% test, where this one shows about 8,200. Either is fine for planning.[30]