Unbounce sold A/B testing. You build two versions of a page, send half your visitors to each, and switch off the one that converts less. That is what the category did, and we did it too.
Here is what the switch does. Say A wins, sixty conversions to forty. Some of the forty who bought on B would never have bought on A. Switch B off and those people still arrive, get A, and leave.
Between 2019 and 2020 we shipped a product that said do not choose a winner. Keep every version running, and send each visitor to the one most likely to convert them. The ceiling is not the best single page. It is everyone getting the page that works for them.
A company whose product was picking winners told its customers to stop. That is the bet. The rest of this page is how we made anyone believe it.
Big companies hire the expertise. Small companies never get it.
A conference floor, several hundred people in the room. About fifteen conversion specialists on one side, our machine on the other. Same pages, same traffic.
To explain why we were on that stage, start with a job.
A conversion specialist is the person who looks at your landing page and knows which headline, which order, which button will make more people buy. Good ones are rare and they are expensive, because what they do compounds.
A few points of conversion is the difference between a marketing budget that pays for itself and one that does not.
If you are a large company you hire one. We could; we had our own. Most of our customers were our size or smaller. If you are a four-person marketing team, you do not, and your page converts at whatever it converts at. Forever. Nobody calls this a problem. It is just the shape of the industry.
We decided to build the expert instead.
Not a tool that gives you advice about your page. A machine that runs the experiment continuously, learns which version works for which visitor, and sends each person to the one most likely to convert them.
The specialist's judgment, as a running system, for people who were never going to be able to hire the specialist.
Then we had to answer the obvious question, and it is not a question you can answer with a chart.
Is it actually as good as a person?
We could have published a benchmark. Everybody publishes benchmarks and nobody believes them, including the people who publish them.
So we put it on a stage, at a conference, in front of the industry.
A live head-to-head. Human conversion specialists on one side doing what they do. The machine on the other. Same pages, same traffic, same question.
It won.
In front of a room of people who do this for a living, and who had every reason to want the other result. Nobody wrote the margin down, so I am not going to put a number on it.
And then the part I actually cared about.
The specialists were never our customers. The people who could not afford a specialist were.
In the alpha, ordinary marketing teams, the four-person kind with no conversion hire and no plan to make one, were seeing 21% more conversions. Not against an expert. Against the page they had before, which was the only alternative they actually had.
That 21% is two things, and it would be easy to sell it as one. The allocation is the visible half. The other half is that we moved the statistics to a Bayesian model, so a result reached significance on a fraction of the traffic.
Most of our customers never had enough traffic to finish a test. The machine was only useful because the math underneath it stopped needing so much of them.
That is the whole product: a little giant. The people it reached were the ones who had been quietly priced out of it.
Nothing on this list worked on its own.
The allocation. Every version stays on, and each visitor goes to the one most likely to convert them. The visible half.
Statistics that needed less traffic. The move to a Bayesian model, so a small company's trickle of visitors was enough to learn from. The invisible half.
An explanation the success team could stand behind. Before the beta, the people who answer the phone asked the question every AI company gets asked now: how do we tell a customer what the system did? A system that decides and does not explain is hard to trust with a budget. The data science team took the question as a requirement, not a support problem, and settled on doubly robust inverse probability weighting, which compares the routed visitors with the rest as if the routing had been random. It never made the explanation easy. It made it defensible, and it was asked before launch, which is the cheapest time to hear it.
Proof people could watch. The head-to-head at Call to Action mattered most to our most loyal customers, because they were the ones in the room.
The data science team filed patents on the work. My part was helping them through the submission. The by-product was bigger than the filing: explaining the system well enough to patent it turned into the company's education in its own product.
The demonstration was a design decision, not a marketing one.
We could have shipped it with a number in a blog post. The reason to do it live, against people who did not want to lose, is that a claim you can watch being tested is a different kind of claim. The audience did not have to trust us. They watched.
A capability only big companies had became a switch a small company could turn on.
Smart Traffic was one of five bets we made instead of cutting the company. The rest of that story is here.