Why A/B testing is So 2010

Written by: Ben Gawiser on 12/4/15 12:23 PM


According to its technical definition, A/B Testing is the “randomized experiment with two variants.”  As we consider the complexity of today’s world, however, we start to understand that the optimal answer may not be A or B and in fact, it likely isn’t.

We need to consider that the complexity of the solution to the problem may in fact need to be more complex than a binary choice.

Today’s marketing involves much more than option A or option B. Instead of one factor to consider against a control group, marketers are having to look at multiple options that span not only customer variants, but channels, messaging and engagement - and how these all may impact each other over time.

Marketers care about not only which campaigns will give them the most leads, but the ones that are driving the highest rate of return on closed business. Predictive marketing provides indicators marketers can use to gain a more concrete understanding of customers and what motivates them. By looking at patterns in historical data and what’s happening with current data, they can statistically forecast the future. If they can predict behavior, they can customize their marketing strategy and hit the consumer with the best messaging at the right time, maximizing their ability to move them through the marketing funnel.

Of course, not everything is predictable. Change and unexpected events happen that are out of our control or our ability to forecast. But we have plenty of data at our disposal that can get us close enough to have reasonable confidence in our assumptions. We can take the guesswork out of our equations and replace it with probabilities we can hang our hats on.

Types of Predictive Models

There are several different types of predictive modeling marketers can use to enhance customer relationships, establish metrics and measurement, and drive better marketing outcomes. Keep in mind, whichever model(s) you choose, they should be integrated with your marketing execution systems to make them actionable.


Customers can be clustered into segments based on certain characteristics, such as behavior, demographics, or preferred products.


Propensity models help marketers see how likely it is a customer will behave in a particular way, such as likelihood to unsubscribe, engage, convert, or buy. This type of model can also determine the lifetime value of a customer and share of wallet.


Recommendation models help marketers determine how effective their suggestions may be to the customer, such as up sell, cross sell and next sell opportunities.

Smooth Forecast

Smooth forecast models examine any number of variables believed to be related to the campaign and identify relationships that forecast a specific numerical outcome.


Scoring models assign a numerical value to a variable, such as a buyer or a campaign, based on historical data. Scores can then be compared to determine probable outcomes, such as likelihood to buy.


Waterfall models look at what worked in the past to close the most business by closely monitoring conversion rates throughout the sales and marketing funnel, from inquiries to marketing qualified leads to sales qualified leads and closed deals.  By working in reverse, marketers can predict how many marketing qualified leads and inquiries they need to create in order to meet a future revenue goal. It’s important to ‘slice and dice’ a waterfall model by region, business segment, product and more, since the conversion rates and velocities can be different in each area.

Finding the right model, or combinations of models, for your organization can be tricky. Hive9 can help.

Icon made by Freepik from www.flaticon.com is licensed under CC BY 3.0

Topics: Marketing Analytics, Hive9, Marketing Stream