3 Reasons Why Multivariate Testing is Vital to Your Strategy

Published by


With an A/B split test you can only test one variable at a time. If you are testing multiple variables at the same time, how can you possibly know what it was that caused an uplift? Maxemail was the first email marketing solution to provide true multivariate testing to its users. In fact we’ve had it as a flagship feature of our product for nearly four years.

Here’s 3 reasons why MVT beats A/B testing

1 – You can’t argue with maths

Let’s say you want to test 5 different mastheads and 5 different CTA’s. You could do two different A/B split tests, but what if masthead 1 only causes an uplift when paired with CTA 3? There’s no way of knowing as long as they are separate tests.

To truly recognise this, you’ll need to build a test that takes into account every possible combination. 5² is 25, that’s 25 splits within your A/B split test. That’s half your week gone.

2 – No really, you can’t argue with maths

Now let’s say that you want to add in a third variable, the subject line as an example. 3 variables with 5 variations of each is 5³. 5³ is 125 and, already, testing something as simple as 3 variables is completely out of the scope of most email service providers.

With multivariate testing in Maxemail, you simply build the variables you want to test and Maxemail works out each possible combination for you automatically. You’d build 5 mastheads, 5 CTA’s and 5 subject line variables that’s 15 things, rather than 125. That’s a HUGE resource saver for you and your team so you can focus on other acctivities.

3 – MVT reporting suite

In addition to your normal reporting, with an MVT campaign you get a new reporting suite. It tells you how each of your combinations performed, with vital stats like open rate, click rate and return on investment. It goes further by showing you which combination performed the best and, importantly, what the combination was- Subject 3, CTA 5 and Masthead 1.

MVT allows you to test beyond the realms of what is possible with an A/B split test and if you’re already doing some weird and wacky split tests, MVT will be a huge time saver for you. .

Arrange a Demo

About Nick Everitt

Senior Campaign Executive at Emailcenter, the UK's largest independent ESP.