Last week we published an article on split testing, which involves testing one variable through multiple versions of your email. But for those marketers who are serious about email testing, you should be thinking about multivariate testing to create that winning email combination.
So, what’s the difference between split testing and multivariate testing? We’ve discussed this previously too, but in short multivariate testing is much more in depth. A split test only allows you to test one variable at a time, such as the subject line or a call to action, but not both.
This means if you want to test lots of variables it could take weeks or months to complete and analyse the results. Even then you can’t really be sure how the different variables will work together. This is where multivariate testing kicks in. You can create a single email and specify the different variables you want to test, resulting in a winning combination for your final version.
If you’re using an email marketing platform like Maxemail, it will calculate all the different combinations for you and send each version of your email to an equal number of subscribers within your sample list. So, if you have three different subject lines, two hero images and three call to actions that would be 18 versions of the different variations. From here you can dramatically increase the speed of your testing.
The benefit of multivariate testing is that you have the technology available to design an email that your subscribers are more than likely to engage with. Sample size is a consideration, but actually you don’t need to test each versions with as many subscribers as you may think. Each sample group can usually be in the low hundreds as you get confidence in the numbers by grouping all sample groups relating to one variant together, such as the subject line.
Multivariate testing is not for everyone though – it does require more effort to implement, which is why it’s not often used on one-off emails. However, is it is extremely popular with repeat and triggered campaigns. Here it’s worth putting the effort in to test and optimise the content of your emails. Also triggers tend to be more responsive, so you get more value for your test.