So you have an amazing MVT tool, now to test your email campaigns – what should you start with? The general rule of thumb is:

• Start with your emails aimed at conversion, re-activation or cross-selling – small changes here will make a big difference, plus the results are more repeatable than newsletters, where each edition will have different content, lessening the reliability and usefulness of some results.

• Think about the objective of the email – how can your tests support this? Is your newsletter about engagement? Then maybe your tests should be around how we get people to read more articles. If your email has a simple single action, like ‘Buy this product’ or ‘Review this purchase’ or ‘Come back and get a special discount’, then your tests should be aimed at maximising this, which usually means evaluating your call-to-actions and key messaging.

Once you have established which emails to test and the objectives for each then here are the key areas to look at:

Buttons & Call-to-actions
Simply, how can you make them stand out more? Should you make them orange? If you have a text call to action, have you tried reversing the text so it is perhaps white text on an orange background? Have you tried different text – how can you improve ‘click here’ – does ‘Buy Now’ work better?

Subject Lines
Subject lines are traditionally the first thing tested in an A/B test programme. However if you have done this you will know that the subject line that fits with the main content at the top of the email, will perform best. So when performing a multivariate test, try testing a number of subject lines, to see which one performs best with different combinations of content within the body of the email. You could find one combination outperforms the rest, even if when measured individually it is not the best performing subject line.

Remove clutter
If you have a single specific goal for your email, any additional clutter will distract reader and therefore should be removed. Try taking things out to see what happens, such as the pre-header and link to the online version, any menus in the header to sections of your website, or even any in-line links in text. If you only want someone to click on one link, test not having other distracting content and you should see an improvement.

Headlines
The headline is possibly the first thing someone will digest when reading your email. Try different text here, as it can make as big of a difference as subject line to overall performance.

Credentials
Why should your customers trust you or what you claim? Does adding logos of the likes of Verisign help persuade people your website is secure? Does adding customer testimonials or review scores improve click-thru rate on product offers?

Product Detail
How much is too much? Should you just have product name, image and price or should you add further information, and if so how much before the added clutter hinders the click-rate?

Product Imagery
Is it worth investing in extra photography, perhaps showing the product you are offering in action against the single box shot offered by the manufacturer?

Offer
If you have a special offer campaign what incentive works best? Does 10% off work better than Free Delivery?

These ideas are just examples of what can be done. If you have an idea of what might impact performance then create an MVT or A/B test.

Maxemail has become the first email solution to have a true Multivariate Testing (MVT) option built in, but why have we added it? Essentially we have added it as it gives marketers much greater power than simple A/B testing.

Here is why MVT Rocks compared to A/B testing, on email marketing campaigns:

Quicker to test
A/B Tests can only handle one change at a time – any more than this and you have no idea what caused the change in performance, and by how much. Therefore it can take forever to test everything you want to test. However with MVT you can keep adding new tests as you think of them, and automatically work out the best performing combination, cutting test time from months to maybe a day.

Easier to identify change between combinations
MVT not only identifies the best variations, but also the best performing combinations. Indeed just because you have identified one call-to-action works better than another, it might not actually perform best against other hero images or headlines you want to test.

Easier to manage
Each A/B test is a different creative, and therefore a different email. While most ESP’s still make this fairly easy to manage, it is still clunkier and less scalable than MVT where more than a handful of test variants are wanted.

A/B Testing still exists though, as it has its place. Where you are perhaps testing fundamental structual changes to a template, or only testing one thing, then A/B testing works fine. However if you are serious about your email testing strategy, then MVT is the only credible option.

Emailcenter are pleased to announce the launch of the Email Industry’s first Integrated Multivariate Testing tool for email.

While ESPs have offered A/B testing functions within their platforms, and some 3rd party solutions are available, this is the first time a full scale MVT solution has been built within an ESP’s offering.

MVT takes A/B testing to a new level by allowing marketers to test more than one thing within their campaign, without losing validity of the results. Now marketers can implement all of their desired testing in much shorter periods of time as a result.

Initial pilots of the tool by Maxemail users have seen click rates improved by as much as 49% versus the original control message. Typically customers are experimenting with button colours, call-to-actions, hero imagery and headlines.

Setup of an MVT email is easy. Simply create the content variants, tag each one as to what part of the email it should appear, and insert the MVT tokens into the relevant sections of your email. Maxemail then calculates all of the combinations between variants, and sends each to equal random samples of your list. Reporting is automatically available against MVT emails, showing both every combination of variants (full factorial) or the best performing variants without taking into account the combination of variants (Taguchi).

Offering both analysis options allows marketers with even the smaller data sizes to start taking advantage of MVT. Indeed a common myth is you need far more data with an MVT than an A/B test to make it worthwhile, however where trying to improve clicks the Emailcenter MVT Calculator shows less data is needed than traditionally thought.

Customers interested in enabling MVT to their account should contact their Account Manager in the first instance.

When reporting on emails it can be useful to group several emails together to get an overall report for a type of email. For example your Christmas promotions might be made up of several weeks of sends, each with various segmented versions.

Also it helps to look at performance between campaigns based upon different attributes. For example, you might want to compare emails by what day of the week they were sent, by what type of campaign (Welcome vs. Newsletter vs. Alert etc), by what type of promotion it was (special offer vs, competition), or by any chosen way of categorising the email send.

The new tagging facility in Maxemail allows you to do just that.

How can it be used?

As well as reporting by tags, they can also be used in other areas of Maxemail:

Behavioural Segmentation

Rather than selecting specific emails within your segmentation criteria you can now also select tags. So you could now select to send to all the people that have clicked on one of your ‘special offer’ emails, rather than selecting these emails individually.

If you are selecting multiple tags then someone could match any of these, they do not have to match all of these. This is the same on every use of tags in other areas of Maxemail.

Data Export

Select additional filters of what emails you are interested in based upon tags selected.

Comparison & Segment Reports

Perhaps when looking at each segments performance, it might be interesting to see how they perform across different types of emails. For example, do certain segements respond better to event information than general news?

Tag Performance Report

This special report in the ‘Insight Reporting’ section of Maxemail allows you to break down where your overall performance is coming from. by tag. For example, what percentage of revenue from email marketing is generated from your re-activation programme versus Welcome emails and Newsletters? You might also want to compare performance between tagged emails this way – for example, you may have tagged emails by what day of the week the email was sent and this allows you to easily see which day of the week performed best.

Insight Reports

In any of these Dashboard and CRM-style reports, you can filter the email selection on which you are basing the report by selecting tags. This is useful in areas such as filtering out special types of emails on your dashboard reports, as these could distort statistics and volumes sent at certain times of year, and cloud any trend analysis.

Creating & Managing Tags

To create and edit tags, simply go to the tools menu and select ‘Manage Tags’.

Here you will see a number of example tags and tag categories to help you get started. Rather than allowing you to freely type tags against each email like on many blogging and CMS packages we have a more controlled approach to creating these. This is to ensure typos, duplicates and mistakes are kept to a minimum as they will cause mistakes in your analysis. Also, throughout an organisation it is good to maintain consistency in tagging to maximise the benefit which tags provide. Therefore you may wish to restrict which users can create and edit tags.

Tips for tagging

We recommend tagging an email with only one tag per category. For example, if your main focus of the message is ‘special offers’ only tag the email with this, even if it contains secondary content items.

Also don’t get too detailed – there is no point in just having one email matching a tag as this will not be very useful when it comes to future analysis.

Adding tags to emails

Simply select these on the email properties screen. It is a good idea to set-up tags in any templates you use, including a ‘Default’ template which allows you to define a whole host of settings, even if the email was not created from a template.

Also for tackling the issue of going through past campaigns and quickly adding tags to all of these we have created a ‘Mass tagging’ function. This will be available shortly by right clicking on the main emails section within Maxemail and selecting ‘Apply tags’. Here you can select your tag, and then tick which emails or folders you wish to retrospectively apply this to. This means in the future you may choose to change your tagging methodology yet still be able to apply this on past campaigns for instant insight.

Getting more help and advice

If you want to tap into our expertise in this area at Emailcenter then feel free to call your Account Manager to arrange a time for our Analysts to provide recommendations on how you go about tagging your email marketing campaigns.

In our previous posts about our Insight Reporting we talked about how email marketing should not be measured or analysed purely on a campaign-by-campaign basis but longer term metrics should be used.

One of our existing reports – Segment and Cross-tab Reporting does this by providing the ability to drill down into segments to identify which segments are performing best, and which ones require further targeting.

We have two new reports though that take this even further.

Lifecycle Report
You will often see a downward trend in open and click rates over time. This is because when people receive your newsletter they might decide it is not for them, as there is perhaps no relevant content or you send it too often, so they make the decision to stop opening when they see it amongst so many other emails. Yes, some people will unsubscribe or hit spam, but generally that involves a lot more effort than people are willing – it is just easier to ignore.

The Lifecycle Report is designed to show how long your database stays engaged after subscribing. You can choose over how many emails you want it to measure – say the first 12 which people receive, and the report will show you an open rate for each of the first 12 emails someone receives.

Like other reports in Insight Reporting, you can filter the emails over which the report is run by ticking them, by selecting tags and a date range. A date range is mandatory, and Maxemail will only select subscribers added after the start of this range. This ensures your report is statistically reliable as only new subscribers are shown, not older subscribers from before you started to use Maxemail.

You can also apply segments to filter on which customers to report on.

What you are hoping to see is a very gradual decline in performance over time. If you have a sharp drop – or say halve the number of opens in several sends – this suggests you will suffer from list fatigue.

If you are seeing a drop, you can identify where you need to correct it. This might mean reducing frequency to new subscribers, improving the relevance by segmenting sends or adding tailored content, or perhaps introducing a welcome programme or other targeted messages at appropriate points in the lifecycle.

The report is also useful when you introduce more advanced elements into your email marketing. New subscribers don’t have any pre-conceived ideas about what is in your messages, whereas older subscribers do. Therefore, even if you do introduce content items, these older subscribers won’t realise you have changed. Looking at just the trends of new subscribers clearly highlights any impact.

Engagement Report
While the Lifecycle Report looks at open rates for each email new subscribers receive, the engagement report gives you an overall health-check on your email database. Against the emails you select, it will group recipients into one of five groups based upon how many emails they have opened.

Therefore you can quickly establish how many of your database have not opened a single email in say the last 12 months.

The report allows you to specify a minimum number of emails that someone must have been sent in the emails you have selected – this just ensures those which only received a handful of messages do not add bias in the report.

You can also select segments to apply, which allows you to look at different audiences – perhaps by the source of the data – and see the true value of each.

Our general rule of thumb is to expect 40-60% of your database to have not opened an email in the last year – anything outside of that is unusual, although you should see this in your most engaged and dis-engaged segments. Our clients who achieve better than this, and also have a higher percentage of ‘Advocates’ on their list, are the ones who run highly relevant email programmes.

The report is really aimed at raising awareness of how many never open the email. This creates the arguments behind the business case for moving to a relevant, automated and optimised email marketing programme. Management are used to seeing 15% open rates and are not inclined to change anything as a result – but show them that the majority of their database have ignored all of their emails in the last year and it shifts their perception. No longer do they over-value their customer database, and the opportunity is there to shift away from ‘batch and blast’ to a more tailored email marketing approach.

To access a list of how many emails each recipient has opened, the existing ‘Engagement Report’ in Data Export allows you to export this.

If you want help in setting up or analysing any of these reports, please do not hesitate to contact Support or your Account Manager.