Full Screen

A/B Testing

More Free Lessons in

Email Marketing View All →

Get cutting-edge digital marketing skills, know-how and strategy

This micro lesson is from one of our globally recognized digital marketing courses.

Start a FREE Course Preview Start a FREE Course Preview
Global Authority

The Global Authority

12 years delivering excellence

Members

245,000+ Members

Join a global community

Certification

Associate Certification

Globally recognised

Membership

Membership Included

Toolkits, content & more

Digital Marketing - Study Notes:

What is A/B testing?

A/B testing, also known as split testing, it's a method of comparing two versions of an email against each other to determine which one performs better. A/B testing is an experiment where two or more variants of an email are shown to users at random. An analysis is then used to determine which variation performed better for a given goal, such as opens or clicks.

So, you can have multiple variations, but you test the same element. If you have five versions of an email, everything remains the same except the one element that you are trying to improve upon.

Take a look at this graphic here to get an idea of how it works. There are two variations, and the metric being focused on is the click.

So, again, A/B testing can be used to test almost any element of an email, but only perform one test at a time to get accurate results. In this scenario, we are testing two emails which are the exact same, but have two different calls to actions to see which one gets more clicks. We send 20% of our overall audience to each email, wait one hour to view the results, and then whichever one has gotten the more clicks, the rest of the subscriber list will receive it.

The test group that got the underperforming email is an unfortunate side effect of your A/B test. However, in your next send with the improved version, you'll know that it will go to everyone, and perform better. This is also why it's extremely important to have a continuously growing database. Without a healthy size database, you will never have a large enough list to actually be able to do your testing.

Best practices

A/B testing best practices are very important to know:

  • Only test one variable at a time.
  • Split your groups equally and randomly.
  • Test early and test often for the best results.
  • Test as large a sample as you can for more accurate results.
  • Trust the data collected, not your gut instinct.

If you're wondering about sample size, I would advise you to go to the blog of ConversionXL (cxl.com/blog) to understand in more depth what sample audience size can and should look like, where it's too small, and all those other questions you may have.

Advantages

A/B testing is one of the most common testing methods in email marketing, as it is, at the end of the day, quite quick and easy to implement, and there's so many advantages.

  • You better understand each of your segments. You can begin to see why this group is different from that group, and why they should be treated as such.
  • You determine the best subject lines so you get to know the language that works for them. This will impact all of your marketing, not just your email.
  • You have improved content. You will start to filter out the fluff that does not matter to your subscribers, and focus on what really delivers value to them.
  • It increases conversion rates because you're sending the right thing, genuinely, at the right time.
  • This all results in more sales. Who can say no to that?
Back to Top

Please note that the module slides are designed to work in collaboration with the module transcript document. It is recommended that you use both resources simultaneously.

Digital Marketing Resources:

Andrea Francis

Inbound marketing manager @ Poppulo

  • Inbound marketing manager in employee communications technology 
  • Worked at Relayr as a senior marketing manager, creating and implementing a global inbound marketing strategy
  • Worked with Hubspot as a marketing manager and funnel optimization specialist focusing on converting leads to qualified leads for the EMEA region
  • Content marketing and blogging with various SaaS startups at Startupbootcamp Amsterdam

The following pieces of content from the Digital Marketing Institute's Membership Library have been chosen to offer additional material that you might find interesting or insightful. While relevant to this module, you will not be assessed on this content.

You can find more information and content like this on the Digital Marketing Institute's Membership Library

ABOUT THIS DIGITAL MARKETING MODULE

Email Marketing
Andrea Francis
Skills Expert

This module begins with the fundamentals of email marketing and how the concepts of segmentation, personalization, timing, engagement, and the legislation and regulations surrounding data protection underpin an effective email marketing strategy. The module introduces key email marketing tools and techniques and explores subscriber list and email design best practices. It covers how to create, test, and optimize an email campaign that maximizes email open and click rates and provides an overview of the value provided by marketing automation tools.