AB Testing: ReActive to ProActive Organizations

Also commonly referred to as multivariate testing, AB testing is a simple concept that yields powerful results. It’s a strategy during which you implement two different versions of the same action simultaneously, testing the results against each other to see which one is more effective. This is usually done as a way to test out improvements that you’re making to some area of your business to garner reaction in real-time, before a larger roll-out is then implemented.

What is AB Testing?

If you were AB testing your website, for example, you would implement two versions at the same time to two different groups of users. The first would be essentially your site as it exists today – also called the control version. The changes you’re thinking about implementing would go live in the other version, also called the treatment. You would then see how one performed against the other, gleaning valuable insight before deciding exactly what to do next.

To really get a better indication of just how powerful AB testing can be, consider facts like these:

• According to Econsultancy, AB testing is the single most used method for improving customer conversion rates for the last two years. 
• An incredible 44% of all companies use split testing during the software development process according to seogadget. 
• In 2011, Google ran more than 7,000 different AB tests on its home page. This included testing 50 different shades of the color blue for the color used in their call to action until they found the shade with the highest conversion rates!
• QuickSprout notoriously runs at least one AB test every two weeks. The result of their efforts is a consistent 20% increase in conversion rates depending on the test. 

How Do We Get Started on Running AB Tests?

To get started running AB tests within your own organization, the first and most important step involves figuring out what you’re actually going to test in the first place. This will vary depending on what specific goal you’re trying to accomplish. AB testing is something that you can use to examine the reaction to nearly anything, but for the sake of example let’s say you’re trying to make the subject line in your next direct email message to clients more effective.

Once you decide what you’re going to test – in this case, an email subject line – everything else falls into place. In this particular example, the goal in your testing is to increase effectiveness, which therefore translates to conversions or open rates (or in many cases, both). 

Next, come up with two different versions of your email subject line to test – your control and your treatment, otherwise known as “Version A” and “Version B.” Your control, which will go out to a standard group of your users, will be a standard email subject line – like the kind you’ve been using up to this point.

“Version B” will represent a dramatic change – either in length, in language, in the use of capital letters or punctuation or some combination of all these things. It should be a large enough change from “Version B” to justify the AB test in the first place.

To use another example, some companies test colors for elements like “call-to-action” buttons. The buttons themselves may look identical, but “Version A” might be red (keeping in line with the current visual theme of a company’s Web presence) will “Version B” might be another color like black (representing a stark contrast).

Then, promote your AB test to specific groups based on your goals. If you’re just trying to increase conversions or open rates among your existing audience, send “Version A” to 100 normal customers and send “Version B” to a different group of 100 normal customers. If you were trying to gain a new audience, you might send “Version A” to existing customers and “Version B” to people who normally fall outside of that audience. Again – this will all vary depending on exactly what it is you’re trying to accomplish.

Once you’ve promoted your AB test, you can begin to gather analytical data to see how you did. Compare the open and conversion rates of “Version A” to “Version B” and look for significant changes. Did your open rates go up or go down between the two versions? Even if nothing appeared to change, this can still tell you a great deal – namely, that your adjustments between the two versions weren’t significant enough to cause any type of spike in the first place. In that case, it’s back to the drawing board for another round of testing. 

Again, you can use AB testing for nearly every aspect of your organization, provided that you have the following core elements:

1. Decide what you want to test.
2. Decide what “success” looks like.
3. Determine how to properly gage response.
4. Test each version on two different groups of people.
5. Gather analytical data to correctly see where you stand. 

Why Analytics Matter

It’s also important to note that analytics play a huge role in terms of AB testing. AB testing gives you statistics, facts and other figures based on the results of your testing efforts. Analytics can then be used to cut beneath those stats, understanding exactly WHY a particular change adjusted user behavior or response the way it did and putting an organization in the best position to implement that on a larger scale.

AB testing might tell you that X, Y and Z changes to your landing page increased conversions by 30%. That’s terrific – but without understanding the “why” of it all, you won’t be able to implement those same principles elsewhere. Analytics and AB testing give you that power in a way unlike most others. 

Who Stands to Benefit From AB Testing the Most?

In terms of which industries can actually benefit from AB testing, the answer is simple: all of them. This type of split testing can be hugely beneficial in nearly all types of organizations, you’ll just have to adjust the actual specifics of the test depending on what you’re talking about.

AB testing has long been used in the world of politics, for example, particularly during campaign seasons. According to event360.com, President Obama was able to raise an additional $60 million dollars to fuel his campaign through extensive AB testing about the marketing collateral he was creating, who was receiving it, when it was being received and more.

While AB testing is widely used by marketing departments in agencies around the world to further their digital efforts, the benefits don’t begin and end in that area. In truth, AB testing is founded on a very simple principle: by putting accurate analytical data and other statistics in your hand, you have the insight you need to make informed, practical business decisions. Because of this, it can benefit nearly every aspect of an organization from the top down.

By using AB testing to optimize a marketing department’s connection with a target audience, they’re able to gather more and higher quality data that can then be fed to the sales department. AB testing is also a valuable way to identify certain gaps that may exist in an IT infrastructure, helping to improve technological assets and align them with long-term business objectives.

If AB testing is at its core about “figuring out what works in the most analytical way possible,” then it stands to reason that ALL departments could benefit from the detailed insight that it provides. This in turn not only benefits every department in a company, but the company’s own bottom line.

Consumers themselves also benefit from AB testing, as ultimately the goal is to figure out what people want and how to better position a business to give it to them. AB testing lets a consumer’s own real-world behaviors and other patterns influence the decisions that a business is making, ultimately creating a stronger and more organic bond between these two entities. 

The Ultimate Value of AB Tests

At the end of the day, the major benefit of AB testing is not unlike the benefit of analytics in the first place: insight. It isn’t just about figuring out whether a particular change will work or not before you implement it on a wide scale. It’s about getting a deeper understanding of the people you’re trying to reach. It’s about eliminating the need to simply look at what your competition is doing successfully and copy it in any way that you can. It’s about being proactive, carving out your own unique variant of success and using the insights generated to carry your entire business into the future.