A/B Testing: Useful or Useless?

A/B Testing: Useful or Useless?

In marketing and business intelligence, A/B testing is a randomized experiment in a controlled environment to test two alternatives: A and B. It can test any pair — of messages, images, layouts, colors, titles, you name it — to see which one works best.

Using A/B testing moves you far, far away from taking risks with your choices — as far away as, say, London is from Mumbai.

A) London B) Taj Mahal
Which would you pick?

A/B testing helps you improve your results. First, you test your variants with a small, controlled group such as an email list. Afterwards, you send the optimized choice out to the masses.

Here’s what you need to know about A/B testing:

  • You can test anything. Really.
  • You can test anywhere, on any platform, as long as you can compare the results in numbers.
  • You must have a controlled test group.
  • You need at least 500 testers to obtain a meaningful result.

And you need to think about:

What you’re testing.

What you normally take for granted, might not actually be true. For instance, with a small AdWords budget you can test keywords, metadescriptions, and page titles. You can check your customers’ price acceptance by testing two offers. You’ll never know unless you test.

The people in your control group.

For a reliable result you need at least 500 responses. With a small slice of your AdWords or Facebook Ad budget, you can test with the first 500 clicks on your message or image. Test different tones of voice on your fans and followers. Try out a pair of subject lines or two unique content messages. See whether A or B attracts more, then optimize your choice to engage the most people with your story. You can test anybody on anything. No problem.

Your reason for the test.

Whether your style is to carefully plan each step or just go with your gut, an A/B test can confirm your decision with data (or prove you wrong with fewer consequences). An A/B test run can give you a little peace of mind before you go big. Or it could give you the confidence to explore a new direction, change the way you you’ve always done things, and shift your accepted boundaries. The more you test, the more you know, and knowledge is power.

No matter what, why, or with whom you test, an A/B test brings an important voice to your marketing communication decisions: your visitors, your customers, and your users. Their choices might speak louder than your preconceptions about message and design. Trust your control group. Let them help you pick the best metadescription, page title, product position — even their favorite button. Best part is they won’t even know they’re helping you.

By the way, thanks for your help! England or India? You just participated in our own little A/B test.

How may I help you?

What is your challenge?
You can call me at +31 (0)20 4233 555 or leave a message.
I will respond within one business day.
Hugo Kalf - Managing Director

Consistency is Key – 5 Communication Strategy Tips