Basics of A/B Testing

AB testing is the comparison between two different versions of a web page to see which one performs better. You compare two web pages by sending the two variants (like A and B) at the same time. For example let’s assume that A and B are two designs. Typically, A is the existing design (called the control), and B is the new design. You split your website traffic between these two versions and measure their performance using metrics that you are concerned with (conversion rate, sales, bounce rate, etc.). In the end, you will be selecting the version that performs best.

What to Test?

Your choice about what to test will be depend on your goals. Like if your goal is to increase the number of sign ups, then you might the test: length of the sign up form, field’s types in the form, display of privacy etc. The goal of A/B testing in this case will be to figure out what prevents visitors from singing up.

• Is the form’s length intimidating?
• Are visitors concerned about privacy?
• Does the website do a bad job of convincing visitors to sign up?

All of these questions can be answered one by one by testing the appropriate website elements.

Almost anything that affects user behavior can be A/B tested. Some of the elements that you can easily test are:

• Headlines and sub headlines (Size and color wording)
• Paragraph Text (Amount, wording, font size and placement of content in page)
• Call to Action Text (I.e. buttons wordings, size, placement and color.)
• Social Media buttons and Links (Placement, size and wording)
• Images (Colors, placement)
• Style and Layout of website pages

Setting up a Test

• Decide what do you want to test
• Set up two designs you want to test (one is the control which is often the original version)
• Don’t be afraid to be bold and test big changes to start with
• Ideally you want a 95% confidence level that your test is statistically significant
• Chose a tool to run your tests or work with a professional company specializing in testing

Tools for A/B Testing

A number of tools are available for A/B testing, with different focuses, price points and feature sets. Here are some:

• Google Website Optimizer
  Free A/B testing tool from the search giant. A great option to get started, but lacks advanced features.
• A/Bingo and Vanity
  Server-side frameworks for Ruby on Rails developers. Requires programming and integration in code.
• Visual Website Optimizer
  An easy-to-use A/B testing tool, with advanced features such as WYSIWYG editor, click maps, visitor segmentation and tag-less integration. (Disclaimer: my start-up.)
• Unbounce and Performable
  Landing-page creators with integrated A/B testing.
• Vertster, SiteSpect, Webtrends Optimize and Omniture’s Test &Target
  Enterprise testing tools.

References

http://www.maxymiser.com/resources/ab-testing