A/B Test Case Study: Homepage

This post is contributed by Janis Lanka (@janislanka, who manages front-end development for Elastic Path Software.

This post is a continuation of a series of posts related to conversion optimization for the Official Vancouver 2010 Olympic Store. Following checkout process and product details page optimization, in collaboration with Wider Funnel, we looked at the store's homepage. Following list of hypotheses were made:

• Too many banner spaces create high clutter
• Secondary (left side) navigation doesn't stand out and is difficult to navigate
• Product photos are too small with no indication on available alternative colors

Control

(Click to enlarge, will open new page)

Variation A

(Click to enlarge, will open new page)

Variation B

(Click to enlarge, will open new page)

As a result, we produced two alternative variations with following changes:

• Reduced banner amount and increased size to improve prominence of each banner
• Increased prominence and clarity of secondary navigation
• Provided color thumbnails to products that have alternative colors
• Increased size of photos and reduced amount of products shown under New Arrivals, Featured Products, and Most Popular tabs

What We Learned

This was a very tough test where even after 2400 transactions we did not have a statistically significant winner. However, a decision had to be made quickly due to the Games fast approaching.

Variation A was chosen as it was converting almost 3% better than the control and had lower bounce rates overall.