A/B Case Study: How MEC Found a 404 Page Winner

Last year, Mountain Equipment Co-op (MEC) shared its case study on 404 "Not Found" pages with Anne Holland of Which Test Won. This test won a Gold Ribbon in the Ecommerce category in the 2009 Testing Awards.

If you missed it, go on and "test your gut" - see if you can guess which design converted best.

If you guessed the 404 page with the image, you are correct. It achieved not only a 3.56% lift but also a 73.2% boost in revenue per visitor.

What you didn't see on Which Test Won were the other versions tested by Amadesa, the conversion optimization firm that conducted the experiments for MEC. Check 'em out:

Control

Click image to enlarge

V1.00 – Text only

• Updated text – messaging why a user might have reached the 404 page
• Conversion rate of 37%

Click image to enlarge

V1.01 – Text ,Image, & Shortened Links

• Updated text – messaging why a user might have reached the 404 page
• Addition of image
• Addition of search
• Shortened link list
• Conversion rate of 33.74%
• Revenue per visit lift of -29.12%

Click image to enlarge

V1.02 – Image, text, & shortened links

• Updated text – messaging why a user might have reached the 404 page
• Addition of image
• Addition of search
• Shortened link list
• Conversion rate of 37.74%
• Revenue per visit lift of 38.44%

Click image to enlarge

V1.03 – Large image, text, & dropdown

• Updated Text – messaging why a user might have reached the 404 page
• Addition of image
• Addition of search
• Product dropdown
• Conversion rate of 38.32%
• Conversion lift of 3.56%
• Revenue per visit lift of 73.2%

Click image to enlarge

The winner

The winning version had a conversion rate of 38.32% and a conversion lift of 3.56%. Overall, this version presented a revenue per visit lift of 73.2%.

MEC found that providing users with an abundance of links (in the first version) did not engage the user, rather overwhelmed them to the point of abandonment. The winning version provided readers with an apology for the error, as well as more appealing imagery and color scheme that better fit the lifestyle of MEC customers. It also provided a search field and drop box that simplified the re-navigation process for users – a successful vehicle for re-engaging.