"One of 15 Entrepreneur Blogs Worth Reading"
- Wendy Bounds, The Wall Street Journal

A/B Test Case Study: Single Page vs. Multi-Step Checkout

This post is contributed by Janis Lanka (@janislanka, who manages front-end development for Elastic Path Software.

A little while back, I wrote on Elastic Path’s Grep Community blog about our decision to change to a two-page checkout process. We piloted this checkout process on the Hockey Canada Store with the main goals being to reduce abandonment and to increase conversion. The results were extremely positive, but we weren’t content to sit on our laurels. So when we started re-working the Official Vancouver 2010 Olympic Store, we challenged ourselves to take it to the next level — and we cut the checkout process down to just single page.

Structurally, the new single-page checkout looks very much like the two-page checkout, with shipping information first, followed by billing and confirmation.

With A/B split testing, 50% of traffic was redirected to the original checkout, while the other 50% was served the new single-page checkout. After only 300 transactions, the winner was clear and we stopped the experiment after 606 transactions. Google Website Optimizer concluded that the single-page checkout outperformed the out-of-the-box checkout by a whopping 21.8%. But what does that 21.8% really mean?

Control: Original, Multi-Step Process

(Click to enlarge, will open new page)

Treatment: Single Page Checkout

The option to create an account after checkout is offered after the order is completed:

The Findings

It’s important to keep in mind that GWO only counts goal conversions and does not link to any ecommerce data in Google Analytics. With a few little hacks, we were able to pass on each test group to Advanced Segments on Analytics for both checkout flows and gather valuable ecommerce data:

  • Successful completion rate for the entire checkout process increased by 257.26%.
  • Overall site conversion rate increased by 0.54%.
  • We also observed some unexpected improvements during this experiment, like an increase of 8.54% in the average order value!

While these kinds of numbers are impressive, they should not be used as the sole indicator of how single-page checkout performs. These are the improvements that we observed when changing from the standard four-page checkout to a single-page checkout process on the Vancouver 2010 Olympic Store. Your mileage may vary, depending on your product, target market, et cetera. There’s no silver bullet checkout process that works best for all business models. Doing your own A/B split testing will give you a better idea of what kinds of numbers you can expect.

Subscription Options



Or, receive new posts by email.
(100% opt-in/out)

Email

24 Responses to “A/B Test Case Study: Single Page vs. Multi-Step Checkout”

  1. Don’t you think that it is also much more clear in the single page version that the user does not need to be registered in order to do the checkout? Maybe it’s not the number of steps, but what the user does in each one.

  2. Great article. Smart marketers and online stores will need to do this in 2010 if they are not already testing.

    I find that the ecom transaction is still fearful for some, especially on a new site. Providing everything alleviates the psychology of being worried about whats around the corner. In my mind, the increase you see is a product of “telling” the customer very clearly, “this is all you need to do and know to complete the transaction”.

  3. Alex Fisken says:

    Janis – that for posting this very interesting test. Would you be able to post all the the screen shots from the multi-step test? I’ve done some similar test and seen the opposite, but as you clearly call out the product, target customer, etc were very different. Also, I’d be curious to see where the abandonment was happening in the multi-step version. By that I mean was there one page that was causing issues or was is fairly well distributed amongst the additional steps. That being said, I really like how you all have designed the 1-page checkout – it’s virtually frictionless, asking for only 15 fields of data, and has all the right reassurances for the user. Thanks again!

  4. Dave Sealey says:

    Hi Linda

    Great research work although I was suprised that the increased conversion rate was only 0.54%!

    You should submit this to http://www.abtests.com.

    Dave Sealey

  5. Maureen says:

    I too was surprised the increased conversion wasn’t more.
    When I checked out the new checkout on the site, I noticed that the email updates box was checked. Previously, you’ve said this should be unchecked or opt in rather than opt out. Have you changed your mind on that or a preference from the client?

  6. Great work!

    Is there a resource out there with instructions on how to segment those tests in Analytics?

    Not being able to track important bottom line metrics like average order value is a big bane on a lot of tests we run.

    Thanks!

  7. DefunkOne says:

    Great article. It makes sense. Why break it into 2, 3, or even 4 steps when you can design a killer layout that fulfills the same purpose on one-page!

    Arrrrgh! I am stuck with the shopify checkout process which is pretty weak at this point. Tiny images and a two-step process that is really not necessary.

  8. Wow, this is a great post. You mentioned Google’s Website Optimizer. Do you recommend any other tools for testing / conversion optimization?


    Jim Lindstrom
    ResearchMob, CEO
    http://researchmob.com/

  9. David Minor says:

    Mike, you can set a custom variable (using _setCustomVar) to a different value for each group, and then you can segment based on these values in analytics.

  10. Mark Simon says:

    The screen shots are what really make this post great because it ties a visual to the data. Great job Linda.

  11. Janis Lanka says:

    @Flávio – certainly clearing up design and copy would help the conversion and initial dropout, but having too many steps gives customer too many opportunities to change their mind.

    @Dave Sealey – yes, I did expect to be a bigger lift for conversion as well

    @Maureen – while there are certain best-practices, it comes down to the preference from the client. As long as it is handled professionally (and there is double-opt-in for this), I do not think that customers will be very upset or convert less.

    @Jim – GWO is a free tool that gives most necessary features you will need for testing. While there are some others out there, I haven’t done any work on them.

  12. Hi Linda, WOW what a result. But is it not a conversion killer if you only accept Visa?

  13. Jestep says:

    We did some similar testing a few years back. We found that on average, a site takes about a 20% loss for every page of a checkout process. Doesn’t matter how well optimized the checkout system is, 20% for each step beyond the view cart page.

    We didn’t find any noticeable drop in sales if the checkout page was the same as or one click from the view cart page. User’s are used to clicking on a checkout button on the shopping cart page. Anything beyond that resulted in a marked decrease in conversion.

    The other thing we noticed is that this decrease in conversion is less defined on ecommerce sites with very expensive or very niche products. If the user has to go through a great deal of pre-qualifying before they pick a product, they really don’t care about how long the checkout process is.

    In a single explanation, I don’t think that single page checkouts “convert better” but rather multi-page checkouts reduce compulsive buying.

  14. Janis Lanka says:

    @Patrick – Visa is the worldwide sponsor of Olympic games and thus there is limitation there. Certainly by allowing other alternative methods of payment it would increase conversion.

    @Jostep – did you publish anywhere your findings? Would love to learn more. If there are any materials you can pass on, would love to read them.

  15. Ernie Sugrue says:

    Hi! Love these comparisons, but I find it odd that an Olympic site (where Visa is the card of the Olympics) uses an older icon for Visa – where the customer puts in there credit card and a link `Why Visa’ – even though the current icon is pictured at the bottom..

    Overall though I love the checkout layout.

  16. Great post.
    Could you precise if the option to create an account after checkout is offered also in the 4 steps checkout ?
    Thanks

  17. Jerome says:

    Well, that’s interesting !

    But I wonder if the results would be the same if the total order was like $456 instead of $45,60.

  18. John in York says:

    The most important lesson on this post is NOT about going single page with your checkout. The important lesson is to TEST whatever changes you make.

    We are constantly surprised by the results of tests. In fact the only thing that is more surprising is the varied excuses people make for not testing ;-)

    Thanks, Janis, for sharing this.

  19. David Vogel says:

    Great article, and interesting to see the account creation option tagged on at the end instead of this decision getting in the way of completing your order. Can you tell us what percentage of people who had not created an account changed their minds when confronted with this page, and what percentage of first time customers created an account on this page?

    I like the elegance of this solution but am a little worried about how many people would engage in an additional step (account creation) after completing their checkout.

    Also, we have found that tracking your order is the most important reason why people change their mind about creating an account, I wonder if it would increase your conversions target of account creation if you emphasised this advantage even more (for example by linking this message with the CTA).

  20. David Minor says:

    @Jestep – Great observation! We are in a niche market and found that most of the “improve conversions 200% by doing x” have very little effect.

    Moving to a single page checkout yielded about a 5% improvement for us. We’ll take it, obviously, but we’ve learned to temper our expectations.

  21. Janis Lanka says:

    @eMerchandising – in multi-step checkout, user registration is only offered at the very beginning.

    @David – good question about tracking order issue. Regards creating creation: it could be a new blog post :) , but our aim was to remove any potential distractions. We would prefer them to checkout and not create account, rather than opposite.

  22. Gareth Brown says:

    Great post and I think the whole process from the cart to success page is nicely designed. One thing that I couldn’t see was any mention of security. Do you think including a ssl security logo or including a link to it in the ’shop with confidence’ section would have increased conversions?

  23. [...] posts related to conversion optimization for the Official Vancouver 2010 Olympic Store. Following checkout process and product details page optimization, in collaboration with Wider Funnel, we looked at the [...]

  24. [...] our case study in conversion optimization for the Official Vancouver 2010 Olympic Store (see last week’s post if you missed it), today we’ll share the results of our product detail page [...]