A/B Test Case Study: Single Page vs. Multi-Step Checkout

This post is contributed by Janis Lanka (@janislanka, who manages front-end development for Elastic Path Software.

A little while back, I wrote on Elastic Path’s Grep Community blog about our decision to change to a two-page checkout process. We piloted this checkout process on the Hockey Canada Store with the main goals being to reduce abandonment and to increase conversion. The results were extremely positive, but we weren’t content to sit on our laurels. So when we started re-working the Official Vancouver 2010 Olympic Store, we challenged ourselves to take it to the next level — and we cut the checkout process down to just single page.

Structurally, the new single-page checkout looks very much like the two-page checkout, with shipping information first, followed by billing and confirmation.

Control: Original, Multi-Step Process

(Click to enlarge, will open new page)

Treatment: Single Page Checkout

The option to create an account after checkout is offered after the order is completed:

The Findings

With A/B split testing, 50% of traffic was redirected to the original checkout, while the other 50% was served the new single-page checkout. After only 300 transactions, the winner was clear and we stopped the experiment after 606 transactions. Google Website Optimizer concluded that the single-page checkout outperformed the out-of-the-box checkout by a whopping 21.8%.

While this number is impressive, it should not be used as an indicator for how every single-page checkout will perform. Your results will vary, depending on your product, target market etc. There’s no silver bullet checkout process that works best for all business models. Doing your own A/B split testing will give you a better idea of what kind of improvement you can expect.

Related Articles

33 Responses to “A/B Test Case Study: Single Page vs. Multi-Step Checkout”

  1. Don’t you think that it is also much more clear in the single page version that the user does not need to be registered in order to do the checkout? Maybe it’s not the number of steps, but what the user does in each one.

    • Martin Wharton says:

      Flavio knows whats up. A huge barrier is removed and the single page design gets all the credit. This is not a valid test to draw conclusions from guys.

  2. Great article. Smart marketers and online stores will need to do this in 2010 if they are not already testing.

    I find that the ecom transaction is still fearful for some, especially on a new site. Providing everything alleviates the psychology of being worried about whats around the corner. In my mind, the increase you see is a product of “telling” the customer very clearly, “this is all you need to do and know to complete the transaction”.

  3. Janis – that for posting this very interesting test. Would you be able to post all the the screen shots from the multi-step test? I’ve done some similar test and seen the opposite, but as you clearly call out the product, target customer, etc were very different. Also, I’d be curious to see where the abandonment was happening in the multi-step version. By that I mean was there one page that was causing issues or was is fairly well distributed amongst the additional steps. That being said, I really like how you all have designed the 1-page checkout – it’s virtually frictionless, asking for only 15 fields of data, and has all the right reassurances for the user. Thanks again!

  4. Maureen says:

    I too was surprised the increased conversion wasn’t more.
    When I checked out the new checkout on the site, I noticed that the email updates box was checked. Previously, you’ve said this should be unchecked or opt in rather than opt out. Have you changed your mind on that or a preference from the client?

  5. Great work!

    Is there a resource out there with instructions on how to segment those tests in Analytics?

    Not being able to track important bottom line metrics like average order value is a big bane on a lot of tests we run.

    Thanks!

  6. DefunkOne says:

    Great article. It makes sense. Why break it into 2, 3, or even 4 steps when you can design a killer layout that fulfills the same purpose on one-page!

    Arrrrgh! I am stuck with the shopify checkout process which is pretty weak at this point. Tiny images and a two-step process that is really not necessary.

  7. Wow, this is a great post. You mentioned Google’s Website Optimizer. Do you recommend any other tools for testing / conversion optimization?


    Jim Lindstrom
    ResearchMob, CEO
    http://researchmob.com/

  8. David Minor says:

    Mike, you can set a custom variable (using _setCustomVar) to a different value for each group, and then you can segment based on these values in analytics.

  9. The screen shots are what really make this post great because it ties a visual to the data. Great job Linda.

  10. @Flávio – certainly clearing up design and copy would help the conversion and initial dropout, but having too many steps gives customer too many opportunities to change their mind.

    @Dave Sealey – yes, I did expect to be a bigger lift for conversion as well

    @Maureen – while there are certain best-practices, it comes down to the preference from the client. As long as it is handled professionally (and there is double-opt-in for this), I do not think that customers will be very upset or convert less.

    @Jim – GWO is a free tool that gives most necessary features you will need for testing. While there are some others out there, I haven’t done any work on them.

  11. Hi Linda, WOW what a result. But is it not a conversion killer if you only accept Visa?

  12. Jestep says:

    We did some similar testing a few years back. We found that on average, a site takes about a 20% loss for every page of a checkout process. Doesn’t matter how well optimized the checkout system is, 20% for each step beyond the view cart page.

    We didn’t find any noticeable drop in sales if the checkout page was the same as or one click from the view cart page. User’s are used to clicking on a checkout button on the shopping cart page. Anything beyond that resulted in a marked decrease in conversion.

    The other thing we noticed is that this decrease in conversion is less defined on ecommerce sites with very expensive or very niche products. If the user has to go through a great deal of pre-qualifying before they pick a product, they really don’t care about how long the checkout process is.

    In a single explanation, I don’t think that single page checkouts “convert better” but rather multi-page checkouts reduce compulsive buying.

  13. @Patrick – Visa is the worldwide sponsor of Olympic games and thus there is limitation there. Certainly by allowing other alternative methods of payment it would increase conversion.

    @Jostep – did you publish anywhere your findings? Would love to learn more. If there are any materials you can pass on, would love to read them.

  14. Ernie Sugrue says:

    Hi! Love these comparisons, but I find it odd that an Olympic site (where Visa is the card of the Olympics) uses an older icon for Visa – where the customer puts in there credit card and a link `Why Visa’ – even though the current icon is pictured at the bottom..

    Overall though I love the checkout layout.

  15. Great post.
    Could you precise if the option to create an account after checkout is offered also in the 4 steps checkout ?
    Thanks

  16. Jerome says:

    Well, that’s interesting !

    But I wonder if the results would be the same if the total order was like $456 instead of $45,60.

  17. The most important lesson on this post is NOT about going single page with your checkout. The important lesson is to TEST whatever changes you make.

    We are constantly surprised by the results of tests. In fact the only thing that is more surprising is the varied excuses people make for not testing ;-)

    Thanks, Janis, for sharing this.

  18. Great article, and interesting to see the account creation option tagged on at the end instead of this decision getting in the way of completing your order. Can you tell us what percentage of people who had not created an account changed their minds when confronted with this page, and what percentage of first time customers created an account on this page?

    I like the elegance of this solution but am a little worried about how many people would engage in an additional step (account creation) after completing their checkout.

    Also, we have found that tracking your order is the most important reason why people change their mind about creating an account, I wonder if it would increase your conversions target of account creation if you emphasised this advantage even more (for example by linking this message with the CTA).

  19. David Minor says:

    @Jestep – Great observation! We are in a niche market and found that most of the “improve conversions 200% by doing x” have very little effect.

    Moving to a single page checkout yielded about a 5% improvement for us. We’ll take it, obviously, but we’ve learned to temper our expectations.

  20. @eMerchandising – in multi-step checkout, user registration is only offered at the very beginning.

    @David – good question about tracking order issue. Regards creating creation: it could be a new blog post :), but our aim was to remove any potential distractions. We would prefer them to checkout and not create account, rather than opposite.

  21. Great post and I think the whole process from the cart to success page is nicely designed. One thing that I couldn’t see was any mention of security. Do you think including a ssl security logo or including a link to it in the ‘shop with confidence’ section would have increased conversions?

  22. [...] posts related to conversion optimization for the Official Vancouver 2010 Olympic Store. Following checkout process and product details page optimization, in collaboration with Wider Funnel, we looked at the [...]

  23. [...] our case study in conversion optimization for the Official Vancouver 2010 Olympic Store (see last week’s post if you missed it), today we’ll share the results of our product detail page [...]

  24. They should have incorporated PayPal Express checkout. This would enable all credit card transactions for non PayPal users. Also even better, the PayPal account holders shipping address would be pre-populated so one less step for them before they confirm -> decrease the dropout rate.. ohh and it is safer for consumers as they do not provide any credit card details to the merchant. Its a win / win / win

  25. Verolina says:

    Hi Janis,

    Thanks for sharing this case!

    Can you tell something more about the “few little hacks” that enable linking GWO / ecommerce data in GA?

    Thanks!

    • Janis says:

      Verolina, it greatly depends on your ecommerce platform, but few points to consider:
      1) we “linked” GWO and Ecommerce data in GA using Advanced Segments (with Regular Expression a.k.a regex)
      2) regex looked at our URLs and filtered something like “audience=A”, “audience=B”, which was appended to our URL.
      3) In order for our URL to show “audience=A”, our developers needed to look at GWO cookie, extract assigned audience, and show this in URL.

      I do realize this is not an obvious clarification, but I hope this sets you on the right track.

  26. Amy R. says:

    I would like to hear more information about this study. I am quite shocked to see these results. They are very uncommon.

    Did more users sign up when using the multi-page checkout? I am a usability specialist working at a few different companies currently, and all of my tests (and industry standards) report that requiring users to register will dramatically decrease conversions. The multi-page design appears, to me, to require a sign up (the optional guest checkout is not very noticable). So, I guess my question is…where did you drop your customers on the multi-page shopping cart?

  27. Jim says:

    Great article. Smart marketers and online stores will need to do this in 2010 if they are not already testing.

    I find that the ecom transaction is still fearful for some, especially on a new site. Providing everything alleviates the psychology of being worried about whats around the corner. In my mind, the increase you see is a product of “telling” the customer very clearly, “this is all you need to do and know to complete the transaction”.

  28. The hypothesis that every redirect loses a few customers has been around for a while. It’s great to have some hard data to back it up.

    As a payment gateway which processes transactions without redirecting cardholders, we look forward to showing these statistics to our clients.

  29. Lelala says:

    Maybe there is now right or wrong in that question:
    I’ve seen both types of checkout funnels beating out each other, depending on scenario/business case. Test test test, is what one should do.

  30. Jason says:

    This is a great test. We did the same test on our site, http://www.promostadium.com and found that we did get less abandonment, but on the flip side, single page widgets/sections can be a little flakey. Take for example, after the billing address is entered, the shipping estimate api will need to be manually refreshed in order to get the updated shipping price.

Leave a Reply

© 2014 Get Elastic Ecommerce Blog. All rights reserved. Site Admin · Entries RSS · Comments RSS