How should you approach split-testing your checkout process?

This question was asked of Bryan Eisenberg in yesterday’s Google Website Optimizer webinar. Bryan recommended split-path testing, reducing the number of steps in your process and using your analytics data to determine what part of your checkout path needs attention.

What is Split-Path Testing?

The definition of a split-path test, according to GrokDotCom:

Split-Path Test — This test will split your traffic among different linear paths containing multiple pages for each path. This is different in that you’re testing the performance of grouped pages against other grouped pages. For example, you could test a checkout process by splitting it into two variations; one with four steps (or pages), and another with only three steps. Each variation of grouped pages will have the same Goal Page (e.g., order confirmation page). Once the data is collected, the winning checkout process will be the one that converted a higher percentage of visitors.

Reducing Checkout Steps

Different ecommerce stores have different checkout paths, ranging from one-page AJAX checkouts to 6 steps or more. Bryan believes less is more - in fact, he recommends going under 4 steps. But you can find out for yourself if this is so for your website by doing your own testing.

I’ve gathered some examples of checkout steps (many are generally the same aside from labeling) that can give you some ideas for how to simplify your process. For example, you may want to test a new path with a combined billing and shipping page vs. your existing separate steps. Or you may want to ditch a step that may be clogging your funnel, such as “Rewards Program.”

Continue Reading:
Checkout Process Split-Testing Tip from Bryan Eisenberg »

Observing users is one of the most under-rated means of improving site usability. Today’s eCommerce managers often rely too heavily on technology to help them make decisions. Faceless A/B and multi-variant testing is used to incrementally increase performance of key areas of the ecommerce store such as product pages and checkout procedures. Though valuable tools, they are only as useful as the deployed functionality.

A very simplified scenario.

Curtis, an online retailer marketing manager, decides that the product pages do not have a high enough conversion rate. In keeping with the mantra “test ’til best”, he wants to see if the ‘Add to Cart’ button is the culprit. Perhaps the color of the button has an effect? A simple A/B split test on two colors is done: green vs blue. In this instance they convert at approximately the same rate. Next, he plays-off blue vs yellow, then orange, then purple, etc. Conversion rates remain somewhat consistent across the board. What could be causing the poor conversion rate?

Mel, a competing online retailer, has the same concern, but she takes a different approach. A simple 30 minute observation of 8 different users given $200 dollars of virtual cash to buy anything they want from the store. 4 out of the 8 users found the product they wanted to buy but were distracted by other things on the page - too much merchandising (up-sells, cross-sells, others bought these items, bundle discounts, warranty), in-store inventory lookup, add to wishlist, add to registry, etc. These users simply wanted to add an item to the cart and they verbally expressed frustration because the page had so many choices. Testing the color of the button would never have reveled this.

Why is this on my mind today?

I am in the process of planning a wedding and honeymoon, so my fiance Amy and I are doing a LOT of online research and shopping. I love observing how she tackles tasks and what she likes and dislikes about the online shopping experience.

While looking for flights, cruises, etc. we have been using a number of sites: Airlines, online comparison engines, and even Google (yes, you can look up flights directly in Google - type Vancouver to New York, the result page gives you input fields to enter dates - cool).

Amy has settled on Kayak.com as her travel website of choice. While seeking out airfares, she was thrilled by the auto-complete on the airport input fields. She started to type v-a-n-c and automatically Vancouver (YVR) was show as an option for the input field (by setting YVR as our home aiport, it is defaulted for future visits as well).

Compare this to other sites where it is often a game of hide-and-seek when looking up airport codes - enter the ‘From’ city, new screen with airport code options, select option, click ok, go back to previous screen where all date information has been magically erased and wash-rinse-repeat on the ‘To’ airport - frustrating and time wasting.

This simple bit of Ajax functionality has made Amy loyal to Kayak for all her travel lookups - a case of usability being the primary cause of loyalty. People want ease of use and intuitive behavior.

The next time you want to improve your online store (which should be always), watch people, listen to what they say, observe their facial expressions. By seeing what makes them smile or grin could save you tons of time on faceless testing.