Comments on: A/B Testing: Giving the Left More Love

By: sam

sam — Fri, 27 Aug 2010 13:25:54 +0000

Great blog post – thanks for the links to the tool as well.
Due to a picture sizing issue we had on our e-commerce site, it had been suggested that we swap the add to cart button (which was on the right side) with the product photo (which was on the left side.This really works and i think we all should test possible design tests.Thanks for sharing..

By: NICK

NICK — Sun, 08 Aug 2010 11:10:54 +0000

great stuff!

Do we have any insights on the post-purchase experience?
Did generating more conversions turned out to generate also more (in %) non-optimal buyer experiences?……….
My company tried a similar test and found that the typical left-hand image, right-hand add-to-cart outperformed the other test cases.

thanks for sharing….

By: Phil

Phil — Wed, 28 Jul 2010 09:23:19 +0000

This really works and i think we all should test possible design tests, i used to do this test with google optimizer combination.

By: Robi

Robi — Mon, 03 May 2010 09:04:06 +0000

I personally think that the position of the call to action is not the only parameter. The color of it, the pictures in the page, the colors, … and lot of other items on the page will all have an impact on the conversion.

To summarize, it will depend on the site and the context of use.

My 50 cents …

By: Linda Bustos

Linda Bustos — Fri, 30 Apr 2010 14:59:37 +0000

@Jack, absolutely, never hurts to test – and if you want to reduce your “risk” – just dilute your test by showing more versions of the same control against the treatment, e.g. A/B/C/D test, only “D” is the treatment, only 25% of traffic exposed to the test. This will, however, make your test run that much longer…

By: Linda Bustos

Linda Bustos — Fri, 30 Apr 2010 14:58:13 +0000

That brings up a good point Giulio that in some cases one might want to tie customer support tickets to user ID flagged by which version of the site test he/she viewed. Especially when you are testing short copy vs. long, for example, or presence of reviews vs. no reviews. Not sure if any testing tools accommodate this.

By: Toni Anicic

Toni Anicic — Fri, 30 Apr 2010 11:47:52 +0000

Hm… this might have something to do with demographics of their visitors. I can’t imagine myself clicking on add to cart button on the left side. It simply makes no sense to me.

By: Chris Goward

Chris Goward — Thu, 29 Apr 2010 16:35:31 +0000

Good questions, Sharon.

This test was run as a controlled experiment on the same traffic mix and with all variations tested at the same time. For every visitor to the page we randomly selected one of the variations to display and then cookied them to ensure they saw the same page template for every page and every time they returned to the site.

The conversion rate to sale was tracked for each variation and the winning page had a statistically significant result at a 95% confidence level.

The original test was also run over several weeks to iron out day-of-week seasonality and then the winning page was tested a second time against the Control to be double-sure that it was indeed a winner.

By: Mike

Mike — Thu, 29 Apr 2010 16:19:29 +0000

This really works and i think we all should test possible design tests, i used to do this test with google optimizer combination with PPC ads which gives me more better idea about the traffic increase

By: Sharon

Sharon — Thu, 29 Apr 2010 09:10:02 +0000

@Chris you are so right, this is what testing is for! Sometimes our gut feeling mislead us.
Regarding results, conversion has many variables that can influence it: time, source of traffic, etc. Did you guys took these in account? Were the conversions from the same sources or different ones? Were there different campaigns?