Thanks to all who signed up and attended today’s webinar I Know I Should Be Testing, But… with Bryan Eisenberg. We did have some technical difficulties with capturing audio, but Bryan graciously agreed to re-record the webinar and you can find the replay here and even download the mp4 file to replay on your iPod or similar device. So big thanks and a hoorah for Bryan!
About our guest speaker: Bryan Eisenberg is a founder of Future Now and an Internet marketing veteran - a ClickZ columnist for 8 years (bit more bio information in that link), blogger for GrokDotCom and author of several books including Always be Testing: Complete Guide to Google Website Optimizer. (PS, the 5 lucky winners of Bryan’s book will appear in our RSS/email footer for all out subscribers).
Without further ado, let’s jump into the summary of the original webinar, including the question and answer period…
No More Excuses!
Great experiences aren’t accidents: why we test
“…a computer can tell you down to the last dime what you’ve sold. But it can never tell you how much you could have sold.” - Sam Walton, founder of Wal-Mart
76.7% of retailers are not testing - how come?
- We’re too busy dealing with urgent, we forget the important.
- There’s a HiPPO in the room (will explain below)
- IT/technology constraints, lack of human resources
- Just plain don’t know where to start!
About the HiPPO: You may understand the value of testing, but are challenged in getting buy-in because there’s a HiPPO in the room - the “highest paid person in the organization.” Hippos are the biggest barrier to testing, according to Bryan, and sometimes you need to take initiative and show the results of testing in order to get more support for it.
All the barriers revolve around 3 key drivers:
TOOLS - Besides Google Website Optimizer (free) there are enterprise tools like Omniture Test and Target and Optimost. GWO may not be as robust as paid tools, but at least no one has the excuse they can’t afford a testing tool.
PEOPLE - Like analytics, your tool is about 10% and the rest is people and process. You should have designated people working on testing so it doesn’t fall by the wayside when the “urgent” pops up.
PROCESS - You can have the hands to do the work, but without a process for determining what to test, how and how to measure success and actually implement the improvements, you’ve essentially got monkeys with typewriters.
Challenges
According to a 2006 JupiterResearch/ERI Executive Survey of decision makers in companies currently using A/B or multivariate testing, the biggest testing challenges include (in order of importance):
- Demonstrating ROI
- Creating test elements
- Modifying pages for testing
- Establishing test scenarios
- Using test interface
- Prioritizing initiatives
- Understanding results
- Acting on test results
- Segmenting/targeting test participants
Example of Tests
You can test any “section” on your website, here’s an example of 3 different headlines and 3 different images:
Amazon is famous for testing anything and everything, including cart buttons:
Just for kicks, here’s a screenshot of Amazon from the late 90’s:
Bryan’s got more details on his blog about the hidden secrets of the Amazon shopping cart. He also explains why retailers shouldn’t just copy Amazon in our recent interview.
2 common mistakes
Bryan advises against any “slicing and dicing.” Don’t take a page and start testing every element on the page concurrently - it will drain your resources and the test duration will go on forever. Start with a headline, start with a cart button. Get some small wins and then test more.
Another mistake is to do testing, but not implement any changes. Commit to continuous improvement - plan, measure and improve. Take data, and make changes!
Process
Process for improvement:
1. Who are we trying to persuade?
2. What action do we want them to take?
3. What action to THEY want to take? (Not always identical to the action you want them to take!)
Step 1 - Focus on Your Customer’s Experience
Use these questions as your blueprint for testing:
- How many different types of profiles will you have that would participate in this campaign?
- How do they buy?
- Are they Logical or Emotional?
- Are they Methodical, Spontaneous, Humanistic, Competitive (4 buying modalities)
- Where are they in the buying cycle?
Develop rich personas based on the 4 buying modalities - for more information on “persuasion architecture,” personas and buying modalities, check out Bryan’s book Waiting for Your Cat to Bark? and our webinar on the subject (video / text recap).
Step 2 - Define the Test/Experiment
- What Action do we want them to take?
- What page will we test?
- Where do we judge success?
Step 3 – Do the Creative
- Create the pages, PPC ads, emails, or advertisements (online and off) that drive your prospects to your landing pages.
- Do have the correct messages for each type of profile and their motivation? You will likely need more than one.
- Do you have the correct message for each stage of their buying cycle? Are they early in the buying process, in the middle, or are they ready to buy?
Check out an example of how Competitve/Spontaneous buyers “see” a page (left) vs. Methodical/Humanistic buyers (right):
Typical behavior from a competitive/spontaneous user – only reads the top part of the communication and typically has shorter fixations. While methodical and humanistic users are more likley to read to the bottom of the communication and make longer fixations.
They read search results differently, too - guess which is which:
More eye-tracking / buyer mode goodness over at GrokDotCom: Eyetracking, Heatmaps and Gaze Plots, Oh My!
Opportunity Cost of NOT Testing
A great example is one of Bryan’s clients, Overstock (maybe you’ve heard of them?) who had a 92% bounce rate on this page:
The problem on this page was the “Kids Titles for Learning and Fun” banner. People thought they were searching only kids’ titles. Changing that banner led to a 33% reduction in abandonment rate, 5% revenue lift overall, not just for DVDs and $25 Million — that’s $68,000 per day. (You might want to show this to the HiPPO in your company!)
What to Test?
What to Test First… If You Have Buy In
- Your Top 5 High Bounce Rate Pages
- Your Top 5 High Exit Rate Pages
- Your Top 5 Lowest Time Spent Pages
- Your Top 5 key pages (i.e., checkout, cart, registration, top product)
What to Test First… If You Don’t Have Buy In
- Pick a few PPC terms and landing pages
- Work on an email campaign
- Work where the BPUs aren’t
Hierarchy of Optimization
Start with function - make sure your site does what you need it to do. Make sure it’s accessible - the site’s not down, you don’t hit 404’s when you omit the www etc. This includes accessibility for the visually impaired. Then look at your usability (not to be confused with conversion) which may involve observing people physically navigating your site and performing various tasks. This is all pre-testing stuff.
Begin testing at the Intuitive stage - what can you do to reduce friction in the buying process? Then explore the Persuasive - do people understand what they are buying?
Setting Up Your Test
1. Create a Descriptive Name
- Point of Action Assurance Test on Lead Form
- Identify if this is site wide test or for a campaign
2. Define Your Goal
- I want to increase conversion…
- How will you measure this? What are the KPI?
3. How Will You Achieve your Goal?
- What are you testing?
- What are the variables?
- What are the variations?
4. Define the Control
- What is your prediction/hypothesis?
- What are you basing that on?
5. Let the Test Rip
6. Measure & Analyze!
- What did we improve or not?
- What did we learn?
- What do we do next?
- What do we do next?
Sample Test Setup - Trust Element
Test Name: Confidence Building: Point of Action by Lead Form
Goal: I want to increase conversion by measuring total conversion rate of leads.
Does adding language about our privacy policy and response time improve lead form conversion?
Control: The control form is our current form (A).
Prediction: I predict the form with the detailed point of action assurance will convert best (B).
Simple Tests to Get Started
- Headlines
- Calls to Action
- Re-order Content
- Simple Graphics
- Confidence Building Language
Improving Product Descriptions
Bryan and Future Now conducted a Retail Customer Experience Survey last year, and found 62% of top retail sites had only a brief blurb for a product description, and only 11% had exceptional product copy. Yet, a study by Allurent reports 67% of consumers who visited an online store intending to make a purchase left because the retailer did not provide enough product information.
The E-tailing Group found in it’s study:
- 77% of online shoppers are “very to somewhat” influenced by the quality of content (descriptions, copy, images and tools) when deciding to purchase from an online retailer
- 79% “rarely or never” purchase a product without complete product information
- 76% believe content is insufficient to complete research or purchase online “always,” “most often” or “some of the time”
- When faced with incomplete information, 72% go to a competitor or research further
Got your attention?
Most people abandon because of lack of information to buy. But you can use customer reviews to dramatically improve product descriptions (GrokDotCom’s Holly Buchanan wrote about this, and so did we) and conversion. Amazon reviews and Buzzillions are great places to mine for this gold. Here are some tips:
- Look for products with low look to book ratios & 3 -4.5 star reviews
- Pull the “trigger words” from each review
- Plot them as Logical or Emotional
- Modify your product descriptions based on the results
And you can plot them on a chart like this if you like:
Here’s an example for a Kenneth Cole watch. Old product description:
Here’s revamped copy, taking in the sentiments of actual people from customer reviews:
Kenneth Cole Women’s Stainless Steel Watch
- This unusual double chain bracelet band and watch is an instant attention getter
- Kenneth Cole design features hypnotic mother-of-pearl face and is pleasing to look at
- The high polish silver hands compliment the shimmery dial
- No worries while washing hands, because this watch is water resistant to 30 meters
- Simple to adjust the size…no jeweler needed
- Secure and elegant jewelry clasp
- Case is a slim 9mm and only 18 mm wide
- Bracelet dimensions: 13mm wide & 7.75 inches long
(If you’re using Amazon, be sure to check out Pluribo and read our post on using Amazon Betsellers for keyword research).
More Conversion Tips
- Show pre-checkout shipping calculations (to reduce cart abandonment by people “just checking”)
- Use fewer checkout steps (test)
- Show assurances during checkout process (test)
59% do not provide shipping cost early in the checkout process and 35% have a checkout process with more than 4 steps.
41% do not provide assurances during the checkout process.
Here’s a post about checkout process split-testing.
Influencing Your Organization
Step 1 - Get the Math Right
If 100,000 people visit your website, and 3% of people convert into a desired outcome - that’s 3,000 total conversions, right? If you say your test improved conversion by 1% - does that mean you know have 3,030 conversions or 4,000?
Make sure everyone understands what a lift means. Talk the language, you had a 25% increase in business if it was an increase in overall conversion rate from 3% to 4%, rather than 3% to 3.03%. That’s a BIG difference!
Step 2 - Take Control of the Visit
100,000 visits
x 3% CR
3000 orders
If you increase traffic (invest more in PPC, for example), without optimizing for better conversion, a 33% increase in visits only produces 990 more orders. If you increase conversion rate by 33%, with only 100,000 visitors, you make 1,000 more orders - and this benefit is permanent - where buying more traffic is a one time deal, and you have to continually buy more traffic.
Think about this:
1. You can’t always control the amount of visits, but you can control what you present to the visitors.
2. What would it cost you to double traffic (if possible) versus doubling conversion rate?
Step 3 - Demonstrate It!
Questions and Answers
Many ecommerce sites use dynamically generated pages? How do you test database driven sites?
Google Website Optimizer works as long as Javascript tags for the test are showing up in browser. It will produce the variables you input to the tool, not what is coming out of your database. More details in the book.
What is the minimum traffic you need to conduct an accurate test?
Keep variations small and look for something that has at least 100 actions per week - if not orders, make it add-to-carts. If you have no traffic it’s hard to do less than that.
If something is out of stock, should you remove the product or announce something is out of stock?
Bryan shares a story about a Sony camera that he loves. Recently at a conference he was sitting with a bunch of people discussing how great this (older) model of camera was. They did a product search on Sony’s website and 3-4 other websites, no one had the camera listed. All the rest were out of stock. Since no product pages exist, you can’t even write a review, and you certainly can’t buy what you can’t find! Bryan suggests rather a retailer should revise the page, explain that the model is out of stock and show options for a similar model. Even tell people that the new model retains the speed (or whatever the features that made the previous one a customer favorite) and is available and in stock now.
I wrote about this topic last month, how to use 301 redirects to preserve your link juice and SEO benefits of an existing product page. Jason brings up a good point that there’s a problem with redirecting visitors because you lose the “scent trail” - you don’t want to send people to a 190 model when they were expecting a 180. You never want to pull a bait and switch or cause user frustration. I’ll keep my eyes out for good examples of retailers who keep pages up for a future post.
Is it better to use a long landing page or multi-step process?
There’s no universal answer, it depends on who you are marketing to and what you do. Though Bryan is a fan of breaking up pages — it’s hard to find exactly what you’re looking for on a long page — you can also break it up badly too.
How do you build credibility for a new site or new brand? Without product reviews, what is the number one way of building credibility?
Have an About Us page to start, and “be real.” People know they are not on Walmart.com. Communicate where you excel in service, quality, unique products, clarity, great experience, better content than everyone else, et cetera. Keep testing and you’ll get better results. Don’t forget phone numbers and addresses, live chat, AIM — transparent / visible contact information.
What are ramifications of SEO and testing, is it a form of cloaking?
It’s not, really. Sure, you could use it for the wrong reasons — you must be careful of what you’re doing, you’re not trying to bait and switch. But from the SEO perspective, when a spider comes, it will spider your “control” page, it has no ramifications to SEO otherwise. If you have a radical improvement as a result of your testing and you implement it, you will have to modify your control which may affect your SEO.
Is there ever a reason you would have a successful test, then go back and retrace your steps and perform the identical test - perhaps at a later date, if customers change, economic conditions, seasonality, technology et cetera?
Yes, Amazon is constantly playing with order and formatting of text and product descriptions, for example. If something causes a significant lift in conversion, it makes sense you want to continue testing the same thing - perhaps in slightly different ways. Don’t retest for small improvements - 3% change, for example. But 15%, 20% or more, it’s worth further experimentation.
Next Webinar, Sign Up Today!
We’re excited to have Bill Mirabito, Founder and Principal Analyst of B2C Partners in the house Thursday, October 23 at 9am PST / 12pm EST for Holiday Wish List for Mobile Commerce
Space is limited, so sign up today!
Subscribe via RSS Feed
Comments(4) 
The following is a guest post from Armando Roggio, a web developer, a marketer, the Contributing Editor of Practical eCommerce and a serial entrepreneur with many micro-businesses. 
A Get Elastic reader asked us this question recently:
According to a 
We’ve all heard Confucius’ famed quote “a picture is worth 1,000 words” at least 1,000 times.
This month’s webinar with Jason Billingsley covered various areas packed with examples from real online retailers. You can catch the 
