Testing the Untestable: An SEO Title Tag Experiment

Words matter. It’s proven that your choice of verbiage can dramatically impact your email open rates, paid search click through rates and landing page conversions. Marketers spend thousands of hours and dollars testing and tweaking text to find out what performs the best. It’s easy with the wealth of testing tools available for your campaigns, but what about organic SEO? Title tags are not just important for ranking, but also for click through. We can expect the majority of searchers to click on the top result, and possibly top few results when searching for information. But for commercial searches, it takes a bit more effort to figure select a search result that’s relevant to the search intent (i.e. an ecommerce site, not a Wikipedia or blog article) and attractive (a familiar domain, a reasonable offer). Your title tag, like your PPC headline, is your small space to shine. But unfortunately, we’re left to make a gut-feel decision on what title tag is best.

Testing title tags: the problems

SERPs (search engine result pages) introduce many uncontrollable variables, such as Google’s penchant for showing different flavors of results, in different areas on the page, for different keyword searches. For example, a search for ‘riding boots’ shows Google Product Search results that can divert a customer from organic listings, star ratings that draw attention to search ads, and brand links to help users narrow their searches.

Notice how the singular “riding boot” and plural “riding boots” show the same page results, but slightly different related Stores results, and omits Product Search results.

If you’re really observant, you’ll notice the related Stores options are different too. Testing is difficult because you have no way of knowing how often these elements are shown or how they may be skewing your results. Not only that, but search positions are always in flux, and vary depending on the exactkeyword phrase queried, the user’s geolocation, whether the user is logged into a Google account (personalized to search history), the “freshness” of results at any given time, and perhaps even involves estimated page load speed and activities of one’s social graph. And did you know that search engines sometimes override your HTML and create their own title for you based on content on your page or what it thinks your page is about? Finally, the biggest hurdle is that even with Google’s own Website Optimizer tool, you cannot tell Google to show one title tag 50% of the time and another the rest. Google will only index your control page. Test versions are served after the referral, if applicable. This makes A/B testing individual pages’ title tags impossible. The only testing you *could* do is in aggregate.

Testing title tags: the workaround

Get Elastic reader Sander Daniels’ team at Thumbtack.com found a workaround to A/B testing title tags in aggregate, described in this case study. Using a home-grown testing platform in its ecommerce back end, the test ran for 7 days, Thumbtack split its URLs into 3 buckets, each containing thousands of URLs. The buckets consisted of a control and 2 challenging formats.

Control: Looking for the best [Service] in [City Name]? Challenger A: Get Free Quotes Today From [Service] In [City Name] Challenger B: [Service] in [City Name] – Get Free Quotes Today

To control for the uncontrollables in search, Thumbtack used the ratio of visits to its experimental titles compared to visits to pages in its control group. Rankings were also monitored, and there was no significant change during the test period. The test ran for 7 days, resulting in a 20-30% drop in traffic to its test groups. Traffic rebounded (for the most part) after reverting to the original title tags.

What to test in ecommerce title tags

If you’re bold enough to try a similar experiment (and have the technical judo chops), here are a few hypotheses you could explore:

  • Keyword vs. store name at the beginning of the title
  • Short vs. long, “keyword stuffed” titles
  • Value proposition such as “Free shipping” or “largest selection of” at the beginning of the title
  • “Shop” at the beginning of the title to differentiate from informational results such as Wikipedia articles and blog posts

Remember, to make it work:

  • You need a fairly large sample size of URLs that you can experiment, and a tool to support it the test
  • Your treatment versions must be tested against your control concurrently, not sequentially
  • Allow enough clicks to accrue to reach a statistically significant result
  • Understand what you are measuring – are search positions affected (for example, if your brand name at the beginning of the title affects your keyword relevance across the board)? Is traffic affected (can you estimate you are getting higher click through)? Are conversions affected (are you attracting the same quality of visitor)?
  • And finally, revert title tags back to normal after the test is over to validate your results

Looking for help with ecommerce strategy? Contact the Elastic Path Research & Strategy team at consulting@elasticpath.com to learn how our ecommerce strategy services can improve your business results.

Tags: , ,


Related Articles

8 Responses to “Testing the Untestable: An SEO Title Tag Experiment”

  1. I’ve always been under the impression that having the store name at the beginning of the title is not as valuable as having the keyword, in terms of conversion, so I wouldn’t like to test that one myself (however would be interesting to see results if anyone else was to experiment ha-ha). The short Vs long keyword would be something I may investigate further.

    • I think it *was* that way, and probably still is for sites that aren’t megabrands like Walmart and Amazon, whose brands at the beginning of the title actually carries more branding benefit than other sites to boot. Search engines seem to be pushing brands more (as we see with the brand and store links that sometimes appear for commercial searches), so there may be some relevance boost to a keyword for searches with brand names in them (for example, UGG boots or Nine West boots are frequently searched, thus giving UGG and Nine West ranking boosts heavier than the weight of having a keyword at the front of the tag). Search engines are continually measuring click through and likely testing positions too, so branded URLs may have a higher click through rate. Unfortunately, we can only speculate!

  2. Mesriani says:

    “And did you know that search engines sometimes override your HTML and create their own title for you based on content on your page or what it thinks your page is about?” – thanks, you shared this, I haven’t known about that.

    Thumbtack.com sounds pretty good for analysis.

    I just want to know how long should I test one title tags at a time?

  3. duran seo says:

    its should be handled extremely careful! messing around with title tags can cause serious ranking shifts. another thing to keep in mind is that once you move if you roll the titles back you are not promised to return to your original rankings…

    btw – from my point of view using keyword stuffed titles is OUT.

    but no doubt that if the shop doesn’t get the results subtle experiments should be made!

  4. Nick Doran says:

    Here’s an alternative method of testing title tag changes that won’t affect your ranking while you test. It was posted by Tom Anthony on SEOMoz and is a method using user testing to measure CTR on search snippet changes. OK, it won’t show you your traffic changes for a given change, but it will give you an idea of the possible impact before you do. That and a whole lot more.
    http://www.seomoz.org/blog/split-test-gather-ctr-analytics-serps

    • Yes I love that article and I found it the day after writing this post. I was going to do a follow up experiment using SERPTurkey, but without a US Amazon account the workaround was a bit tedious. I highly recommend Tom’s method, though. Very cool.

  5. Paul says:

    An interesting approach to a challenging issue. The title tag is the single most important part of your SERP result. If you’re blogging, there is an excellent WordPress plugin called Yoast which allows you to preview how your article will appear in the search engines. Not only does this give you a full results preview, bu itt also gives you keyword density on your target phrase.

Leave a Reply

© 2014 Get Elastic Ecommerce Blog. All rights reserved. Site Admin · Entries RSS · Comments RSS