Loading Form...
Apr 25, 2011 | 8 minute read
written by Linda Bustos
I asked some of the top conversion optimization gurus what, in their experience, are the biggest challenges for enterprise testing initiatives. There was unanimity in their responses – the number one hurdle is gaining support from the higher-ups.
“WhichTestWon.com readers have told us it's the largest enterprises that have the hardest time testing, not because they don't have ample traffic or internal resources, but because of office politics. The bigger the organization, often the bigger the political hoops you have to jump through to get something as high-profile as a web page or landing page tested.”
- Anne Holland, founder, Marketing Sherpa and Anne Holland Ventures / WhichTestWon.com
“In our experience, the biggest challenge for testing in large enterprises comes down to culture. Most enterprise organizations' cultures are still risk-averse, opinion-driven and brand-centric. Testing introduces risk that many large organizations are not prepared to culturally embrace - employees are simply not allowed to 'fail.’ Testing is only successful in an environment where risk is acceptable.”
- Brooks Bell, Brooks Bell Interactive
“Organizational challenges present the biggest roadblocks. Most organizations don't have the same level of structure or clearly defined process for making design decisions as they do for enforcing HTML coding standards (even though better design/UX could significantly impact business results far more underlying code structure.) The reality is that most design decisions in most large companies are made either by committee or via the HiPPO method (Highest Paid Person's Opinion).”
- Lance Loveday, Closed Loop Marketing and co-author of Web Design for ROITesting in large organizations may be more difficult because there are more layers of approvals, less tolerance for risk, a lack of well-designed processes for design decisions and a tendency to rely on experience/gut-feel rather than data. If your organization does not come with a testing culture baked in, you may have your work cut out for you in winning support. But there are strategies for winning over your HiPPOs.
“HiPPO” as defined above has also been defined as the “highest paid person in the organization.” Thus, it’s important you sell your case for testing to the HiPPO. However, you’ll find in large organizations, there are many HiPPOs that you need to win over, in many departments and in different levels of your organization. They are not necessarily the highest paid, but they may be anyone High in Position, with Potential Opposition to testing. This could be your boss, the brand manager, the IT department, the finance department, sales or even human resources.
Winning over your HiPPO depends on knowing your HiPPO and seeing things from his/her perspective. What keeps your HiPPO up at night, and how can testing make life easier? What are the fears/uncertainties and doubts (FUD) he/she may have about testing, and how can you educate and motivate? Let’s look at these concerns from the perspective of various HiPPOs.
FUD: Testing is a waste of employee time and moneyThough testing is typically “free” aside from the cost of enterprise testing tools (and potentially consultants), there’s always an opportunity cost when you assign talent to one activity at the expense of another. Testing often involves multiple departments, so there has to be a compelling reason to back up your proposal. Your best bet is to literally “show them the money.” Use visual aids to illustrate the increase in bottom line even a conservative improvement in conversion can have vs. “buying more traffic” through email campaigns, paid search, display advertising, SEO campaigns, etc. A great example from Lace Loveday and Sandra Niehaus’ Web Design for ROI (chapter 2) compares the potential ROI of investing $100K in buying more traffic vs. an equal investment in conversion optimization. (I’ve adapted it into the little table below).
Here you can see even a 2.5% increase in conversion rate results in a 400% increase in ROI vs. the traffic buy. Remix this into a fancy chart and you have a powerful argument! Strengthen your case with examples of other sites that have benefited from testing (especially your competition), and examples of KPI benchmarks that fall behind industry averages. Great resources for test examples are WhichTestWon.com, ABTests.com and the Marketing Experiments Quarterly Journal. FUD: You’re challenging my opinion By suggesting there’s a better way than the status quo, you may be stepping on HiPPO toes. If your goal is to bring decision making authority under your own domain, you always want to be sensitive that a) HiPPOs have feelings and b) the HiPPO is an intelligent person with valuable opinions, often based on experience. Never attack the existing site or decisions, rather approach it from the perspective that not testing leaves other good ideas on the table, and testing gives you the ability to truly understand what works best for your site and your customers. Offer to involve the HiPPO in the creative process. In reality, your ideas may not be any more effective than the HiPPO - you never really know until ideas are tested against each other. Don’t give the impression that you or your team has better ideas or that you want to “fix” the existing site. Rather, you want to test various things to learn more about your customers, their behavior and the changes’ impact on usability, conversion and profitability. FUD: This looks risky…Educate your HiPPO on the benefits of testing. Fist, demonstrate it’s too risky NOT to test (provide examples from other sites/competitors). Second, reassure the HiPPO that you can reduce risk by testing smaller samples (for example, 20% of traffic rather than 50%) for the first test or two, until you can demonstrate some wins and gain the confidence to run tests with larger splits.
FUD: Making changes will undo years of brand building FUD: Some elements on the site are untouchable for legal reasons Both of the above FUDs are valid. Legal is easier to win over – just gain approval before running any test. Branding, on the other hand, is a bit more challenging. “Brand-building is a long-term investment and depends on static, consistent messages. Testing takes the polar opposite approach - rapidly changing, sometimes wildly different messages. Brand-centric cultures are frequently uncomfortable with the idea of presenting inconsistent experiences to their customers. Brand, legal, and IT teams often have political sway, and can easily undermine commitment to a data-driven testing methodology.” – Brooks Bell. Many organizations, irrationally, will protect the brand at the expense of driving sales. Here’s where executive sponsorship can help you tremendously. If you have the support of the highest of the higher-ups, to whom branding is accountable, you’re more likely to get co-operation. Working with brand managers to gain a better understanding of where they are coming from can also help. If you can understand why branding decisions are made, it may change your opinions of what to test, and branding will be more trusting of the optimization group when it knows their concerns are considered. In the end, your hands may be tied when it comes to testing copy, colors, slogans or value propositions, but it doesn’t mean you can’t test anythingon your site.
FUD: This is extra work, and will pull resources from more “important” projects
“If it usually takes 2-3 months for a simple design change to be pushed live, the timeline probably doubles when you're looking to test multiple iterations. And if you're implementing a new testing platform at the same time you can probably double it again due to all the approvals required, paranoia about 3rd party code on the site, troubleshooting tracking issues, integrating with analytics, etc. So if the normal web development process is like herding cats, testing is like getting the cats to stand up and do a synchronized kick line. Not easy.”
- Lance Loveday Furthermore, IT and marketing are not always BFFs. Often marketing views IT as a roadblock to progress, and IT views marketing as fickle and unaccountable. As with the branding team, it’s important to get to know what keeps your IT leaders up at night, and understand their processes and existing projects. Don’t take them for granted or take advantage of them, or expect IT to be at your beck-and-call. In order to make testing an integral part of your company, you’ll need to work very closely with IT.”
FUD: We’re experts. Are you challenging our expertise? While UX teams are the experts in web usability and design, they can also be the most likely to make gut-feel decisions based on generally accepted design principles and “best practice.” For example, usability experts may perform heuristic evaluationsand recommend changes based on experience rather than customer behavior. While observed usability tests can be helpful to identify problems on a page, they are no substitute for quantitative data on what customers with real purchase intent do on your site. Because of the UX team’s influence on design and functionality, it’s KEY that they drink the testing kool-aid. Help them understand that site testing is not undermining their expertise. Rather, their expertise is critical in developing and evaluating tests, and that testing allows them to actually expand their skill set through experience of what really works and doesn’t, as opposed to theory, what Jakob Nielsen says or what worked for “site XYZ.”
You may encounter other HiPPOs, but these are the main ones that could hinder your efforts to launch a testing program. No matter who you need to convince in your organization, make sure you understand what their fears, uncertainties and doubts are when pitching testing. (Not unlike optimizing your ecommerce site, is it?) Be armed with a strong value proposition for testing (how it will improve their lives – not just yours), and be willing to work co-operatively with them (which may include their inputs along the way until confidence is built). Looking for help with A/B and multivariate testing? Contact the Elastic Path consulting team at consulting@elasticpath.com to learn how our conversion optimization services can improve your business results.