Cookie Permission: Should You Test That?

Last post we looked at the European Union's Privacy and Electronic Communications Directive and what that means for businesses in the EU, and the UK in particular in light of new regulations surrounding disclosure of the use of tracking cookies, and gaining permission from site visitors.

To recap, to comply with new and impending regulations, websites must gain explicit consent to set most cookies (especially those deemed non-essential and potentially invasive, like web analytics, personalization and third party advertising cookies), or face possible consequences. Asking for permission is most likely done through pop-up / lightbox windows or accordion strips at the top or bottom of web pages. This not only adds an extra layer of friction to the user experience (user must make a decision when faced with the prompt), it also creates anxiety (what are cookies, and why is this a privacy risk) and confusion (what happens if I accept or don't accept cookies).

Ultimately, we can expect site abandonment to rise, at least in the near term until EU residents get used to sites asking all-the-time. This means lower conversions, revenue and site loyalty. Most of the time, when a site element has such potential to harm user experience, web optimization experts will tell you to "test it."

Why AB test now?

Testing tools depend on cookies to serve the right content to the right bucket of test visitors. Once new laws are enforceable in your jurisdiction, you cannot test without being de facto non-compliant by setting analytics cookies without explicit consent. You may workaround this problem by setting analytics/testing cookies initially upon site arrival, disclosing in the prompt and linking to information how to opt out of tracking. (The ICO itself sets an essential cookie at the beginning with the option to delete all cookies). Technically, since an analytics cookie is not essential to the functioning of the website, you may be walking the fine line of non-compliance.

That means to be above board, testing must occur while there is still grace time.

Secondly, testing now allows you to measure the impact on performance metrics with and without asking for permission. Once the grace period is over, you can no longer use a control group that does not see your popup/accordion prompt to accept cookies without being non-compliant, and you'll never be able to quantify the negative impact on your business. You will know what to expect when it's rolled out fully to your site.

More importantly, consider testing variables such as the clarity and persuasion of your request's copy. Learning what converts best now means you will be able to reduce your risk long-term. Again, you will not be able to properly split test without cookies being set upon visitor arrival in one year's time.

But testing your opt-in requests today also has many caveats.

Why testing cookie permission is a challenge

The big problems are:

• Testing something you know is going to slaughter conversions is usually not pushed to the front of the queue
• Allowing test users to opt out of testing and analytics cookies means no tracking after the conversion event (in this case opt in, out or abandonment)
• If users opt out, you are expected to honor their request forever. Allowing them to opt out before you absolutely have to ask means losing out on data and disabling site features that could improve conversion rates
• If running a cookie permission test concurrently with other tests, it's not guaranteed the tests will play nicely together.
• This type of test may have a short half-life, meaning that consumer behavior is expected to change over time. As people get used to encountering permission requests, they are likely to become more tolerant, and respond to prompts differently than today.

Like all quantitative analytics, AB tests can only tell you the what, not the why. When you optimize your permission process, it's important to identify why users abandon a site. What anxieties do they have? What are their attitudes toward cookies? Do they even understand what the prompt is asking? Would they abandon a site to look for alternatives? These are questions traditional user testing can answer.

However, user testing as we know it is also an imperfect solution. Recruiting users, bringing them in for a test and crunching the data takes time and money. If you want to test the clarity and persuasiveness of your permission request across a number of alternative versions, you will need to perform several rounds of user testing. Likely not worth the effort.

A hybrid approach

An alternative that combines the qualitative benefits of user testing with the multi-version, hands-off approach of AB testing is using a service like FiveSecondTest.com, EasyUsability, 3rdPartyFeedback.com or Feedback Army, which allow you to upload an image to be tested by a panel of user testers. (See Bryan Eisenberg's smokin' list of low-cost testing tools for a description of each). With FiveSecondTest, for example, the testers see the image for 5 seconds and must answer a question you create. You could run 4 tests with 4 different combinations of copy/design and ask testers what they understood the site to be asking, and what action they would take at that point. You would only need to use a screen shot of what your prompt would look like, with no changes made to your website at all (a digital version of the "paper prototype.")

What to test

You may test any variable that may impact the "conversion rate" for accepting cookies, including:

• Presentation (popup/lightbox vs accordion)
• Copy (tone, length, motivators / value props for accepting cookies, headline)
• Calls to action (label, size, style, color, placement)
• Point-of-action assurance (copy, motivators)

ICO's prompt uses an accordion presentation at the top of the page

A lightbox presentation in the center of the page

The lightbox example is courtesy of David Naylor. Check out the rest of his satirical permission requests for a good laugh.

When optimizing your copy, it's critical that your message is informative, persuasive and eases the visitor's anxiety about cookies and your use of them. Next post we'll look at tips for writing your "permission slip."

Looking for help with ecommerce? Contact the Elastic Path consulting team at consulting@elasticpath.com to learn how our ecommerce strategy and conversion optimization services can improve your business results.