Q&A From Ratings and Review Webinar

The following is the question-answer period of a webinar we did with PowerReviews. Unfortunately we do not have a transcript of the webinar itself or a replay due to technical difficulties, but there is still value in the following information if you're curious about customer reviews for ecommerce sites.

Questions and Answers

What about moderating "dirty words" from reviews? How much of an effort is that?
Jason Billingsley:PowerReviews offers a profanity filter and it is basically automatic. Other review solutions like Bazaarvoice likely have the same capabilities. If you are using an out-of-the-box system included with your ecommerce software, it will vary - there will be no 1st level moderation by a service provider so you will be on the hook for ALL moderation. I have seen numbers in the 10% range for the amount of reviews that need some level of scrubbing.

What about spam like "get this product from www.competing-site.com instead of buying it here"?
Jason Billingsley: Retailers often state in policies that linking to other sites is prohibited. In the moderation process, reviews with such content can be flagged and removed or edited. I suggest contacting the reviewer to let them know of the policies and why the review was not accepted. Always follow up with negative reviews if possible. It gives you an opportunity to learn, correct mistakes, and turn an unhappy customer around.

How do you handle negative reviews or low ratings?
Jason Billingsley: We cover this at around the 25 minute mark of the webinar.

Is there any comparative information about the use of overall ratings and broken-down ratings by criterion?
Jason Billingsley: I am assuming this is referring to the benefits of having one versus the other. There have been no numbers published to my knowledge, but I highly suggest using criteria when asking for a rating. Criteria help a user quickly identify what is relevant to their needs. Shoppers place value in different areas. A vacuum that is quiet may be more important to me than one that is versatile, durable, or powerful - criteria based ratings help identify where a product excels or fails.

We are a publisher of education books. I am wondering about how our authors would feel about us publishing comments on their books, as it's different from a product manufacturer relationship?
Jason Billingsley: Authors must have a thick skin for reviews in general. The truth is, books are already being reviewed on Amazon and other book retailers' sites anyhow. By working with the author to disseminate the review information, future content can be corrected or supplemented (I am assuming multiple editions if they are educational texts). We were recently at Book Expo America and saw first hand how the author/publisher relationship can be tricky. Likewise, the author is essentially a manufacturer of content (The Book), so the same dynamics should apply as a retailer-manufacturer relationship.

Do you have any examples of reviews being effective for clothing/apparel?
Jason Billingsley: Many top-end apparel retailers are implementing reviews. I worked closely with James Covert on a story for the Wall Street Journal on this topic called Retailers try on online clothes reviews for size (The Post Gazette has the full article re-published without a registration required)

Any experience in not-for-profit or government sectors using reviews to rate customer satisfaction?
Jason Billingsley: None off the top of my head, but there are consumer review sites like Yelp that address services rather than products.

How do you verify someone who bought the product in your store?
Jason Billingsley: There are typically two ways or levels of verification. The first is a check on whether the review was submitted by a human and not a bot. This can be checked with email verification. The second is by comparing the reviewer data with your store data to make sure they have actually made a purchase. This can account for offline purchases as well if you are a multi-channel retailer. No wonder so many stores are asking for email addresses at checkout these days. If the concern is to only accept reviews from YOUR customers and not people who bought the item elsewhere, then you must chose to only display verified purchasers reviews. Keep in mind, this will likely hinder your review acquisition though.

Where can I go to get the CompUSA and Petco case studies?
Jason Billingsley: Both of the companies are clients of Bazaarvoice. I could only find the original Petco case study, the details on the other were gathered from the CompUSA press release.

We can't spend any time on moderation. How does someone like PowerReviews moderate the reviews? What criteria do they use?
Andy Chen: In addition to automated profanity filtering, PowerReviews provides a 2-level human process. The first level of moderation if performed by in-house, PowerReviews employees. Our analyst team works with clients to develop standards for "constructiveness" (positive or negative) of the reviews based on that specific product category, and we ensure that the review is primarily about the product vs. the service or price. Additionally, we flag reviews with special statuses such as: customer service, safety issues, testimonials, product content error, and many more.

After PowerReviews has moderated the review, our clients have the option of performing a second level of moderation OR they can moderate by exception (low ratings, escalations, etc). Clients can also add a "merchant response" to any review. At any point, the client can log on to their Dashboard to pull down a review that has been published.

If I sell the same product as my competitor, will customers have a greater tendency or inclination to purchase the product from my store just because I have user reviews on my site?
Andy Chen: Great question. We have not studied this in-depth yet. However, we have done a few test with our clients and discovered that the presence of customer reviews greatly enhances the Credibility and Trust for that site (i.e. they help answer both "is this the right product", but also "is this the right place to buy").

Any filters for obscene images in reviews?
Jason Billingsley: There are no automated filters for this to the best of my knowledge, but level 1 moderation by a reviews service provider will pick this up. No questionable material should get through to the site as it will pass through a human first.
Andy Chen: Our analyst team moderates ALL images for nudity, obscenity and relevance.

How about rating a user?
Andy Chen: We currently do not provide that capability. In our backend, PowerReviews DOES monitor which user reviews have been flagged as inappropriate/helpful/not helpful, but there is no public display of the user's "credibility"

For B2B, can there be a way for the buyer to review the seller’s ability on delivering the product with the qualities that they said they will. At the same time, can the seller review the buyer’s performance on how good of a customer they are, such as how often they buy, and how often they return product?
Jason Billingsley: This seems like a marketplace feedback mechanism. Very eBay feedback like. Though reviewing the seller is beneficial (some comparison engines do it and Yahoo! Shopping has seller ratings too), the product is where the review has the most benefit.
Andy Chen: Great idea. However, we do not currently have a robust "eBay style" capability to review the sales process or the buyer.

How do the panelists feel about some type of system that allows manufacturers or brand representatives to discuss a negative review or resolve an issue - something like eBay's feeback system - to keep the reviews fair for both parties? Has this been done before?
Jason Billingsley: Google just introduced this type of functionality in Google News (people mentioned in a news story can clarify remarks - it is all moderated, but nonetheless). I have not seen this within review content, but I like the idea. Currently, most companies deal one-on-one with negative reviewers. Your suggestion could help them close the loop.

Approximately how many customers will take the time to leave positive reviews? What about when you're initially start off and have zero reviews? Wouldn't this give a bad impression to new customers?
Jason Billingsley: Current estimates have positive reviews (4 and 5 star ratings) at over 80%. We talk about acquiring reviews and the impact of having too few reviews at around the 33 minute mark of the webinar.

Will you discuss the need to tie-in ads that show product reviews on them with the relevant product page (or category page) once the customer clicks through? We've seen a lot of companies get that wrong lately.
Jason Billingsley: Maintaining a scent for the shopper is really important. If we are attracting a user to the site by dangling review data in-front of them, we better land them on a page that actually contains that information. The folks over at GrokDotCom demonstrate a retailer who is using review data in ads, but do not have review data on the landing page - ouch. We talk briefly about using review data in title tags and seeing what effect it has on click-through and conversions from search engine results pages. We forgot to mention the benefit of social bookmark sites and social commerce sites automatically including the rating data in listings if it is in the title quite often.

What's your opinion on aggregating Amazon reviews to get your review page started (I've seen this on some sites) does this impact SEO postively or negatively? (in terms of duplicate content?)
Andy Chen: As part of Amazon's AWS license, you must include a "buy it at Amazon" link with the content that you display. This is a good idea if you want the content, but you may inadvertently divert sales away from your site. As for SEO, our believe is that the benefit is mostly attributed to the owner of the content, in which case, that would be Amazon. Our advice would be to send an email to your previous customers to ask them to fill out a review on your site. This generates lots of review volume (~5% response rate to the emails), provides an opportunity for a repeat purchase (give a coupon?), and creates proprietary content for your own site.

How do you keep product reviews from being used for flame wars?
Andy Chen: Because most of the reviews from our clients are generated via a "follow-up email" (e.g. 3 weeks after the order ships, the client sends an email to the customer requesting a review), we have not seen any flame-wars occur. That's the value of the "Verified Purchaser" concept...the most credible reviews are from customers who have actually purchased the product vs. users who want to rant.

How was data collected about indication that 63% of users are more likely to buy? Is it opinion, or did you look at data of actual buying?
Jason Billingsley: Those numbers were gathered from a study by iPerceptions and CompUSA in a case study on reviews

Does it matter what type of product has reviews?
Andy Chen: PowerReviews has over 180 clients that sell products from Office supplies (Staples) to Furniture (NetShops) to Underwear (Essential Apparel)...and we are always surprised about how reviews are used to help other customers make the right purchase decision. We find that just about every purchase decision is important to that shopper. For instance, this simple Stapler has 18 reviews already.

Do you pre-populate the keywords/tags, or is it entirely from customer responses?
Andy Chen: PowerReviews works with its clients to pre-seed the tags based on the merchants' expertise. However, we have consistently found that the customers create more relevant and useful tags than even the best merchant can predict. PowerReviews manages over 2000 category- specific templates that would have been impossible for create and maintain without the help of actual customers.

Do you find that because some terms are presented as "these are commonly-used," that people just tag these in a "lazy" fashion? Are they perhaps just going along with the crowd?
Andy Chen: Yes, we find that there is a bias toward the listed tags vs. adding a new tag. We are continually testing the interface to determine how to best minimize the impact of this effect. Understanding this, as we promote/demote tags, we take into account the fact that a "add your own" entry has more weight than a checked tag.

I think it would be hard to figure out how to keep negative reviews up if you have control over the reviews - isn't there a great temptation to suppress things that make you uncomfortable?
Jason Billingsley: You are right, so it takes dedication to an increased level of transparency and engaging customers at a new level. There has to be complete buy-in to the strategy that negative reviews help balance positive reviews and add authenticity and realism. They actually help the positive reviews.

Is there any ability for reviewers to re-post reviewed products on their blog/site if their is an affiliate arrangement?
Andy Chen: Absolutely...and no affiliate arrangement necessary. Our clients have been using this functionality since January and have garnered hundreds of blog postings in this manner. Here's a great example of a camera review blog post.

By the way, this is a tremendous SEO tool because each blog posting links back to your site and to your product page.

What do you believe are some of the best review programs? What are the best freeware ones?
Jason Billingsley: PowerReviews has a 'free' option that is performance based and also has an enterprise option. Bazaarvoice is another vendor of review solutions, but is not free. There are some simple JavaScript based apps out there too but no true freeware solution I know of besides what comes bundled with some ecommerce platforms.

About what percentage of posted comments can one expect to be disingenuous, from sites/suppliers attempting to market through review systems?
Jason Billingsley: It really depends on the market. Electronics is a nasty market (vendors) vs sporting goods where it's very cooperative. Regardless of the market however, it a tiny fraction of a percent.