Zappos: where your review makes you look smarter than you are

Posted by & filed under Reviews, Technology.

One of my friends shared a link to the Slate article about review editing, which referred a study about the impact of well-written reviews on sales by Panos Ipeirotis. Well, the study was about improving product prediction, but as a side note authors noted, that well-written reviews, with no grammar and spelling errors, improve sales with Amazon and TripAdvisor. Which is understandable — a review that is overburdened with errors is worse, than something that is easy to read, and if all you have is a handful of barely understandable reviews, you probably won’t buy that product, no matter what you read.

However, in the paper the readability was given linear weight, rather than logarithmic, which I think would be more appropriate. Specifically, if you have two reviews, where one has one grammar error (or misspelling) and one has none, would it really stand out that much. Additionally, data set was primarily electronics- and entertainment- oriented (DVDs, cameras, audio-video), versus, say, fuzzy slippers sales, or sales of things for puppies or kittens (I’ve seen a number of sites where pet owners communicate from the name of their pet, and while I don’t care much that review author claims to be “Fluffers” and coughing up hairballs every other weeks, it is still a bit… odd).

But all of the above is just a prelude to the real revelation. That Zappos is fixing their reviews (yes, I’m late to this party — news about it hit the web at the end of April 2011). Editing them, fixing grammar and spelling. I couldn’t believe my eyes. Frankly, I still can’t, because a spot-check of reviews for these sandals revealed this not-quite-perfect specimen:

I guess this review will magically be “fixed” in a little while, if Zappos continues the practice of sprucing up the reviews in hope of increasing sales.

In my eyes any alteration after the review has been submitted by the user without an explicit indication is lying to readers. Essentially Zappos simply sold out its integrity for a few extra bucks of profit.

Yes, it’s nice when grammar and spelling are perfect. No, it doesn’t mean you’re free to re-hash things afterwards, because you are altering user’s voice without user’s explicit approval. If someone has a hillbilly speech pattern, and you “fixed” it to be nice and clean, you have also changed my perception of who just used your product (and I bet it was your intention all along, right?).

I know I’m not a typical customer, but to me reading a somewhat awkward review of fishing boots would add a few “accuracy” points to the actual usage account versus something that is well polished and borders on marketing-speak. I don’t expect everyone to be an English major and speak eloquently and elegantly, I just expect to have an accurate account by a real genuine customer. If person can’t spell but is great at fishing, so be it, we’re not reviewing English dictionary here. His or her opinion in my eyes will not be lesser because of typos and wrong punctuation.

You also have to ask, how far such “fixing” could go. Swap a few sentences here and there, add/correct epitets, comb over description of how the item was used, and you get an equivalent of payola. At that point you could just ask customer to give a few adjectives, numeric rating, and have “a team of highly skillful writers selected by our marketing department, create a perfect review that will be published under your name”. And unless the page with review has that giant disclaimer at the top, I won’t believe a word I read.

The only possible scenario, would be an extra step when review is submitted — you get back email that says “we have noticed a few spelling errors, here is how we would like to fix it, is it alright we publish review in this form?” and you can click “Yes” or “No” thus giving permission to show “fixed” version of your review.

6 Responses to “Zappos: where your review makes you look smarter than you are”

  1. Oliver

    Yep, I agree. The way something is written is often an indicator of a number of aspects. Intelligence and level of education of the writer (whether accurate or not) are influenced by how something is written. I think it is misleading to transform something improperly written to be correct because you end up implying a higher level of intelligence (and perhaps some other positive traits) than should be. It’s no wonder that that increases conversions, but I don’t feel it’s kosher.

    Reply
  2. John

    ” in the paper the readability was given linear weight, rather than logarithmic, which I think would be more appropriate. Specifically, if you have two reviews, where one has one grammar error (or misspelling) and one has none, would it really stand out that much.”

    I guess you mean exponential weight? Having logarithmic transformation would bring closer the reviews with small and large number of errors, not separate them as you want?

    Plus, readability is not really about grammar and spelling, but rather indicates the required level of education to understand the content of the review.

    Reply
    • Max Smolev

      I’m not sure about the exponential weight. I was thinking that small differences should be compressed. In other words when you compare review with one error with a review with 200, you get significant difference, while adding on top of 200 errors won’t change index much.

      Readability certainly is more than grammar and spelling, but in case of Zappos I haven’t seen anything that would indicate they try to fix style beyond that (throw away seldom-used words? re-write everything in basic English).

      I believe the kind of product that is being reviewed will also severely impact the readability requirements.

      Reply
      • John

        Just took a quick look at the paper. From what I see, they seem to use “log(spelling errors)” (but not log of readability).

        But the argument “if you have two reviews, where one has one grammar error (or misspelling) and one has none, would it really stand out that much” is different than the “when you compare review with one error with a review with 200, you get significant difference, while adding on top of 200 errors won’t change index much”. The first one is an argument for compressing differences with small numbers, while the second one is indeed an argument for logarithmic scaling.

        Reply
        • Max Smolev

          You’re right, so just logarithmic scaling wouldn’t work, needs to be a bit more advanced than that.

          I wonder if there is anyone doing research on more non-tech oriented sales (plush toys, pet food etc — I suppose shoes are not tech, but I haven’t noticed the real numbers in the second paper, though I just skimmed it without reading thoroughly)

          Reply
  3. Cropsey

    My review wasn’t edited for any grammatical errors, but the fact the I mentioned that the product I was reviewing was made in China. They removed that sentence.

    Reply

Leave a Reply