The biggest problem with Tripadvisor and other user reviews — reliability

by Janice Hough on July 29, 2011


No, it’s not to my mind all the fakes, although that is a continual problem. Yet another article, this time England’s Travel Weekly indicates.

The article notes that the Sunday Times posted an article that talks about sites where hotels are offering to pay writers for fake reviews. I have heard stories of hotel guests being offered freebies like meals or drinks to write a positive review. But, this is the first time I have heard about ads soliciting fake reviews.

I sense that there is another issue that’s potentially a bigger impediment to online review credibility — generally only the extremes prompt travelers to write.

Speaking from personal experience, and I’m a part-time travel writer, I don’t usually write travel reviews after staying at hotels. It’s not an aversion to the idea, but there are only so many hours in the day, etc.

I know that some travelers are religious about writing reviews of all their hotels. But most people don’t work that way. They write if something is fantastic and they write if something is awful.

Plus, if you read between the lines “fantastic” may have more to do with who they were with at the hotel or one great gesture by an employee. “Awful” can be one employee’s rudeness, getting sick, or having a hotel not upgrade a discount room.

I’ve also noticed that Priceline and other “opaque site” type bookings engender a lot of comments, either from people who are thrilled to get a decent sized room with indoor plumbing for a good price, or from people who don’t read the fine print saying that bedding and views cannot be specified.

Recently, I wrote a review where the hotel was good but not fantastic. But, honestly, the reason I wrote it was because the hotel, an airport Marriott, had so many poor reviews.

I wanted to simply add a positive opinion. “Yes, it is an older property and the bathrooms aren’t big,” I wrote, “But the staff was nice, the rooms were perfectly comfortable and the bedding was new.” I concluded, “Travelers with an early or late flight would have a decent stay at a relatively reasonable price.”

My sense is that Tripadvisor realizes the problem too. Because they have been sending me regular emails since, telling me how many people have read the review, how many think it is “helpful,” etc., informing me that if I keep writing I will be a higher level contributor.

As far as combating the problem, Tripadvisor and other sites would say that people just need to start writing more reviews to help their fellow travelers and themselves — that is essentially free labor.

Failing that, my advice is simply that as much as possible, take aggregate opinions over individual reviews and try to read between the lines. In addition, get opinions from multiple sources. If you have a travel agent, ask them if they have any special resources. (Some travel agents, for example, use Star Service, which is still opinionated since it’s written by individuals, but they are paid and assigned the reviews.)

Consider that even in the age of seemingly unlimited user-generated content, look at actual professionally written guidebooks. Finally, if you really are just looking for a clean, comfortable room, remember such rooms don’t usually inspire rants or raves.

Print Friendly
Be Sociable, Share!

  • Manisfeld

    I agree with most of what you write except for “aggregate opinions”. The problem with aggregate scores – the big number displayed – is that often they are the average of 1 or 2 or 3 reviews, which is not sufficient to garner a range of views to effectively aggrgate/average (you’d need at least 20 reviews to do that).

  • SgFm

    I do read individual reviews on TripAdvisor, but only if that individual has written at least 2 or 3 reviews. I skip right over the ones that are written by someone who clearly could be a shill (or doesn’t care to repeatedly contribute), and therefore has no reliable online reputation on TripAdvisor.

  • http://www.facebook.com/profile.php?id=576001031 Jeff Linder

    I look for patterns.  If there are a number of reviews complaining about X (or praising it) I consider that as a factor.

  • janice

    Good point, should have said “aggregate when there is a significant number of reviews.”   Otherwise you are better off with an AAA book or something.

  • SoBeSparky

    A planted review on Trip Advisor has many obvious clues.  One is that the reviewer just joined TA and posted only one or two reviews. 

    Second, it is that everything was just perfect, but nothing is specific.  “The rooms were stylish and the front desk always helpful.”  This review is garbage and the score discarded.

    Third, as noted above, when one thing tees off a reviewer, then just stop reading.  Strong emotional responses overpower facts every day of the week.

    Fourth, consider the cultural context of the reviewer.  If a reviewer did not fill out a location and TA profile, that is another reason to just forget the review altogether. 

    As a tip to all seeking hotel review reliability, most hotels have an accompanying “%” number in the upper right on the rating tab.  That is the number of people willing to recommend the property.  That is the acid test, not the bullets or stars.  One or five fake reviews out of 100 will not substantially tilt the % recommendation rate.

    Some five-star properties have 55% recommendation rates, while two-star properties have 90% rates.  Which is better?  The former did not meet the expectations of almost one out of two people.  That latter was consistently of high quality, given its category and amenities.  I would rather be in the second with a clean simple room and a great helpful staff, than a luxe room with dirty carpets and a rude staff.

  • Bob Cowen

    I also consider the date of the review.  Hotels often change owners so a bad review years ago does not carry much weight if the recent ones are good.  The converse is also true.

  • Marilyn

    I interpret the suggestion to look at the aggregate as seeing the trend in the reviews rather than just looking at the numerical score at the top.  If repeated reviewers complain of the same thing, then you can probably count on its accuracy.  I also look at the types of things that people complain about.  Complaints about lack of cleanliness or noise weigh far more than complaints about the decor or view which aren’t of much concern to me since I don’t plan to live at the hotel.

  • Ton

    “that is essentially free labor. “i don’t agree with that. What you do is adding to a source of information that will benefit others and will benefit you if you need advice.

    You give and you get back. I agree that some information is difficult to value, but if there is a trend it is usually right, information about the area can be usefull to. I have been saved twice, 1 time the review came with pictures (burns around sockets, fire exits blocked, dirty rooms. The other one alerted about the fact that the street the hotel was in was also used for street prostitution.

  • Anonymous

    I don’t just look at the star ratings or any one or two reviews.  Like others have said, these can be shills or just a really unhappy camper.  I want specifics or I consider the review to be pointless and not worth my consideration.  

    I also look at the content because it can also be too specific and I look at dates of the reviews.  Old reviews aren’t necessarily reliable other than to track a consistent problem.  But when I see  a bunch of ratings all grouped up like 3 in a week but none before and none since, I’ll figure someone was trying to bogey the scores.  Ratings cheaters tend to pool their results in a short period of time, they tend to repeat some of the same words or phrases from review to review, particularly to describe the same thing.  When I see reviews that tell me how great Bill the front desk manager is or that Charlene the waitress gave them the best service they’ve ever had at any restaurant ever and forever, I actually count these reviews against the establishment because they are self-serving ads, not reviews.

    But establishments can also get a really bad rap.  As hard as TripAdvisor works to make sure people understand the star system, there’s always that one or two people that don’t get it, and they think 1 star is the best rating.  I’ve also seen 1-star ratings because the user said, “the Egyptian cotton towels could have been softer”.  It’s sad when a nincompoop screws up the ratings and reputation of a well -run and high quality place because of something so foolish.

    I do however, also look specifically at all the really bad ratings and reviews.  If the reviews all mention the same problem, I know it’s something to look out for and I want to know if it’s something I’d care about.  If they got all these lousy reviews only because their airport shuttle royally messes things up, and I’d be depending on them for ground transportation, then I can’t stay there no matter how nice the place is.  Conversely, if I’m not taking a plane, then these lousy ratings don’t matter to me.  If everyone complains about the pool and I’m not there to swim, then I don’t care.

  • Anonymous

    I don’t just look at the star ratings or any one or two reviews.  Like others have said, these can be shills or just a really unhappy camper.  I want specifics or I consider the review to be pointless and not worth my consideration.  

    I also look at the content because it can also be too specific and I look at dates of the reviews.  Old reviews aren’t necessarily reliable other than to track a consistent problem.  But when I see  a bunch of ratings all grouped up like 3 in a week but none before and none since, I’ll figure someone was trying to bogey the scores.  Ratings cheaters tend to pool their results in a short period of time, they tend to repeat some of the same words or phrases from review to review, particularly to describe the same thing.  When I see reviews that tell me how great Bill the front desk manager is or that Charlene the waitress gave them the best service they’ve ever had at any restaurant ever and forever, I actually count these reviews against the establishment because they are self-serving ads, not reviews.

    But establishments can also get a really bad rap.  As hard as TripAdvisor works to make sure people understand the star system, there’s always that one or two people that don’t get it, and they think 1 star is the best rating.  I’ve also seen 1-star ratings because the user said, “the Egyptian cotton towels could have been softer”.  It’s sad when a nincompoop screws up the ratings and reputation of a well -run and high quality place because of something so foolish.

    I do however, also look specifically at all the really bad ratings and reviews.  If the reviews all mention the same problem, I know it’s something to look out for and I want to know if it’s something I’d care about.  If they got all these lousy reviews only because their airport shuttle royally messes things up, and I’d be depending on them for ground transportation, then I can’t stay there no matter how nice the place is.  Conversely, if I’m not taking a plane, then these lousy ratings don’t matter to me.  If everyone complains about the pool and I’m not there to swim, then I don’t care.

  • J.J

    I do read and/or glance others’ reviews before I plan a trip. I admit that there are some just “complaining to complain”. Most of them have legit reasons, I feel. When I write a review, I try to keep a positive tone of voice, unless the situation is really bad. For example, my son and I were bit by bugs in a LaQuinta Inn in Peru and THAT WAS BAD. I uploaded pictures on corresponding websites as well as wrote emails to La Quinta corporate.

Previous post:

Next post: