Due to the speed at which travelers are sharing their experiences on the site, TripAdvisor found instances where a newly listed property was able to skyrocket to the top of the rankings on relatively few 5-bubble reviews. They called these properties “fast-risers”. Over time, TripAdvisor travelers would submit more reviews on the fast-risers, causing them to settle into more stable, accurate rankings. Unfortunately, while this transition was happening, TripAdvisor consumers may not have been seeing the most accurate rankings for the destination. Fast-risers would temporarily enjoy higher positions before falling significantly in the rankings, and other businesses would appear lower than they would have otherwise.
To improve the site experience for travelers and businesses alike, TripAdvisor has enhanced its Popularity Ranking algorithm. The enhanced ranking results in a more accurate representation of a business’s performance over time. These changes were carefully designed and tested to improve the rankings algorithm in some very specific ways, while maintaining the accurate standings of existing properties with great reputations among TripAdvisor members.
In designing the enhancements, TripAdvisor analyzed the hundreds of millions of reviews received over the past 15 years and how they impacted the rankings of properties over time. Based on this research, they redesigned the enhanced Popularity Ranking algorithm to value the quantity and consistency of reviews more significantly than in the past. They believe doing so helps stabilize the ranking for all businesses, reduces fast-riser behavior and creates a more accurate overall ranking for travelers. The enhanced algorithm has been tested extensively – both internally and by analyzing how travelers interacted with the site as they’ve gradually rolled it out.
When did this happen?
In January 2016, TripAdvisor switched to using the enhanced Popularity Ranking algorithm for Restaurants and Attractions. Between February and April, they rolled out the algorithm for Hotels. During the roll-out, there were some cases where the rankings would move independent of new reviews being collected. Those ranking changes were due to the enhanced algorithm accounting for more and more of the calculation for each property over time.
The enhanced algorithm is now being used on the vast majority of the site, and TripAdvisor claims the resulting Popularity Rankings have stabilized.
How the Enhanced Popularity Ranking algorithm works
The goal was to design an enhanced algorithm where a property settles into a stable ranking more quickly and avoids the fast-riser behavior. The Popularity Ranking continues to be based on the quality, recency and quantity of reviews that a business receives from travelers. The final algorithmic input, for a small set of properties that violate the guidelines, are fraud penalties that exist to protect the integrity of the content on TripAdvisor.
Quality
The bubble ratings provided by travelers as part of their reviews continue to be used to rank properties. All other things being equal, a property with more 4- and 5-bubble ratings will rank higher than a business with lower bubble ratings.
Recency
TripAdvisor believes that recent reviews are more valuable to travelers than older reviews. They give a more accurate representation of the current experience at the property. To take this into account, more consideration is given to fresh reviews over those that were written in the past. This means that reviews – even excellent ones – that are more dated will not count as much towards a property’s ranking as a review written last week. Even though these reviews do not have as much weight in the ranking, they are still visible to travelers in the Traveler Rating bar chart, in the overall bubble rating on each listing as well as in the review history.
Quantity
The number of reviews is a critical indicator to TripAdvisor travelers about a property. TripAdvisor consumers read multiple reviews to help form a balanced opinion on a business and tend to have more confidence in their decisions when they see agreement across a large set of fellow travelers’ reviews.
The enhanced algorithm quantifies this intuitive notion of confidence in a statistical way. As TripAdvisor accumulates more reviews on a property, they have more confidence in the potential experience it provides to guests. Once a critical mass of reviews is reached, they’re able to more accurately predict that property’s ranking.
When TripAdvisor talks about quantity of reviews, it is important to note that a property doesn’t have to have more reviews than others. It just has to have enough reviews to allow for a statistically meaningful comparison to other properties. For instance, a business with 110 reviews isn’t necessarily going to be rated higher than one with 100 reviews that were received in roughly the same period of time. This is because they both have enough reviews to make the company confident in the potential travel experience they can deliver.
In summary
- More reviews are better than fewer reviews;
- Good reviews are better than poor reviews; and
- Recent reviews are better than older reviews.
These factors interact to determine a property’s Popularity Ranking.
For example, the quality and quantity of reviews are compared to assess the consistency of the experience provided by the property. The new algorithm does a better job of rewarding consistently good behavior: A property that has many consistently good reviews will rank higher than one that has many reviews, some of which are good and some of which are poor, all other things being equal. That’s because we can have more confidence in our ranking if a large number of travelers are reporting consistent experiences at that property.
Similarly, recency and quantity of reviews are closely linked. When a property has lots of recent reviews, they are more highly-valued in TripAdvisor’s calculation because they improve the confidence in the current experience at the property. A large number of reviews that are several years old will not provide TripAdvisor or travelers with as much confidence on what the property is like today.
The rankings are re-calculated daily based on new reviews received that day. At the moment, the sub-ratings (cleanliness, service, etc.) do not factor into the ranking calculation. Note also that management responses are not factored into Popularity Ranking. However, research shows that when an owner responds promptly and professionally to a review, addressing any specific complaints as well as the positive comments, it can make a big impact on prospective customers.
Some smaller properties have expressed concern about the increased weight on the quantity factor. In practice, TripAdvisor’s analysis found that while smaller properties tend to receive fewer reviews because they have fewer customers, the personalized attention, service and care that they can provide often results in more of their guests being willing to write reviews. As a result, they’ve found that smaller properties are still able to compete with larger competitors.
