As a hotelier, it is hard to ignore the constant emphasis on testing from the likes of Booking.com and Expedia. It is no wonder that many hotels are convinced that A/B testing is the right choice for them – they have seen it employed to such great success by their biggest rivals. At ITB 2018, Booking.com’s CEO Gillian Tans made no bones about the extent to which her team are constantly iterating on their product. “Everything you see in our product, in our advertising or on our website has been tested. If it’s there, it’s been tested. “
The idea of a ‘perfectly tested product’ is increasingly integral to the organizational identity of many OTAs, and Booking.com in particular. This creates a pressure on hotels that is difficult to ignore.
There are several reasons, though, why hotels and OTAs aren’t on the same playing field when it comes to testing. But that doesn’t mean hoteliers should lose hope. Let’s take a look at some of the main differences between hotels and OTAs when it comes to A/B testing.
1. Test sensitivity
OTAs have the necessary economies of scale to invest heavily in data infrastructure and achieve accurate testing results for minute UX (user experience) changes. They can change the colour of a button and test the impact, because they can achieve a sample size that enables them to detect tiny uplifts.
The vast majority of hotels don’t have the necessary traffic to be able to do this. While measuring tiny uplifts (such as a 0.1% increase in conversion rate) is worthwhile for huge OTAs, it almost certainly isn’t for most hotels. Hoteliers should be focused on identifying areas for significant improvement rather than testing each button change.
2. Data scale
Due to their scale and the breadth of their user base, OTAs have a far greater number of reference points by which test results can be assessed. Traffic to OTAs is made up of millions of different types of consumer, which means OTAs can test the impact of a change on any number of different user groups. OTAs can see which changes have the biggest effect on which type of person, and why. The largest hotel groups can perform this kind of analysis, but a single property or a small group only has a very narrow traffic base to test on. It is difficult to capture many confounding variables on such a small base.
3. Knowing the guest
Hotel websites are about far more than just converting a guest. They’re about providing a great experience. One thing an OTA will never be able to do is actually meet and speak to the person who is booking a room through them. While you might not have millions of visitors to your website each month upon which to test, your hotel is full of potential research subjects. One of the simplest things a hotelier can do is just ask each guest about their booking experience as they check in.
The hospitality industry is defined by human relationships. While OTAs are only focused on one specific part of the guest ‘journey’, hotels are necessarily invested in the entire trip – from booking, to pre-stay, to stay, to after the guest has gone. The knowledge and understanding that this gives hotels is often underestimated. A hotelier can apply their ‘soft’ knowledge of guests to how they optimize their website in a way that OTAs cannot. A hotel website does not have to be all things to all people, as an OTA does. Really understanding your guests will set you well on the way to an optimized website.
About the author
Lily McIlwain is the Content Manager at Triptease.ÊAs A/B test providers to some of the world’s largest hotel groups, Triptease is uniquely poised to comment on the myths and misconceptions surrounding split-traffic testing. ”Spotlight onÉ A/B testing” sets the record straight on why independent hotels are unlikely to be able to run statistically significant tests, why hotels and OTAs are on a very different playing field when it comes to testing, and why a 10% uplift in an A/B test doesn’t translate to a 10% uplift in your bottom line.
For a free download of the full Spotlight on… A/B testingÊreport, click here.
