Raveable: hotel appraisal

Raveable is a new review aggregation site which specializes in hotels. It claims to provide information for 55 thousand hotels in the US culled from more than 35 million reviews. The site mines these reviews to automate the summarization of opinions from guests for the hotels.

The data

Firstly, where does the data come from? Paging through a number of hotel sheets, it initially appeared to me as if most of the data comes from TripAdvisor. The site states that the data comes from a number of sources, including TripAdvisor, Expedia, Travelocity, etc. as well as individual blogs. Browsing the site brought more of these sources into view – in fact, the experience suggests that some hotels get more data from one source than the other. It would be interesting to know the demographics of those sources and to consider a more representative summary taking the demographics into account. Has Raveable formed relationships with those sources?

The UI

In general, I found the UI clean and pleasantly put together. One of the most important aspects of presenting data for a choice like hotel booking is to get some understanding of the ranking of the candidates and the coverage of the system (how many of the competing hotels are in the system). The UI goes some of the way here by providing an overall ranking:

However, it would be nice to get a bit of additional geographic context that shows other hotels near by. In addition, with the added layer of appraisal (summaries of reviews) one could get a picture of the general quality of the hotels in the neighbourhood. Am I looking at a good hotel in a forest of good hotels? A good hotel in a sparse neighbourhood?

The Interesting Part

Of course, the interesting part about this whole thing is the automated summarization and extraction of opinions written in natural language. With 636 reviews available, on average, per hotel, there should be good opportunity to deal with accuracy issues by doing something sensible with summarization and presentation (note that I’ve not yet seen a hotel with > 600 reviews so I’m not sure if all the reviews are assigned to hotels in the system).

My expectations for the extraction of opinions was that the positive stuff would generally be pretty accurate, but negative comments – always harder than positive to recognize – would show some amount of error. Some anecdotal examples (pulled without too much hunting):

• Our room was on the 3rd floor we were offered a room at the back or the front, we chose the front whats the point of being in new york with out the noise. [Casablanca Hotel]
• The rooms are not large but are more than adequate. [Casablanca Hotel]
• You can see the old bank vaults in the basement and the registration area on the first floor is where the tellers used to be. [Drury Plaza Hotel]
• Housekeeping was called and eliminated the two problems. [Camelback Inn]

These false positives are interesting in terms of what they tell us about the algorithms that compute the sentiment. There is some background value associated with certain terms (noise in the first example triggered a negative tag; large in the second and problem in the third). The trouble comes with the valence associated with the terms by the surrounding context.

I don’t feel that these problems are really a weakness in the system. There are always going to be errors. I believe there are two ways to combat this problem in general. The most obvious is figure out a way to introduce another level of summarization in the UX. There is high level summary for each major category. One could try something simple, like a tag cloud, that summarizes within each category in a way that would allow the user to get a finer grained, but still aggregate summary of the content.

Comments

[Dmitriy]

This is by a couple of ex-MSFT folks I know.

[Vijay R]

Have you seen uptake - http://www.uptake.com (previously kango)? They seem to have a better ontology to search hotels on - for instance, they understand "kid friendly" while raveable doesn't. They also have activities. I haven't evaluated how the opinion mining of the two companies compare, though.

It will be interesting to see what work exists on using opinion mining to flesh out domain ontologies.