Data Mining: Text Mining, Visualization and Social Media

Better Beaches

Having recently returned from a trip to Kauai where I used my beach search engine with middling success, I've now got a few updates out on the site.

Firstly, there is a full map showing either all the beaches in a location, or all the beaches from a search within a location. This was a pretty obvious missing feature.

Secondly, as this is an active map, you can zoom and pan the map which interactively restricts the set of results.

There are some minor improvements to other elements of the site as well.

Note - something that always interests me is the relationship between back-end data quality and the presentation of the data. By having a complete map of beaches, it highlights cases where there are duplicates in the results (a topic for another post).

If you are heading to Hawaii - give it a try and let me know how you get on.

Related articles

Snorkel* and Surf* in Kauai and Maui

The State of Hawai'i Demands a New Search Engine

Beach Search Engine Demo

I've written recently about building a perfect beach search engine. Here is a brief example of using the site.

Let's imagine you want to find a beach that offers snorkeling, but you want to find one that is shallow because you have small children with you. A query for 'snorkeling AND shallow' brings up the following results:

(currently the data on the site is limited to Kauai and Maui).

From these, I might decide that I'm more interested in Kauai than Anini (it does appear to have more of the beaches that I'm looking for after all).

Clicking on Kauai, then selecting Anini beach leads me to the following:

Here I can see that there is plenty of content describing the beach. Some of the content contains highlighted terms that lead me to confirm that it is indeed described as a beach that offers snorkeling and that it is shallow. Content from both Fodor's and FourSquare supports this:

A great family park, Anini is unique in that it features one of the longest and widest fringing reefs in all Hawaii, creating ashallow lagoon that is good for snorkeling and following the occasional turtle.... [Fodors]

A nice, shallow place for snorkeling, but can get tiring to swim around if the current is too strong.... [FourSquare]

There are many issues with the site that I'm slowly addressing, but the basic task of finding beaches seems to be up and running. If you are interested, please take a look and let me know what you think - the site is available here.

Building the Perfect Beach Search Engine

Currently, as I've mentioned in previous posts, beaches are a strangely under-served segment of the local search space. Searches on Google and Bing for beaches are fielded by entities such as resorts and restaurants that happen to be matches for certain beach related terms. If you search for 'beaches in kauai' you will get hits for beach resorts, etc.

There is plenty of content about beaches, from the many dedicated locale sites to general travel related community sites (like Trip Advisor) and editorial sites (like Fodor's). In addition, there are a number of resources that aggregate structural data about beaches. These include open data resources like GeoNames and GNIS but also proprietary resources like Foursquare.

Unfortunately, there is nothing that brings all these things together. There is not product which provides an aggregate view of the set of beaches or the collection of things said or otherwise reported about them.

With an upcoming trip to Hawai'i at the end of the year, I wanted to make sure I was getting the best value for my travel dollars. I've build a prototype beach search engine which provides the following.

• a partly curated set of beach data covering approximately 12, 000 international beaches
• aggregation of beach related content
• search funtionality (so you can search for kid friendly beaches that offer good snorkeling)
• summarization of Flickr images so that an impression of what it's like to be at the beach can be formed

I believe there is plenty of potential for such a system. I've already found some hidden beaches that I wasn't aware of at our destination that I'm excited to check out when we get there. My goal is to make the system public in the next few weeks (my trip will be a forcing function for this!).

For now, here is a screen shot of part of the experience.

How Apple's Local and Mapping Investment Will Lead to Web Search

[I work at Microsoft where I work on projects that drive data quality in our local search experiences on Bing and other clients.]

Most of the civilized world, by this time, has heard about Apple's fumble with their new mapping and local search capabilities in iOS. Apple replaced Google's application - which is possibly the largest investment in cartography, imagery and local data ever made - with a home grown solution reportedly rolled out of maps from a number of providers including TomTom and local data from providers including Yelp.

As Apple has realised, there is a lot to learn for an entrant in this space. The hardest lesson they are learning now is actually not about data sources but about metrics and how to assess the quality of the product - something which they don't appear to have invested in in a manner fitting to their global user base.

Apple will soon learn another lesson. Once the fog has lifted over the state of their entity data set (e.g. fixing the location of cities and ensuring coverage for local businesses), Apple will have to start worrying about ranking search results. When a user asks for {kid friendly sushi in seattle} which of the many sushi places ought they to return. They will be presented with a choice between specialized providers - with whom they will actually be in competition - or creating the resources required for relevance ranking themselves.

A key aspect of providing appropriate indexing and ranking features is the association of content with the entities. Where does this content come from? The web. How is it acquired? Through large scale crawling, understanding and indexing.

Apple will likely find that as they pull on the thread of local search, their scope will have to open up to quite a different world, another world which - like local - they haven't yet the expertise in.

The State of Hawai'i Demands a New Search Engine

We will soon be embarking on a short trip to Hawai'i. Naturally, I'm turning to search engines to find out about the best beaches to go to. However, it turns out that this simple problem - where to go on vacation - is terribly under supported by today's search engines.

Firstly, there is the problem with the Web Proposition. The web proposition - the reason for traditional web search engines to exist at all - states that there is a page containing the information you seek somewhere online. While there are many pages that list the 'best beaches in Hawai'i' as the analysis below demonstrates these are just sets of opinions - often very different in nature. An additional problem with the Web Proposition is that information and monetization don't always align. Many of the 'best' beaches pages are really channels through which hotel and real estate commerce is done. Thus a balance is needed between objective information and commercial interests.

Secondly, beaches are not considered local entities by search engines. While the query {beaches in kauai} is very similar in form to the query {restaurants in kauai} the later generates results of entities of type <restaurant> while the former generates results of entities of type <businesses that have beach or kauai in their name or associated content>. While local search sounds like search over entities which have location, it is largely limited to local entities with commercial intent.

Finally, there is general confusion due to the fact that the state of Hawai'i contains a sub-region (an island) called Hawai'i.

To get to the answer to my original search query, I reviewed 8 sites which resulted in a search on Bing or Google for the query {best beaches hawaii}. I then reviewed each of these and created a spread sheet tabling all the beaches and whether they were voted for by the site.

Of the 57 beaches that were mentioned on at least one site, the average number of mentions was 1.89. This indicates a general lack of consensus regarding which are the best beaches. In fact, most beaches (38 out of 57) have only a single vote. Consequently, while there might be a set of pages returned by search engines for queries looking for such information, a user will be reading - in isolation - very different opinions with no aggregate view summarizing them.

The top beaches are summarized in the following table showing the beach and the total votes.

Search engines could do a far better job by:

1. Generalizing local search to include any entity which has location, not just commercial entities.
2. Leveraging editorial content (like that reviewed in this post) so that variance may be exposed to the user but aggregates can also be synthesized.

In addition, there is a very large opportunity here in analyzing the content associated with these local entities to determine which beaches are best for different activities, their accessibility, and so on.

Search Engine Land's Mediocre Post on Local Search

A colleague brought to my attention a post on the influential search blog Search Engine Land which makes claims about the quality of local data found on search engines and local verticals: Yellow Pages Sites Beat Google In Local Data Accuracy Test. The author describes surprise at the outcome reported - that Yellow Pages sites are better at local search than Google. Rather, we should express surprise at how poorly this article is written and at the intentional misleading nature of the title.

The article describes an analysis done by Implied Intelligence. The analysis looks at 1, 000 local businesses in the US. Here is the first problem - these businesses exclude chains and franchises. In addition, if a website wasn't known for the business, it too was excluded. With some general assumptions about the definition of local business, it is safe to assert that firstly there are many instances of chains and franchises out there and secondly that many (if not most) businesses don't have a website (the distribution varies by category of course). Quite where the original sample of 1, 000 came from is not reported.

This biases the analysis - Google, like Bing is intersted in all local entities.

The initial part of the analysis is reasonable - looking at coverage (% in the sample found on the site) and quality (duplicates, phone number errors and adderss errors). Note, however, that this is a measure of the local data, not of local search. A search product includes a relevance component and it is quite possible that a well tuned relevance algorithm might suppress duplicates.

The last table in the analysis sees us swinging back to bad reporting. It describes the percentage of records that have a certain attribute: URL, Hours of Operation and 'additional info'. Did you see what they did there? This is what we call the coverage of an attribute, and it tells us nothing as to the quality of the value. I can quite easily populate a local database with 100% coverage for all attributes. They might all be wrong, but the coverage could be 100%. Consequently, this table is reasonably close to meaningless. If they had included the precision of these values then coverage can be used to compute recall, but that wasn't done.

In summary, an important search publication has either written an intentionally misleading article, or has demonstrated that it doesn't really get data.

Did Web Search kill Artificial Intelligence?

The most commonly referenced definition of Artificial Intelligence (AI) is probably the Turing Test which avoids the tricky questions like 'what is intelligence' or 'what does it mean to think' and replaces them simply by the test of recognizing an anonymous agent as a fellow human (and, therefore, intelligent).

A key aspect of this is, of course, that the interrogator is communicating with an equal in terms of the mode, pace and structure of dialogue (it is possible that a computer could succeed at this deception but reveal itself by being implausibly smarter than a human, but that is a question we can enjoy a little later).

As we have since learned, creating AIs is extremely hard and requires a very large portion of the smart people coming out of a broad spectrum of disciplines including software engineering, robotics, psychology, cognitive science and linguistics.

Another enabler is industrial context which brings the pragmatics of real world problems, and the scale and funding that is often not available to academics and if so doesn't benefit from stark focus required to make progress that is provided by industry.

However, we currently have the following:

• Search engines that don't understand language and which attempt to mediate between people (searches by people and documents by people),
• The best and the brightest coming to work for document oriented web companies.

I can't help but wonder where the AI project would be today if web search (as it is currently envisioned) hadn't gobbled up so much bandwidth.

Bing Visualizes Corporate Diversity

I've just noticed this feature on Bing's finance pages which summarizes visually the diversity of a company's assets in terms of the stock price.

Microsoft:

Google

I can imagine a mashup of this data using eggs and one or more baskets.

Bing Has The Answer

Initially, web search engines were restricted to returning results which were simple pointers to web pages. Modern search engines often include an 'answer' on the first page. This might be a block of information about movies (if you searched for movies) or a small map with some restaurant information.

Bing and Google both present this type of result. If you are a skier, you'll know that before heading out you want to know the condition of the slopes. Searching on Bing for 'snoqualmie' or 'whistler' does just that:

Google doesn't have any answer for skiers as far as I can tell. What I did note though was that if certain snow report websites surface in the results, summar data is extracted:

This is a weaker approach, however, as it relies on the site having the right content and ranking as opposed to Bing strong analysis of intention and curration of appropriate answer data.

Bing, with its rich image and video based home page, gets to show its personality, and these answers shouldn't surprise anyone whose seen today's video:

BTW, there's a line at the end of Gosford Park spoken by the head of the staff which goes something like 'I'm the perfect servant - I know what they want before they know it themselves.'

Web Search - It's Worse Than You Think

While it seems like everything in the online space is hunky dory and progress is making predictable strides towards our inevitable AI infested future, I often see such utter failures in search engine results that makes me think we haven't even started to lay the foundations.

Here's the story: as I've become interested in mining the news cycle for various reasons, I've started attempting to understand who the editors of major news sources are. The current version of the Hapax Page on d8taplex tracks the attribution of article authors and editors (I conflate the concept of writer, reporter and un-typed contributors under the term 'author' while explicit editors are tracked separately). From this analysis, I see that there is someone called Cynthia Johnston who is often associated with articles from Reuters (in fact, she is currently at the top of the list ranked by count of articles).

So, I want to know who Cynthia Johnston is.

I search in Google for 'Cynthia Johnston Reuters bio', and I get the following:

The results present a number of options:

• The result of a search on the Reuters site for the name 'Cynthia Johnston'. It turns out that Reuters has a blog reserved for most of its people but not for CJ.
• An article about 'Bio Gaudiano' olives posted on Reuters and edited by CJ - this is actually duplicated in the results, one with the normal link and the other a mobile version.
• An article posted on Reuters by CJ.
• Various teaser pages from people aggregation and search engines with lists of Cynthia Johnstons.

It is important to note here that there is no evidence that the search engine understands the trivial difference between an article written by someone and an article written about someone.

What would have been a good result here? In the existing 'I can only show you a web page' paradigm, the only reasonable page I can find thus far is here, which contains an out of date bio for CJ.

Cynthia Johnston
Correspondent
Reuters Egypt

Cynthia Johnston is a correspondent with Reuters News based in Cairo. This is her fifth international assignment for Reuters, and her second stint in Egypt. She has previously worked for Reuters in Beirut, Jerusalem and London, and has also reported on short-term assignments from Syria, Iraq, Sudan and many other locations in the region. She covered Israel's 2005 withdrawal from the Gaza Strip and reported from Gaza during several Israeli military incursions there. She was in Beirut for the Syrian pullout from Lebanon, and covered the death of Yasser Arafat from outside his hospital in Paris and then returned to see him buried in Ramallah.

Johnston first joined Reuters in 2000 fresh out of graduate school as part of the Reuters graduate training scheme, a program that involves a year of intensive on-the-job training in London in financial and general news in preparation for future assignments. Her assignments have included a mix of field reporting, writing and editing. Much of her work for Reuters has also involved financial journalism, from covering currency devaluations to inflation crises and privatization efforts.

Johnston holds a Bachelors degree in journalism from Northwestern University and a Masters degree in Middle Eastern and North African Studies from the University of Michigan. She spent two years studying at the American University in Cairo -- one year as a study abroad student and a second year in the Center for Arabic Study Abroad (CASA) program.

But what do I really want?

Firstly, I want the search engine to be able to understand the query. I am asking for a biography of Cynthia Johnston who (I believe) currently works for Reuters. This is not to be confused with

• Articles by Cynthia Johnston published by Reuters.
• Lists of people called Cynthia Johnston who may or may not work for Reuters.
• Failed search results from other search engines.

Secondly, I want the search engine to work with entities, not pages. I also want the search engine to really understand how the web works. A 'page' (HTML data) is not a static resource like a page in a book. It is, in our current version of the web, often an active resource and search engines need to know the difference. The results I show above give me a choice between static data (articles on Reuters), search results (the end point or midway point of some unknown interaction), etc. A good result would present me with a list of entities, below which should be aggregated pages that support the identity of the entity and justify its distinction from the other entities.

Thirdly, I don't want the search engine to be used as a patsy for other sites trying to get my attention (e.g. people search engines).

Fourthly, if the search engine believes it understand the query but thinks it doesn't have the answer, I want to know that. A softer requirement would be presenting to me how confident it is in the top results. If I can see I'm starting off with poor results I will more rapidly modify my query.

My view on things-we-currently-call-search-engines going in to 2012 is that the web-page paradigm is not only outdated, it is fundamentally wrong for lots of scenarios. The entity based view of the world (which accommodates web pages, sites and activities as things in the world just as it does people and places) is already up and running (Yelp for places, Amazon for consumer products, Wikipedia for documents) and search engines need to flip sooner rather than later.

Update: note that now this post is out there, the link I provided to the bio has increased in relevance, thus you will probably not be able to replicate the search experience I describe in the post. Kudos to Google for their real time indexing and reraking technology!