Data Mining: Text Mining, Visualization and Social Media

Re-Educating Search Users

Briefly, there are a couple of products out there that are intent on getting search users to change there 2.4 words per query habit. The most visible might be Siri (though recently it was operating at a zero word per query rate due to an outage). But perhaps a more subtle one is 'A Google A Day'. At the site quiz questions are presented which essentially require long form queries (and often more than one) to solve. By creating such a site, Google is indicating that it wants to train users to leverage what it believes are its strengths and what will be viewed as the emerging area of natural language search.

Bing Local Gets Social

There was an announcement yesterday from Bing describing a very broad set of new updateswhich bring social features powered by Facebook to the Bing experience. These features hit both the main web search experience as well as verticals.

In local search, we have put together a feature which - when available- associates either the Facebook page or Twitter identity (or both) with the entity surfaced on the details page. Below, for example, is the description of John Howie in Bellevue.

 
We believe this feature is going to be useful to users who want to connect with the establishments that they are interested in. However, there is something more subtle at work here. By connecting these three different identities, we are providing a mapping across three distinct models of the world. Many exciting opportunities for data quality and user experience are enabled by this association.

When Data Mining Goes Horribly Wrong

Historically, the local search space was defined by aggregates of business listings purchased from companies whose original focus was various types of advertising and direct marketing. As the space has evolved, search engines have looked at ways to gather better data and, importantly, to merge different sources of data into a single unified view of the restaurant or business being indexed and served as the result of a query.

Consequently, when you see a details page - either on Google, Bing or some other search engine with a local search product - you are seeing information synthesized from multiple sources. Of course, these sources may differ in terms of their quality and, as a result, the values they provide for certain attributes.

When combining data from different sources, decisions have to be made as to firstly when to match (that is to say, assert that the data is about the same real world entity) and secondly how to merge (for example: should you take the phone number found in one source or another?).

This process - the conflation of data - is where you either succeed or fail.

Take this example from Google's local search product.

The entry is for a company called the West Seattle Blog - certainly one of the best hyperlocal blogs I've come across. This is clearly the identity of the corporation that publishes the popular hyperlocal blog covering news and events in the West Seattle neighborhood.

As the places page indicates it is an owner verified listing, which means that the business owner has confirmed and contributed some of the data.

However, because Google gets data from many sources, it has conflated these additional data sets with this authoritative listing. One of the local entities that it has conflated with the West Seattle Blog is the West Seattle Bowl - a bowling alley in the same neighborhood but at a different address. While it is easy to see how this has happened (the names are identical apart from a few difference between the words blog and bowl which could easily be accounted for by a typo on the part of either data provider) it is unusual in that the addresses of the entities and many of the other features conflict.

By adding the attributes of the two businesses together, we see that the hybrid entity has the categories 

 

Again, due to the mashing together of the data, the images - for a blogging company - are actually those for the bowling alley. In addition, a search on Google for 'west seattle bowl' brings up this listing and no listing for the bowling alley per se.

Because Google's local search, like most local search companies, provides reviews for businesses, this search result contains snippets of text reviewing the company. But compounding the problems of mis-associating these two entities, because the blog itself posts reviews about other local businesses, these too have been associated with the Blog / Bowl informing the user of the great chinese food, etc.:

Another feature of this result is that there are links (listed under reviews) to articles in the Seattle PI (a local newspaper). I imagine these have come about due to the journalistic nature and linking of the West Seattle Blog. However, the article one immediately reaches via the Seattle PI link on the page is entitled Shooter in West Seattle slayings reportedly had mental problems - not exactly ideal copy for potential customers of a bowling alley in that neighborhood.

I certainly believe that the mining of web content and the inference performed on multiple signals of data are two crucial technologies in the local search space, and I have plenty of respect for what Google has achieved in these areas. However, this example (which I came across via some original local search I don't recall) just seemed like an amazing example of a mutated associated graph with interesting consequences worthy of sharing.

Language Really Does Matter For Search

While most pundits regard the deep, formal semantics promised by the likes of Powerset as not important to search I feel that I am personally finding search dead-ends in my long tail queries that clearly indicate the need for this type of feature. I will commit the sin of using a single example to support my point.

The query 'embedding javascript in feeds' returns results on Google for using javascript to embed feeds in HTLM pages. While there are only 4 words in the query the order of these words and the interpretation of the word 'in' are vital in getting the results I'm looking for.

Perhaps there are no actual pages online that will answer my query and the incorrect (they are not alternate, they are wrong) interpretations are backfilling. But this is useless to me as a user. I'd rather know that the engine understands what I'm looking for and can tell me 'you won't find what you are looking for in our index.'

For clarity, the issue I'm interested in here is the behaviour of most feed readers of stripping javascript and other attack vectors from RSS and Atom feeds. I know there are resources out there that discuss this.

Update : here is a suitable hit for this query : element should not contain script tag from the w3c's feed validator documentation.

Data Diligence: More Thoughts on Google Books' Ngrams

Since my earlier post on the new trending tool provided by Google Books, I've been thinking more about the service. While I've found plenty of interesting trends (more of which later), I've also been considering the underlying data and interface.Many of these considerations are common to any trending or other data probing interface (such as BlogPulse).

While there have been lots of reasonably visible copy written about the opportunities presented by the data set - the potential to understand trends in our culture and linguistics - this enthusiastic data geekery is somewhat lacking in data diligence. The original article in  Science, for example, doesn't describe the data in the most basic terms.

At the very least, the data needs to be described in terms of design, accuracy and bias.

By data design I mean the intentions of the data. These intentions are somewhat exposed in the interface (where one can choose from things like 'American English', 'British English', etc.). I'd love to understand the rationale behind some of the corpora - e.g. English 1 Million - and the reason for missing corpora (we have English Fiction but not English Non-Fiction).

The accuracy of the data, with respect to the design, can at least be considered in terms of the current specifications. How accurate are the years associated with the articles? How accurate is the origin of publication? In addition, as Google points out, the accuracy of the OCR is also of great interest, especially for older texts (Danny Sullivan has an interesting post on this topic).

Finally, given any data designed along a set of dimensions, one can always take another set of dimensions and see how they are distributed and correlated - if at all. For example, what is the mixture of fiction and non-fiction in the English corpus? What is the distribution of topics? Are these representative with respect to historically accurate accounts of linguistic and cultural shifts (e.g. the introduction of the novel, the impact of the enlightenment on the mixture of fiction and non-fiction). What is the sampling from different publishing houses and is that representative of the number of books, or the number of copies sold? This last point is intriguing - does a book with 1 million copies in circulation have more 'culturonomic' impact than a book with only a single copy out there.

While the data sets are clearly labeled as 'American English' and 'British English' the books in those collections are not actually classified as such. Rather they are defined by their country of publication. With this in mind, how do we interpret the color v colour graph from my earlier post? As Natalie pointed out in an email, the trend in 'British English' of the difference between these terms could be described either by an underlying cultural shift towards the American spelling, or by a change in the ratio of American books published in the UK without editorial 'translations'.

Searching for foreign terms in certain languages brings up hits for the foreign language (e.g. 'because' in the Spanish corpus, 'pourquoi' in the English corpora).

Regarding the English Fiction corpus, I was surprised to see mentions of figures and tables in works of fiction.

Drilling down on these in the interface surfaces what are clearly non-fiction publications (but it is not clear if this search is filtered by the various corpora visualized in the ngram interface). It is also important to bear in mind when looking at these anomolies the volume of hits. Here we are seeing very small fractions of the overall corpus containing what look like terms indicating false positives.

Another subtle, but easily missed (I missed it!) aspect of the interface is that it is case sensitive. This allows us to do interesting queries like 'however' versus 'However.'

How do we interpret this? The most obvious interpretation might be that 'however' at the beginning of a sentence is becoming more frequent. We could also conclude that 'however' in general is becoming more frequent (imagine if we could combine the lines). Alternatively, it could mean that sentence length in the corpus is shifting. Given that we don't know the exact cultural mix of the 'British English' corpus, it could be somehow related to the mixture of American and British content. Finally, it could be due to the mix of fiction and non-fiction. Interestingly, the 'American English' corpus has quite a different signal.

When investigating temporal data, it is always interesting to try to discover things that don't change over time. What words would we expect to be relatively stable? From a simple initial probing, it seems like numbers and days of the week are reasonably stable. In looking at this, I did find that certain colours come and go in a very correlated pattern.

Overall, I find this to be a hugely exciting project. I'm disappointed in the general lack of analysis given to the data set before jumping to conclusions, but perhaps this is more a reflection of the blogosphere and the quality of writing. I'd love to see a more in depth analysis of the corpora provided by the team that wrote the Science article.

 Update: Read more at The Binder Blog (1, 2), and at the Language Log.

Trending Terms in Google's Book Corpus

I love this new feature in Google's book search product which allows you to look at the time series trends for terms according to the publication dates of books. The example below shows the trend for the tokens 'colour' and 'color'.

This type of statistical analysis brings up lots of questions, simultaneously about the occurrence of terms in books in general, and the distribution of books in Google's collection. Does it show a decline in the ratio of British to American publications? or a decline in the British spelling of colour? or a bias in the corpus towards recent American publications and earlier British publications? Hard to say, and interesting to ask if probing only via this tool one could find out.

 Update: I'm actually quite serious in being keen on understanding both the distribution of terms in our language and the nature of the collection. While this article on ReadWriteWeb rightly celebrates the insights that this data set can bring, it lacks in any questioning on the representative nature of the underlying data set.

Update: It seems Chomsky throws the tool off. In searching for 'manufacturing consent' I'm shown this artwork:

This is probably a positive sign indicating a lot of interest in the tool - though surprising to see this in a Google product.

Excel-ence in Search

These days, I find myself executing many complex queries and - like everyone who is employed by a search engine - maintaining my search behaviour as far from the norm as possible. My focus of late has been in exploring the world of excel documents available online. Consequently, I've been looking at the difference in search experience between Google and Bing.

Firstly, to find excel documents, one can execute a 'filetype:xls' query in the search box. This works on both Bing and Google. Right there, one is exposed to major differences. Bing estimates it has 38, 100 hits, peanuts compared with Google's claim of 18, 000, 000.

Both Google and Bing provide some ability to inspect documents without having to load the actual excel data. The semantics are, however, a little different. Bing's 'cached' page (which all results appear to have) is actually an HTML version of the excel document. This is very useful for making quick judgments about the value of the hit. Google has both 'cached' and html versions of the document.

From the point of view of intuition, I find the Google design to be a little simpler, being explicit about cache and html viewing.

However, the problem with Google's implementation is that their cached pages are not always present:

In addition, not all Google pages have a 'View as HTML' option. So on the one hand you have fewer results in a less intuitive but consistently implemented interface, and on the other, a far larger collection with clearer semantics but with inconsistent implementation.

Comparing the HTML views of the spreadsheets, I find Bing's to be a little better, with Google's rendering often appearing with reams of whitespace and strangely repeated elements.

Unfortunately, I don't even believe all of the above summary. If you include a search term in the query - 'filetype:xls per' - you get back from Bing an estimate of 2, 810, 000 results, and from Google 828, 000 results. Who knows which of these numbers is correct - but Bing's indication of a super set of the more general 'filetype:xls' query is troubling.

Much of the evaluation of search engine technology focuses on the 'head' - do they get searches for movies right, do navigational searches for well known brands work, etc. However, I think by looking in the niche areas one can get an understanding of the rich opportunity that lies in the data available online as well as the manner in which this opportunity challenges the popular optimizations of search engines.

Search, Semantics and Shopping in Paris

It is quite common, in the social circles of the technorati, to observe individuals madly accumulating piles of random household goods while rapidly snapping away with mobile devices in order to demonstrate the quality of applications like Google's goggles, or Bing's iBing. These applications, among other things, provide image based search. That is to say, using images as input to search.

On one such occasion, I and a friend were testing Google and Bing's abilities with a certain CD cover: Martino Tirimo's recording of Debussy's Suite Bergamasque. The CD cover features Basile Lemeunier's "Shopping in Paris" and is pictured to the left.

Imagine our surprise when using this CD cover in image based search brought (I'm sure you've guessed it) the book "April Lady" by Georgette Heyer, pictured below.

Of course, the confusion was caused by the assumption of unique artwork used on covers. The Google application faired better of the two as it has access to a larger database of possible matches. This database covers not just products (that can be keyed off cover art), but also art works (possibly via online poster retailers). Bing, on the other hand, was only aware of the book. The features used to key the products are, firstly, overwhelmed by the nature of the artwork and, secondly, not intended to distinguish between types of objects. They don't 'see' a CD cover or a book or a poster.

This seemed like an interesting issue, and I bookmarked it for this blog post. However, in getting the facts straight for writing this, I had to recover the original art work. I had the picture of the CD cover and vaguely remembered the name of the art poster that Google picked up (which I thought to be "Shopping in April"). However, I couldn't for the life of me encourage the definitive information of the name of the picture or the author out of either Bing or Google. 

Searching on Google for "shopping in paris" brings up various lurid images of Parisian consumerism and a certain American socialite. Adding in the terms "fine art" didn't help at all. It was only when I came up with the idea of searching for art poster online stores that I managed to home in on the original picture, and the artist: Basile Lemeunier (France, 1852-1922).

 

 

The Power of Bing (t-shirts)

I was chatting with a neighbor this weekend. He's an architect, and, upon noticing that I was wearing a Bing t-shirt, mentioned that now at his firm most everyone has switched to using Bing. The reason being that the image search was superior to Google's. Bing's image search, in addition to providing different - in this case, better - results, innovated the page-less scrolling of results.

The model of attracting people via a key feature, which beats the competition, then encouraging them to stay for all the other features is, naturally, what the fight for users is all about. Here's an article from the New York Times, via Techmeme, which illustrates the feature war with a number of similar stories. Such stories are, of course, anecdotal, but each user is one by their own anecdote.

Google's Bing-like Image Background

Fascinating to see Google' come out with a Bing-like image background.

Note that some work is needed on the fit and finish to avoid readability issues like this: