Data Mining: Text Mining, Visualization and Social Media

Rich Client, Poor Client

Recently, in discussing clients for text mining applications, I've found the distinction between Google Maps and Google Earth to be a useful point of comparison.

With Google Maps, we have a think client which is little more than a script enhanced browser capable of pulling images from a server and intercepting textual and gestural input. It doesn't know anything about the geographic or physical world per se. In fact, the browser in which it is implemented is designed to understand documents, and doesn't even do that well.

With Google Earth, we have a rich client which has an intimate understanding of the geographic and physical world. It is capable of drawing pictures in 3D, of dealing with logical descriptions of cartographic features (such as borders) and has a rich form of interaction.

Using this analogy with text mining clients, we can compare browser based interactions with those provided by a rich client. The browser implemented client has some obvious attractions, such as the ability to access your application almost anywhere. However, it doesn't know anything about textual data. With a rich client, you can build capabilities into the application that enable a richer and deeper analysis. The caveat with this approach is that the data, or some portion of it, needs to be situated on the client machine. Our ability to enable this is a function of bandwidth, server response times, disc speed and CPU power - things which will always be improved.

In my opinion, the rich client path leads to a far more exciting application - compare Google Maps with Google Earth and see what I mean. By extension, I believe that the web in general is bottlenecked by the browser - a client that doesn't really understand the data that it provides interactions with (hyper text documents). There will be a time in the future when the client for the web will be unrecognizable when compared with today's browsers. The question is: are we going to get there by evolution, or will there be some innovation that allows us to leapfrog to that point?

Web Site Tracking

Lifehacker has posts to two different web site tracking systems: Feedwhip and DeltaSpy. I've not played with these, but note:

• Tracking web sites like this should be a major tool for anyone bent on corporate intelligence.
• This is not a new idea - Intelliseek in its early years - produced a very popular product called BullsEye which did just this, Whizbang labs also had a product (not released) called Web Diff which also provided this functionality.
• Why can't I get these results in an RSS feed?

[I see that Lifehacker earlier pointed to InfoC, another one of these tools].

Google Analytics Update

John Battelle recently posted a conversation he had with SimplyHired CEO Gautam Godhwani in which Google's ability to deliver applications was questioned. Battelle's position takes the view that applications need to be built on top of search, which is, from a technology point of view, not really accurate. Search is an application built on the analysis and representation of data. Other applications can also be built on similar or differing analyses. For example, if I am going to build an application to deliver automated product review summaries, I need something to analyse the data to detect product mentions and opinions - this is not built on top of search, it is text mining, computational linguistics, etc.

Anyway, the point about Google's ability to deliver applications is timely. Their recently launched free version of Urchin - the web site stats tool - couldn't be more embarrassing. Firstly, the service died on the first day leaving even paying users without stats. Secondly, I keep getting a message telling me that data will be available within 12 hours. That works for the first 12 hours, but after that first period it just looks silly.

The rest of the deployment is also poorly executed. When clicking the link to learn about who has access to my data, I get the message:

The requested URL /support/analytics/bin/answer.py?answer=26786 was not found on this server.

What is forcing Google to deliver such rubbish? Didn't they test it?