Data Mining: Text Mining, Visualization and Social Media

Microsoft Delivers Clustered Search Results

This is new to me: Microsoft Research Asia appears to be working on clustered search results a la Vivisimo. I saw this URL: http://rwsm.directtaps.net/ in my referer log which give search results with the familiar clustered tree on the left.

It looks like this has been around for a while, at least since the end of January.

Update: Ok this is old - but still new to me. Here is a post from July 5th 2005 (!) mentioning the site.

 

Attenex

Attenex, a company specialising in data/text mining, has what appears to be a slick document clustering/visualization product. I haven't found any clean screen shots, but the one below gives you some idea of what they produce. Their product pages also show some standard, but well presented, components that deliver key words and phrases.

The Strength of BlogAnalytics

A while back, I wrote about how dangerous trend mining over blogs could be in the wrong hands. Stephen Baker writes something similar in a recent post, but he gets the message completely wrong. Stating that blog analytics are weak is not helping anyone. All that is being said in the post is that keywords offer no insight to or protection against the inherent ambiguity of words. Consumer facing search engines treat documents as objective, surface form data: sequences of characters forming words and phrases. Ambiguity is observed when the intent to express two different meanings results in the same sequence of characters being used. Or, to put it the other way around, a word has two meanings.

This is not a new problem, and any competent text mining outfit is going to take care of this issue. That is why any company providing non-trivial analytics over blog data - or any other data for that mater - has already solved this problem. The reason it is not yet visible in the consumer facing search space is often attributed to the lazy user problem. However, I believe it is more to do with the lazy developer problem - not looking for the right interface. Vivisimo is currently leading the way here with explicit representations of ambiguity in the form of clustering analytics layered on top of search results. Google's take on the problem is to make the underlying search engine so powerful you won't need to worry about this type of problem. This is fine if you are prepared to await the arrival of Deep Thought...

Personalize Search Smack-Down

Something is going on with Vivisimo. First there was the Murdoch rumour, then I hear them interviewed on NPR (in a story relating to immigration), now I see this paper by Raul (via Battelle). The paper makes a good argument for why personalized search won't happen for underlying technological reasons. I have to say, I find myself agreeing with Raul's point to some extent. I would add that personalization is another example of making the computer smarter (while preserving the sacred simplistic interface) rather than empowering the user. Vivisimo's clustering engine is a great example of extending the interface: giving the user more control and more information.

I put the question: why haven't other search engines gone for this? on the same stack as: why can't we browse news by maps?