Data Mining: Text Mining, Visualization and Social Media

NSA Coverage: Update

I see that Technorati is now doing some limited trending over tags:

The above is the trend for the tag NSA (Google and Technorati need to get the search term in the graphic if they want to make these things portable). I say limited trending as the link associated with the chart that is meant to take you to the full chart interface (as with their normal search) is broken.

A Tale of Three Tag Clouds

With the Wall Street Journal's redesign, we now have 'tag cloud' views of three different publication genres: blogs, boards and online news.

Here is a recent cloud from Technorati:

Here is a board cload from BoardTracker:

And here is the Wall Street Journal's cloud generated from search results:

It would be nice to believe that we could use these tag clouds to compare what is going on in either the respective media or the users of those media. However, the term 'tag cloud' has broken out of the constraints implied by the word 'tag'. Technorati's tag search tokenizes tags into individual 'words' (thus making it possible to search for the 'tag' the). I'm not sure if the cloud itself contains these pseudotags. The treatment of tags as plain words or strings means that the tags don't really share a common and distinct semantic space but are more like keywords. Note, however, that a search interface for tags (i.e. searching the space of tags, not the space of tagged posts) would be the proper place for this type of tag decomposition.

The BoardTraker tag cloud shows tags provided by users of the search engine. The tags are at the thread level.

The Wall Street Journal's tags are not really tags, but search terms. Here the term 'tag cloud' has come to mean the form of the visualization, not the type of data being visualized. In some sense, this demonstrates the strength of the visualization, but further weakens the notion of tags and tagging.

Christopher Brooks presented an interesting paper at the recent Symposium on Computational Approaches to Analysing Weblogs:

An Analysis of the Effectiveness of Tagging in Blogs, Christopher H. Brooks and Nancy Montanez

Which discussed some of the issues surrounding tags and the notion of folksonomies.

WordWorks on Tag Clouds

WordWorks has an interesting deep dive on tag clouds. There are plenty of examples in the post of different approaches to tag clouds and their various merits. Interestingly, all of these examples are for reasonably well behaved tags (atomic symbols). I'd love to get an opinion on the nightmares of multi-word tags, morphology and so on.

On a related note, due to the broken nature of tags, Technorati (and possibly other tag search services) have to tokenize the tags to index the tagged posts in blogs (actually, the don't have to, but they do). This means that you can search for the 'tag' the and get a result. Great: I think I really need to know what people are saying about the these days - is it an emergent tag I wonder? After all the debates about the value of a free, unconstrained tagging space, the fact that you can implicitly tag posts with contentless words says alot.

Technorati Provides Blog Search

Technorai's blog posted about their new find-a-blog service:

By default, the blogs are presented in order of authority, which means highly-linked blogs appear first. So each of these Blog Finder pages is like a mini Top 100 for any topic you can imagine. You can also sort each tag by how recently the blogs have updated, or alphabetical by title.

This looks like a response to all the uber list complaints, and interesting in light of my earlier post on the opening in the blog space for just such a service as well as discussion on what tags really should be used for. I'm not convinced that ranking by tag authority is the right way to go for the ideal solution (a brief scan of some results suggest this is a light-weight solution), but getting something going in this area is something of a milestone.

Fun with Tags

In a recent post (August 5th) I finished off with:

The big win with tags is going to be the applications that are built on top of inferences made from them.

I think a link to Flickr's new tag clustering would have been appropriate (launched on August 1st). This is just what I am talking about (putting aside the argument that tags on photos are real meta-data and tags on text aren't). I've played with the usual tests (jaguar - it's a car and a cat, etc.) and haven't yet found results that I really didn't like. Some examples:

• Edinburgh: the city, the G8 protests, ...
• Golf: VW car, sport,...St Andrews
• Vacation: beach,..., Italy, Paris, Hawaii,...

With automated clustering, it is not too difficult to start doing searches and pulling things apart critically. For me, I found the system useful and fun.

Trouble with Tags

I was very pleased to read this post by Om Malik (profile) on tags. It is hard to find criticism of tags, but this does a good job of summarizing the problems (as well as pointing to some other posts on the topic). Sifry's recent post on tags quotes Clay Shirky:

This is something the ‘well-designed metadata’ crowd has never understood — just because it’s better to have well-designed metadata along one axis does not mean that it is better along all axes, and the axis of cost, in particular, will trump any other advantage as it grows larger. And the cost of tagging large systems rigorously is crippling, so fantasies of using controlled metadata in environments like Flickr are really fantasies of users suddenly deciding to become disciples of information architecture.

First of all, tags on text are not metadeta. You can not legislate metadata by a syntactic construct. In other words, tags are just words. It is only when tags become references to a new system of symbols with a clear relationship to object data that they can be called metadata. Okay, so I'm making a slightly stronger case here than even I believe in, but the point has to be made. The systemic result of this is that there are many tags that you need in order to capture everyone's idea of metadata. It is ironic that Sifry's post on tags includes such repeats as blog, weblog, weblogs, just to make sure that everyone gets the idea of what the post is about. Clearly these all refer to the same thing. In addition, they all mean the same thing to David. David knows that they mean the same thing to others, but he still has to use them all.

Shirky talks about cost. But he is only talking about write time cost, not the cost that the spaghetti has on the system over the course of the life of the content.

And I am not done yet. When I look at my referrer log, I can see which tags bring more traffic. With a simple utilitarian model of a blog author, how do you think I would react to that knowledge?

The big win with tags is going to be the applications that are built on top of inferences made from them. The tags will become the object data and the inferences will become the new metadata. But hang on, can't we just do that from the text, like tagCloud does?

UPDATE: Kevin Burton comments with a pointer to his posts on tagging issues where he discusses the problem of third party tagging. Note that this is yet another issue - the stuff I am talking about is more to do with the semantics of the author's tags, though the semantics issue doesn't go away with third party tags.