Data Mining: Text Mining, Visualization and Social Media

Tags in Board Space

A post on a BoardTracker forum pointed to the use of tags on Slashdot. In addition, the post highlighted the use of tags on BoardTracker's own forums. Sounds like there will be more tag based features to come:

Tagging is very popular these days, particularly among bloggers, and of course BoardTracker has brought tagging to forum threads too as you can see in the 'topic tags' above. For an example of these tags displayed on boardtracker have a look here..

http://www.boardtracker.com/search/?search=site%3Aboardtracker.com

We are rolling out several new tag related features for boards soon also.. watch this space! 

BBC adds Categories/Tags to News Feeds

Kevin Hinde, of BBC News Interactive, just posted this to the backstage mailing list:

Dear Backstage,

quite a few of you have asked for item-level categories on our RSS feeds
on this list in the past, and since Monday we've been producing them for
all our News and Sport feeds. Hope you like them, let us know what you
think, sorry it's taken so long.

Cheers,
Kevin.

This is pretty exciting. Looking at the categories, they appear to be a mixture of topics and locations. I'm wondering how I'm going to mix them into the datasphere view of news.

Technorati Kitchen

Library Clips points to Technorati Explore, the first offering from Technorati's Kitchen. The kitchen looks like an alpha/beta feature testing ground, similar to Google Labs, or the BlogPulse Showcase where we initially launched trend search and conversation tracker:

This is where we put the new stuff we're cooking up while we're still testing it out. Be warned! The stuff in the kitchen may not be perfect - that's why we haven't served it up yet! But if you'd like to give it a taste, we'd love to know what you think.

Technorati Explore gives a view of what people are talking about in the blogosphere very similar to the original idea behind BlogPulse. However, they add two additional features. Firstly, it provides categories, grouping hot links into a number of classes including Gadgets, Search, Business, Design, etc. Secondly, it is updated more frequently:

This page shows what's hot in blogs on specific topics right now.

The market is getting pretty full of link ranking/recommendation systems, both automated (memeorandum, blogdex, BlogPulse) and supervised (TailRank, del.icio.us - currently down FYI). Evaluating them is not easy - they mostly appear to give interesting but different results. One very obvious issue with Technorati's offering is that the number of links for the top ranked posts in each category is very low. In addition, the times associated with the recommendations seem pretty old when you consider the 'right now' promise. This is partly due to the fact that the default ordering is by authority (in fact, it is something called recent authority which, I imagine, is a function of authority and recency). It goes to highlight the fact that mixing authority or relevance with recency is really problematic with a standard list-like interface.

The top ranked post in the Gadgets category was posted 13 hours ago and linked to by 2 other posts. These two posts were posted 7 and 11 hours ago. The post is from Gadget Spy, who ranks 6, 159 in Technorati. In fact, Gadget Spy has cornered the Gadget category with the 4 top posts at the time of writing.

The Future of Tags: Cloudy

Generally, an informative visualization of any data should:

• Make as much of the data discoverable (e.g. by providing some form of indexing)
• Have clear relationships between the values associated with  variables and the relative amount of ink used in rendering symbols of those variables.

Of course, there are many more similar statements that we might make, but these, in my mind are two of the most important.

Tags - which I generally distrust - are often represented by tag clouds. Tag clouds, in a certain popular form, index their content by alphabetising the tags - you can scan through the tags to find a certain tag. They indicate the frequency (or other variable) of the tag by the size (generally the font size) used to render the tag.

Considering this last point - it means that the amount of ink that a tag receives is a function not only of the frequency of the tag, but also of the length of the string. In general, if the tags are all of a similar length, then this is not an issue. However, when people start using tags with length greater than, say, 10 characters, they can gain visibility not consistent with their frequency of use.

As I said, I don't like tags too much - or rather, they are being hyped for qualities inconsitent with their usage. However, I would offer the following tag guidlines:

1. Don't be tempted to use morphological varients. E.g. Sifry's use of 'tag', 'tags', 'tagging' is noisy (and only goes to further the argument that folsonomies are broken).
2. If you find yourself placing conjunctions in your tag (e.g. 'blogging and search') then you probably want to use two tags. Multiple tags may indicate a conjunction of topics.
3. Don't decorate your tags with syntactic elements: 'Always-on Panopticon...or Cooperation Amplifier' (Howard Rheingold - fellow Corante Network blogger).
4. If your tags are made public, chose tags that are informative to not just your regular readers, but to a reasonably informed new reader (see #3).
5. Be consistent. Don't use 'blogs' in one post and 'weblogs' in another. If you find yourself using a disjunction in your tag, it probably means you don't have a clear description of your topic, or that you are spamming tagging systems by using synonyms.

Tagging is a very broken idea (I mean, capitalization!). I've always believed that they are worth keeping around because someone will build neat tools on them (e.g. flickr's tag clustering is pretty cool). Tag clouds are interesting, but can amplify the types of problems I have outlined above. Have a look at the tag cloud on Corante's Web Hub, and compare it with that created by Joi Ito. But then, it is no surprise that Joi gets tagging...

Riffs, Tags, Amazon

Playing a little more with Riffs - it looks like they have imported a lot of Amazon's catalogue. If you do a search, you are likely to find a lot of items that no human has entered. These items all have direct links to Amazon's product pages. The weird thing is that they are still associated with 'common tags'. These tags look very machine created. Are they pulled from Amazon somehow? If so, it sort of breaks the whole point of folksonomic tagging (unless you consider Amazon to be one big intelligent agent - which would be cool!)

For me, this somehow breaks the whole image of the human created, community content that Riffs wants us to think it is all about. Is it just another Amazon affiliate?

Fun with Tags

In a recent post (August 5th) I finished off with:

The big win with tags is going to be the applications that are built on top of inferences made from them.

I think a link to Flickr's new tag clustering would have been appropriate (launched on August 1st). This is just what I am talking about (putting aside the argument that tags on photos are real meta-data and tags on text aren't). I've played with the usual tests (jaguar - it's a car and a cat, etc.) and haven't yet found results that I really didn't like. Some examples:

• Edinburgh: the city, the G8 protests, ...
• Golf: VW car, sport,...St Andrews
• Vacation: beach,..., Italy, Paris, Hawaii,...

With automated clustering, it is not too difficult to start doing searches and pulling things apart critically. For me, I found the system useful and fun.

Trouble with Tags

I was very pleased to read this post by Om Malik (profile) on tags. It is hard to find criticism of tags, but this does a good job of summarizing the problems (as well as pointing to some other posts on the topic). Sifry's recent post on tags quotes Clay Shirky:

This is something the ‘well-designed metadata’ crowd has never understood — just because it’s better to have well-designed metadata along one axis does not mean that it is better along all axes, and the axis of cost, in particular, will trump any other advantage as it grows larger. And the cost of tagging large systems rigorously is crippling, so fantasies of using controlled metadata in environments like Flickr are really fantasies of users suddenly deciding to become disciples of information architecture.

First of all, tags on text are not metadeta. You can not legislate metadata by a syntactic construct. In other words, tags are just words. It is only when tags become references to a new system of symbols with a clear relationship to object data that they can be called metadata. Okay, so I'm making a slightly stronger case here than even I believe in, but the point has to be made. The systemic result of this is that there are many tags that you need in order to capture everyone's idea of metadata. It is ironic that Sifry's post on tags includes such repeats as blog, weblog, weblogs, just to make sure that everyone gets the idea of what the post is about. Clearly these all refer to the same thing. In addition, they all mean the same thing to David. David knows that they mean the same thing to others, but he still has to use them all.

Shirky talks about cost. But he is only talking about write time cost, not the cost that the spaghetti has on the system over the course of the life of the content.

And I am not done yet. When I look at my referrer log, I can see which tags bring more traffic. With a simple utilitarian model of a blog author, how do you think I would react to that knowledge?

The big win with tags is going to be the applications that are built on top of inferences made from them. The tags will become the object data and the inferences will become the new metadata. But hang on, can't we just do that from the text, like tagCloud does?

UPDATE: Kevin Burton comments with a pointer to his posts on tagging issues where he discusses the problem of third party tagging. Note that this is yet another issue - the stuff I am talking about is more to do with the semantics of the author's tags, though the semantics issue doesn't go away with third party tags.