Data Mining: Text Mining, Visualization and Social Media

Techmeme, Memeorandum's Most Discussed

Mary McGlohon has been working with me this summer on analyzing and comparing the spread of information in Usenet and the (political) blogosphere. As part of this work she has been looking at how long links stay around on Memeorandum (and once you’ve done it for Memeorandum, Techmeme comes for free). By looking at the accumulated discussion links for any lead link she has put together a list of the most discussed topics on both sites.

On Memeorandum, scandal really is the name of the game; on Techmeme, gadgets and particularly smartphones rule the roost. Below is a teaser list, be sure to visit Mary’s blog for the details.

Memeorandum:

1. "For McCain, Self-Confidence on Ethics Poses its Own Risk" [McCain and scandals] 219
2. "Spitzer is Linked to Prostitution Ring" 178
3. "Embattled Attorney General Resigns" [Gonzales and scandals] 170

Techmeme

1. "Dear early adopters: Sorry we made iPhone available to the proles. Here's some iTunes. Love, Steve Jobs [smartphones] 152
2. Microsoft's bid for Yahoo! [acquisitions] 138
3. "Dear iTunes customers: No DRM-free music for you. Love, Steve Jobs [IP] 122

The News Width

One of the basic properties of old world programmed media was the fixed time slot. This was a particularly defining feature of the news. There had to be exactly 30 minutes (or whatever) of news to fill the slot. If that day was slow, the news was padded with light stuff, if that day was hot, then the light stuff was dropped as were other moderately important features.

You would think that that approach to media would be one of the first to drop online. We may, perhaps, be forgiving of the online presence of mainstream media - there is a fixed real estate on their front page. However, having become aware of the issue, I'm less forgiving of Web 2.0 aggregators including: memeorandum/techmeme, TailRank and even our own BlogPulse. Take techmeme for example. Stories are ranked according, in part (I'm guessing) to how many citations they get from which bloggers. However, on one day, a story with citations from A and B may appear on the front page whereas on another it may not - depending not on how important that story is absolutely, but how important it is relative to all other stories.

I say that we should be looking for interfaces to information that reflect how important that information is and which don't persist artifacts of the very media that we are (apparently) trying to escape.

TailRank Update

TailRank released some new features on the 24th of March, most importantly, sub-sections on specific topics. Can you guess which ones? Of course, tech and politics - the core duopoly of the blogosphere. Looking at TailRank's Alexa numbers, it looks like the honeymoon is over and it is going to be interesting to see how Kevin transitions it from the next new thing to a major player in the whatever-this-space-is-called space.

A similar decline can be seen in memorandum's reach stats - what does this suggest for the space?

Note that digg.com does not have the recent decline that these two sites show.