Data Mining: Text Mining, Visualization and Social Media

PubSub linkRank Confussion

I have to admit - I'm confused about PubSub's stats. A while back, I had a suspicion that historical values were not stable. In other words, it seemed to me that the values for a few days in the past were not the same when I looked at them a few days ago. Compare these two graphs below. The first I saved on the 28th of September, the second is from today, the 5th of October.

The first has my percentile between (approx) the 8th of September and the 26th of September clearly decreasing. The second has my percentile between (approx) the 9th of September and the 4th of October clearly increasing. The linkRank graph appears to match, but the percentile graph has changed retroactively in some way.

Is this a bug, or is this something I'm not getting about the measurement? If it is something that I'm not getting about the measurement, I think it is still a bug - a bug in data visualization design.

PubSub top 1000, Part 2

Bob Wyman has posted a follow up to my post on PubSub's 1000. I've now figured out the analysis I want to do on the data, but will have to hold off as I am about to go on vacation. Putting that aside, I've been thinking about the middle ranks. As Bob points out, the middle ranks fluctuate in terms of linkRank. I hypothesize that there is some subset of all sites for which a given site has a stable linkRank. This set is effectively the community that the ranked site is a member of. In other words, by having linkRank run agnostically over all sites, the upper, stable sites act whimsically on the linkRank of the lower sites, making them appear unstable when in fact from the community point of view they have a very stable rank.

Think of it this way. If you are a middle rank blogger with a solid community blogging on a specific topic, your in links will be relatively stable - coming from peers in the community. And then one day you blog about something that catches the eye of an A-list blogger. This causes your link rank to fluctuate wildly until the popular topic that you posted on goes beyond the attention span of the blogosphere.  There are two important ideas here:

• We need blogospheres plural, not a single blogosphere - these may be fuzzy, mercurial things, but they make a lot more sense than this top heavy being which we might term blogosphere 0.0.
• We need to think about influential ideas and influential blogs/bloggers as two separate elements of  any model backing ranking of any sort.

PubSub top 1000

A while back, Mark Wagner from PubSub pointed me to their top 1000 linkRanked sites and asked for some comments. I haven't yet got round to doing anything interesting with this stuff, but can at least show the following graph which displays the Average linkRank for 30 days (x axis) versus 15 days (y axis). It suggests, not surprisingly, that the higher you are ranked, the less likely you are to change rank - or the more stable the rank is. I can probably do something far more interesting on measuring these changes, but that will have to wait.

Note that those values below the line represent sites whose rank has decreased (i.e. have gone up in ranking - very confusing I know) when taking the 15 day average rather than the 30 day average, where as those above represent those which have increased (i.e. fallen in rank). I've added some labels as landmarks - gnuplot is not that good at labelling, but they are positioned so the left most extreme of the label is at the point being labeled. For example, littlegreenfootballs.com is above the line whereas techcrunch.com is below, therefore littegreenfootballs 30 day average is of lower rank than its 15 day average.

Is Prospective Search different from Search Feed Subscription?

I've seen a number of posts talking about how PubSub's system is fundamentally different because it does prospective searching, unlike the stodgy old backwards facing search that Technorati, BlogPulse and others provide. But how is prospective search any different from having a subscription to a search feed? Technorati and BlogPulse (not to mention everyone else) has this as part of a standard feature set. Cindy Sullivan says:

PubSub is a prospective matching engine, meaning it's only concerned with information you want to be alerted to from this very moment forward.  I'm by no means the expert here just really fascinated with what PubSub is doing.

Isn't that a description of a search feed? I have to be missing something here.

PubSub describes a system that would require a representation of all queries in a form that can be efficiently matched against each new post (or rather the content of an RSS feed):

The heart of the PubSub service is a powerful, proprietary Matching Engine that makes it possible, for the first time, to match millions of search queries against thousands of new pieces of information every second.

A naive implementation of this would simply place the new data in the index and then run a time restricted search. A more interesting implementation (theoretically interesting) would be one which keeps no object data state, just query state. The criterion that drives this is the time taken from a post being published to notification of the search subscriber. This time becomes more and more critical given the importance of the issue and/or the prominence of the poster. PubSub have worked on the later issue with their linkRank system. I don't believe any of the free public services allows for a request like 'let me know if someone says anything libelous about my product.'

 

Feedback on PubSub's LinkRank Graphs

Bob Wyman recently slipped a mention of the new LinkRank analysis from PubSub into the discussion of top N blog lists, and now the analysis is live. In general, I have liked PubSub's approach to profiling. They solved an interesting problem (how to put all the information on a single page) in a reasonably elegant manner. The latest addition essentially adds an extra column to the table (with two new fields per cell) and an additional graph (again, with two new variables tracked).

As Wyman mentioned in his initial post, the new graph displays the LinkRank and the percentile of the blog. For illustration, here is the graph for this blog as of today:

It took me a while to really grok the graph. Upon reflection, I realised that I was used to seeing percentile information expressed as a shaded region on a graph (as percentile in the contexts I have seen implies the range of instances that are subsumed by the instance under consideration - thus a child in the 80th percentile in height is taller than 80% of all other children of his age). In addition, as in the child example, the number is expressed as the inverse of that used by PubSub. My understanding may be entirely incorrect here, but it is the reason I had some difficulty at first.

Another issue is the range of values. I range from 25, 000th in LinkRank to somewhere around 500. For blogs of a more stable rank at the top of the list, it is hard to make at any fluctuations in the small numbers, this is even more the case for the percentile value. Here is the graph for Battellemedia.com:

The small number issue is interesting as it illustrates the relationship between the metric and the pragmatics of presenting the metric in an intuitive and consumable graphic. It may be that for this metric, a non-linear scale would produce a better presentation.

As is discussed elsewhere, Wyman's metric gives more weight to recent links. This means that regular posting is required to keep a high rank. For me, this highlights an important part of the whole ranking discussion: what is the meaning of the rank? Wyman's rank conflates volume/time and in-links and so, if it is used to judge quality, or authority, it means that more posts more frequently are better than fewer. I'm not sure if this is how I judge the quality of a blog. Blog posts are time stamped, but not all posts contain time-constrained information.

I'm not going to comment on rank algorithms here, but will say that there are many types and uses for blogs. Just as tags might have been a good idea, it is easy to see how they have become noisy and spamable - becoming things in their own right rather than descriptions of object data. If ranking algorithms like this do the same thing then we will artificially value features of blogs which perform well in such rankings.

Anyway, I have to stop here as I need to write another five posts today to keep my rank up...

BTW, the link to 'learn more' on the site is broken.