Data Mining: Text Mining, Visualization and Social Media

Google Reader Bugs?

I posted recently about some problems with Feedburner and stats from Google Reader. While information from the Feedburner side suggested that these problems were intermittently experienced (an issue with Google's feed fetcher's reporting of subscribers), the problem may run deeper. A reader of this blog says:

I am a subscriber to your feed through google reader, and I have actually not been getting updates made to your feed up until today (so all of a sudden I had precisely 10 unread updates) -- then I read your post about the google stats for feedburner, so I think maybe the two are related.

Today I see a drop of over 1, 000 readers from my normal count of Google subscribers. A little while after Google reader launched, some improvements to the system garnered considerable positive commentary. I wonder if the system is having scalability issues (not in the number of readers, but in the number of feeds perhaps).

If anyone else is having problems with reading in Google reader, please leave a comment - if you can read this, that is!

GeoRSS in Google Maps API

Nigel Tao (a former colleague from WhizBang!Labs - hi Nige) has just posted the news that Google's mapping API is now conversant in GeoRSS. Basically, this means that geolocated RSS feeds can now be rendered in Google Maps. Nigel gives the example of the SlashGeo feed. Below is a screen capture of viewing this feed and reading a post.

Latest Backstage Project: BBC Touch

Chris Riley has posted an interesting site to the backstage mailing list: BBC Touch. What the site does is compare the news stories on the front page (which are described as the stories the beeb 'wants' us to read) and compares these with the most popular (most read) stories based on the most read stories RSS feed (this is described as the stories 'we' want to read). On top of this, some key term extraction is done via the Yahoo term extraction API.

The site is definitely fun, however, one needs to have a bit more perspective as to the function of the BBC and its relationship to other news media services. Research by Ethan Zuckerman demonstrated that the global coverage of the BBC news (that is to say, the distribution of countries in articles published by the BBC) is quite different from that for other (US based) media. Note that currently, Ethan is not supporting his Global Attention Profile research page, so you may have to dig a little to get the details. Secondly, one has to be careful in interpreting popularity. The BBC has a clear obligation to  provide informative and broad coverage to the nation (and the world). Guiding news via popularity is in part an admission of news-as-entertainment which is a very slippery slope. Remember the BBC is not a commercial enterprise.

One could almost look at this site as indicating how in touch readers are :-)

An Alternative to Bloglines

I've been using Bloglines for a while now and though I'd registered at a number of other sites, I've never really used them. Today that changes. The reason that I've decided to switch to something else is that, although there are many features of Bloglines that I like, there are more that simply don't work. These have become too annoying so I'm moving on.

The main annoyances are:

• Inaccurate reporting of fresh material: Steve Rubel doesn't really write 30 posts a day, but Bloglines is always telling me that my subscriptions have new posts even though, when I visit them, there is nothing there.
• Navigating my folders: folders are important, but Bloglines, at least as I experience it in linux/firefox simply doesn't present them well at all. The names of the folders and the presence of absence of new posts, are often hidden and I need to scroll the frame around to see if there is anything to read.
• Non trivial OPML exporting: ironic that this would be a reason to leave, but I find the path to exporting OPML from my own account poorly designed - so much so that I'm sure I've missed something. The only way I know to export my own OPML is by going to a view of my account as others see it.

Having reported the above, things that I will probably miss:

• Bookmarking posts: it is a great way to collect posts for further reading and blogging.
• The 200 post limit: this limitation is a good one - when I see a feed reaching this limit, I know that it is losing my attention and a candidate for deletion.

So far, I've looked at two alternatives. The first is Attensa's free online service. It pretty much fell over when I uploaded my OPML. The foldering structure doesn't appear to be there (which is weird, when you consider OPML). In addition, many (most) of the feeds haven't come through and are stuck with a 'downloading...' place holder.

The second that I have looked at is Rojo. I looked into this very early on and didn't really do much with it. Now, however, I find that I'm really starting to like it. It had no trouble uploading my OPML; the folders are all there, I can read the name and status of all the feeds; the reading space is very nicely designed. I'm going to keep with Rojo for a while and see if it sticks.

Measurability, Measurement and Accuracy

Shel Holtz writes about the frustrations of being in the PR world where measuring the results of actions is a prime focus on selling new ideas, but where the ability to make accurate measurements, and the ability to determine what to measure is of secondary concern.

My feeling is that the blogosphere (actually, the whole gamut of online personal media, or open media) offers something else to measure without there being any inherent features that result in better measurements. This is why consumer facing products, such as BlogPulse, Technorati, OpinMind, while having a great role in attracting people to the space, cannot, without adult supervision, really be used when betting the farm. Will users consider when and how new blogs are added to the index when looking at Technorati's absolute count trends? Will they really take seasonal variation into account when looking at a 30 day window? Will users moderate OpinMind's positive and negative counts with a known background of precision and recall for each category of opinion?

Monitoring buzz, guaging polarity (aka sentiment) of online opinion is, for some time to come, going to require either expert users, or an enterprise service layer appropriate for the cost of decisions being made (here I am thinking of things like 'do I pull the product now?', 'do we re-target our ad campaign to the teen-female segment?', etc.)

On a related note, calls like this for better consumer services indicate another level of disconnect between the users and the pragmatic, technical world of web 2.0. To think that those working in blog search (and, let's face it, everyone working in this space is working the enterprise angle as well) have not been considering comment search is ridiculous. This is precisely why I wrote my When Worlds Collide post earlier this year.

A great enabler of blog search has been the RSS-ping duo. There is only one blog publishing software which currently allows a good RSS based comment stream. Getting to the time granularity of Technorati for comment search is a reasonable engineering challange. Consider this in the face of the scalability challanges that consumer facing blog search engines have faced and overcome this year.

What is needed is not just the right motivation, or someone to grant us the vision, but some platform alignment and modifications to infrastructure. Of course, having said that, there is an opportunity for a blog hosting system that also provides blog search to lead the way here. Such combinations, however, for some reason tend not to be leading the pack in terms of innovation.

Having got there, the problem of understanding when comments indicate an interesting conversation and when they don't is going to add another layer of complexity which will lead in many misguided inferences being made. Many A-list bloggers turn comments off to remove certain types of work load. Are their posts important?

Bloglines Update

What is more likely: Steve Rubel created 75 new posts in the last 4 hours, or Bloglines is getting its feeds in a twist? Much as I appreciate the creative output from Mr Rubel, my money is on Bloglines. Recently I have found that the feed reading portal has been all over the place in terms of quality and yet has stood still in terms of development.

The main quality issues I am aware of are:

• The interface tells me there are no new messages for a certain feed. However, when I click on the item, I do see new messages.
• The interface tells me that there are new messages for a certain feed and when I look, they are all old (as in the case above).

In addition to these quality issues, I'm somewhat at a loss to understand why they haven't developed any of their features or provided any new ones. Why am I still sticking around? Because Bloglines doesn't provide a simple way to export OPML!

MyToday.com

Every now and then, something turns up in one's referer logs that points to some new product or company. Today I see 'w.mytoday.com'. From what I can tell, it looks like another RSS reader. The home page provides links into the following categories: Business, China, City Buzz, Education, Entertainment, Food, Fun, Gaurav Agarwal, Headlines, Health, India, Mobility, Rafat Ali, Rajesh's Feeds, Recipe Collection, Search, Software, Sports, Technology, Telecom, Top 100, Travel, Wellness, Wireless.

There are some names in there that may point to where this project is from.

For what it's worth, I really like the interface. Very simple, very intuitive. Each tab brings up a collection of feeds for that topic. Clicking on the feed name opens and closes the titles and snippets for that feed.

Update: It seems that Gaurav Argawal and Rajesh Jain are both IIT.

Update: Netcore Solutions, which launched Blogstreet, India, owns mytoday.com, so I'm guessing it's a new product from them. They certainly seem to be in the right space.

What Ever Happened To: RSS Ping

RSSPing makes a lot of sense. How come no-one is talking about it? Google BlogSearch 16, BlogPulse 34, Technorati 48. As far as I can tell, it started sometime in August (judging by the earliest posts found):

RSS Ping combines RSS item metadata with site update information. It enhances the current ping specification, adding information about exactly what has been updated or published, with the notification that something has been updated. RSS Ping reduces the need to revisit or work done by aggregators and search engines when revisiting the publisher site after being pinged, to provide truly real-time updates for information on the Web.

Sphere Update

Sphere will soon be providing RSS feeds for their blog search. This is a must have feature, of course, but I'm starting to get the feature that Sphere is going down quite a different path from that which I originally imagined. Mary Hodder, one of the reasons I suspected Sphere was going to deliver something disruptive, has voiced some very interesting points:

I believe that blog search, at this point, is a baseline for any company in the space. You have to do it. But it's not so interesting to me, compared to making a leap forward toward something like topic browsing of communities or sophisticated weighting of bloggers. I'm less interested in 'yet another blog search' tool. The ones we had already were fine. But I've very interested in what Sphere is really here for: changing the ways we can view small topic communities and the bloggers within them in sophisticated ways that take us ahead of where we are now, which I equate to the place websearch was in in 1997, before Google.

I get the impression that Mary is a little disappointed with Sphere. Perhaps Mary should take TailRank for a spin...

A Tale of Two Trends

Steve Rubel posted about the Audible file format issue. In his analysis, he posted a trend from IceRocket's Blogs+RSS search system:

So Audible largely won in the eyes of the media - the sole basis for how we used to define PR. Of course, we no longer live and die by traditional media alone. In the blogosphere it was an entirely different story, as this IceRocket trend graph shows.

I can't quite figure out what Steve means here: the trend shows what? That Audible didn't win in the eyes of the blogosphere? How does this graph show that? Another point of interest is the difference between the IceRocket trend and the BlogPulse trend (note that BlogPulse only shows hits for Blogs, not arbitrary RSS, so there are going to be some differences.)

 

I see two major differences here. IceRocket shows two large spikes - one in late August, the other in mid October. BlogPulse shows a single spike in early September. The other is the two smaller spikes in the BlogPulse trend (note that IceRocket has a couple of zero data points around the same time). BlogPulse allows you to drill down on specific dates. IceRocket doesn't have this, and as their custom date range query from the advanced search page is broken (it doesn't appear to work in either FireFox or IE), I can't yet determine what the spike in the IceRocket trend actually is.

The large spike found in BlogPulse is accounted for by quotes of a post by Brian Williams. The other two spikes appear to be associated with the Audible Inc. story - as this trend graph shows:

 

Here is the same trend on IceRocket:

 

I get a big kick out of trend graphs. However, it takes some care to work with them in the blogosphere. For example, every mention of the word 'audible' is not a mention of Audible Inc. - in fact, it probably the reverse. In addition, depending on what you trend over (blogs in the case of BlogPulse, RSS in the case of IceRocket) you see quite different results. If you are going to make any detailed use of trend graphs, you need the ability to inspect interesting features (either by drill down on the graph or by date specific queries).