Measurability, Measurement and Accuracy

Shel Holtz writes about the frustrations of being in the PR world where measuring the results of actions is a prime focus on selling new ideas, but where the ability to make accurate measurements, and the ability to determine what to measure is of secondary concern.

My feeling is that the blogosphere (actually, the whole gamut of online personal media, or open media) offers something else to measure without there being any inherent features that result in better measurements. This is why consumer facing products, such as BlogPulse, Technorati, OpinMind, while having a great role in attracting people to the space, cannot, without adult supervision, really be used when betting the farm. Will users consider when and how new blogs are added to the index when looking at Technorati's absolute count trends? Will they really take seasonal variation into account when looking at a 30 day window? Will users moderate OpinMind's positive and negative counts with a known background of precision and recall for each category of opinion?

Monitoring buzz, guaging polarity (aka sentiment) of online opinion is, for some time to come, going to require either expert users, or an enterprise service layer appropriate for the cost of decisions being made (here I am thinking of things like 'do I pull the product now?', 'do we re-target our ad campaign to the teen-female segment?', etc.)

On a related note, calls like this for better consumer services indicate another level of disconnect between the users and the pragmatic, technical world of web 2.0. To think that those working in blog search (and, let's face it, everyone working in this space is working the enterprise angle as well) have not been considering comment search is ridiculous. This is precisely why I wrote my When Worlds Collide post earlier this year.

A great enabler of blog search has been the RSS-ping duo. There is only one blog publishing software which currently allows a good RSS based comment stream. Getting to the time granularity of Technorati for comment search is a reasonable engineering challange. Consider this in the face of the scalability challanges that consumer facing blog search engines have faced and overcome this year.

What is needed is not just the right motivation, or someone to grant us the vision, but some platform alignment and modifications to infrastructure. Of course, having said that, there is an opportunity for a blog hosting system that also provides blog search to lead the way here. Such combinations, however, for some reason tend not to be leading the pack in terms of innovation.

Having got there, the problem of understanding when comments indicate an interesting conversation and when they don't is going to add another layer of complexity which will lead in many misguided inferences being made. Many A-list bloggers turn comments off to remove certain types of work load. Are their posts important?