Technorati/Edelman Survey

Technorati and Edelman have created an online survey:

in an attempt to better understand how blogging and traditional PR intersect, and what bloggers think about communication from mainstream companies.

(according to Niall Kennedy, Edelman). The results will be out in a month - should make for good reading.

I hope that they include some discussion of the methodology. I'm particularly interested in the issue of self-selection: how the results are biased to profiling the type of people who elect to take such surveys. One of the promises of measuring communication and publication in the OPM (online personal media) space is the reduction of this problem. Evaluations and white papers are being thrown out at a considerable pace with, I'm afraid, a worrying lack of sound methodology.

The white paper will get a lot of attention from bloggers, just as comScore's Behaviors of the Blogosphere did. The reaction to that publication was not entirely positive due to a lack of methodology and ultimately some very dodgy looking results.

(It is a shame that both Technorati's and Edelman's blogs don't support trackbacks.)

Comments

[niallkennedy]

I will be posting more background information about the survey on my website soon and my blog supports TrackBacks.

The first few questions are meant to frame the conversation. I believe we will see a different set of answers based on why people blog, and how often, and answers to the first few questions should add context to the rest of the survey.

I am one of the organizers and a speaker at Webzine 2005 this weekend, but more soon.

[Matthew Hurst]

Niall - thanks for the comments.

Yes - your blog supports trackbacks, but I didn't notice this post there, so not much I can do with that info. The problem of self-selection can be illustrated as follows: if the survey is designed to see how many people blog for reason A and how many for reason B, then if A types have a tendancy of 80% to answer surveys of this nature, and B types have a tendancy of 20% to answer this type of survey, the results are going to be skewed accordingly. The problem is, you can't measure that bias.

Now consider a system that measures something in the blogosphere (e.g. classifies type A and type B bloggers). This system *can* have measurable accuracy, and thus can give sound results (including error bars). The point of surveys like this is that there is an assumption that the thing you are trying to measure is for some reason only measurable by asking questions of bloggers, not by observing their behaviour and output. Quite reasonable given some of the questions in the survey.

[Constantin Basturea]

Speaking of methodology -- I was wondering if you can provide any information about the methodologies used by Intelliseek for the last two white papapers published with Edelman:

-- Talking from the Inside Out (September 2005), which claims that "nearly 70 percent of companies have no policies or guidelines in place for employee bloggers," but offers no other information about how the study was done

http://www.intelliseek.com/releases2.asp?id=139

-- Trust MEdia (April 2005), which ranks "the most influentials blogs" in 6 key industries without providing information on how these rankings were compiled

http://www.intelliseek.com/releases2.asp?id=126

[Matthew Hurst]

Constantin,

No I can't - I have absolutely nothing to do with them. You should feel free to contact any of the authors listed in the publication.

If you are interested in *my* take on methodologies, I'd be happy to refer you to any of my personal research publications or theses.

[Constantin Basturea]

Thank you, Matthew. Since Pete Blackshaw is the only person from Intelliseek listed in the two studies, I'll try to contact him.

Of course I'm interested in your take on methodologies, and I would appreciate if you could point me to any relevant materials. Surveys based on nonprobabilistic sampling are mushrooming these days, and there's little discussion of how the data should be interpreted.