The Diva Marketing Blog has an interview with Bill Neal - a marketing research guru. There are a number of problems with Mr Neal's responses regarding the value of social media anlaysis as well as the sophistication of the technologies involved.
[T]hose out there in the Internet world who are generating their own media are self-motivated to do so and are not representative of any defined population of buyers. And, given the fact that they have taken a public position on a particular product or service, it means that they more often than not have exceptional or non-typical attitudes about those products and services. The information they generate may be true, or not true – there is no way to discern which. Therefore, the information generated by those folks is neither credible nor reliable. So, as researchers, yes, we should be listening, but we must be very cautious and skeptical about its veracity and its usefulness.
One of the fundamental problems with the term Consumer Generated Media, a term created by those firmly on the marketing/advertising side of the fence, is that it forces a view of the data (blogs, boards, etc.) with the assumption that an individual acting as a consumer of a product is creating the media. One can see a trace of this in the above comment: the fact that they have taken a public position on a particular product or service. If I state that I ate a pizoid pizza while watching the Superbowl, what position have I taken about that product? The notion that consumers are writing this stuff specifically to take a position wrt a product is simply wrong. In addition, it misses the aggregate value of a million LiveJournal users mentioning pizoid pizza in passing.
The information they generate may be true, or not true – there is no way to discern which. This statement seems to suggest that other forms of inquiry are more reliable. I don't follow the logic that something I write in a journal would be less true than something I say in response to a market researchers question.
As someone on the outside of this emerging space, Mr Neal makes the following statement:
Let me give you an example. I own two Ford trucks – a 1997 Expedition with close to 200,000 miles on it and a 2004 F-150 with 25,000 miles. In the blogosphere I’ve seen a bunch of postings on Ford trucks depreciating their quality and reliability. Yet, both of the vehicles I own have been exceptional in quality and reliability. I don’t take the time to post those positive experiences, but some who have had problems are very vocal about their supposedly negative experiences.
So what is the truth of the matter? Did these negative experiences really occur? Was it the fault of the manufacturer? Or the Dealer? Or the buyer? Are these generators of consumer media about Ford trucks really being the dispassionate arbiters of truth, or do they have an agenda? That’s the key issue – that information gleaned from the blogosphere is simply not reliable, and in many cases it is not valid.
By carrying out large scale aggregate analysis, we can identify those vehicles which do attract lots of positive mentions, and those which attract more negative mentions. The class of vehicles which tend to attract negative mentions are those which tend to be commodity vehicles (e.g. the Ford Escort). In other words, they are very good performers in their price range and do a reasonable job of getting from A to B. This class of vehicles over indexes on negative comments, but by modeling this expected behaviour we can determine when there is an outlier from the class for which the negative (or positive) signal is abnormal, thus gaining insight.
Finally, there is the following:
Bill: Honestly, I don’t know a lot about them and have not used them in my consulting practice, nor do I address them in our forthcoming book.
But to the best of my understanding, they are primarily counting product/service mentions and, in some cases identifying the major sources of those mentions. The basis for their business model is the belief that consumers have a higher trust of consumer generated media then they have for company generated media.
I think that might be more of a reflection on the stupidity of much of the advertising and promotion that permeates today’s traditional media. I’ve already talked about the problems with simply counting the number of brand “hits” and how that can be so misleading.
And, as consumers mature in their understanding of how consumer generated media can be manipulated by those with less than honorable intentions, I think their trust in those sources of information may wane considerably.
As an expert in this space, Mr Neal owes it to his readers to be more informed in this area. The level of technological sophistication involved in the types of analysis we do at Nielsen require a more mature understanding before they are dismissed. In addition, the statement [t]he basis for their business model is the belief that consumers have a higher trust of consumer generated media then they have for company generated media is incorrect. The trust issue is only a small part of the story.