Data Mining: Text Mining, Visualization and Social Media

Tracking Brand Sentiment

Nothing could be more satisfying than this article in the International Herald Tribune which mentions the use of our technology. It is not primarily the mention of Intelliseek (BuzzMetrics) but the fact that there is a prominent mention of sentiment (analysis). Missing from the article is the fact that the results are derived from social media, but still, a very validating piece:

Advani said Lenovo had tracked brand awareness in nine countries and noted that an Intelliseek survey showed that in the past six months the association between ThinkPad and IBM dropped substantially, while the association with Lenovo rose. And, he said, sentiment for the ThinkPad "remains just as strong."

[Please note my optimistic bias - I'm interpretting the mention of sentiment as refering to the automated analysis of opinion and evaluation.]

Monitoring Social Media: Your Name Here

It is surely a sign of a maturing market when you can take the copy from one company web page and insert any of the competitors' names in it without significantly changing the text's validity. Take the following, for example:

COMPANY continually tracks sources such as online newsgroups, blogs, company web sites and syndicated sources. Data mining technology searches the collected data for trends, clusters and sentiments which are then used to compile reports according to client requirements.

Who could this be? Umbria? BuzzMetrics? It is actually a reasonably new player - Attentio (circa late 2004, early 2005). Also, there's nothing like a good job posting to tell you what the company is up to:

Candidates should have unique skills and experience in at least one of the following areas:               

    - Text-based Information retrieval
          - Web mining, data mining
          - Machine learning/AI techniques
         
          - Web application development
          - Search engine, crawler and wrapper development
          - Web services, XML/RSS, Microsoft .NET, Sun J2EE
          - Scalability, distributed systems, clusters, grid computing
         
          - Object oriented programming techniques, Java, C++
          - Relational databases/DBMS, SQL
          - OS experience incl. UNIX/Linux
          - Project management, software specification, methodology

Simon McDermott, co-founder, also keeps a blog.

 

I Work For a Giant

It's interesting to see an article on BlogPulse's top news stories for yesterday about Intelliseek/BuzzMetrics (and by implication, BlogPulse). It's also interesting to hear that BuzzMetrics is referred to as a 'giant' in this space:

To capture the chatter, Nielsen BuzzMetrics, a giant in the industry, uses software that collects hundreds of thousands of comments a day. The technology can scan for specific companies, products, brands, people -- anything searchable. It can slice data into a range of categories to quantify the number of times a subject was discussed online, the individuals who mentioned it and the communities where it appeared.

Hundreds of thousands is probably an underestimate. Also the 'anything searchable' is an undervaluation of our technology. The expectations of search are far weaker than what true text mining, nlp and categorization technology can do. Anyway, nice article.

Sentiment Mining

There is no doubt that Mood News is a cool idea. I've been tracking it for a while as a member of the backstage community (from the beeb) - it was recently picked up by Infosthetics. However, it also illustrates the difficulty of developing this type of feature. The current most negatively ranked news is:

Algerian bomb suspect on trial

An Algerian man accused of involvement with bombings in Paris in 1995 goes on trial in France.

I would think that this was actually positive news. However, as the system uses keyword spotting to determine what is good and what is bad (bomb==bad), the orientation can escape the system. In addition, there is an issue of point of view. To whom is it good/bad news? Take a look at the current highly ranked positive news:

Palestinians hail EU aid deal

An EU aid package worth 120m euros is hailed by Palestinians and the US but condemned by Israel.

Sounds like bad news for Israel at least.

In analysing social media in the context of marketing intelligence, I tend to think of a number of categories of textual features, including:

• Sentiment: evaluative statements  (I like it, It sucked)
• Objective Favourable/Unfavourable statements (It is broken)
• Textual features: these are signals in the text that indicate some sort of mental state, especially emotion/affect (I can't wait to see it - indicating anticipation/excitement)

Generally, what we need to deliver is some mixture of these. The question raised by mood news is: what is the mixture appropriate for journalism? and, of course, what is of value to the user?

There is another set of considerations from the research side which needs to connect:

• Analysis that is useful to a customer/user of the output
• Psychological models of emotion/affect, etc.
• What is achievable with text analytics

That is a story for another post.

Relevant Noise: A New Blog Search Engine?

I'm seeing a few hits from this URL: http://sonar.relevantnoise.com:81/blogsearch/search.aspx. Looks like it might be a new (or homebrew) blog search engine.

Okay - perhaps not new. I find citations to relevantNOISE as early as November last year. The domain itself doesn't respond just now. Further digging provided this press release:

Startup Will Leverage Proprietary Data Mining, Natural Language Processing and Tone Extraction Technologies

NEW YORK, NY -- (MARKET WIRE) -- 11/07/2005 --  relevantNOISE™, a technology company dedicated to mining web logs, or blogs, for business intelligence, officially launched at ad:tech New York this morning. Using proprietary data mining, natural language processing, and tone extraction technologies, relevantNOISE helps clients monitor the quickly growing "blogosphere" in real time to track the level of volume around key topic areas -- including industry, brands, products, executives, and competitors.  The company further assists clients by determining whether those conversations are positive or negative and how that changes over time.

The company was founded by Dan Lynn, Scott Goldthwaite and Steven Goldthwaite, with Lynn serving as CEO. The three also founded Web marketing firm DigitalGrit, which recently ranked third in Deloitte's Technology Fast 50 for New Jersey.

"Forbes magazine recently published an article, 'Attack of the Blogs,' focused on the damage they can do to brands and reputations," notes Lynn. "While I don't necessarily agree with their perception of blogs, we're here to help those brands, people and organizations gain awareness of what the blogs are saying so they can be proactive and address any problems in a timely and appropriate manner. Even more importantly, we can help companies identify emerging opportunities and trends associated with this powerful new marketing tool called the blogosphere."

relevantNOISE's services provide tools that allow these parties to:

--  Measure the volume level of the "noise" across the blogosphere for
    specific issues -- and track changes over time
--  Perform "Public Opinion Mining," monitoring real-time changes in the
    positive and negative tone within the blog "conversation"
--  Forecast emerging product and industry trends and opportunities
--  Map changes in conversations to key business indicators, such as stock
    price, sales volume and marketing expenditures.

So they appear to be an Intelliseek/BuzzMetrics/Umbria competitor. The fact that their domain is not responding could indicate that they came and went before I even got a handle on them. Shame on me!
Note also that they appear to have signed up for the structured blogging initiative. I'm posting this with a reminder to myself to dig a little deeper and connect the dots with DigitalGrit.

Update: the domain is back, check out the blog.

Nielsen: Not Just a Number

A lot of commentary regarding the recent Intelliseek/BuzzMetrics VNU deal, and Nielsen's position wrt the new company have focused on two things: measuring media consumption and using that measurement to inform advertisers (and thereby, for those whose glasses are half-full, influencing the content that hosts the adverts). This perspective is, perhaps, forgivable. However, I would encourage anyone with a true interest in the consolidated company, and the implications for both the user-generated media anlaytics space as well as the blogosphere in general to both look at the offerings pre-deal described on Intelliseek's and BuzzMetrics' sites and to look in greater depth at the range of products and services offered by A.C. Nielsen. They are more than just numbers.

On a related note, refer to Matt's post, and this raving lunatic to get some perspective on what happens when the consumers of consumer-generated media are recognized as the producers, oh - and customers as well. The issue is pertinent: in the traditional media model, there are two distinct classes - producers and consumers. The new world just isn't like that.

Conversation Gap Wizard

Niall Cook, of Hill and Knowlton,  writes about using free online tools to study, among other things, the conversation gap. The end of his post has a form which asks for some simple data and creates trend graphs using BlogPulse' trending system. Pretty cool. I'll note, however, that the quality of results will vary greatly with respect to the ambiguity, etc.,  of terms. Looking for 'apple' will likely pick up mentions of the company, but also mentions of fruit.

Intelliseek, BuzzMetrics

Stay tuned:

            VNU Agrees to Acquire Majority Interest in BuzzMetrics (Jan 17)

            

Company enhances portfolio of marketing and media information businesses by adding leader in measuring online consumer dialogue

Haarlem, the Netherlands – VNU NV (ASE: VNU), a leading global information and media company, today announced it has agreed in principle to acquire a majority interest in privately held BuzzMetrics Inc. as part of BuzzMetric’s plan to acquire Intelliseek Inc. The move enhances VNU’s media measurement capabilities and extends its reach into the fast-growing sphere of consumer-generated media, such as blogs and public message boards, which are measured to understand the impact of word-of-mouth behavior and influence on product sales, entertainment choices and other consumer-related activities.

Following the merger of BuzzMetrics and Intelliseek – two leaders in tracking and analysis of online consumer-generated media – VNU will have a 50.1% stake in the new company, to be known as BuzzMetrics. Headquartered in New York, BuzzMetrics will market its services using the Nielsen BuzzMetrics brand under a special licensing arrangement with VNU. VNU is the parent of such renowned research names as ACNielsen and Nielsen Media Research.

The companies said they expect to close the transaction within 30 days. Financial terms were not disclosed. While VNU will have a majority interest, BuzzMetrics will be managed by Jonathan Carson, current CEO of BuzzMetrics, who will retain that title with the new company, and Michael Nazzaro, Intelliseek’s president and CEO, who will become president and COO.

The transaction marks the third in a series of word-of-mouth research investments supported by VNU. In February 2005, VNU acquired an equity interest in Israel-based Trendum, a leader in media and linguistic analysis technologies. In September 2005, Trendum acquired BuzzMetrics, and the new company adopted the BuzzMetrics name.

BuzzMetrics serves a broad spectrum of industries including media and entertainment, automotive, electronic, healthcare and consumer packaged goods. Key clients include Canon, Comcast, Ford, General Motors, HBO, Kraft, Mazda, Microsoft, Nokia, Procter & Gamble, Showtime, Sony, Target and Toyota.

About VNU

VNU is a global information and media company with leading market positions and recognized brands in marketing information (ACNielsen), media measurement and information (Nielsen Media Research) and business information (Billboard, The Hollywood Reporter, Computing, Intermediair).

VNU is active in more than 100 countries, with headquarters in Haarlem, the Netherlands and New York, USA. The company employs 38,000 people. Total revenues under IFRS amounted to EUR 3.3 billion in 2004. VNU is listed on the Euronext Amsterdam (ASE: VNU) stock exchange.

For more information, please visit the VNU website at www.vnu.com.

Press contact          Will Thoretz   New York    +1 646 654 8133

Investor Relations     Peter Wortel   Haarlem     +31 23 546 36 00
                              Arnout Asjes   New York   +1 646 654 5031

       

Technorati Survey

Technorati is currently running an opt in survey aimed at a numbers of areas including customer satisfaction and online monitoring. Questions of interest:

*Please rank this statement for you: "I regularly use Technorati to monitor who's saying what about my company or place of business."

*Please indicate the extent to which the following statement is true for you: "I would be interested in seeing content from mainstream media sources along side content from blogs and bloggers."

*Please indicate the extent to which the following statement is true for you: "I would like to see the widespread adoption of a standard format for things like contact information, reviews (for movies, for instance), and event calendars, and for Technorati to surface search results for these."

*Please indicate the extent to which the following statement is true for you: "If Technorati were to offer advanced member services, advanced search and reporting tools, or other advanced features, I would be willing to pay a nominal fee for these."

*Please indicate the extent to which the following statement is true for you: "If Technorati were to offer specialized tracking and analytics products to help me meet objectives at my place of business, my company would be willing to pay a reasonable fee for such products."

The questions regarding monitoring are of particular interest. Technorati does have a product called Technorati Blog Tracker about which little information has surfaced. Also of note is the question regarding movie reviews etc - a clear microformats/structured blogging related question. Certainly, this year we will start to see some products built on this potential if it is really going to fly.

Regarding the Yahoo/Microsoft acquisition rumours, such an acquisition, I believe, would change how the company approaches the monitoring business. If you don't believe either Yahoo or Microsoft would be interested in the space (other than for their own products), then the acquisition either doesn't make sense, or would see the end of this line of product development. On the other hand, if you do believe that GYM are interested in the space, then it speaks volumes as to the predicted size of the market.

If I were to place a bet, my money would be on Microsoft.

The Business of Marketing Intelligence

John Stith writes about monitoring the blogosphere. I imagine we will see more and more articles on the actual business rather than the potential of the data throughout 2006 and beyond. His article mentions Umbria and their BuzzReports product:

Recently, a small company called Umbria Market Research got a profile in Forbes Small Business. Their service Buzz Reports was the focus. Buzz Reports is a blogosphere monitoring service costing upwards of $60,000 or more a year. And they maintain they're the cheap ones.

A hat top goes to B.L. Ochman for this one. She noted the article compared Buzz Reports to the products of rivals Intelliseek and BuzzMetrics. She notes the article says the two bigger entities "can easily run into the seven figures."

I have a few comments on  John's article. Firstly, the blogosphere is just part of what Umbria (and others) monitor. The tools and products are actually aimed at a far broader content space - anything that is generated by users/amateurs/consumers/call-them-what-you-will. This includes blogs, but also boards, review sites, usenet, listserv data and beyond.

Secondly, the issue of price is presented a little confusingly. The difference in cost is not between like products. It would be like saying 'Honda sells cars for $12, 000 but Ford sells cars for $30, 000.' You probably can get a low/mid-range Honda for 12k and a higher-end Ford truck for 30k, but Ford also sell cheaper cars and Honda also sells SUVs. Services oriented markets allow customers to create contracts out of a range of features and options, which can potentially add up to large contract prices.

Thirdly, I'd love to be able to read B. L. Ochman's original post, but unfortunately a) John's link is to Umbria's site (as I have cut-and-paste quoted it) and b) The Ochman's blog appears to be unavailable at the time of writing. I found the post on BlogPulse (here's the link) so I'll come back to it when and if the blog is back in action.