Data Mining: Text Mining, Visualization and Social Media

Now With More Matterhorn

I'm not sure why I'm getting caught up in this, but here is the best rendering of the Matterhorn that I've yet seen:

Found on Viewfinder Panoramas.

More On The Matterhorn

I posted recently on the use of the Matterhorn as a test for geographic visualization tools. James Fee gives an enterprise perspective via the ArcGIS tool which, he notes, could easily trump consumer facing tools - but he's looking for data.

Umbria Offers Mobile TV Report

Briefly, Umbria has a new mobile tv report based on analysis of posts in the blogosphere:

       

Based on the wealth of conversations from the blogosphere and consumer generated media, Umbria offers quarterly reports focusing on topics directly related to the rapidly growing mobile TV industry. In these reports you will gain insight into both the age and gender of speakers, what they discussed, how they interact with this product, and the implications for companies serving this industry.

       

Here’s a few examples of Umbria thematic insights and recommendations:

       

• How do bloggers speak about portable media players vs. traditional mobile devices?
• What is the preferred method to access content, when, and why?
• What are their biggest frustrations and concerns?
• What are the preferred content types, by demographic?
• What implications are there for increasing adoption?

Unfortunately, their registration mechanism for getting more info on the report is currently not quite functioning (asks for email address, once entered it asks again for your email address...). I'd love to get a copy of this to see what findings they've made.

Judging by the URL (http://www.umbrialistens.com/products/syndicated.php) this is part of a syndicated strategy - a key way to scale in the social media monitoring market.

Happy Feet, 007

Last week was the thanksgiving holiday here in the US. Like Labour Day, this has an impact on movie sales. The graph below shows how Happy Feet (an awesome movie about musical penguins) is getting more buzz than the new 007 film. In addition, it shows how Happy Feet buzz increased compared to its opening weekend while 007 decreased despite the boost that the holiday could have given it. I've included The Illusionist as well to show how the holiday weekend gave it a little shove (the last bump in its line).

International Conference on Weblogs and Social Media

An update on ICWSM:

• We have some great new sponsors joining us - the full set now includes Yahoo, Google, Microsoft Live Labs, NEC, Attentio, Sphere, Klostu, as well as those involved in founding the conference (Nielsen BuzzMetrics, Umbria and UPenn).
• The conference is affiliated with both the IW3C2 (which runs the WWW conference) and AAAI.
• We have three excellent speakers lined up: Ev Williams, Andrew Tomkins and Danah Boyd.
• Finally, the submission page is up and ready for your papers!

We are really looking forward to making this a great event - see you in Boulder.

BTW, if you want to blog about the conference, please use the tag icwsm.

BlogMonitor

BlogMonitor is a (new to me) blog monitoring service from Infuz.com. They provide a service which is characterized by the following steps:

SEEK
First, we use our expertise to help you prepare a search concept key word list that meets your objectives. Then, based on those key words, we use exclusive tools to comb the Internet for relevant data.

SIFT
Next, we sift through the data to filter out information that is bogus or irrelevant to you.

SORT
A professional reviews and categorizes the remaining content to meet your specific needs.

REPORT
We present user-friendly aggregated data reports with our evaluations and recommendations for action. You may choose to receive immediate notification, in-depth analysis on a monthly basis, or both.

It is not clear which, if any, of these steps involves any significant technology. They appear to be positioning themselves as a competitor to clipping services rather than the more established players in the social media monitoring space (such as my employer BuzzMetrics).

Search engines and clipping services may use technology to gather data, but they don’t discriminate when it comes to sharing it with you. Blanket, untargeted Web searches don’t effectively tap into “authentic media.” Often they’ll send you meaningless spam results (up to 60%!), inaccuracies, and noise. Then you are forced to spend your time and resources sorting through it all.

However, the issues that they highlight are also those which social media monitoring systems handle in a more automated and scalable manner. Ultimately, the report is the important thing, and from their website, they appear to be more qualitative then measurement based:

• Understand what the buzz is regarding your company, industry, or your competition.
• Discover honest perceptions consumers have about your brand or company, both positive and negative.
• Find new product ideas, innovations, and marketing concepts.
• Identify and track market trends and other insights that are important to your business.
• Help corporate governance by finding out what your partners and employees are saying about you publicly.
• Gather competitive intelligence by finding out what bloggers say about your competition.
• Identify your loudest friends and foes!

TailRank Info Porn

Kevin over at TailRank has put up a graph showing the cyclical nature of story volume from TailRank's POV. The graph shows the number of stories per day that TailRank ingests - that is to say, the number of threads that it considers when creating its real-time view of the blogosphere. Typically, TailRank's home page shows 30 such stories. TailRank appears to be looking at something like 6k stories on a typical week day. If we guess that there are 5 links per story (the head is going to have many more, but the long tail will lower the average number) then TailRank would be considering something around 30k posts per day. This is, however, the number of posts involved in a thread. TailRank will actually be crawling many more (though I'm guessing that it works with a moderated blog list - that is to say it doesn't crawl any old blog out there).

Two Minute Rule: Intelligent Email Clients Needed

Fernando gives some more commentary on how technology has failed us in the area of email:

Some of us have been doing a bit of work on tools for helping manage email. Just a beginning. I totally agree that dropping any email that needs more than 2 minutes to respond is filtering all worthwhile thought from your exchanges, maybe from your day. What I would like is a convenient way to slow down email, especially the kind of email that doesn't need much thought. For example, I'd like an adaptive delay for my outgoing replies. If an outgoing reply is delayed by 8 hours, the reply to the reply will arrive tomorrow. Of course, some replies I want to go out right away. that's why intelligence and adaptation are needed for this purpose.

In addition, I think that one of the big problems with email is there is no direct relationship between the time stamp on an email and when the sender expects or requires a reply. This can be inferred in part by various priority settings, but these are easy to ignore and often mean different things to different people  (an ontological commitment problem if ever there was one!). Perhaps analysis of the content as well as past interactions with the sender could be used to provide a better sorting function...

Down With The Two Minute Rule

Scoble writes about the two minute email rule:

I love this new two-minute rule for email from Eric Mack (delete all email that takes more than two minutes to answer). I woke up today to find 48 new emails waiting. Damn, and it’s Sunday. Imagine how many I’ll have on Monday.

This is the problem with answering email — it generates more email.

Marc Smith of Microsoft Research calls the standard time based sorting of email clients the ADD sorting. This two minute rule is the type of thing that makes me recoil in shock and give up hope for a sane future for my daughter. Shouldn't we be removing email that takes less than two minutes to answer? I feel that something that doesn't require attention to interact with is worthless - I'd rather spend my time engaged with concerns that require real thought! Remember, if you get a reply from Robert Scoble it means he hasn't spent any time thinking about the answer.

What I think is really going on here is that for some reason email clients have never provided any real tools for content analysis and management - the market is actually wide open for some quite trivial new functionality (this is what happened with desktop search applications).

Powerset In The Observer

Briefly, Powerset gets a writeup in The Observer:

Barney Pell is pursuing the dream of a 'natural language' search engine. He says that today's products, such as Google, search only for keywords and cannot, for example, distinguish between 'book for children', 'book by children', and 'book about children'. But a natural language search could identify 'function' words, understand that word order means something and respect the importance of small 'stop words'. Pell, 38, believes Powerset's search engine, soon to be launched, will be a catalyst for the 'semantic web'.