Data Mining: Text Mining, Visualization and Social Media

iSpam

It seems that my prediction about buzz was pretty much on the money. iPhone buzz jumped to 1.4% yesterday (launch day). The thing is all over TechMeme and Tailrank. A new part of the story which I haven't read elsewhere yet is iPhone spam. I received an email with the text 'AT&T has activated X's new iPhone, the number is...' I'm not sure if this was generated via the set up process, or if it was hand rolled. The fact that it mentions AT&T suggests the former.

It is probably worth pointing out the case for NLP in the context of buzz monitoring. How would you surface all the blog posts from people who now own an iPhone? A keyword approach is going to be inaccurate. A few searches on Technorati for 'bought iphone' or 'my iphone' demonstrate this. Being able to pull out the type of relationships that Powerset talks about is the way to go.

I notice that Cymfony has a position open for a computational linguist. It has been interesting to track the transformations both BuzzMetrics and Cymfony have undergone post acquisition (by TNS and Nielsen respectively). A cursory review of their websites suggests that Cymfony is racing ahead. Expect a deeper dive on this competition a little later on.

What's That Ringing Sound?

Do you hear a ringing sound? As I suggested earlier, the iPhone launch is going to be a case study vehicle for social media anlaysts. Hey presto - BuzzMetrics put out a press release and will follow up with a web cast. In addition, they will be offering a syndicated report.

Interestingly, while the press release includes a BlogPulse trend graph, it fails to support the bullet point factoids with any data. For example, it describes the visual voicemail and the music library as being hotly anticipated features. Here's my graph:

My prediction is that chatter in the blogosphere about the iPhone will hit 1% tomorrow.

Update: Relevant Noise is also blogging about this from an analytics point of view, Max (BuzzMetrics), Pete (BuzzMetrics).

Reality Mining

Juan Carlos Baranhona is presenting some interesting data about the Reality Mining project at the MIT media lab. The system works via the use of cell phones. 81 individuals from 2 different but neighbouring communities were given these cell phones. The data collected is being analysed for a number of different features. Below, for example, is a visualization of the location of the cell phone carriers.

Visualizing the Signatures of Social Roles in Online Discussion Groups

I'm at the Communities and Technologies conference to participate in a workshop on the data collection aspects of social media analysis. Marc Smith, my colleague at MSR with whom I suffered a horrible day of travel yesterday from Seattle, WA to Lansing, MI, is giving a keynote this evening. Timely, then, that his recent paper 'Visualizing the Signatures of Social Roles in Online Discussion Groups' is available online and highlighted in this Infosthetics post.

I like the classes of authors that are described in this paper:

There are many important social roles in online discussion groups: local experts, answer people, conversationalists, fans, discussion artists, flame warriors, and trolls (Burkhalter and Smith 2003; Golder 2003; Turner et al 2005; Herring 2004; Haythornthwaite and Hager 2005).

Communities and Technologies 2007

I'll be traveling to the 3rd International Conference on Communities and Technologies tomorrow. See you there?

iPhone: Video a Big Deal?

While the iPhone is generating a Star Wars like anticipation in the real world, the online world is buzzing with discussion. The ad hoc analysis shown below suggests that the ability to play videos is a big deal among potential customers.

BuzzMonitor - Open Source Buzz Monitoring from the World Bank

Pierre Guillaume just dropped me a note with a link to an open source buzz monitoring system developed at the World Bank. I've not yet had time to do a deep dive on it, but the motivation and history of the tool certainly look interesting. Essentially, the WB was interested in monitoring discussion about itself and developed some tools to do just that - by pulling feeds. They then rolled they system, which is built on the LAMP stack, out as an open source project.

Pierre pointed me to a Salon article which covers the story.

BlogBelt

I saw this graphic linked to from Steve Rubel's blog and couldn't resist posting it. I guess I like the clean lines and the compromises made between geographic projection and space. I have no idea if the data is really interesting or even accurate. The data comes from FeedBurner which carries <1MM feeds, of which I'm not sure how many are blogs.

Data and Community

Jeffrey Heer et al's work may well be familiar to readers of this blog. I've been pointed to this paper almost simultaneously by two colleagues so I thought I should pass this on. I've not read the paper, but the video(linked to from the bottom of the page) makes for a very clear explanation.

Twitter Facts: A Blog Analysing Twitter

Bruno sent me a pointer to his new blog: Twitter Facts. I'm way behind the curve on noticing this - a great source for statistics on Twitter.