Data Mining: Text Mining, Visualization and Social Media

Technorati Indexing Twitter

I just did a search for ICWSM on Technorati and it surfaced a tweet by Dave Sifry from his Twitter account. David is giving an invited talk at the conference!

Some Perspective on the Twitter Paradigm

A referral in my log lead me to this BBS page which (as far as I can make out) contains links to a number of Chinese Twitter clones:

• fanfau.com
• jiwai.de
• taotao.com
• v2ex.com/ing
• weazone.com/say.do
• wulog.com

I'm guessing this is just the time of the iceberg for Chinese Twitter sites - I wonder if there is an index of all Twitter clones, just to get an idea of how big this new paradigm is. Turns out there is some information here: Twitter Fan Wiki; Wired wrote about some others way back in May 2007.

A cursory eyeballing might suggest that there are certainly 10s of micro-blogging platforms out there, possibly more than 100?. It seems clear that aggregating and analysing this data is going to be a requirement for social media analysis firms throughout 2008 and beyond.

Twitter, BBC, Rugby

I just noticed that the beeb has integrated Twitter on to one of its blogs covering the English team's march to victory in France.

Social Media Visualization: Twitter Blocks

Twitter has released a visualization which renders either the public timeline, or your own personal timeline in an interesting manner: Twitter Blocks.

The visualization has generated some modest buzz:

• Stowe Boyd:   A cool way to wander over your friends networks, looking for interesting people. This ... is very helpful.
• Information Aesthetics recognizes the debate: meanwhile, a heated discussion exists on TechCrunch about the purpose & usefulness of such fancy data visualizations. what is your opinion?
• TechCrunch points out that the project was done with Stamen Design, the people behind the Digg visualizations (which are also somewhat controversially on the boundary between almost useful and only pretty).

Scoble (mis)Reads the Twitter Tea Leaves

I'm currently reading Fooled by Randomness by Nassim Nicholas Taleb which may explain my lack of appreciation on this, but I'm having a bad reaction to Robert Scoble's Fast Company post on Twitter and other micro-blogging platforms. My general problem is that it is all anecdotal, heavy breathing and hearsay when in fact it could have been simply founded in data. Scoable says:

[Micro-blogging] services mix contacts, instant messaging, blogging, and texting, and they're poised to make email feel as antiquated as the mimeograph.

It is pretty standard to underestimate the massive volume of email that we generate. According to Yahoo Answers:

As best as we can figure then, the number of emails sent each day far exceeds 2.25 billion. It may be approaching 62 billion.

Ballpark, blogging is around <small int> millions of posts per day, and Twitter is less. As for its growth, I don't see significant growth now in Twitter (though ask Akshay for a more informed opinion).

On the marketing potential of mining these <141 character tweets, Scoble states:

Sales and marketing are lagging in seeing the potential here. When I used all these services to tell the world that my wife and I were expecting a child in September, I anticipated hearing from the world's largest consumer-products companies begging me to try their latest diapers, food, car seats, and financial instruments. What came back? Nothing.

Firstly, there are two major paths in handling unsolicited expressions regarding products or product opportunities online (connecting with everyone or connecting with the influencers). Given that blogs and message boards contain far richer data and thus far more accurate and relevant mining results produced from it why would one jump on Twitter long before that space had been fully established?

Secondly, while I'm sure Scoble has plenty of anecdotal evidence, how much of the online conversation is relevant? Did he use Twitterment or some other search service to estimate the signal to noise ratio? If so he doesn't mention it in his post.

Well, perhaps when you have over 1000 blogs to read each day there isn't time for any really deep analysis and on must, as one so often sees on the blogosphere, go on instinct.

Personally, I think there is huge value in the Twitter data, but it is at the aggregate level (most likely intersected with geographic information and other filters).

Twitter Saves TinyURL

To pass a link via Twitter uses up valuable resources (140 chars or fewer). Consequently, many users turn to URL shrinking services such as TinyURL. It looks like Twitter may have stopped the fall in TinyURL's popularity:

Research Paper on Twitter

Akshay's paper, to be presented at KDD, is now available on line. Lots of interesting data about Twitter. Interesting to compare the rate of growth of users and the rate of growth of tweets.

[via eBquity].

Twitter Facts: A Blog Analysing Twitter

Bruno sent me a pointer to his new blog: Twitter Facts. I'm way behind the curve on noticing this - a great source for statistics on Twitter.

TweetVolume

A little tweet told me that Wowza has an interesting comparison tool for Twitter term volume.

TwitterViz

Time for one more (other Twitter stuff here)? I subscribed to Voisen.org after I read this post on an app he is working on which displays Twitter data. Now the application - TwitterViz - is available. TwitterViz logs in to your Twitter account and then displays the tweets of the people you follow (or is it your friends, I get confused with that). It turns out, for me, those people aren't tweeting a whole lot. Here is a screenshot provided by Voisen: