Twendz: Twitter and Sentiment

Waggener Edstrom (who do plenty of work for Microsoft) has released a Twitter topic/sentiment tool called Twendz (other coverage here and here). The site shows both a tag cloud of related terms and attempts to surface positive, negative and neutral posts on the topic. Thus far, I’m not too impressed with the sentiment analysis on a per tweet level. Here are some postitive false positives tweets about Jim Cramer:

• The most important video you'll watch this year: Jon Stewart ends Jim Cramer's career Video
• oh wow, John Stewart probably ended Jim Cramer's career last night, it was amazing.
• Please take 21 minutes and 12 seconds to watch this video. http://is.gd/nahG Jim Cramer and John Stewart discuss financial markets.

Here’s some of the negative false positives for Jim Cramer:

• Jim Cramer is not at fault! He'll agree to be blamed, but YOU PEOPLE are the ones investing! Be accountable! (still love J. Stewart, though)

Here’s some negative false positives or watchmen:

• Fair point But that is intensely faithful to the original graphic novel. It's dark, dark socio-political commentary. #watchmen
• back to designer mode and crazy to see Watchmen!
• Saw #Watchmen. Must watch for anyone interested in politics/visual effects. The complex philosophical plot kept me at the edge of my seat.

It is interesting to see this in the context of recent posting about the twittersphere making things easier for sentiment mining. While the system should be doing the right thing with valence shifters (negation, etc.) these examples also demonstrate the problem of topic association. In addition, the examples with Watchmen demonstrate the problems between the nature of the plot and the evaluation of the film. And, of course, like nearly every tool in this space, by providing a direct link to the raw data, the errors get surfaced and the user begins to question the value of the results.

Summary: I’m really excited to see that WE is building tools in this space. However, it looks like they are starting at the shallow end when it comes to sentiment mining.

Comments

[Nigel Legg]

I think automatic sentiment flagging is still very much in it's infancy, and I haven't come across any system that I can be sure is reliable. I've got a hunch that the requirement for getting sentiment tagging right for social media will lead to some major advances in computer understanding of language.

[Kevin Murphy - Digital Experience Director]

Thanks for kicking off this discussion. I'm one of the folks at Waggener who worked on this. You're absolutely right on seniment analyis being in its infancy. On this project we particlarly struggled with intepreting sentiment with such a small data set to work with (140 or fewer characters). We do hope that people find this a good monitoring tool to look at aggregate sentiment and identify interesting discussion points trends and related topics.

Thanks,
Kevin Murphy

[Brendan Cooper]

Hi,

I checked out the sentimenting for the Jade Goody search today. At first it showed overwhelmingly negative, which I think could be accounted for instances where "Sad that Jade died" would be interpreted as negative even though, really it was positive.

However, at the end of the day (literally), it said sentiment was mixed - which did in fact reflect the opinion of articles and interviews on the subject. So, did it work? Yes, this time. I think it's an interesting approach and will be testing it out on other issues.

Thanks
Brendan