Data Mining: Text Mining, Visualization and Social Media

Opinion Mining and Sentiment Analysis; Pang and Lee, 2008

Congratulations to Bo Pang and Lillian Lee for getting their monograph on subjectivity published. The authors have been good enough to put the pdf version of the book online (here). The page for the book itself, from which you can order your copy, is here. The abstract:

An important part of our information-gathering behavior has always been to find out what other people think. With the growing availability and popularity of opinion-rich resources such as online review sites and personal blogs, new opportunities and challenges arise as people can, and do, actively use information technologies to seek out and understand the opinions of others. The sudden eruption of activity in the area of opinion mining and sentiment analysis, which deals with the computational treatment of opinion, sentiment, and subjectivity in text, has thus occurred at least in part as a direct response to the surge of interest in new systems that deal directly with opinions as a first-class object.

This survey covers techniques and approaches that promise to directly enable opinion-oriented information-seeking systems. Our focus is on methods that seek to address the new challenges raised by sentiment-aware applications, as compared to those that are already present in more traditional fact-based analysis. We include material on summarization of evaluative text and on broader issues regarding privacy, vulnerability to manipulation, and economic impact that the development of opinion-oriented information-access services gives rise to. To facilitate future work, a discussion of available resources, benchmark datasets, and evaluation campaigns is also provided.

Quantum of Solace

The next James Bond film is starting early with publicity, taking advantage of the 100 year anniversary of the birth of Ian Fleming. The Times has an interactive timeline and map (note an earlier BBC interactive tool from 2006).

The film will be release in November (IMDB, 007 site).

3D Video Mapping

A nice vide found on TechDrool which shows a tool for composing multiple real time video streams in a 3D model.

(Note the sensor alerting sound – is it taken from Metal Gear Solid?)

Lillian Lee AAAI Talk on Subjectivity

Briefly, Lillian Lee’s invited talk at AAAI 2008 on subjectivity is available online. I’ve not yet had time to look through it, but am hoping it looks beyond the simple lexical approaches that have been the mainstay of this field for way too long at real models of opinion and subjectivity.

The Surprise of Success

I was intrigued by this comment captured by Wired writer David Germain:

"We knew it would be big, but we never expected to dominate the marketplace like we did," said Dan Fellman, head of distribution for Warner Bros., which released "The Dark Knight." The movie should shoot past the $200 million mark by the end of the week, he said.

The simple tea leaves of BlogPulse would have helped…

And Now For Something Completely Different

The print version of Wired magazine points to GraphJam – the equivalent of LOLCats for data miners?

Summer Movie Madness

Here’s a chart of a selection of recent movie releases.

Queries for each movie are basically the name of the movie (plus variations). For example, (batman OR “dark knight”).

If you want to check in on things as they progress, the link is here. No surprise that Batman broke opening night records. BoxOfficeMojo says (via email):

'The Dark Knight' raked in $18.489 million in its midnight opening, according to distributor Warner Bros. That sets a new midnight opening record, beating 'Star Wars: Episode III - Revenge of the Sith's benchmark of $16.5 million. 'Sith's full opening day ended up at $50 million.

This marks 'The Dark Knight's second record after posting the highest theater count ever: 4,366.

We were disappointed to find that Mamma Mia was sold out last night! It was released on ~3,000 screens. FlixPulse is giving Batman a 98% score currently, though Mamma Mia hasn’t even registered there.

Batman is currently sitting at 1.75% of blog attention. For comparison, the second Pirates of the Caribbean movie came in at 1.25, the Golden Compass also hit 0.7%, the first Narnia movie peaked at 1.25%. Have a look at other related posts. Note that there may be some variation in how these graphs were created. Watch out for cases where a modifier like ‘+movie’ or ‘+(movie OR film)’ has been inserted to restrict matches. For comparison, here is a repeat of the above graph with the modifier ‘+movie’.

Batman

It’s not news : judging by the anticipation – the new Batman movie is going to be a monster hit!

Watch for it on FlixPulse!

Visualizing Radiohead

Radiohead’s new single has a video created using only range data (in a mixture of scales).

There is a good write up of this by Andrew Trice at InsideRIA, including a link to an interactive version of the data. Here’s a related post from a while back which features videos with a data/info graphics them, including the excellent Remind Me by Royskopp.

[Radiohead story via TechMeme]

Batman Anticipation Growing

Here are’s a graph showing attention around Hellboy 2, Wall-E and Batman.

Batman: The Dark Night will be released on July 18th, but you can already see growing attention. For calibration, Wall-E did $23MM on the biggest night of its opening weekend (Saturday 28th June) and Hellboy 2 did $13MM (Friday 11th July).

The first peak for Batman is around the untimely death of Heath Ledger, I’m not sure what the second is.