Data Mining: Text Mining, Visualization and Social Media

TechNet August 2009

TechNet’s August 2009 edition features on of my network visualization on the cover.

Visualizing Music

I just came across this video on Mary Godwin’s Body Electric blog.

inwit’s channel on YoutTube as a few more examples, like this:

Every Sparkline Now

Every Moment Now has a nice effective example of sparklines in action.

Bush: It’s all downhill from here

The Beeb has a nice little interactive time series showing multiple poll results for Bush’s approval rating.

As Seen in the Guardian

Briefly, one of my blogosphere visualizations appeared in the Guardian on Thursday 8th January 2009.

not even the rain

Chris Riccomini pointed me to a visualization he has worked on with his sister, Alex Riccomini, of ee cummings' work.

The visualization shows the complete work. Starting at the 12 o'clock position, each bar at the edge of the circle represents a poem. Poems progress in a clock-wise direction. Each ray represents a line in the poem. The coloring of the bar represents the theme of that poem. The thickness of the bars represents the word count for that poem.

 

I'd be interested to know more about how the themes of the poems were determined and if an automated approach would lead to even more interesting presentations.

Twitter Venn Diagrams

[Update: I was having issues with this app due to corporate proxy rules – it’s all good from home.]

Jeff Clark over at Neoformix has come up with a really simple but compelling application/visualization. For 2 or 3 keywords, it displays the venn diagram of search results over Twitter for the sets and their intersections. Of course, there is no reason why it needs to be pointing at Twitter – the data could come from anywhere.

Unfortunately the app is down right now (not sure if it is the client or Twitter search that is broken), but you can get a good idea of what it does from Jeff’s blog posts here and here.

FeedViz: An Integrated Feed Visualization and Reading System

Jason Priem recently pinged me with a link to a project he’s working on: FeedViz. FeedViz provides several dimensions along which to explore and consume feeds: time (via a time series), tags (via a linearized tag cloud) and specific blogs (via a list). Selecting on any of these dimensions updates the display of the other 2. Finally, you can read posts that exist and the intersection of (the settings for) these dimensions.

The tag cloud is generated using “two numbers for each word:

• The first is frequency. Frequency says how many times a word is used per 1000 words. If you hover over a word, you'll see its frequency to the left of the frequency change value.
• The second is frequency change. Often, a word will be more (or less) popular than usual in a certain time period (for instance, "election" in early November). Frequency change measures that difference as a percentage: greener words are unusually popular; redder words are the opposite.”

While I really like the design, animations and implementation, I’m not convinced that the above approach is the best way to surface keywords. Of course, it depends on what the purpose of the keywords is (descriptive, discriminative, or trendive), but I’d love to see this stuff running on something like BLRT or TF.IDF.

Zoetrope: That is the Web that Was

Update: Here’s the video

Eytan et al have been working on Zoetrope for a while, but here is a new article/video that shows of how far they’ve come. Zoetrope is a combination of a browser and a web archive that allows the user to manipulate the temporal point of view of a web page (or parts of a web page).

The Internet contains vast amounts of information, much of it unorganized. But what you see online at any given moment is just a snapshot of the Web as a whole -- many pages change rapidly or disappear completely, and the old data gets lost forever.

"Your browser is really just a window into the Web as it exists today," said Eytan Adar, University of Washington computer science and engineering doctoral student. "When you search for something online, you're only getting today's results."

Now, Adar and his colleagues at UW and Adobe Systems Inc. are grabbing hold of the fleeting Web and storing historical sites that users can easily search using an intuitive application called Zoetrope.

"There are so many ways of finding and manipulating and visualizing data on what we call 'the today Web' that it's kind of amazing that there's no way to do anything similar to the ephemeral Web," said Dan Weld, a UW computer science and engineering professor who also worked on the application. One service, the Internet Archive, has been capturing old versions of Web sites for years, but the records for the stored sites are inconsistent, Weld said. More importantly, there's no easy way to search the archive.

(Note to UW – make your videos trivially embedable).

Overnewsed but Uninformed

Overnewsed but Uninformed has to be one of the best titles out there. I’m still not sure what the site is (I don’t speak German), but it has some interesting stuff.