Data Mining: Text Mining, Visualization and Social Media

I Can Haz Prime Directive?

Fascinating, ironic, timely (wrt Future of News and Personal Democracy Forum) post on William's blog.

Noah Smith and I are co-supervising Tae Yano on a project involving analysis of political blogs, and Tae left a pile of results and code on her CMU web site as a way of communicating with us...world-readable. Surprisingly someone at one of the blogs she spidered, Little Green Footballs, actually noticed, leading to a lot of investigative work in this fascinating thread:

Anyone know what this page at Carnegie Mellon means? It’s some kind of experiment that involves comments posted at LGF, and I have a feeling it’s not friendly.
...
comment #5: Maybe it's post-modern poetry, academia-style? Using LGF comments as gibberish to transcend interpretation?

Perceptive Pixel: Update

I'm very late to this. When I first heard about Perceptive Pixel, I was very excited. Several videos of the very slick, very fun wall-mounted multi-touch based interface zipped around the blogosphere last year. But then there didn't seem to be much going on. Meanwhile, a number of companies started popularizing some of the elements of that interface, Apple with the iPhone (multi-touch), and Microsoft with the surface computer to name two obvious examples.

It turns out that Perceptive Pixel were in fact deploying their product. I don't watch broadcast tv so I had missed the fact that it was being used as a differentiator in discussing the primary elections in the US.

Here's an example.

Here's another one:

Future of News at Princeton

I'm back in Seattle after attending the excellent Future of News workshop hosted by David Robinson and Ed Felten. There were many contributing factors to the success of the workshop, not least of which was the cross disciplinary nature of speakers, panelists and attendees. In addition to participants in my line of work, or with similar areas of interest, such as David Blei (former colleague at WhizBang!Labs) and Kevin Anderson (whose career spans both the BBC, with some involvement with Backstage, and the Guardian, an organization that is not caught with with fretting about the past; and who blogs on Corante, for his sins) I met representatives (and survivors of) traditional newsprint organizations, individuals centrally involved in transforming news in the Web 2.0 world, academics with impressive access to the entire trajectory of media evolution, hackers and so on.

Given the rich range of voices, while we may not have solved any specific problems (despite Ed's closing remarks) I certainly feel as if we aired a reasonably good sample of them. On reflection, I stand by yesterday's summary: there is plenty of pessimism around old media structures and plenty of optimism around the opportunities that new sources and new forms of information, combined with new ways to filter, analyse and aggregate this data presents. Bridging the two positions, there is concern around issues of quality and value with respect to the nature of the content (that is to say, a contributor's ability to provide transparent and supportable content) - will the new information ecology support reasonable ideals for news? Note, to me, those ideals centre on making the reader better informed and more efficient at selecting content.

For other coverage of the event, a great starting place would be Kevin's posts (starting here with an account of Paul Starr's opening talk). Also Steve Boriss, Tim Lee, and Jack Kemp. Note that the presentations and discussions will soon be available online.

Industry Standard's Top B-Z Blogs

The Industry Standard has produced a list of B-Z blogs of note.

These are the blogs you won't see on the Techmeme Leaderboard, Technorati's Top 100 blogs, or the CruchBase BloggerBoard ... at least not yet. They include VCs, entrepreneurs, coders, experts, and observers, and they bring a delicious mix of insight, experience, and passion to their blogs. While they may not have the right amount of link love, they need to be on your radar screens.

I'm very happy that this blog has been included on the list!

Future of News: Day 1 Summary

Had a great day of presentations, panels and discussion today at the Future of News event here in Princeton (which reminds me a lot of Cambridge). In summary, I heard both optimism and pessimism regarding the future of news. Things that seem to be of concern:

• The collapsing of the newspaper model (which has had plenty of coverage) - though the Guardian's model was held out as an exception.
• Of greater concern: the lack of watchdog journalism implicit in "decentralized non-market" forms of media.

Optimism was expressed largely by those who were actively trying to push the evolution of the space (the best way to predict the future is to invent it).

I was mistaken for a BBC employee - can I put that on my CV?

Worldwide Telescope

Microsoft Research's Worldwide Telescope is available for download. I'd encourage you to go and take a look. I would have been writing up some details of it for this blog, but it is so easy to spend time exploring the sky and the planets that I'm left with no time right now! Note that it does provide planet models (including earth) as well as astral data.

Brand Tags

This is a great idea: the user is shown branding and asked to provide a single tag. You can then click through to see the tag clouds (which could be displayed a little better) for each brand.

[Via Nathan]

Powerset Factz, Star Wars

One of the best things about Powerset is its Factz feature. If you look at a page for a movie, you can see a pretty neat, completely automated summary of the plot. Have a look at Star Wars.

Powerset Launches!

Powerset, which provides a new relationship with web data via innovative interfaces and natural language processing, launched this evening. Take a look at this video:

I'll write more later, but for now, check out other posts I've made on Powerset and NLP. I'll try to keep abreast of the commentary as it comes in. Meanwhile, I'm waiting for Fernando to pounce.

Update: ok, some comments. A couple of things that people are going to get hung up on. Firstly, writers seem to be referring to the technology as context or contextual search - why not call it NLP. Not sure where that is coming from. Secondly (actually, this is more important) pundits are going to write about the wikipedia-only issue. They're not getting it. 90% of search results come from a tiny fraction of web pages due to the huge redundancy on the web and the differences between searcher needs and author/publisher intents. The task isn't to always search that huge set, but to get the answers to the user.

Spectra Visual News Reader

Another interesting find from Information Aesthetics. News classes, selected via the top menu, populate a rotating column of articles that are then read at the bottom of the display. Fun - not sold on the utility.