Data Mining: Text Mining, Visualization and Social Media

Animated Zoom at Seattle International Film Festival

I love the Zoom books by Istvan Banyai, and I really like the SIFF trailer which animates the Zoom concept over classic cinema moments:

Whose Oil Spill is it Anyway?

While to most of us, the first name that comes to mind when we hear about the oil spill is BP, as this article in the New York Times indicates, there are a number of commercial entities involved, including BP, Transocean, Schlumberger and Haliburton.

Bloggers are spending most of their attention on BP.

The stock market appears to have divided their attention more evenly (among those entities in the public domain), with Transocean (RIG) taking the deepest dive.

 

A Visual Guide to the International Conference on Weblogs and Social Media (ICWSM)

I’m continually impressed by the breadth of content presented at ICWSM. This conference is truly achieving a unique position for social media and social network researchers: bringing together social science and computer science. I thought it would be interesting to capture the conference visually, so below is a selection of the visual representations of data and results from some of the papers from the conference. Each is linked to the online paper page, so click through to learn more!

 

The Full Wiki Marries Mapping and Wikipedia

Due to an increased amount of traffic from Factbites, I visited the site today and found their new product: The Full Wiki. The Full Wiki extracts locatable concepts from Wikipedia articles and provides an accompanying map annotated with these locations.

Ethnicity and Geography of Facebook Users

ePluribus: Ethnicity on Social Networks, by the Facebook data science team includes some interesting estimates of the geographic distributions of Facebook users. My guess is that these are not dissimilar to those found in the census data, confirming the team's suggestion that the ethnic distribution of Facebook users is approaching that of the internet population in general.

 

The Buzz on Facebook's Open Graph

Here's what Blogpulse says about Facebook's Open Graph:

What is Data?

Of all the concepts that are thrown about in the web/data mining space, data, information, knowledge and content seem to be the most important, but also the most overloaded and slippery. In many situations, this isn't too much of a problem, but when dealing with the creation of large scale systems that are intended to derive one (e.g. knowledge) from the other (e.g. data) it becomes vital to get some idea of where the boundaries are.

The definition in Wikipedia for data is a reasonable start, but, for me, not firm enough:

The term data means groups of information that represent the qualitative or quantitative attributes of a variable or set of variables. Data (plural of datum, which is seldom used) are typically the results of measurements and can be the basis of graphs, images, or observations of a set of variables. Data are often viewed as the lowest level of abstraction from which information and knowledge are derived. Raw data refers to a collection of numbers, characters, images or other outputs from devices that collect information to convert physical quantities into symbols, that are unprocessed.

I need a bit more.

To me data must include

• a representation: the set of symbols used to capture the data
• an intention: the reason for capturing the data
• an object concept: the thing about which it is data of
• a semantics: a set of 'rules' for interpreting the representation

If you think of a sequence of numbers representing a sample of frequencies recorded from a microphone pointed at a song bird. We might represent those frequencies as numbers. The numbers have to be represented as symbols and those symbols have to be known to map to frequencies of sound waves in a certain unit of measure. The object concept is the sound wave and the intention is to capture the song to determine (for example) the characteristics of the song.

I believe this is all important in a definition as I look at data from the perspective of archiving and discovery. If I come across a document with numbers written on it, is that data?

Of course, the interesting thing about capturing these details of data is that they can then be layered to arbitrary representations. Electrons to binary, binary to digital, digital to unicode, unicode to xml, etc.

Manufacturing Consent in the Google Age Part 2

I finally bit the bullet and wrote some code to pull news stats from Google News to provide something more attractive than the charts that Google News Archive provides. I should be living up to other posts on this blog and present the data in HTML 5 charts - I'll get to that. In the mean time, here are two charts which capture mentions in the news of some key terms present in the main stream media during the US's invasion of Vietnam. Note, of course, how these terms are getting plenty of press attention now.

 

The y-axis is the number of documents Google reports in its archive. The x-axis is without labels (Google docs seem determined to copy some of the worst aspects of Microsoft Office's charting controls). It represents the years from 1960 to 2010.

Some more context to this thread can be found in my earlier post.
 
 

Doraemon Puzzle Applications

A friend visiting from Japan showed me how the creators of Doraemon have a number of free apps which when arranged on the iPhone create a full image. Interesting to think of such strategies to drive app downloading.

Two Views of UK Election Trends

Briefly, the two charts below show firstly mentions of the party leaders without collocation with mentions of the other leaders, and secondly mentions of the party leaders without this restriction (i.e. without requiring that the name appear in isolation).

Gordon Brown's microphone gaffe is clearly visible (the second blue peak). The first graph can be thought of as a description of each candidate's recognized identity. Notice how Brown is isolated in the first graph around the unfortunate microphone story.