Data Mining: Text Mining, Visualization and Social Media

Visualizing Jane Austen Part 2

A better way to do this is to format with the columns indicating the chapters (rather than arbitrary flow based layout).

Pride and Prejudice, Collins

Emma, Churchill

Sense and Sensibility, Willoughby

Visualizing Jane Austen

A simple experiment - render a book in a summary form and highlight specific terms by changing the color of the paragraph.

Here is Jane Austen's Pride and Prejudice with the term Collins highlighted.

And here is Emma with the term Churchill highlighted.

Finally, here is Willoughby from Sense and Sensibility.

Note that this form of rendering has an artificial clustering of colours when highlighted paragraphs happen to be adjacent across columns.

Visualization of Google Plus Graph

I took a sample of the Google Plus graph and loaded it up in to the excellent Gephi graph rendering package. Due to the simple manner in which I crawled the data, this doesn't represent a true breadth-first crawl of the graph (rather it captures the random set of in and out links that appear on a user's profile page). However, what is interesting is that there is a clear tightly connected component that is distinguished from a more dilute area. I suspect, without any real investigation, that this is the core area of Google employees and some of their immediate connections as well as alpha users. This is complete conjecture, though.

In capturing this data, I also observed the complete in and out degree of each node. The top ten that I came across where (in order): Mark Zuckerberg, Vic Gundotra, Robert Scoble, Tom Anderson, Matt Cutts, Markus Persson, Pete Cashmore, Thomas Hawk, Evan Williams and Om Malik with links ranging from 456k to 33k.

Finally, I noticed that 48% of users haven't yet posted.

National Geographic's Egyptian Twitter Graphic

I love getting the physical edition of the National Geographic. I spotted this graphic in a recent copy - it shows some Twitter activity in Egypt (can you spot when the government turned of the intertubes?).

The National Geographic's Twitter account promoted it as an 'awesome' visualization. This made me think a little about what it means to be an awesome visualization - I'm not sure I know.

The Data Viz Quiz

As an introduction to a talk I recently gave I threw in some slides to form a data viz quiz. They were approximately as follows.

1. Basic Types : name the type of data presentation

1.1

1.2

1.3

1.4

1.5

2. Semantics - What does this represent?

2.1

2.2

3. Historical Personalities - Who created this?

3.1

3.2

While this was just intended as a warm up before the talk, I believe that anyone working in a data driven organization, or with the title 'data scientist' should be able to nail these questions on the back of their passion and curiosity around data.

Answers

1.1 scatter plot

1.2 bar graph

1.3 tree map

1.4 choropleth

1.5 cartogram

2.1 map of deaths from cholera due to contaminated water supply, by John Snow (1813 - 1858)

2.2 map of Napolean's march on Moscow, by Charles Joseph Minard (1781 - 1870)

3.1 Florence Nightingale (1820 - 1910)

3.2 William Playfair (1759 - 1823)

 

Visualizing Correlations as Small Multiples - Part 2

An alternate to the first small multiples approach to visualising correlations that I posted about earlier is to group variables vertically. This provides an obvious place for the variable name.

The vertical bar is intended to indicate the groups of correlated terms.

[Thanks to Eytan for discussion on this.]

Visualizing Progress in the Canadian Grand Prix

The official F1 site provides this visualization of the position of the drivers on a lap by lap basis through the race.

With a little more work (or a little eye strain on your part) you should be able to see the progress of Mr. Button.

Mapping Locals and Tourists Behaviour with Flickr

The Dohop blog points to some great work by Eric Fischer which provides a visualization contrasting a locals view of the city with that of a tourist. By looking at the frequency of photographs uploaded by a user, Fischer figured out approximately which were taken by locals (many pictures over time from the same city) and which were from tourists (only for a short period of time in a city). The resulting plots give an indication of how these different personas view cities.

Blue points indicate concentrations of images by locals, red, by tourists.

New York

Tokyo

Seattle

Via Thomas Lento

Analyzing a Week of Foursquare Data

The Wall Street Journal has created an animated presentation of a week of Foursquare checkins focused on New York. The analytic covers other regions with top checkin events, gender differences and a comparison between New York and the Bay Area.

Number Picture: Meta Tools for Data Viz

New to me is Number Picture:

Number Picture is a web application that enables you to come up with, easily create, and share fresh and interesting tools for visualizing data - that others can then use to visualize their own data.

These tools we call templates and they take in anybody else's data, shake it around a bit, add some sugar and spice, and then output it structured in a certain way in the form of a picture that hopefully is nice to look at, keeps the viewer engaged, and provides a refreshing change to all the millions and billions of bar graphs and pie charts that we see in our everyday lives.

We provide you with the tools to easily create your own templates and it is our wish to make it as easy as possible for you to do this. All templates that are created are free for anyone to use and all Number Pictures made are freely available for anybody to view. If privacy is an issue for you: you can also sign up for a premium membership and keep your Number Pictures private.

Take some time looking through their gallery.