Data Mining: Text Mining, Visualization and Social Media

GeoRSS in Google Maps API

Nigel Tao (a former colleague from WhizBang!Labs - hi Nige) has just posted the news that Google's mapping API is now conversant in GeoRSS. Basically, this means that geolocated RSS feeds can now be rendered in Google Maps. Nigel gives the example of the SlashGeo feed. Below is a screen capture of viewing this feed and reading a post.

Geolocating Bloggers

TechCrunch posts an update on Outside.In, a service which brings a local twist to the blogosphere by associating blogs and posts with specific locations. This is a topic very close to my heart - I will be presenting a paper at ICWSM on geolocating bloggers. Outside.In uses user provided information to associate location with content - the information is provided specifically for the service.

We took a different approach - scraping profiles for user provided location information. By doing this, we created a reasonably large (and still growing) database of locations associated with bloggers (and, in fact, other social media authors).

We then started looking at an even more interesting problem: automatically classifying the location of an author based on their content. As far as we could determine, there is no real work published on this problem currently out there. Matthew Siegler, who did much of the work on this problem, found the initial results very encouraging. However, what started as a simple exploration of residence classification revealed itself to be a problem intersecting language, location, nationality and ethnicity at least!

Outside.In is not the first service to offer some form of local blog search. BlogDigger started a service based on meta geo information over a year ago.

The Time When

The BBC's experiment in retro blogging (thetimewhen.co.uk) just got a very important upgrade. The site collects articles which capture memories of events contributed by the general public. The RSS feeds for this site originally just had the time of posting, and the content. Now they have been augmented to include the time of the memory and the geographic location.

The two tags (dcterm:temporal and dcterm:spatial) are both recommended elements from the Dublin Core. One of the problems with the spatial tag (of which there can be a number) is that it essentially marks free text capturing the location. There is always a balance between getting accurate structured data and the interface for doing so. Flickr has approached this in a more comprehensive manner using a drag and drop interface to help people locate their photos. This too has its disadvantages - the structured information is only lat/long. In other words, there is no real information about the city, county, state, etc. In addition, as a brief experiment just showed in which I placed photos in the UK, it suffers by association from Google's poor international coverage.

Mapping Avian Flu

James Fee points to an application of Google Earth to track in (near) real-time the spread of avian flu.

Declan Butler, the chap behind the effort, has a great post on the topic which includes a list of othe rmaps that visualize the cases of the desease. I've posted previously on mapping the location of news articles about avian flu.

James' blog, Spatially Adjusted, is a great resource for GIS in general, but also from the POV of ESRI products and how they are reacting to the spread of consumer facing mapping products from GYM.

Zonetag: Geocoding Pictures in Flickr

You knew this day would come: you can now upload images to Flickr from a location enabled device and Flickr will take care of the geocoding. ZoneTag works by using cell tower information:

ZoneTag can automatically tag your photos with the location, based on the cell tower, in which they were taken. The location of the cell tower is not known until our user community updates the city, state, or zip code of their photos on Flickr. ZoneTag is then able to map the cell tower to the location.

on the following phones:

Currently, ZoneTag only works on Nokia Series 60 phones. ZoneTag is known to work on 6620, 6670, 6680, 6681, 6682, 7610 models, and it is likely to work on 3230, 6630, 6260 as well.

Note that you need to install an application to get all this geo-goodness to work. Via SEJ.

Reverbiage

Miss the last geolocated news interface? Don't worry, another will be along shortly...here it is: Reverbiage (via infosthetics). The difference with this interface is that it pans and zooms the map as new data arrives. This is impressive - at last some novelity in news mapping interface. However, I don't much like it. It makes my head spin and brings attention more to the interface than to the data.

I really appreciate the great work that Infosthetics does in tracking down bucket loads of interesting interfaces and data visualization work. However, I wish Andrew would cast a more critical eye over the pieces that he highlights.

Andrew - what do you think of Reverbiage? How about The Dumpster? Does the simplicity of the What's Up map make for a better display of located information than, say, NewsQuakes? I'm sure others would love to hear your opinions.

Where's the Snow?

On the 12th of February, New Yorkers received the biggest dump of snow the city had every experienced. Looking at the blogging activity around this event we can see a clear peak reflecting that fact. In addition, there is another larger peak. Memory being what it is, I wondered where that other snow fall had hit. However, as you can see from the trend line for New York, when people blog about the weather, they don't seem to explicitly state where it is. People in New York don't say 'it's snowing in New York', they just say 'it's snowing'. Only a small percentage actually provide both the meteorological  and geographic information in the text. This means that, yes, knowing where people are located is a key dimension in analysing online data - otherwise, when the aliens land, we won't know where they hell they are.

Actually, the fun way to view this is to consider how one picks up the smaller signal with the intersection of weather and location information automatically.

This trend, showing discussion about earthquakes, suggests that these disasters are blogged about with more geographic information. However, on the one hand the story is international and on the other, the earthquake in question is historical and being discussed in the context of hurrican Katrina.

Geocoded User Generated Media

Steve Rubel points to the next step in Topix' use of geocoded forum posts:

t's not enough that marketers/PR pros know what's being said about our clients/brands, what language this commentary originated in, how loud it is and if the speed is accelerating or decelerating. We also need to be able to pinpoint just where the conversation is strongest or weakest. Topix.net has taken the first step I have seen this direction. (Disclaimer: Topix is a CooperKatz client). They have created a real-time map that shows where the contributions from citizen journalists are coming from inside their walls. I think we'll see tons more innovation in this area this year. This is data that marketers die for.

It is a pretty cool map (see below). This is not the first effort to geolocate user generated media (not by a long shot). I agree with Steve, however, that this is certainly a must-have feature for analysts of this type of user generated data.

What's Up?

Infosthetics points to yet another geolocated news applications: What's Up? This one is a little different in that it uses a very stylized map and some nice flash interface design. The map itself has no information about countries (one can't tell in which country a dot happens to appear), but the bubble that shows the news title includes both source information and location (city, country) information.

Jeroen Wijering, the creator of the site, states that it takes news feeds from dozens of news sources (I'm assuming via RSS, and that there are multiple feeds per source).

One of the key questions in the usability of interfaces that geolocate news (or any other stream of information for that matter) is the mode of use. Something like Jeroen's interface suffers a little from the security monitor UI problem - you have to be sitting there watching the stuff pass before your eyes. An improvement on this would use the interface to archive and access information. Thus, I could look at the map and see how much news on what topics appears at each location. This is something I've attempted in the past to do with DataSphere.

Topix Discloses Poster Location

Members of message board communities disclose plenty of information on their profile pages, as do bloggers. In all cases, they have the option to omit or hide certain fields. Topix now discloses the location (based on IP lookup) of posters contributing to their forums. This gives a fascinating insight into what topics are important to people from which countries, however it also bypasses a very standard level of privacy - how can contributors opt out of this?

Via BuzzMachine