Data Mining: Text Mining, Visualization and Social Media

Vayama: Visual Interface for Flight Exploration

I've just been playing around with Vayama. Vayama provides a novel interface to searching for flights. Essentially, it allows you to enter origination and destination information by selecting locations on a map. It then shows you the results in the form of routes drawn on the map.

For example, let's say we want to fly from Seattle to Barbados. First, we select Seattle. By browsing the global map, zooming in exposes airport locations.

Here we see Seattle has been registered as the port of origin.:

Having selected Barbados as the destination, hitting select shows the results on the map:

I suspect that by mapping the results, users will be able to spot odd solutions to their flight search quite easily.

[Via Emergence Media]   

Tech Rumble

On May 28th, Microsoft pushed out some nice updates to its Virtual Earth mapping system (my post here). The next day, Google dropped streetside imagery into Google maps (note that Microsoft had a demo of this a while back). Today, Microsoft announced Surface Computing.

From what I've seen of the surface computing product, it looks like Microsoft may have beaten Perceptive Pixel to the punch.

Perceptive Pixel

I have to admit, I'm charmed by the technology and demonstration of Perceptive Pixel's multi point touch screen (shown earlier at TED and lately highlighted in Fast Company). I'm charmed, but I'd like to know a little more about the use cases. You see, if you watch the TED video, or the new one produced for Fast Company (direct link here) you'll see that the applications being demoed are, in themselves, pretty neat. Google Earth, some 3d model editing, graph/tree layouts, etc. However, there are other parts of the demo where Jeff appears to be just shuffling stuff around on a screen (mostly pictures). I'd like to know how it performs with a reasonably complex interface (like SketchUp).

I think that Jeff Han is a visionary, and I'm excited to know where this innovation is heading - at the same time, I'd like to better understand the nature of the disruption that is implied. As it is a tactile interface (well, sort of), perhaps this can only be answered by experience.

Global-i: DataSphere Lives!

Readers of this blog will be familiar with my occasional post on a back burner project I have going by the name DataSphere. As a reminder, here are a couple of screen shots:

Infosthetics points to InfoMagnet's Global-i application - incredibly close to DataSphere in many respects. The interface has two basic functions: an interactive, animated globe which allows zooming and rotation, and data overlays. Here's the interface:

Here is an example of Global-i showing export destinations for Oman:

The application is very well put together, the animation is smooth and the interactions are intuitive. There are some issues with certain views where, perhaps, aesthetics have been chosen over understandability or simplicity, but overall a really nice piece of work.

In terms of implementation, there are two basic challenges that delivering this type of application poses. Firstly, the rendering engine - drawing interactive 3d graphics in Java requires plenty of ingenuity and optimization. Secondly, the streaming of data from the server - this has to be done in a way which doesn't punish the client too much. Part of the solution to this problem is to simplify the data and compress it. The major simplification required is to the GIS data - specifically the country shape files. This is pretty straightforward. As far as I can tell, unlike DataSphere and other geographic data browsers, Global-i doesn't use multiple scales of data - a single data set is used at all zoom levels.

Global-i doesn't provide image overlays (such as satellite images) and uses 3d line drawing. DataSphere uses an image as its basic rendering mechanism and projects it using a custom 3d aliasing mechanism. At the granularity that Global-i uses, 3d line drawing is an appropriate choice and has produced elegant results.

When 3d rendering is limited to a sphere, there are some tricks that lead to greater efficiency. One is to essentially slice the globe with a plane parallel to the plane of view and only render things on top of, or in front of that plane. One of the downsides of this is that graphical data elements that transition from the visible hemisphere to the invisible hemisphere get truncated. This can be seen in Global-i when 3d arcs are used to indicate links between locations. DataSphere, which also uses this partitioning approach, renders arcs on the sphere's surface which essentially avoids this problem.

The Map Room: Comparing Online Mapping Services

Jonathan Crowe at The Map Room (profile) provides a good summary of links to comparisons for online mapping solutions. The post contains a link to Ryan Jonasson great page providing a side by side comparison of Google and Microsoft's web hosted maps (thanks to Matt Siegler who pointed me to this originally).

Reviewing the “big four” in online maps — Google Maps, MapQuest, Yahoo! Maps and MSN MapPoint (yes, in that order) — is problematic: they rely on the same map data and routing engine for directions, according to Webmapper, so it basically comes down to the user interface and extra features layered on top. But that apparently makes a big difference; via Webmapper again, a report concludes that people believe Google Maps is more reliable because of its user interface. Presentation matters, in other words; data along isn’t enough. It may be that Google’s clean pages, lacking the cruft of other services whose ads drown out a comparatively small map window, matters more to the general user than the hackability and other virtues we, as map enthusiasts, focus on.

It is interesting to note that Google and Microsoft share the same map data (something which Ryan's page really highlights. I noted earlier that Google at least uses Landsat imagery (all be it somewhat cleaned up).

Here is the BlogPulse view:

DIY Virtual Earth, Part 1

Everyone is busy sticking pins in the earth via either Yahoo's or Google Earth/Google Maps' APIs. But what does it take to build an application from scratch? There are three basic components:

• Data - lots of it; satellite data and GIS data.
• Maths - projections (mapping projections and 3D projections).
• Code - you need to build a client server architecture.

For satellite data, you need global coverage and you need detail. Currently, most of the great coverage - i.e. the coverage that produces the 'wow, have you seen this?' reaction, is limited to certain areas of the planet and certain types of locations. Google's data, for example, is heavy on cities, but not so heavy on places that Matt Hurst wants to go on vacation. This makes a lot of sense as the first thing people do is to look up where they live to see it from the sky. Populations are denser in cities, so more satisfied customers.

So, how do you go about getting Google Earth-like data? Landsat is your friend. Compare these two images, the first from Google and the Second from Landsat:

These pictures are comparable in quality. Of course, downloading and storing all the Landsat data would require significant disc space. However, if you have the right architecture, you could potentially use the web interfaces to this data to act as your server. Whatever you want to do, you want to have a look at https://zulu.ssc.nasa.gov/mrsid/mrsid.pl.

Next GIS data - you want boundary information and place name information. The first allows you to draw the polygons around countries, the second allows you to pinpoint cities (and act as a resource for location searching and geocoding). For this, turn to the world of potatoes. Specifically, the country data found via the Centro Internacional de la Papa. This will give you all the country boundaries as well as a goodly amount of admin1 and admin2 level data (i.e. states and counties or similar).

For location data, there is little to rival that provided by the Government of the USA. The Geographic Names Information System (GNIS) provides very large datasets of locations, names, alternate names, etc.

Coming in part 2, pointers to the mathematics needed to quickly play with this type of thing.

Battle for the Planet

This graph shows the percentage of hits on blogpulse for a number of terms related to recent announcements and releases in the mapping application/api world:

It suggests that Google is way ahead with its mind-share, though it will be interesting to see how the now-you-see-it now-you-don't application fairs in the weeks and months to come.

Map Madness

The world is going map crazy. I'm seeing a lot of posts recently about Google Earth (of course), job openings at Microsoft for their Virtual Earth product, the Where 2.0 conference, Yahoo's API, a9.com's new mapping/local search page (via Greg Linden)...

Eventually, things will calm down and people will start looking more at the value of these systems and (perhaps) less at the eye candy. One thing that is going to be interesting at that stage is the way that mapping and local search work together. A local search which presents results as a list of links has the potential to hide, or make less obvious, mistakes. However, now with Google Earth, when I enter 'london, canada' and it takes me to London, UK, I am presented with a very immediate way to see this error.

Google Maps/Projections

Google Blogscoped mentions the global Google map. Once at this level, the type of projection (or lack thereof) that Google offers is somewhat disappointing. The world doesn't look like this:

Geographers, politicians and other activists have battled along time to counter the inequities of this type of projection and the bias is gives to richer, more developed nations. The fact that Google hasn't explored other projections that would still be usable as an interface to their applications is something of an oversight given their intended 'do no evil' image.

[update] Andrew Hitchcock has some information that suggests that the projection issue is being addressed.

It is interesting to note that zooming out to a certain level gives a wrapped image. This is something that you need to do when drawing lines between arbitrary points, or zooming in and out of arbitrary locations.

Got Keyhole? GetMapping

If you like keyhole, you are going to love this product from GetMapping and NEXTMap. From the online sample it looks like the elevation data is far more detailed resulting in a far richer fly-over experience.