Data Mining: Text Mining, Visualization and Social Media

What I Did On My Vacation

While on vacation, I took no computer, had no internet and didn't blog - this was all good. However, sitting on the beach (just like swimming laps or dozing in the bath) is often when the best thinking gets done. Here are some of the things that came up:

• How can one use mapping applications to help with getting a good snorkel? I looked at Google maps a little and found that it is actually hard to tell coral from turtle grass in many cases - especially close to the beach. I've done some searching and found some resources that I'll write about later on the topic of remote sensing and coral.
• How can one make a better selection of a hotel or other accommodation? I want to be able to visit my accommodation in a SketchUp form, look out of the window, see the view, walk to the beach.
• How can search mix data types? For example, why can't I search for 'a beach near a marina and near good coral' and have the results come from text and image data - this requires inference, something which search engines don't touch.
• When you meet people on vacation, there is a good chance that there are plenty of similarities - where are the online tools that can continue these relationships of trust so that one can visit other spots on the planet that you are very likely to enjoy? I actually have a good answer to this, but it will be a while in coming...

I'll try to explore these topics over the next few days (or weeks or months...).

Getting the Name Right: Farecast Update

I recently got an email from Farecast - the site that predicts the movement of air fares and advises the user to buy or wait for future travel. The cool thing is that they now include Pittsburgh in the list of 55 cities they offer.

I recall hearing that the C*O of Netflix indicated that the company was not called 'DVDs by Mail' as their business is about getting movies to people, not about mail based DVD distribution. One day we will be streaming movies from our Netflix accounts. Farecast could potentially target any market where the price of the product fluctuates over time. In particular, they could target the usual basket of things that any travel site offers, including hotels. However, their name is shrink wrapped to transport ticketing which may require an name change in the future.

Learn to Fly

Farecast, a sytem that uses hi-tech to give fare chasers more power has been in the works for a while:

Farecast, formerly known as Hamlet Inc., will utilize predictive technologies and data-mining techniques to deliver an innovative, leisure-focused, online travel experience. The company has evolved from a University of Washington research project, led by computer science professor and Internet search expert Oren Etzioni, into a venture-backed company. Farecast is located in Seattle, Washington, and is funded by Greylock Partners, Madrona Venture Group and WRF Capital.

TechCrunch just posted about FlySpy - a company that looks like it might make it to market sooner than Farecast. The TechCrunch overview contains an image of a timeseries showing dates and ticket price changes. One thing to keep in mind, this is not like predicting weather: a natural, but complex, system. There are folks on the other side of the price changes that are also running their datamining, predictions and optimization algorithms. It's going to be more like an arms race.