Data Mining: Text Mining, Visualization and Social Media

Tracking Microsoft Buzz with Blogs, Twitter, Bitly and Videos

Microsoft is an incredibly diverse company. I've just celebrated 5 years here and still don't have a full appreciation of the breadth and depth of products and innovation that the corporation generates. After BlogPulse was unplugged, I felt something of a hankering to continue to follow the buzz around Microsoft, partly as a way to better follow what the company is doing and how it is perceived in the online world.

I'm a big fan of TechMeme, but it has some challenges when it comes to tracking news and trends around a specific company. Firstly, I don't know the sources that are used and the ranking mechanisms in place, so it is hard to really understand quantitatively what it represents. Secondly, with limited real estate, while a big story may be happening for a company of interest, it can be crowded out by other events. Thirdly, I can't help but think it has a strong valley culture bias. Fourthly, it hasn't evolved much in the years that I've been visiting it.

So I've put together an experimental site called track // microsoft which follows a few blogs, clusters posts that are related and uses Bitly and Twitter data to rank the articles and clusters of stories. In doing this, I observed that many posts in the blogosphere about Microsoft would contain videos (be they of Windows 8 demos or the latest research leveraging the Kinect platform).

The site has three basic columns. The first contains established stories, represented by clusters of articles. The second represents a more timely view of posts. Both of these columns use Bitly and Twitter statistics to rank, with a bias to recency. The third column shows videos which have been embedded in posts multiple times.

Thus far, I find the stories and videos that surface here to be very interesting. This is where I first learned about:

• Microsoft's site for the new Hunger Games movie.
• The dynamics of Nokia's presence in the UK cell phone market.
• The release of the iPlayer on the Xbox (how cool is that!)

The nice thing about track // microsoft is that it should be easy to repurpose it to follow any topic - all you need is a list of feeds. I'm hoping to put together pages for taiko, skiing, Kauai, ...

Please take a look at track // microsoft and let me know what you think.

msnNow Offers Real Time Trending, Recalls BlogPulse

Microsoft has just pushed out a new product under the MSN brand called msnNow. It is a combination of data mining for trends and topics and editorial content. Surfacing trends has long been a staple for the real time web / social media, but it is an area that I feel has never really been done well. Twitter trends, for example, tend to be less than informative and more of a reflection on the constraints and structure of the twittersphere. msnNow has the potential to take trending seriously, especially if there is commitment behind the product and the contribution of the editorial team.

Parts of the product remind me strongly of the recently decommissioned BlogPulse, for example this module:

 

Take a look!

 

First Day with Windows Phone 7

I've now had a Windows Phone 7 device (Samsung Focus/At&T) for 24 hours and thought I'd share some of my thoughts. Note that this is replacing my iPhone 3G.

I've been looking forward to using the new phone for a while now. Certainly, working in Bing Mobile has brought me pretty close to plenty of the excitement around the new product, but Microsoft's generous offer of a free phone pretty much sealed the deal. The first hurdle was getting the device. The AT&T store was sold out and it had to be ordered and delivered. Once in my hands I was immediately frustrated by AT&T's activation mechanics. I had to take three different paths before finally getting the thing up and running. However, that frustration quickly dissolved as I started setting up and playing with the new phone.

When using a piece of technology that takes a completely new approach to an existing problem, one has to bear in mind that things that seem initially awkward are really just reminders of how one has become trained to use the previous stuff - it is a relative measure of difference, not an absolute measure of design. The biggest difference between the Windows Phone 7 UI and the iPhone is that, from an engineering perspective, it is closer to an object oriented metaphor. The tiles that one can place on the home screen are really references to objects of a variety of types. I can place a location resulting from a local search right on the home surface for easy reference. I can place a person on the home screen for quick calling and email. Each of these differently typed objects is associated with different types of actions (one can ask for directions to a location, or for the contact information for a person). Getting this is key to getting the most out of the Windows Phone 7 experience. Personally, I think this metaphor can go even further, with a more comprehensive exposure to the various types of objects.

The first app that I loaded was OneBusAway - a must for any serious public transport commuter in the Seattle area. Initially, I was disappointed with this version of the application. I think it is going to take some time getting used to, but I'd like to see some improvements. Note that, as I mentioned earlier, this is also a reminder that my first experience with the iPhone version of this app was also confusing and it took time to figure out the sweet spot in its interactions to get what I wanted out of it.

Next up I started playing with the browser. All in all I found it to be pretty snappy and a reasonably rounded implementation. Surprisingly, the dynamics of the mobile version of techmeme didn't work. Hitting the 'more' and 'new' tabs don't appear to load any additional content. I'm not sure if this is a bug or a limitation in the javascript (note that with javascript, optimization for the peculiarities of non-standard browser - i.e. all browsers - can impact the use of other browsers, so this may be an issue that techmeme can fix independently of any issues present in the phone browser).

I've played with most of the functionality offered by the phone and have really learned to enjoy the experience. The calendar is great - I much prefer it to the iPhone version from a design point of view; opening Microsoft Office documents is a very smooth experience; the Facebook integration can clearly drive more use of the social networking sites features, and I really liked the idea of linking profiles across different contexts.

However, the biggest surprise to me was the Zune functionality. This really impressed me. I found the experience of playing music on the device quite delightful and the desktop Zune product is also a very well designed and engaging system. I was also very happy with the way in which the phone integrates data with the cloud, including pictures and OneNote documents.

There is one feature that I wish existed - a simple way to transfer my contacts from the iPhone to the Windows Phone. I've read various summaries on how to do this - it is possible - but it should be far easier.

Bing Maps Continues To Innovate

Take a moment to check out this video which shows some of the content that Blaise presented at TED today. There are two key features: integration and matching of photographs to our human scale imagery, and the integration of the world wide telescope. These features are pretty cool, but ultimately it is the whole idea of the mapping ecosystem that is the real winner.

The Street Shots app, which matches images to our human scale experience has a couple of really nice emergent qualities. The human scale imagery, in some sense, is more useful the more objective it is - matching images can really bring a place alive by capturing a human moment or event.

Secondly, when someone has uploaded an historical image, once can experience a location with a view to a different age. Here is a picture of Vancouver from 1890.

 
Bing maps - the ecology - is only just getting going! 
 

Local Lens: Bing Gets Hyperlocal

Today Microsoft rolled out a new version of our map site. There are many new features to explore. Firstly, the site is entirely powered by Silverlight. This means that the user can explore mapping and local data with the full power of deep zoom technology – a more sophisticated look and feel. Secondly, the platform has been re-architected to deliver an extensible set of features via an application framework. For example, traffic is not a feature of the map control itself, but an application that is running in the mapping framework.

While the mapping base layer may look familiar to most users, the zooming model now demonstrates the complete integration of maps, overhead imagery, oblique birds-eye imagery and, new to Microsoft, Streetside imagery. The Streetside imagery introduces a novel 3d experience which allows for a richer and more immersive experience.

One of the applications that is rolling out in this new framework is Local Lens. This application provides access to hyperlocal blog content by identifying key blogs associated with cities and neighborhoods, and by analysing their content to understand the locations (addresses, businesses, neighborhoods, schools, etc.) that are mentioned in their posts.

You can access this application through the gallery of applications by clicking on the

in the lower right hand corner of the left hand pane.

Local Lens, along with a number of other applications, can then be enabled by hitting the application card.

Local Lens covers a number of major US cities and allows navigation down to the neighborhoods within those cities. Here, for example, is a view of West Seattle. Each pin point represents a location (or several locations) mentioned in one or more blog post.

The shaded area under each pin represents a clustering of proximate locations. By interacting with the pin, you can access a mini browser showing summaries of the posts:

The left hand pane can be used to page through days of posts and to access a fuller reading experience.

When a post is selected in the left hand panel, locations are highlighted. Interacting with these highlights will emphasize the location on the map and a click will navigate you to that address, business or neighborhood.

Note how the pins are carried through to the streetside images to provide access to local content right on the street:

In addition to the hyperlocal application, check out the Twitter mapping application.

Additional commentary here.

Update: here is a nice video introducing the new system.

Visual Search is Semantic Browsing

I really like Bing’s new visual search feature. I like it partly because of what it does, but I like it even more for the fact that Microsoft is taking risks and exploring new ways to help users interact data.

If you’ve not yet seen it, got to http://bing.com/visualsearch.

However, some of the reaction to this new feature haven’t really understood what it is. ReadWriteWeb for example writes:

Bing will now give users the option to visualize the search results on a 3D grid with icons or photos.

In visual search, when you are looking at the visualization of icons, you aren’t looking at search results but a representation of the object. For example, this view of digital cameras:

is not a picture of a bunch of blue links but a matrix of all the cameras (camera concepts if you will).

You can filter this set by features of the concept type (in this case type, megapixels, optical zoom, brand and the qualitative ‘good for’).

Once you have selected one – by finally clicking on it – you then generate a search string which is passed to Bing’s traditional search interface. This will then bring you to the standard set of blue links.

Chris Dannen at Fast Company, in the course of writing a somewhat complaining article, makes an interesting comment also relating to the naming of the feature and the implications:

[V]isual search isn't very deep; when I click on something, I want the visual goodness to keep going, and it doesn't.

TechFest 2007

TechFest 2007, an annual showcase for projects at Microsoft Research, kicked off yesterday. This is great timing for me as I started this week at Live Labs and can jump start forming a map of research and innovation here at Microsoft.

Contrary to popular belief, I don't expect that working for Microsoft is going to really impact the content of this blog. In the case of TechFest, this is to some extent a press event (today was the press day), so once I figure out the exact guidelines I'll probably write something about it. Having heard about a few of the projects on display, I'm certainly psyched to attend.

Microsoft Takes on Google Earth

Microsoft has released a new feature of its mapping services (requiring a download and install for Internet Explorer) that takes direct aim at Google Earth. I was half way through writing a post about how cool this is (and I still believe that it is, BTW), when it managed to crash my PC. After rebooting, I'm having major problems trying to run it again (I try to launch, it tells me to install it, I try to install it, it fails).

Anyway - as I've lost time with this, the details will have to wait for another post. The key thing to note is that Microsoft has gone for a level of detail in its 3d city scapes that leapfrogs Google's current technology. In addition, they are integrating advertising with this 3d world in the form of billboards.

Microsoft Delivers Clustered Search Results

This is new to me: Microsoft Research Asia appears to be working on clustered search results a la Vivisimo. I saw this URL: http://rwsm.directtaps.net/ in my referer log which give search results with the familiar clustered tree on the left.

It looks like this has been around for a while, at least since the end of January.

Update: Ok this is old - but still new to me. Here is a post from July 5th 2005 (!) mentioning the site.

 

Microsoft: Origami Project

I'm not fanning the flames of hype, just tracking them: