Data Mining: Text Mining, Visualization and Social Media

Social Business Design

Kate Niederhoffer, and her colleagues have posted about her groups move into a space they term Social Business Design. In a sense, what they describe is a reaction to the siloing of thinking around social media brought on by the term ‘media’ itself. This is, perhaps, an extension of my dismissal of the term ‘consumer generated media’ due to its origination in the world of marketing and advertising. Kate and friends are making the point that the social part of the future of online ecologies is not just about ‘media’ per se, but about the entire end-to-end, front-to-back dissemination, assimilation and synthesis of information and its integration with business processes and models.

While this is an interesting direction, it reminds me of terms like ‘democratization’. As anyone in a large organization can attest, capitalism doesn’t scale through democracy. Good companies succeed via Philosopher Kings. The idea that social effects will permeate a large organization without accommodating hierarchy is a little far fetched.

Niederhoffer et al’s thesis centres on four core notions. As Kate puts it:

• Ecosystem - a community of connections
• Hivemind - the socially calibrated mindset of individuals
• Dynamic Signal - the constant multi-faceted means of collaboration
• Metafilter- a method of finding signals in vast amounts of noise

The later two are clearly about information flow. Hivemind (which I’ve always taken to mean the aggregate psychological process of a group, not that of an individual moderated by social context) is also about information diffusion and the assimilation of social cues in moderating that information and republishing.

Perhaps the key to understanding what this topic is about is to understand the scale at which is intended to be implemented. I’m excited to follow where their thinking leads and what impact it will have on reshaping existing businesses and forming new ones.

Raveable: hotel appraisal

Raveable is a new review aggregation site which specializes in hotels. It claims to provide information for 55 thousand hotels in the US culled from more than 35 million reviews. The site mines these reviews to automate the summarization of opinions from guests for the hotels.

The data

Firstly, where does the data come from? Paging through a number of hotel sheets, it initially appeared to me as if most of the data comes from TripAdvisor. The site states that the data comes from a number of sources, including TripAdvisor, Expedia, Travelocity, etc. as well as individual blogs. Browsing the site brought more of these sources into view – in fact, the experience suggests that some hotels get more data from one source than the other. It would be interesting to know the demographics of those sources and to consider a more representative summary taking the demographics into account. Has Raveable formed relationships with those sources?

The UI

In general, I found the UI clean and pleasantly put together. One of the most important aspects of presenting data for a choice like hotel booking is to get some understanding of the ranking of the candidates and the coverage of the system (how many of the competing hotels are in the system). The UI goes some of the way here by providing an overall ranking:

However, it would be nice to get a bit of additional geographic context that shows other hotels near by. In addition, with the added layer of appraisal (summaries of reviews) one could get a picture of the general quality of the hotels in the neighbourhood. Am I looking at a good hotel in a forest of good hotels? A good hotel in a sparse neighbourhood?

The Interesting Part

Of course, the interesting part about this whole thing is the automated summarization and extraction of opinions written in natural language. With 636 reviews available, on average, per hotel, there should be good opportunity to deal with accuracy issues by doing something sensible with summarization and presentation (note that I’ve not yet seen a hotel with > 600 reviews so I’m not sure if all the reviews are assigned to hotels in the system).

My expectations for the extraction of opinions was that the positive stuff would generally be pretty accurate, but negative comments – always harder than positive to recognize – would show some amount of error. Some anecdotal examples (pulled without too much hunting):

• Our room was on the 3rd floor we were offered a room at the back or the front, we chose the front whats the point of being in new york with out the noise. [Casablanca Hotel]
• The rooms are not large but are more than adequate. [Casablanca Hotel]
• You can see the old bank vaults in the basement and the registration area on the first floor is where the tellers used to be. [Drury Plaza Hotel]
• Housekeeping was called and eliminated the two problems. [Camelback Inn]

These false positives are interesting in terms of what they tell us about the algorithms that compute the sentiment. There is some background value associated with certain terms (noise in the first example triggered a negative tag; large in the second and problem in the third). The trouble comes with the valence associated with the terms by the surrounding context.

I don’t feel that these problems are really a weakness in the system. There are always going to be errors. I believe there are two ways to combat this problem in general. The most obvious is figure out a way to introduce another level of summarization in the UX. There is high level summary for each major category. One could try something simple, like a tag cloud, that summarizes within each category in a way that would allow the user to get a finer grained, but still aggregate summary of the content.

Twendz: Twitter and Sentiment

Waggener Edstrom (who do plenty of work for Microsoft) has released a Twitter topic/sentiment tool called Twendz (other coverage here and here). The site shows both a tag cloud of related terms and attempts to surface positive, negative and neutral posts on the topic. Thus far, I’m not too impressed with the sentiment analysis on a per tweet level. Here are some postitive false positives tweets about Jim Cramer:

• The most important video you'll watch this year: Jon Stewart ends Jim Cramer's career Video
• oh wow, John Stewart probably ended Jim Cramer's career last night, it was amazing.
• Please take 21 minutes and 12 seconds to watch this video. http://is.gd/nahG Jim Cramer and John Stewart discuss financial markets.

Here’s some of the negative false positives for Jim Cramer:

• Jim Cramer is not at fault! He'll agree to be blamed, but YOU PEOPLE are the ones investing! Be accountable! (still love J. Stewart, though)

Here’s some negative false positives or watchmen:

• Fair point But that is intensely faithful to the original graphic novel. It's dark, dark socio-political commentary. #watchmen
• back to designer mode and crazy to see Watchmen!
• Saw #Watchmen. Must watch for anyone interested in politics/visual effects. The complex philosophical plot kept me at the edge of my seat.

It is interesting to see this in the context of recent posting about the twittersphere making things easier for sentiment mining. While the system should be doing the right thing with valence shifters (negation, etc.) these examples also demonstrate the problem of topic association. In addition, the examples with Watchmen demonstrate the problems between the nature of the plot and the evaluation of the film. And, of course, like nearly every tool in this space, by providing a direct link to the raw data, the errors get surfaced and the user begins to question the value of the results.

Summary: I’m really excited to see that WE is building tools in this space. However, it looks like they are starting at the shallow end when it comes to sentiment mining.

Is Authority Migrating?

There’s an interesting discussion going on between Brian Solis at TechCrunch and Dorrion Carroll at Technorati. Brian’s thesis is that the number of inlinks to blogs from other blogs is dropping because microblogs and other attention sources are proliferating. Rather than link to something from your blog, you link to it from Twitter, for example.

However, a disruptive trend is already at play. While blogs are increasing in quantity, their authority–as currently measured by Technorati–is collectively losing influence. For instance, just last November, Technorati counted 32,493 links towards gadget blog Engadget’s “authority.” Today, it counts half that amount (16,326). Even TechCrunch’s link authority as measured by Technorati is down by several thousand links, yet its relative position in the overall ranking (No. 3) hasn’t moved.

Dorrion responds by pointing out that it is the quality of Technorati’s analysis that has improved: by removing a lot of spam from their index, their numbers are slimmer and better. Technorati, and now Google, indexed the whole blog page in many cases, not just the post. Consequently, link counting was confused with links intended to be consumed in a different manner (e.g. blogrolling). This too, Dorrion suggests, has changed recently at Technorati.

While Brian may have got it wrong in terms of Technorati’s data, it should be clear that any type of social measurement needs to consider as many sources as possible. In addition, when analytics are being sold, they need to be sold with a deep understanding of the meaning of the different types of data sources and how to interpret them. Is a link from a tweet worth the same as from a blog?

Sentimine, Twittrratr

Briefly - a couple of new to me sentiment related things: Sentimine (for sentiment in general) and Twitrratr for twitter (c.f. flixpulse.com). Not only is the sentiment/opinion mining space starting to fill up, but so is the name space: Sentimine, Sentimetrix, SentimentMetrics, ...

Modeling, Managing and Mining of Evolving Social Networks

The programme for this workshop is now available. Alexey and I have a paper on blog crawling. I won’t be making the trip to Shanghai unfortunately, but if you go please say hi to Alexey!

ScoutLabs offers 30 day free trial

Having completed their beta phase, ScoutLabs has decided to roll in a 30 day free trial period to their offering. ScoutLabs blog posts is here, and CNet covers the story here. On their plans page, ScoutLabs lists the type of data they analyze: Blogs, open social networks, photo and video sharing sites, Twitter. It is interesting to see that they don’t include any message boards – where a lot of the gold is.

A while ago, I predicted that with the acquisition with some of the major players in the space (Cymfony, BuzzMetrics, Umbria), feature innovation would be seen from the less encumbered entrants. This is exactly what I’m seeing with ScoutLabs (see their feature list page).

Social Networks and Web 2.0 Papers at WWW 2009

[updated with Tagommenders paper – thanks Shilad.]

The organizers of the World Wide Web conference recently announced the list of accepted papers for this year’s event. In the Social Networks and Web 2.0 track (chaired by Elisa Bertino and Lada Adamic) the following papers are listed (where the paper is available online, I’ve provided a link to the PDF):

• Sharad Goel, Roby Muhamad and Duncan Watts. Social Search in "Small World" Experiments
• Ulrik Brandes, Patrick Kenis, Juergen Lerner and Denise van Raaij. Network Analysis of Collaboration Structure in Wikipedia
• Yutaka Matsuo and Hikaru Yamamoto. Community Gravity: Measuring Bidirectional Effects by Trust and Rating on Online
• Shilad Sen, Jesse Vig and John Riedl. Tagommenders: Connecting Users to Items through Tags
• Jiang Bian, Yandong Liu, Ding Zhou, Eugene Agichtein and Hongyuan Zha. Learning to Recognize Reliable Users and Content in Social Media with Coupled Mutual Reinforcement
• Jérôme Kunegis, Andreas Lommatzsch and Christian Bauckhage. The Slashdot Zoo: Mining a Social Network with Negative Edges
• Dietwig Lowet and Daniel Goergen. Co-browsing dynamic web pages
• Anon Plangprasopchok and Kristina Lerman. Constructing Folksonomies from Userspecified Relations on Flickr
• Jose San Pedro and Stefan Siersdorfer. Ranking and Classifying Attractiveness of Photos in Folksonomies
• David Crandall, Lars Backstrom, Daniel Huttenlocher and Jon Kleinberg. Mapping the World's Photos
• Munmun De Choudhury, Hari Sundaram, Ajita John and Doree Seligmann. What Makes Conversations Interesting? Themes, Participants and Consequences of Conversations in Online Social Media
• Thomas Karagiannis and Milan Vojnovic. Behavioral Profiles for Advanced Email Features
• Cristian Danescu Niculescu-Mizil, Gueorgi Kossinets, Jon Kleinberg and Lillian Lee. How opinions are received by online communities: A case study on Amazon.com helpfulness votes
• Meeyoung Cha, Alan Mislove and Krishna Gummadi. A Measurement-driven Analysis of Information Propagation in the Flickr Social Network

The Business of Mining the Twitter Stream

While mining Twitter data for business and marketing intelligence (trend/buzz analysis, sentiment/opinion mining, authority/influence analysis) looks like a compelling path to explore for a business model, it is important to consider the proposition from the point of view of the customer. Enterprises have been working with vendors in this space (mining social media content for BI) for well over 5 years and already have expectations regarding the features and quality of reports that these analytics needs to deliver to be useful (actionable).

• Domain coverage: how broad is the topical space available in the solution? Crawling all data sources is the way to win here.
• Demographic coverage: the broader the demographic coverage (and the accuracy with which the demographic features of the content authors can be determined) the better.
• Content Analysis/Text Mining: how well does the solution take all the unstructured content and deliver structured interpretations that can then act as the input for further data mining. This is generally a matter of applied research (taking the current state of the art in text mining and making it work with the greater variety and complexity of social media content).
• Timeliness: how timely is the analysis. This is generally a function of how timely the data is collected. Blog data, for example, can be gathered in a very timely manner thanks to the ping/feed  mechanism. However, the reality of real time mining is that the consumer of the data is the real calibrator - real time may mean 4 hourly, not second by second.

If the business model for Twitter is going to be mining the Twitter stream for BI/MI, then they will be competing with companies that gather very large data sets (weblogs, usenet, message boards, reviews, groups, mailing lists, etc.). Seth Grimes suggested that the short texts of the Twitter stream may make hard problems like sentiment mining simpler as the limited space requires the author to be concise. However, this is a double edges sword as it means that the depth of analysis will be far shallower.

I believe that mining Twitter data will be a very exciting experiment, but I think that if Twitter goes down this path, it will have to either provide analytics over the other data sets, or partner with an existing company (say Visible Technologies). In fact, such a partnership would take the burden of building out an analytics engine away from the small Twitter team allowing them to continue to focus on infrastructure and ensuring the flow of this valuable data stream.

Is Twitter Special?

I'm not sure I get the big distinction that everyone is making between mining twitter specifically rather than social media in general. Ultimately I don't see it trumping blog mining or forum mining in timeliness or quality, though it is a welcome new signal.