Data Mining: Text Mining, Visualization and Social Media

Bing Local Gets Social

There was an announcement yesterday from Bing describing a very broad set of new updateswhich bring social features powered by Facebook to the Bing experience. These features hit both the main web search experience as well as verticals.

In local search, we have put together a feature which - when available- associates either the Facebook page or Twitter identity (or both) with the entity surfaced on the details page. Below, for example, is the description of John Howie in Bellevue.

 
We believe this feature is going to be useful to users who want to connect with the establishments that they are interested in. However, there is something more subtle at work here. By connecting these three different identities, we are providing a mapping across three distinct models of the world. Many exciting opportunities for data quality and user experience are enabled by this association.

A Visual Guide to the International Conference on Weblogs and Social Media (ICWSM)

I’m continually impressed by the breadth of content presented at ICWSM. This conference is truly achieving a unique position for social media and social network researchers: bringing together social science and computer science. I thought it would be interesting to capture the conference visually, so below is a selection of the visual representations of data and results from some of the papers from the conference. Each is linked to the online paper page, so click through to learn more!

 

Ethnicity and Geography of Facebook Users

ePluribus: Ethnicity on Social Networks, by the Facebook data science team includes some interesting estimates of the geographic distributions of Facebook users. My guess is that these are not dissimilar to those found in the census data, confirming the team's suggestion that the ethnic distribution of Facebook users is approaching that of the internet population in general.

 

Social Psychologists in Las Vegas

I've just returned from a brief trip to Las Vegas where I had been invited by Sam Gosling and Kate Niederhoffer to participate in a panel at the annual meeting of the Society for Personality and Social Psychology. I was very happy to participate with Tom Lento from Facebook and Winter Mason from Yahoo Research. The panel topic was Forging connections between Social Media and Social/Personality Psychology and was intended to help bridge the gap between researchers in the social-psych field and those of us in industry working with social and personal data.

Perhaps the most salient point that came out of the discussion was the difference between current social-psych methods and the data scale of the online social world. To help understand this new world, and for social-psych researchers to make the transition, an ability to code and to deal with large amounts of data is required. There was genuine interest in this (provoking questions about the specifics we had in mind, such as working with scripting languages) and I believe a real interest in the opportunity that lies in this data rich online world.

In terms of facilitating interactions, Winter probed the audience regarding knowledge of key conferences for web data analytics, etc. such as WWW and ICWSM. I consider the later to be a perfect forum for this discussion to continue, and with Sam's help I think this can really happen.

The conference itself is run in a manner quite different to that which computer scientists are used to. No papers are submitted and acceptance (which runs high) is based on short abstracts (generally paragraphs). I managed to see one of the poster sessions and was fascinated by the topics (extremely focused accounts of certain behaviours and traits) and the methodologies.

I feel that there is a lot to learn and the the ultimate winners in the social space will be those companies farsighted enough to really embrace this field and who figure out how to integrate such insights about users into their products.

The Mathematics of Modern War

When I started watching the TED talk below, I felt that it was wholey unoriginal (we are just getting through the period of network theory where every presentation presents a power law). However, when I got to the end I felt that there was a real insight, that the narrative was a great example of the value of data visualization and mining and that the speaker - Sean Gourley - had structured the talk (7 mins) perfectly. (via Tim)

If I Read This Book, Will It Make You Happy

Picking up Connected: The Surprising Power of Our Social Networks and How They Shape Our Lives by Nicholas Christakis and James Fowler, you might be forgiven for thinking ‘another book about goldfish jumping from one bowl to another?’ I’m currently through the first chapter, and while the authors don’t go far enough in terms of setting the scene for their work in terms of basic social network theory and where it comes from (Barabasi is not mentioned until page 265), they do offer some interesting anecdotes about the type of information that passes between the socially connected.

With a clear focus on avoiding the obvious and well trodden paths of those that have gone before them, the authors pull their stories from their own work and other accounts of things like cascades of emotion (someone 3 degrees away from you can cause you to be happier or sadder) and epidemics of laughter (mass psychogenic illness).

I’m enjoying this book (provided by the publisher) more than the previous book that I attempted to review. This may be entirely due to my frame of mind, or being a better fit for the audience. Or – perhaps someone three degrees away from me read it and liked it.

I’ll write more when I’ve finished.

Zooming in to Twitter

David Lojudice Sobrinho, who blogs here and who pointed me to another magazine cover (on Veja in Brazil), directed me to @mlemos’ Twitter visualizations. I took the liberty of uploading one of them to the new SeaDragon site which hosts (large) images and provides a zoomable image experience.

Observant readers will notice the similarity between my Twitter image and @mlemos’!

The image is published under the creative commons share alike license.

BusinessWeek on The Social Graph

Take a look at Stephen Baker’s BW article ‘Learning, and Profiting, from Online Friendships’. Independent of the actual content, I really like the way that BW integrates a video of highlights from the author and danah, SN illustrations and a weird photo of Cameron.

Note the last line in the video ‘…if our friends like [the ads], we will too.’ and factor in Watt’s work on Friend Sense which suggests that we don’t really know our friends as well as we think.

Influence – not as simple as Gladwell would have you believe!

The invited talks at ICWSM were especially good this year. I want to highlight a few points from Duncan Watt’s talk and Jon Kleinberg’s talks.

1. Social influence makes the selection for success less predictable. In other words, judged against independent measures of quality, if an audience is influenced by knowledge of community behaviour, it will select or promote with less correlation to quality than you would think. You may think ‘so much for the wisdom of the crowds’ but, of course, WOC is all about aggregating over independent judgments, not socially influenced ones – see Experimental Studies of Inequality and Unpredictability in an Artificial Cultural Market.
2. We know less about our friends than we think we do. In the Friend Sense experiment, it was demonstrated that we project our opinions onto friends about whom we make assumptions regarding political beliefs. Watt’s concerns about the misrepresentation of polarization might be contrasted with the experiments reported in Nick Carr’s book The Big Switch in which a) small preferences lead to deep segregation and b) homophilly leads to extremism.
3. Diffusion of information may ‘long circuit’ the small worlds of social networks. In Kleinberg’s presentation regarding the study of the largest internet chain mail (a petition) he described the role of the threshold model of diffusion in which we require multiple receipts of a stimulus (e.g. a chain mail letter) to pass it on, we are more sensitive to our immediate community – our strong links – than to small-world building weak links. This seems to have some relationship with Watt’s work on Challenging the Influentials Hypothesis and both his criticism of the disease analogy and his focus on the importance of the network structure, not some magical power of the ‘influential’.

Modeling, Managing and Mining of Evolving Social Networks

The programme for this workshop is now available. Alexey and I have a paper on blog crawling. I won’t be making the trip to Shanghai unfortunately, but if you go please say hi to Alexey!