Data Mining: Text Mining, Visualization and Social Media

The Long Tail Arrives

My copy of Chris Anderson's The Long Tail arrived today - stay posted for a review.

The Effect of Word of Mouth on Sales: Online Book Reviews

Today, NPR featured a brief article about a paper written by Judith Chevalier and Dina Mayzlin of the Yale School of Management entitled: The Effect of Word of Mouth on Sales: Online Book Reviews. The abstract reads:

The creation of online consumer communities to provide product reviews and advice has been touted as an important, albeit somewhat expensive component of Internet retail strategies. In this paper, we characterize reviewer behavior at two popular Internet sites and examine the effect of consumer reviews on firms' sales. We use publicly available data from the two leading online booksellers, Amazon.com and BarnesandNoble.com, to construct measures of each firm's sales of individual books. We also gather extensive consumer review data at the two sites. First, we characterize the reviewer behavior on the two sites such as the distribution of the number of ratings and the valence and length of ratings, as well as ratings across different subject categories. Second, we measure the effect of individual reviews on the relative shares of books across the two sites. We argue that our methodology of comparing the sales and reviews of a given book across Internet retailers allows us to improve on the existing literature by better capturing a causal relationship between word of mouth (reviews) and sales since we are able to difference out factors that affect the sales and word of mouth of both retailers, such as the book's quality. We examine the incremental sales effects of having reviews for a particular book versus not having reviews and also the differential sales effects of positive and negative reviews. Our large database of books also allows us to control for other important confounding factors such as differences across the sites in prices and shipping times.
 

The radio article summarized the findings, saying that negative reviews do impact sales of books. The study uses two online stores (Amazon and Barnes and Noble) to help add control to the experiment. Chevalier has also co-authored another paper using similar data, with Austan Goolsbee entitled: Measuring Prices and Price Competition Online: Amazon and Barnes and Noble.

The Word of Mouth paper is new to me, but it was actually published in 2003.

It would be interesting to compliment a study like this with an anlaysis of opinion regarding books expressed in blogs. That is to say, quasi-reviews published on sites not associated with the books. How different would this data be? What effect would negative or positive opinion have on sales. The NPR site notes something they call the NPR effect (styled after the slashdot effect) where they monitor the increase in sales rank of a book after it has been featured on NPR.

New Data Visualization Book

Scientific Visualization: The Visual Extraction of Knowledge from Data appeared in a Bloglines search feed today. It looks very interesting, but I'm blogging about it as the page it comes from is far from a blog page. Bloglines (which, I think, calls anything with an RSS feed a blog) refers me back to this feed: http://www.springer.com/sgw/cda/rss/0,,1-10042-0-0-0,00.xml. The whole problem of typing RSS feeds (i.e. indicating if the feed is a blog, a news feed, a book release feed, etc.) is nothing new, but I'm wondering why no-one has jumped on this. Clearly, any explicit tagging approach is open to exploitation, but what would the exploitation be in this case? Spammers would, whatever tag they chose, be lying anyway (assuming that they don't label themselves as spam!)

Anyway, here's the abstract:

One of the greatest scientific challenges of the 21^st century is how to master, organize and extract useful knowledge from the overwhelming flow of information made available by today’s data acquisition systems and computing resources. Visualization is the premium means of taking up this challenge. This book is based on selected lectures given by leading experts in scientific visualization during a workshop held at Schloss Dagstuhl, Germany. Topics include user issues in visualization, large data visualization, unstructured mesh processing for visualization, volumetric visualization, flow visualization, medical visualization and visualization systems. The book contains more than 350 color illustrations.

I'm really excited by the relationship between data mining - in which the computer makes inferences and selects which results to report - and data visualization - in which the designer determins a visualization of the data that best describes some set of relationships between data elements.