Data Mining: Text Mining, Visualization and Social Media

Visualizing Wikipedia

A Beautiful WWW has posted an updated version of their viz of Wikipedia. This version specializes on science and technology articles. The map is generated by first laying out the graph of Wikipedia (nodes are articles, edges are links between articles) and then decorating that substrate with colours (for topics: maths, science and technology) and images. Given the broad swath of science (green) articles, a question that springs to mind is: what is divided by this class - what topics are to the left and right? Could it be that science articles, due to some quality of their authors or topic are more interlinked and consequently take control of the layout algorithm?

Wikipedia Search Results - Reality Check

Steve Rubel recently posted on (what he considers to be) the influence of Wikipedia wrt brands on the web. The basic point he wants to make is that search results for a certain set of brands often include Wikipedia results and so Wikipedia has a lot of influence. He further notes some entries which contain potentially damaging content about brands.

However, Graywolf followed this up with a look at the click through rates for searches on these brands in the AOL search data and found that people don't generally click through to those results. It isn't too much of a stretch to then claim that Wikipedia actually doesn't have any influence - perhaps people see the result and decide not to go there due to some preconceptions regarding the content found on Wikipedia.

I suspect that part of the problem here is a lack of understanding of intent: search engines, due to their narrow interfaces, have no ability to disambiguate intention. If searches for a brand were intended to find items for sale, for example, Wikipedia would not be a good result. If they were intended to find blog posts or reviews, again, Wikipedia would not be a good result. If, on the other hand, they were looking for historical or other factual information - no problem.

[Via Emergence Media.]

Wikipedia Visualization

I haven't yet fully grokked this visualization, but it certainly is going to be worth it:

 

 

This is from the paper 'Analyzing and Visualizing the Semantic Coverage of Wikipedia and Its Authors' by Todd Holloway, Miran Bozicevic, Katy Börner:

This paper presents a novel analysis and visualization of English Wikipedia data. Our specific interest is the analysis of basic statistics, the identification of the semantic structure and age of the categories in this free online encyclopedia, and the content coverage of its highly productive authors. The paper starts with an introduction of Wikipedia and a review of related work. We then introduce a suite of measures and approaches to analyze and map the semantic structure of Wikipedia. The results show that co-occurrences of categories within individual articles have a power-law distribution, and when mapped reveal the nicely clustered semantic structure of Wikipedia. The results also reveal the content coverage of the article's authors, although the roles these authors play are as varied as the authors themselves. We conclude with a discussion of major results and planned future work.