Data Mining: Text Mining, Visualization and Social Media

Peeking into the Future by Text Mining: Zapaday, Recorded Future

Briefly, Zapaday is a new (in alpha) online tool which mines the web (with a bit of editorial thrown in) for future events to create a global public calendar. This puts Zapaday in some overlap with Recorded Future, but with a clearer message and more narrow scope.

 

Here's the Recorded Future video for convenience:

 

How To Write an Academic Paper in Text Mining

I'm completing a set of reviews for a reasonably high quality conference that touches on data mining and text mining problems. Perhaps the industrial setting has jaded me with respect to academic papers, but there seems to be some key points that - for me - really matter in the writing of a good paper (and implicitly in the selection of interesting areas of research).

1. Be clear on the context. Many papers I see start of with some general, weak and unsupported statements about some big trend in the data domain (e.g. the huge growth of social networks) and the imperitive for analysing that data. These statements, however, are rarely supported with actual statistics. Nor are they accompanied by comparison to the last big imperative. For example, it seems we must analyze twitter for getting insights into what people are thinking - but what does this data give you above and beyond other corpora like blogs?
2. Be clear on the contribution. Given the current state of the art in the particular field and a formulation of the problem (which has to cover the linguistic qualities of the corpora). what specific areas require progress (or have been currently untouched) and how has an analysis of that specific problem space motivated the proposed solution?
3. Be clear on the utility. So what - perhaps you have applied your favourite technique to a problem and moved the needle a little. What do we now know that we didn't before? Do we know something about the problem itself (20% of the corpus had this particular construct, my solution solves for that construct), do we know something about the technique (previous applications of this technique suffered from this fundamental conceptual problem, I fixed that by introducing this significant evolution).
4. Be clear! When you've figured out the above, think hard about the most efficient and impactful way of communicating the research. Examples, using real text, are very useful for getting straight to the heart of the matter (and be careful to communicate how representative these examples are). Don't throw in assumptions and leave the reader standing (every vague assumption will cause concern for down stream results). Make sure the evaluation is transparent (you remembered to include an evaluation, right?)

Pragmatic Philosophical Technology for Text Mining: Categories

Modeling the world is a key part of text mining. Modeling is an activity that comes after someone makes a statement like this :  'there are restaurants in the world, let's mine some web data and find out where they all are!" This is because someone else will ask "what is a restaurant?"

If we step back a little, we might think about the value of generalization in language, which is clearly linked to what we often call 'meaning'. When I say "there is a restaurant on 3rd at 5th, let's meet there" I am assuming that the things in the world that you use the word "restaurant" to describe are reasonably similar to the set that I would denote with the same term. This remarkable assumption - that we will behave in similar ways in the future in a manner determined by an act of language - is pretty much what holds the whole idea of language together.

The idea of assigning categories to things (or things to categories) can be analyzed at a number of levels.

Firstly, we might think of some natural, or objective quality of the objects in question and determine that if something has this quality, then it is in the category. This is what biologists have been doing for quite some time.

Secondly, especially in the case of various ownable things, we might consider the manner in which the owner describes it (this is a sushi bar, damn it!).

Thirdly, we might consider how people refer to the object (Japanese fast food!).

Finally, we might consider how people (or other agents) interact with the thing in a non-linguistic manner.

All of these come with various caveats. That being said, the difference between the first and all the others is the difference between objectivity and subjectivity. Someone int he UK might refer to an establishment selling a limited range of fried food for immediate consumption (but not on the premises) as a "chippie" whereas the same place in the US might be called a "fish and chip shop". These names don't necessarily change the nature of the thing in question. With the other options, we have to consider the subjective nature of the audience. This leads to a set of categories which differentiate both the space of things and the space of speakers (and even the space of contexts in which people are communicating).

The next challenge when trying to put things in buckets is specialization. We might feel that concepts like restaurant and plumber are pretty straightforward (we know what to expect when we engage with such businesses). In addition, we might have some idea of what a loung is. But what is a "Thai Restaurant and Lounge"? Is this a restaurant that is also a lounge? Or is this a restaurant that has a relationaship with another entity which is called a lounge? Is a "bar & grill" the same as a "grill & bar"? Is it a bar with a grill or is it an atomic concept in which the nature of the bar experience is altered by the intimate relationship with the grill?

The value of categories is that they help us project assumptions onto concepts about which we know almost nothing. If you tell me that your mum runs a business that is fine, but if you tell me she runs a restaurant, I will be around for some discounted food.

Ultimately, the things we can do with entities are defined by a matrix of properties. Categories are really a projection from this matrix. This is how I know how to interact with a taco truck, a kaiten sushi bar and a french restaurant the same for some things (paying for goods) and differently for others (sitting down at a table, at a bar or not at all). This matrix of fine grained facets can be used to project onto a model of convenient concepts (e.g. concept_134 for which I use the word 'restaurant' to denote, among others). There are many different terms we may use to indicate these concepts and we may do so imprecisely (it is more efficient to use a category we are all familiar even for an object that has a 2% variance in the details of some of the common facets).

Pragmatic Philosophical Technology for Text Mining

In text mining applications, we often work with some form of raw input (web pages, web sites, emails, etc.) and attempt to organize it in terms of the concepts that are mentioned or introduced in the documents.

This process of interpretation can take the form of 'normalization' or 'canonicalization' (in which many expressions are associated with a singular expression as an exemplar of an set). This happens, for examples, when we map 'Barack Obama', 'President Obama', etc. to a unique string 'President Barack Obama'. This is convenient when we want to retrieve all documents about the president.

In this process, we are associating elements within the same language (language in the sense of sets of symbols and the rules that govern their legal generation).

Another approach is to map (or associate) the terms in the original document with some structured record. For example, we might interpret the phrase 'Starbucks' as relating to a record of key value pairs {name=starbucks, address=123 main street, ...}. In this case, the structure of the record has a semantics (or model) other than that of the original document. In other words, we are mapping from one language to another.

Of course, what we want to do is denote the thing in the real world. It is, however, impossible to represent this as all we can do is shuffle bits around inside the computer. We can't attach a label to the real world and somehow transcend the reality/representation barrier. However, we can start to look at the modeling process with some pragmatics.

The model of the 'semantic' or 'knowledge' language governs a space of statements that can be made. These statements may or may not be true when compared with the real world, but they should at least aspire to a system of behaviours that does mirror the real world. For example, if two entities have the same address, then they should be located in the same space. If two entities have the same phone number then one should connect with the same agent when dialing that number. A set of entities describing a restaurant can include a statement about an entity described as a chef just as a restaurant 'has' a chef. A set of statements describing a clinic should not have any associated statements about chefs just as clinics don't have chefs.

It is convenient to associate some proxy identifier with sets of statements associated with real world entities. We can then use that identifier to 'mean' the real world entity. When we do this we have to be careful of two things. Firstly, the statements may change to better reflect the 'truth'. Thus the identifier stays the same, but the phone number statement changes. Secondly, we mustn't confuse this proxy behaviour with the real world entity. For example, I may have two proxy identifiers for sets of statements I later realise are statements about the same real world entity. Thus I will have to then make some sort of statement about the relationship between the identifiers (whatever id1 indicates, id2 indicates).

I tend to think of the world generatively. The world generates documents that somehow reflect imperfectly its nature. Text miners are in the business of synthesizing this secondary representation.

Tell me something I don’t know!

There were a couple of papers that I really liked at www. One was Mapping the worlds photos by David Crandall et al, another was SOFIE: a self organizing framework for information extraction by Fabian Suchanek et al. Both of these papers (while operating in different domains) used features from orthogonal spaces (images, tags and geo coding in the first case and entity, relations, text patterns and logic in the second) to automatically mine new facts from a data set.

However, after the initial impression of how great these things are (and they are great) one realises that the facts that they have surfaced are already known. In the first case, which excels at discovering landmarks and images of landmarks, this is a well known knowledge set – landmarks, by there definition, are discovered facts. In the second case, the fact used in the example (which we shouldn’t, of course, judge the entire system by) was also well known.

What one would want to see in these papers and certainly in their presentations, is the long tail of facts. The head of the knowledge-sphere is well known by definition, rather than discovering it, we should assume it. The long tail will have weaker signals and it is there that we really care about the power of these systems.

Other papers in this and related areas from the conference include:

• Exploiting web search to generate synonyms from entities by Surajit Chaudhuri et al
• Measuring the similarity between explicit semantic relations on the web by Danushka Bollegala et al

DoodleBuzz

DoodleBuzz is worth taking a look at. A very different way of interacting with textual data (annotated with concepts).

[thanks to IanG for the tip.]

Text Analytics Summit 2008

Here are my slides from me presentation today at the Text Analytics Summit 2008. I have to say, I was impressed with the event. I had initially thought that it was going to be less technical and more of a sales/networking event. However, it turned out to be very engaging, with a great mix of practitioners from all over the map. Of particular note was the clear trend in mining social media, and the importance of both sentiment analysis and influence/authority analysis.

| View | Upload your own

Text Analytics Summit

I'll be speaking at the Text Analytics Summit (June 16th, 17th in Boston). The idea for the talk is as follows:

• Examine what I consider to be the two pillars of social media analysis: subjectivity (content) and authority/influence (structure). I have a story that pulls these two together.
• Characterize the current model for most social media analysis products (the consumer versus the consumed) and describe what I think will (or ought!) to be the next model for analysis in this space. The basic idea is that a) the producers of stuff shouldn't just be interested in the point of consumption, but in the entire ecology surrounding their product space, b) if you look hard enough you can aggregate the voices of many participants in production of goods and services.

The Surface/Symbol Divide

I noted William's post on a new paper (to be published at ICDM) which his student, Richard Wang has written. The paper describes a system which uses a smart combination of wrapper induction, set (list) discovery and graph based ranking to materialize, as if by magic, expanded sets of terms. For example, enter Ursa Major, Ursa Minor, Orion and you will get back a nice list of related terms (including Taurus, Gemini, etc.).

I love this type of thing. It is close to some of the things I worked on during my thesis (mining information from tables in text) and is an example of a very active research area (which includes quite a bit of work on mining Wikipedia for entities and relationships).

However, there is no free lunch. This approach to knowledge discovery is fixed at the surface level of text (and the surface level of the representation language of documents, to be complete). Consequently, the performance of the system highlights both what is good about statistical surface techniques (little training required - which is often the case for systems that work with both document structure, textual data and high precision seed input; works in (m)any language(s); fast) and what is bad (has no real knowledge of language).

An example of this problem can be seen when we give the seeds {obama, clinton} to the system. The following results appear:

#	Entity	Weight
1	obama	1.00000
2	clinton	1.00000
3	edwards	0.13000
4	romney	0.11125
5	mccain	0.10493
6	he	0.08484
7	giuliani	0.07974
8	the	0.06658
9	bush	0.06585
10	hillary	0.06373

While many of these results are fine, there are also errors which illustrate the separation of surface and symbolic processing: he, the.

Automated versus Human Judgments

A couple of posts provoke an interesting discussion: William Cohen points to the issue of the popularity contest approach to ranking which may have undesirable consequences; The Measurement Standard provides an interesting angle on the fallibility of human judgments. In the area of sentiment analysis, one often hears skepticism around the quality of results (or even the possibility of automating this task). It is always informative to see how well humans do at these tasks (most reports in the literature of inter-labeler agreement for sentiment are pretty poor). My feeling is that while an automated approach can never be error free, the systematic nature of the errors lead to a more manageable result than the randomness of human error generated by poor methodology.

As for the issue of automated ranking of web pages. The problem cited above exposes the frailty of addressing a content problem (finding a document whose text is appropriate) via an orthogonal structural solution. The structural solution (counting links and propagating results) may do well in some domains where it is regarded as a proxy for measurements of 'authority', however, the ambiguity in the structure cannot be determined, leading to the type of problem William cites. This is where solutions like Powerset come in.

Note: I'm really happy to see that William is writing - grab his feed!