Innovation Ecology

What is innovation? I’ve been chewing on that question for a while, and a recent post from Hal has finally nudged me to write something. Hal’s post, in summary, discusses the nature of ownership in progress and breakthroughs (that was my idea!). The reason that I care about innovation is that I really care about environments that are intentionally set up to deliver innovation. While we might often hear ‘I work in an innovation centre’, or something similar, does that really have any meaning?

At best, innovation is something that we (that is to say, users) recognize, and I think this is key to understanding the nature of innovation processes in the internet space. Innovation is not just the idea (ideas are free!), nor is the implementation (often the hardest part) sufficient – connection with the user and the recognition of the value by the user is key. Thus, I think the key elements to an innovation centre are (at least):

1. smart people: to generate ideas
2. engineering excellence: to implement the ideas
3. connection with users: the people that recognize ultimately determine the innovation

The last part is the hardest as it requires some sort of faith. The reason being that one has to find the right users, a tasks which requires an interesting mixture of skills, perseverance, tenacity and luck.

One can possibly generalize the three components above. So in the case of Hal’s discussion, the issue of how freely one publishes ones research results is part of the mechanism for connecting with users. You may have solved some important problem, but if no-one knows, then there is no impact. Of course, the reason this is more complex than Hal’s post describes is the economic system. Breakthroughs in Google or Microsoft are not socialized in the same way their academic cousins.

Looking back at the environments I’ve worked in, I’ve witnessed situations where we had plenty of 1 (lots of smart people) but the company failed, and situations where the users have been breaking down the doors while we’ve been scrambling to match the smarts with the solution that would satisfy them. In addition, there are plenty of cases where 1 and 2 are clearly not a problem, but where the current behaviour of the users (driven in part by the existing solution paradigms) presents a barrier to connection.

The 3 part innovation system works at many levels. For example, the academic innovation system feeds not only academia and the public good of science and knowledge in general, but also industrial systems. Industrial R&D labs themselves interface often with academia, but also produce internal innovations to product group customers and so on.

Comments

[Eyal Sela]

It is important to remember that, while no one denies the importance of innovation today, it is not always the needed solution. People want the *right* solution to their problem, not the innovative one. It is true that many times the right solution is also innovative, but innovation per se is not a prerequisite.

[Eyal Sela]

It is important to remember that, while no one denies the importance of innovation today, it is not always the needed solution. People want the *right* solution for their problem, not the innovative one. It is true that many times the right solution is also innovative, but innovation per se is not a prerequisite.

[Chris Brew]

I think there's 1a) Generating ideas that could be worth pursuing; then 1b) Deciding which ideas to pursue.
This second bit involves guessing how 2) engineering and 3) connecting to others might turn out. Many smart people don't like guessing, so good ideas can die at this stage. This is as much about attitude and confidence as smarts. An environment that finds ways of supporting the transition from 1b to 2 and 3 will succeed.

One way to build confidence is to make things easy. If you have good, scalable, comprehensible frameworks for doing stages 2 and 3, you make it easier for potential innovators to be confident in their judgment when at stage 1b. Hence the value of ACL memes like "turn everything into supervised machine learning" and more industry-oriented ideas like MapReduce, LINQ, database-backed web frameworks. This is more 2 than 3.

Maybe the same thing, but more 3 than 2, are the shared tasks like TREC, BioNLP and SemEval, because they come with the assurance that some community cares and there will be an audience for sensible work. The ability of Yahoo, Google and LiveSearch to deploy things fast provide a similar assurance that an idea can be placed in front of relevant eyeballs.

Finally, a sensible attitude to trying things is crucial. Success must be encouraged, but not everything pans out. Toxic environments, where every idea is required to succeed, produce a culture in which only liars and fantasists can "succeed". There's an article in the New York Times this morning about the Iraq reconstruction effort, and the lies and fantasies which were occasioned by the horrid realization that no-one at any level of the organization was willing to accept the realities of the increasingly ugly situation. Better not to replicate that.

[Joshua R. Simmons]

I fully believe that the prevailing prerequisite of innovation is a diverse disciplinary background! The moving of data out of silos and on to the 'web', trimming the incredibly vertical hierarchies in companies and facilitating cross departmental collaboration, etc. These are just a couple manifestations of the same phenomena.

You did capture many other pivotal variables in the equation, however!

[Dmitriy]

Joshua, you win the buzzword bingo, hands down. Here's some recommended reading for you:

http://www.mtholyoke.edu/acad/intrel/orwell46.htm