One problem with this scenario is that stable environments can turn turbulent quickly and without warning. Think about the housing markets in the US and the UK in 2008 (and all the markets related to those two). This is the basis of Nassim Nicholas Taleb’s black swan theory.

Lee Smolin makes some important points about coping with these types of shifts. The main thrust of the article is that scientific research may be too conservative these days as researchers focus more on taking a safe approach to facilitate publishing, while not spending enough time on the riskier big questions. While black swan events (things with a low probability of occurring, but which have a large impact if they do actually arise) are often negative in financial markets, they are sought after in science and innnovation.

I found the article through the a good post by Roland Harwood. The angle that he thought was most interesting was that the article proposes the formation of markets for research. That is an interesting angle, but not the one that grabbed me. I was struck by this section:

Some scientists, he suggests, are what we might call “hill climbers”. They tend to be highly skilled in technical terms and their work mostly takes established lines of insight that pushes them further; they climb upward into the hills in some abstract space of scientific fitness, always taking small steps to improve the agreement of theory and observation. These scientists do “normal” science. In contrast, other scientists are more radical and adventurous in spirit, and they can be seen as “valley crossers”. They may be less skilled technically, but they tend to have strong scientific intuition — the ability to spot hidden assumptions and to look at familiar topics in totally new ways.

To be most effective, Smolin argues, science needs a mix of hill climbers and valley crossers. Too many hill climbers doing normal science, and you end up sooner or later with lots of them stuck on the tops of local hills, each defending their own territory. Science then suffers from a lack of enough valley crossers able to strike out from those intellectually tidy positions to explore further away and find higher peaks.

This is a nice description of innovating on fitness landscapes, an idea that John and I have been talking about in our classes. And it is particularly important now. There are numerous reports coming out showing that companies have become increasingly conservative in their innovation efforts as a response to the global financial crisis. In Smolin’s analogy, companies are reverting to only undertaking hill climbing. Many have never thought much about valley crossing in the first place, while many others are pulling resources out of more speculative ideas.

This is as wrong for business as it is for science. Innovation is central to survival, and successful innovation requires both hill climbing and valley crossing. Now more than ever it is critical for firms to figure out a way to continue to invest in areas that have breakthrough potential. Newspapers and record companies have collectively been very effective at getting to the top of their hills. And these industries are in big trouble now because they have completely missed on out the discovery of new, better hills. It’s a challenging process to manage in good times, and probably even harder now. But it’s still essential.

One way to cope with this problem is to maintain an innovation portfolio. The idea with this is that we always need to investigate ideas that both help us climb hills and cross valleys. A large percentage of our efforts do need to go into hill climbing. This leads to incremental innovations that keep us competitive.

However, it is essential to always be working on a few ideas that will lead to valley crossing. This can prepare us for times when our environment becomes turbulent unexpectedly. This turbulence can come from an unexpected financial shock, a natural disaster, or the introduction of a disruptive innovation.

In all of these cases, maintaining an innovation portfolio will make you better equipped to deal with this change.

On the innovation journey, we need to be climbing hills, but we also have to cross a valley every now then as well.