black swan innovation


After John & I yelled at him for an hour last friday, our PhD student Sam MacAulay was still gracious enough to continue talking with me. In the course of discussion he asked a good question – what actual impact complexity science has had on management studies? We talked about it for a while, and concluded that complexity has primarily been used as a metaphor. However, I did point out as an example that Didier Sornette has done superb work in modeling the stock market as a non-linear complex system. Elizabeth Garnsey has also done some really nice research in this area (see for example the interview that is linked to on her webpage, or the excellent book she co-edited with James McGlade Complexity and Co-Evolution). But in the main, I had to agree with Sam that we probably haven’t made as much out of complexity theory as we could.

Nevertheless, Sornette’s book Why Stock Markets Crash is one of the best pieces of practical complex systems thinking that I’ve run across. So I was very pleased to see his work discussed in this article from The article also talks about the work of Nassim Nicholas Taleb and Lee Smolin, two other people I’ve got a lot of time for. The main thrust of the article is that scientific research may be too conservative these days as researchers focus more on taking a safe approach to facilitate publishing, while not spending enough time on the riskier big questions. While black swan events (things with a low probability of occuring, but which have a large impact if they do actually arise) are often negative in financial markets, they are sought after in science and innnovation.

I found the article through the excellent Nesta Connect blog. The angle that they thought was most interesting was that the article proposes the formation of markets for research. That is an interesting angle, but not the one that grabbed me. I was struck by this section:

Some scientists, he suggests, are what we might call “hill climbers”. They tend to be highly skilled in technical terms and their work mostly takes established lines of insight that pushes them further; they climb upward into the hills in some abstract space of scientific fitness, always taking small steps to improve the agreement of theory and observation. These scientists do “normal” science. In contrast, other scientists are more radical and adventurous in spirit, and they can be seen as “valley crossers”. They may be less skilled technically, but they tend to have strong scientific intuition — the ability to spot hidden assumptions and to look at familiar topics in totally new ways.

To be most effective, Smolin argues, science needs a mix of hill climbers and valley crossers. Too many hill climbers doing normal science, and you end up sooner or later with lots of them stuck on the tops of local hills, each defending their own territory. Science then suffers from a lack of enough valley crossers able to strike out from those intellectually tidy positions to explore further away and find higher peaks.

This is a nice description of innovating on fitness landscapes, an idea that John and I have been talking about in our classes. And it is particularly important now. There are numerous reports coming out showing that companies are becoming increasingly conservative in their innovation efforts as a reponse to current economic conditions. In Smolin’s analogy, companies are reverting to only undertaking hill climbing. Many have never thought much about valley crossing in the first place, while many others are pulling resources out of more speculative ideas.

This is as wrong for business as it is for science. Innovation is central to survival, and successful innovation requires both hill climbing and valley crossing. Now more than ever it is critical for firms to figure out a way to continue to invest in areas that have breakthrough potential. Newspapers and record companies have collectively been very effective at getting to the top of their hills. And these industries are in big trouble now because they have completely missed on out the discovery of new, better hills. It’s a challenging process to manage in good times, and probably even harder now. But it’s still essential.

Note: The photo at the start is from the very useful resource, the Australian Bird Image Database.

Student and teacher of innovation - University of Queensland Business School - links to academic papers, twitter, and so on can be found here.

Please note: I reserve the right to delete comments that are offensive or off-topic.

10 thoughts on “black swan innovation

  1. I wish I’d heard of that book sooner. I’m giving a speech tomorrow on ‘Economic Crises and their Root Causes.’ Would’ve come in handy :)

    Thanks for the links!

  2. And I am still gracious enough to comment on your blog 😛

    Just a quick comment as I am in the middle of work (don’t tell my supervisors). I wonder about your comment that,

    “While black swan events (things with a low probability of occuring, but which have a large impact if they do actually arise) are often negative in financial markets”

    Why would negative black swan events be necessarily more likely than positive ones in financial marks?

  3. Hi Amber – It’s a really good book. Fairly technical, but I still think it’s one of the best finance books I’ve read in a long time.
    Sam – there probably are positive finance black swans, but they take the form of bubbles, and so they’re expressed over much longer time periods than crashes. When looking at the biggest single-day stock market movements over the past 50 years, something like 70% of them are crashes, and all of the positive moves are bounces that follow a crash…

  4. interesting…I wonder if a bias towards downside risk aversion plays a role in this…

    Thanks Tim!

  5. Taleb is really good. Fooled By Randomness is a bit better than The Black Swan, but they’re both worth reading. I hope your studies are going well Amber!

  6. Hey Tim,

    Glad the post on our blog provoked something. I agree that the hill climbers/valley crossers bit is highly evocative and relevant. Not nearly enough valley crossers for my tastes.


  7. Thanks for reading the post Roland! I get something out of nearly every post on your blog – I just wish you guys wrote more. -Tim

Comments are closed.