Where Do Bad Ideas Come From?

There’s been a lot of buzz about Steven Johnson’s book Where Good Ideas Come From. An article in Foreign Policy by Stephen Walt addresses the opposite question: Where Do Bad Ideas Come From?

He is talking about bad ideas in foreign policy, such as the domino effect, which have been used to justify policy but which have failed to be supported by actual facts or events.

In part, Walt thinks that bad ideas result from an inability to learn from error:

All countries have obvious incentives to learn from past mistakes, but those that have successfully risen to the status of great powers may be less inclined to adapt quickly in the future. When it comes to learning the right lessons, paradoxically, nothing fails like prior success.

This wouldn’t seem to make sense. After all, strong and wealthy states can afford to devote a lot of resources to analyzing important foreign-policy problems. But then again, when states are really powerful, the negative consequences of foolish behavior rarely prove fatal. Just as America’s “Big Three” automakers were so large and dominant they could resist reform and innovation despite ample signs that foreign competition was rapidly overtaking them, strong and wealthy states can keep misguided policies in place and still manage to limp along for many years.

If you think about this, it sounds a whole lot like the Innovator’s Dilemma, doesn’t it?

The basic premise is that success reduces the incentive to innovate, which results in the propagation of bad ideas.

Bill Easterly wrote a very interesting post today called How Ignorance Dooms Autocracy, which explains in part how this happens. He includes this table:

Tier Type of knowledge Recommended actions System Compatible with autocracy?
(1) Certainty (known knowns) Just do it Administration Yes
(2) Probability (known unknowns) Hypothesis testing Academic freedom Temporarily Yes, eventually No
(3) Ignorance (unknown unknowns) Decentralized feedback and accountability Individual liberty No

Here is how Easterly describes the three tiers:

Autocrats defend themselves by claiming they live in a word of certainty, where they can solve problems with known solutions (Tier 1 in the above table) through sheer administrative effort.

If the world is really more in Tier 2, where academic freedom is necessary to test and reject hypotheses, then autocrats sometimes try to carve out the space for it, while restricting other kinds of freedom. This can sometimes succeed for a while, but a House Divided against itself cannot stand forever — it will eventually revert to no freedoms or all freedoms.

Much of the development problem is really in Tier 3, where you don’t even know the probabilities of solutions to problems working. Then you need entrepreneurs for business, inventers for technology, and political reformers for institutions, all using a trial and error method where they are accountable to positive and negative feedback. In other words, you need unhindered democracy and markets to support continuing innovation for development to keep proceeding to the highest levels.

Like the powerful countries discussed by Walt and the Autocrats discussed by Easterly, successful firms act like they exist in Tier 1, with no uncertainty at all. When you are in Tier 1, everything is simply an engineering problem, and you don’t have to worry about innovation at all.

The problem is that except for a handful of extremely stable markets, most firms actually operate in Tier 3, with plenty of genuine uncertainty. And Easterly’s prescription for economic development holds for businesses as well – when you face uncertainty the best thing to do is to experiment.

In another excellent post from earlier today, Dave Gray explains one way to address this issue. He starts by discussing data from John Hagel and John Seely Brown that shows that the average lifespan of firms in the S&P 500 has dropped from 75 years in 1937 to 15 years more recently. In other words, the corporate world is getting less stable – they are facing more uncertainty. Gray’s recommendation is to combat this by developing a Connected Company – one that functions more like an ecosystem than a machine.

A connected company is a lot like one built on Jon Husband’s idea of wirearchy, and Steve Denning’s concept of dynamic linking. Here is Gray’s recommendation:

To design the connected company we must focus on the company as a complex ecosystem, a set of connections and potential connections, a decentralized organism that has eyes and ears everywhere that people touch the company, whether they are employees, partners, customers or suppliers.

Social Business Design is a new discipline, but some basic rules are already emerging. These emerging rules have less in common with traditional business design, and more in common with urban design and city planning. It’s not about design for control so much as design for emergence. You can’t control a complex system, but you can manage its growth, and there are a lot of things you can do that will position it for success.

He then goes on to outline a number of practices that support building a connected company – I definitely recommend that you read the whole post.

Here are the key points:

  • Large systems tend to stagnate over time: this is true of powerful countries, and of established firms. Stagnation is dangerous, because it eventually leads to decline.
  • These organisations stop innovating because they act like they exist in a world with no uncertainty: if there is no uncertainty, everything is an engineering problem. You don’t need good new ideas. This allows bad ideas to take hold.
  • One way around this problem is to manage connections rather than hierarchies. Whether we believe or not, we are operating in an uncertain, complex world. One of the best ways to do this is to manage connections. This leads to the development of new ideas, which can lead to the renewal of even large, old organisations.

Bad ideas come from bad structures. One of the best ways to eliminate bad ideas is to build new, better structures. Managing connectivity is a great tool for doing so.

Student and teacher of innovation - University of Queensland Business School - links to academic papers, twitter, and so on can be found here.

Please note: I reserve the right to delete comments that are offensive or off-topic.

7 thoughts on “Where Do Bad Ideas Come From?

  1. “Bad ideas come from bad structures. One of the best ways to eliminate bad ideas is to build new, better structures.”

    Oops, how would you know if they are “better?” I would say that to show up bad ideas for what they are, build new structures and EXPERIMENT like your life depended on it (because it does :)

  2. Tim,

    Thanks for this. Gray’s article is indeed very good.

    Your friend Easterly, however, shows what happens when academics start mouthing off about things which they know nothing about.

    Autocrats run into problems for one very simple reason: They care about power more than they do about solving problems.

    Their structures of governance are optimized for power, incorporating patronage, corruption, etc. An enlightened leader, like Lee Kuan Yew can avoid this, but those people are very, very rare (I think the one before was Marcus Aurelius).

    So it’s no use sorting through which problems autocracies are good or bad at solving. Solving problems simply isn’t what they are set up to do (and some corporate structures are the same).

    Sorry to get in a blather about this, but the recent deluge of “autocrat experts” has me a bit annoyed.

    – Greg

  3. Thanks for the comment Greg. That particular post might not reflect it, but I think Easterly knows a fair bit about autocrats – he’s done a huge amount of research in autocratically run countries. He’s a polarising figure in development research, but I think that in particular, his ideas about the value of experimentation, data, and scaling up small trials to increase development are very good.

    You can get a bit of an idea of what he’s working on here:

    http://williameasterly.org/academic-work/peer-reviewed-publications/

  4. Hmmm. I took a quick look through and wasn’t overly impressed. I guess that we’ll have to agree to disagree on this one.

    I, for one, am invigorated. We’ve been talking about the importance of having a diversity of voices, now we’re actually practicing it! 😉

    – Greg

  5. I guess my point is that while I understand your exasperation with armchair autocrat experts, Easterly isn’t in that category. Unlike all the people like Shirky, Gladwell etc. (some of whom I respect a fair bit), he’s spent 40 years or so doing actual research in the region. So he’s more qualified to comment than most. You still may think he’s full of crap, but he’s not making up stuff from his office.

    In any case, the diversity point is a good one! 😀

Comments are closed.