In the 1850s, infection rates in maternity wards were very high, and this was a big problem. No one knew why, and no one knew how to fix it. Ignaz Semmelweis wondered “what if everyone washes their hands before the come in contact with patients?”

It was an experiment. And it worked – infections rates dropped from 15% to less than 1%. Semmelweis went from not knowing anything, to knowing that something worked, but he didn’t know why. The last point was important – it took 20 years to answer that question with the germ theory of disease, and it took 70 years before hand washing in hospitals became widespread.

In the 1890s, no one knew how to separate zinc from the sulphurous rocks that contained it, and this was a big problem. Despite the high value of zinc, no one knew how to get it separated. Guillaume Delprat wondered “what if we dissolve the rocks in water and then try to precipitate the zinc out of solution?”

It was an experiment. And it didn’t work – the zinc wouldn’t sink. Instead, it floated. When Delprat realised that floating zinc was actually separated, he combined that insight with ideas from a few other people, and the process of flotation was born. Delprat went from not knowing anything, to knowing that something worked, but he didn’t know why. However, unlike with hand washing, flotation almost immediately was put into use, even though it took another ten years to figure out why it worked.

Last week at the CEEC Workshop that I spoke at, Geoff Garrett, the Chief Scientist of Queensland, gave the opening keynote. And he asked “why don’t we use what we know?”

Both Semmelweis and Delprat learned something new. One idea went into use quickly, but the other didn’t. Why?

The Gap Between Knowing and Doing

Geoff made another important point in his speech. He talked about the gap between what we do and don’t know. That’s the gap that is bridged by science. But he pointed out that the gap between what we know and what we do can be even bigger. That’s an innovation diffusion problem.

I’ve been thinking about his talk for the past week, and this is how I’ve ended up framing my thoughts:

Along the horizontal axis, we have things we know and things we don’t know. On the vertical, things we use and things we don’t use.

The bottom left square is a great place to be – it’s things we use based on things we know. This quadrant is evidence-based – and we need to do more of it.

The top right is where we’re bridging that gap between what we don’t know but we want to. That is where we ask “what if…?” like Semmelweis and Delprat both did. Then we try an experiment.

If the experiment works, and we know why, and we start using it, then we end up in the evidence-based quadrant.

If the experiment works, and we don’t why, but we start using it anyway, then we are in the bottom right quadrant. These are the practices that we follow, but they’re not based on solid knowledge. In the case of flotation, for the first ten years in practice, it was something that just worked.

If the experiment works, and we know why, but we don’t use the knowledge, then we are in the top left quadrant – we have a diffusion problem. This is where hand washing was for a long time.

One of the big innovation questions is why does diffusion take so long? We know that it follows an s-curve, and unlike the idea of flotation, most new ideas spread much more slowly than we expect – like hand washing.

I think that part of the problem lies in the other half of the bottom right quadrant – we often follow practices that are wrong. When they are wrong, and they don’t work, this is a big problem. Dislodging these practices is often extremely hard.

What DO We Know?

Here’s another example – watch this talk by Dan Pink – it provides real insight into how to motivate creative people:

The key point here is that the research evidence on how to motivate innovation is overwhelming. It’s not through paying people to incentivise innovation. Instead, it happens when we pay people an adequate amount, and then give them rewarding work to do. This is work that provides a sense of mastery, in an environment of autonomy, and that is imbued with a sense of purpose.

We know this, but few people use it. One company that does is Valve, and their employee handbook explains why.

They have a flat hierarchy, with no managers. That’s huge autonomy. Rewards in this system are based on your team-mates’ assessment of your performance. This requires mastery. Why? This:

In 1996, we set out to make great games, but we knew back then that we had to first create a place that was designed to foster that greatness. A place where incredibly talented individuals are empowered to put their best work into the hands of millions of people, with very little in their way

That’s purpose.

Valve uses what we know, and they have been enormously successful in doing so. So has W.L. Gore, and Google, and Amazon, to name a few.

Valve’s handbook closes with an interesting question:

Q: If all this stuff has worked well for us, why doesn’t every company work this way?
A: Well, it’s really hard. Mainly because, from day one, it requires a commitment to hiring in a way that’s very different from the way most companies hire. It also requires the discipline to make the design of the company more important than any one short-term business goal. And it requires a great deal of freedom from outside pressure—being self-funded was key. And having a founder who was confident enough
to build this kind of place is rare, indeed.
Another reason that it’s hard to run a company this way is that it requires vigilance. It’s a one-way trip if the core values change, and maintaining them requires the full commitment of everyone— especially those who’ve been here the longest. For “senior” people at most companies, accumulating more power and/or money over time happens by adopting a more hierarchical culture

In other words, it’s easier to be wrong, even though it’s much less effective.

There’s actually a lot that we know about innovation. The research that Pink cites about compensation is one of the big things. And we know that innovation is a process, not an event. And these are just two evidence-based ideas that aren’t implemented nearly as often as they should be – there are many others.

It’s time to stop practices based on ideas that are wrong. Let’s start using what we know.