It’s sad but true that most initiatives designed to improve innovation within organisations fail.

This is frustrating, because we actually know a fair bit about what makes innovation work.  Our results should be better.  But still, countless organisations are having brainstorming sessions, adding idea management systems, building idea diffusion networks or implementing some other well-meaning innovation idea, only to fail to deliver better results.

Why?

I think that there’s a missing ingredient in many of these efforts, and it can be summed up in this diagram:

It’s The Innovation Loop – and it is this core set of skills and processes that you need to underpin a successful innovation effort.

Innovation starts with ideas – they are the lightning bolt that ignites things.  The ideas can be for products and services, new ways of doing things, new markets to serve, or new business models.  If we have better ideas, we are more innovative.  If we have lots of ideas, we are more innovative.  Taking steps to improve both the quality and the quantity of ideas in our organisations is important.  However, good ideas are necessary, but not sufficient for innovating.

The other parts of The Innovation Loop are also critical.  These are the monkey wrench – the hard work that makes the idea real.

Next, we need to formulate a hypothesis about our idea – what change do we expect to see if we implement it?  Will people buy more of our stuff? Be happier? Tell their friends about us?  We need to link our ideas to expected outcomes.

Now – test the idea!  Pilot it, prototype it, build a minimum viable product, or a wire-frame, or a mock-up.  Figure out how to test your idea – preferably as cheaply and as easily as possible.

Experimenting is the core innovation skill.

Ok, you’ve run your experiment – now, measure the results.  Was your hypothesis correct?  Did you get change in the direction (positive or negative) that you expected?  As much change as you expected?  Did something cool but completely unexpected happen?  Or something bad but completely unexpected?

Figure out if the idea worked or not.

If your idea did work, now you have to scale it.  Do more of it, apply it in a different setting, get people to buy into it.  All of these are parts of scaling.  It’s not straightforward – but since you’re taking an experimental approach, at least you’re not trying to sell a guess.  You’re pitching data-driven results instead, which is a bit easier.

As you start to scale, you will have many new ideas about what will work and what won’t – put them through this process too.

What if your hypothesis was wrong?  Well, then, you have to learn from it.  Every experiment yields data – and this helps us learn.  Why didn’t it work?  Was the idea bad, or was it just the wrong setting for it?  Are we targeting the wrong people?  Do we need better execution? Does it not fit with our skills or values?

Learning is the key to making the experimental approach work.

The Innovation Loop is at the heart of successful approaches to innovation.  It is a core component of the ideas in lean start-up, and in discovery-driven growth.  It is well-documented that using these approaches will increase your chances of success.  Your idea management system will work much better if you have this loop working underneath it, and so will your stage-gate process.

Experimenting effectively builds your innovation capability.

There are other factors that are also important.  Nilofer Merchant and I are working on identifying what these are.  You need to open up your network to more (and new) people – this helps with getting better ideas, and it helps with scaling.  And you need to have purpose – this is what helps you devise hypotheses about what will create value for people.   You have to get these two things right to innovate.  The Innovation Loop is simply the process that drives success.

The key point, though, is that this is simple, but not easy. This is a very simple model.  But executing it is not easy – it often requires significant change in the way that we do things.  Experimenting means that we have to be comfortable with seeing some of our ideas fail.  Or, at least, some won’t work in the way we expected them to.

It also means that we have to push decision-making authority out towards the front-line.  Innovative cultures are experimental – but experimenting well means that everyone has to be involved with the process.  This was the key to the success of the Toyota Production System.  Jeffrey Pfeffer and Bob Sutton point out that:

The Toyota Production System is about philosophy and perspective, about such things as people, processes, quality, and continuous improvement. It is not just a set of techniques or practices: On the surface, TPS appears simple…

What is important is not so much what we do—the specific people management techniques and practices—but why we do it—the underlying philosophy and view of people and the business that provides a foundation for the practices.

The Innovation Loop is a tool that will help you build a philosophy and perspective that supports innovation.  It also appears simple, but if you implement it, the changes may be profound.