How to Use Experiments to Build an Innovation Culture

Build Measure Learn

When you were learning how to ride a bike, did you work out the theory of how to do it, jump on the bike, and start riding flawlessly? No. You started with training wheels and worked it out through trial and error.

When Thomas Edison invented his version of the light bulb (the 24th to be patented!) did he discover that tungsten was the best thing to use as a filament through logic?  No. He famously found 9,999 things that didn’t work before settling on Tungsten – through trial and error.

When Caterina Fake and Stewart Butterfield launched Flickr, did they work out the perfect business model in advance, then launch the site?  No.  They started out building massively multiplayer online game – Game Neverending, which became relatively successful, just not successful enough to keep the company afloat. So the team built a site around a photo-sharing protocol that they developed for the game, and that became Flickr.  It was business model development through trial and error.

Experimenting is a critical innovation skill.  None of us think twice about learning to ride a bike through trial and error, so why is it so rare in business?

There are two main reasons.  One is risk aversion.  In a work setting, people don’t like to make mistakes, and they don’t like to look foolish.  Trial-and-error can cause both of these things to happen when things don’t work out as expected.  The second reason is that our organizational cultures are often not amenable to experimenting.  In larger organizations, we are often trying to improve efficiency.  Doing this means that we must reduce variation.  But innovation and experimentation increase variation.  There is a tension between efficiency and innovation.

But the benefits of experimenting outweigh these issues.  The problem that experimenting solves is this: it’s nearly impossible to know in advance which ideas will work and which won’t.  If we experiment, instead of guessing which ideas will work, we can test them.  This helps us get better making decisions based on data.

Steve Blank recently wrote Why the Lean Start-up Changes Everything and he talks about this as the fallacy of the perfect business plan.  The approach is built around identifying your assumptions about how your idea will work, and then gathering data from customers to validate or disprove these hypotheses.

Eric Ries has outlined a methodology for Lean Start-up – Build-Measure-Learn.

Build Measure Learn


This is the core method for experimenting.  This version works for tech-based startups, but the general principles apply more widely.  We need to use this build-measure-learn loop more in larger organizations as well.

There are many advantages to experimenting.  The first is that it keeps failures small.  The key to this is building prototypes and testing them.  As Diego Rodriguez says:

 Anything can be prototyped. Prototypes aren’t just for physical products. I routinely see people prototyping services, complex experiences, business models, and even ventures.

You can prototype with anything.  You want to get an answer to your big question using the bare minimum of energy and expense possibly, but not at the expense of the fidelity of the results.

Prototyping is the Build-Code steps in the Lean Start-up loop, and it leads to better results than predicting does.

The second advantage to experimenting is that it gets us out of the trap of over-thinking.  The further away we are from our customers, the easier it is to believe that we can just figure out what they need most through logic.  But we can’t.  We have to test the ideas out – trial and error helps us find what will really work.

Finally, experimenting is a great way to get rid of air sandwiches.  That is what Nilofer Merchant calls the gap between management aspirations and the reality down at the coalface.  This is a frequent problem with innovation.  It is rare to find a CEO that doesn’t want their organization to be more innovative – but few have tangible plans for how to achieve this vision.  Experimenting can help.

If we build a culture of experimentation, people will be more comfortable with examining and testing their assumptions.  This is a great way to build the skills that you need to innovate, and it is also useful for finding the best way to make new ideas real.

Here is a method that anyone can apply:

  1. Idea: Think about how much you can get away with – if you manage a budget, how much discretion to you have? If you don’t have a budget, what are the parts of your job that you control?
  2. Build: Make a list of 10 things that you can do within the current scope of your work that will make things better for the people with whom you interact – customers, co-workers, bosses, whoever.
  3. Prototype: Do those things.
  4. Measure: Figure out which ones worked, and do those more.
  5. Data: Figure out which ones didn’t work, learn why not, then forget about them.  Do more of the ones that did work.
  6. Learn: Apply what you learned to the next set of ideas.

If you’re a manager, encourage those on your team to do this as well.  As people get better at, things will start to improve.  And you’ll start to build a culture of experimentation.

It’s a lot like learning to ride a bike.

Note: I’d like to thank Sarah Green from Harvard Business Review Blogs for help with editing this post.


Enhanced by Zemanta

Student and teacher of innovation - University of Queensland Business School - links to academic papers, twitter, and so on can be found here.

Please note: I reserve the right to delete comments that are offensive or off-topic.

8 thoughts on “How to Use Experiments to Build an Innovation Culture

  1. Super post Tim – its a pity you have to even say this though – its common sense in the real world but in the bubble world of institutions in particular it has to be spelt out.

    Its a result of the engineering approach left over from the last century – the language permeates all through business from “Human Resources” instead of people through to “Structure and Restructure” instead of organise.

    • Good points Martin. I agree that it should be a basic and obvious idea, but, I spend a lot of my time talking about things that should be basic and obvious!

Comments are closed.