Mistakes versus Experiments

One of the reasons that people try to avoid failing is that it seems like they’ve screwed up if they fail. This can certainly be the case, if your failure is major. But if you set up experiments to test ideas out, and you learn from them, then failing can be very productive.

Here is how Hugh MacLeod put it in his daily newsletter:

The cartoon pretty much says it all. Here’s what he wrote to go with it:

I also love Esther Dyson’s great line, “Always make new mis­ta­kes” …
It’s all about the same stuff: That our abi­lity to suc­ceed and to thrive is in direct pro­por­tion to our abi­lity to make mis­ta­kes and learn from them.
It ain’t roc­ket science, but it’s easily for­got­ten by some. Myself inc­lu­ded. Ouch…

But there’s actually an interesting distinction between mistakes and experiments. Mistakes are things you do even though you know better. Experiments are tests designed to expand your knowledge. The big difference is that you learn from experiments (or at least you should).

I ran across a great post by Maddie Grant today that addresses this – Embracing Failure Does Not Mean Embracing Mistakes. She and her colleague Jamie Notter talk about a case study of NTEN that they wrote. Jamie said:

If we fail at something, it will fuel learning, which will enable us to do it better next time. Failure is good.

But during the session, someone asked Amy a question about NTEN being comfortable with making mistakes, and Amy was quick to jump in and clarify. “We don’t define failures and mistakes the same way,” she said. A mistake is when you do something wrong, even though you knew the right way to do it. Failure is when you are trying something new, and you don’t know ahead of time how to make it successful. A typo in a conference brochure is a mistake. It’s not like you didn’t know how to spell the word correctly. NTEN is not “comfortable” with mistakes as it is with failures. They work very hard to eliminate mistakes. (Though I doubt they are the kind of place to ruthlessly punish people for making mistakes either.)

But they are okay with failure. They love to learn from what they are doing. They recognize that if you don’t fail some of the time, then you aren’t pushing hard enough. You aren’t growing. Eliminating failure would mean doing ONLY what you already know how to do. You don’t grow that way.

So this is an example of a mistake (well, at least three mistakes by my count, four if you include “getting a tattoo in the first place”):

Now, there’s a bit of a conflict between these quotes. Do we want mistakes or not? Are mistakes failures?

You Must Learn From Failure

To me, there are two key issues to consider with failing. One is that failure is valuable if and only if you learn from it. The best way to do this is to experiment – this way, the learning is built into the process. Experimenting keeps you conscious of the fact that you don’t know exactly how things work, and you’re trying to figure that out.

But you can also learn from mistakes. I remember the first time I was at fault in a car accident. I was looking for a cassette tape that I dropped and bumped the car in front of me. Fortunately, no major damage or injuries. It wasn’t experiment to test whether or not I could get away with driving while distracted – I wasn’t intentionally gathering data. That was a mistake. I knew better than to not pay attention, I just let myself get distracted.

But I learned from it. It was my first at fault accident, and so far, my only one. The fact that it was caused by a cassette tape tells you how long ago it was! One of the reasons that I haven’t had any more is that I learned from the first one – the mistake.

So failure can be productive, as long as learn from it. Experiments are the best way to do this.

Fail Quickly and Cheaply

Which leads to the second key point about failing – you need to do it as early in the process as possible. I’m doing some work with a collaborator right now trying to put together some tools aimed at helping small businesses. This scheme has several assumptions built into it. One big one is that the problem that we’re trying to solve is big enough that these firms will be willing to pay for help.

But will they?

We flat out don’t know. So we’re trying to build an experiment to help us figure out if they will. We’re doing this because it is much better for our assumption to fail now, than for our entire scheme to fail when we launch because the assumption was wrong.

If we try an experiment and the small firms won’t or can’t pay, then we an re-design our program targeting larger firms. Or a particular sector. Or whatever else our data tells us might work.

The key point is to force faulty assumptions into the open as soon as possible. If they’re wrong, we don’t want them to torpedo the whole operation later on.

Failure is always a contentious issue. If you fail, you lose resources that could been used elsewhere. The problem is, if every idea that you try works, then you’re not trying enough new ideas. So some ideas will need to fail. This can come through experiments, or mistakes. Experiments are better, but both can help as long as you make sure that you learn from them.

Student and teacher of innovation - University of Queensland Business School - links to academic papers, twitter, and so on can be found here.

Please note: I reserve the right to delete comments that are offensive or off-topic.

15 thoughts on “Mistakes versus Experiments

  1. It’s so tricky this theme.

    I know some examples where managers spoiled a lot of money from companies to whom they worked for,because they made bold decisions with lots of money involved. When owners questioned the results, they showed papers like this post saying:
    You see, failing is part of business. Next time will be better, luckier
    And the owners, humbled by the awesome number of papers praising failure had to retreat…
    It’s so tricky.
    Of course I agree with you, of course I agree with most of those papers, the problem is people using the ideas to support their crazy or ill-mattered ideas.
    More care should be used when speaking about failure and relating them to experiments, not to full and expensive moves
    It’s not enough to separate mistakes from experiments/failures, one must also distinguish stupid experiments from clever experiments. Wasting a lot of money with an experiment can jeopardize the future of the companies. Tour lost point “cheaply” should always be underlined and given top attention.
    Sorry my english

  2. Thanks for the thoughtful comment Carlos (and your English if fine!). It is really tricky. I think that the main point is that the best approach is to figure out in advance what your approach to learning will be. Using posts like this after the fact is more of an excuse than a strategy. Much better to design experiments right from the start.

    But you’re right – it is very tricky.

  3. Hey CCZ,

    I’d like to know the owners faced to a situation with lots of money spoiled who retreat humbly when showed paper like this.

Comments are closed.