How to Make Good Lean Startup Hypotheses

When teams start out with lean startup, they often build hypotheses that are too precise – we assume we know more than we do. Imagine that we’re trying to build a startup called ScotchFinder – it’s like AirBnB, but for Scotch! Our first hypothesis often looks like this:

We believe that: people will be happy to share their rare scotch with others.

We will test this by: interviewing 30 people outside a liquor store.

We are right if: more than 70% or people say they like the idea of ScotchFinder.

Obviously, it’s better to start with this than it is to go into interviews blindly. However, this hypothesis/test combination can be improved in several ways.

First off, who are the “people?” We often try to approach interviews as though we are looking for a representative sample of the population. This is especially true for scientists, as this is how we’re trained to do surveys.

But that’s not the best way to do customer development. Remember, we’re looking for our first market segment – a specific group, with a specific problem that we can solve. Once we zero in on this group, our sample might end up being very unrepresentative, and that’s fine.

The second problem: what kind of liquor store? Are trying to make something for people that already buy premium scotch? Then we’ll want to do our surveys outside a store with lots of good Scotch. What if we’re trying to get people to start drinking Scotch? Then we might be better off doing interviews at places selling lots of beer. What if we’re aiming to get people that normally drink Scotch in bars to start drinking it at home? Maybe we need ScotchFinder drones!

For each of these three segments, we need to find people in different places, because they’re all living their lives differently. ScotchFinder may eventually serve all three groups of people – but right now, we need the group that is dissatisfied with what they’ve currently got.

The final problems are with the last question. If we set up the hypothesis like this, then we’ll spend too much of our interview time pitching ScotchFinder, and not enough learning about what problems people are running into when they try to drink Scotch.

Finally, again, this isn’t a poll. The majority doesn’t rule. We’re looking for a market segment that we can absolutely dominate. What if we do our interviews (or do a survey), and 97% of people say that they (correctly) think that ScotchFinder is a stupid idea. Then it’s dead, right?

Not necessarily. What if the 3% that do like ScotchFinder are all the same? What if they’re all well educated, double-income no kids couples, living in the same part of town, and for some reason, that part of town has no good liquor stores? The ScotchFinder Drone may well work! And if it does, this might be a segment that we can dominate, which is a great way to start out.

In our early interviews, we need to set the bar pretty low for our hypotheses. We don’t know enough yet to use percentages, as though our 30 interviews are the same as a mini-survey. We’re actually looking for themes.

This is what Eric Ries says about this in his upcoming book The Leader’s Guide:

Don’t bog new teams down with too much information about falsifiable hypotheses, and try to resist the urge to critique teams’ hypotheses when they are early in their Lean Startup journey. Because if we load our teams up with too much theory, they can easily get stuck in analysis paralysis. I’ve worked with teams that have come up with hundreds of leap-of-faith assumptions—page after page after page in a tiny spreadsheet of all the things that had to be true for the project to take effect. They listed so many assumptions that were so detailed and complicated that they couldn’t decide what to do next. They were paralyzed by the just sheer quantity of the list.

Strategy: Simplify.

Take the photo-sharing product example. We might start out by saying “100% of parents want to share photos.”

Now we have a prediction that we can test. Creating an MVP has gotten much easier. I can test out my assumptions with the first ten parents I find—and I can find them anywhere: online, in a coffee shop, or wherever. Over time, I can come up with more specific, sophisticated experiments.

Remember, our goal is to get our teams to write down whatever it is that they believe already—no matter how sloppy, ill-conceived, or foolish those beliefs are. It’s extremely difficult to talk entrepreneurs out of a bad idea; it’s much better to let reality be their teacher.

Here’s another way to frame our initial hypothesis that might yield more usable learning:

We believe that: double-income no kids couples in high-power jobs need a great way to socially share Scotch.

We will test this by: interviewing 30 DINKs (and include a guess about where to find them)

We are right if: we find a common Scotch-related problem shared by several (more than 6 or so) of them, which they are currently trying to solve.

That looks vague, but that’s where we start. Once we have the problem identified, then things will become a lot more specific as we move to IRL 4 and above.

We need to start our lean startup process with discovery – and that is harder to hypothesise. But we can’t look for false precision, that will lead us down the wrong path.

Note: Over the past year, I’ve been running (with help, of course!) a bunch of Lean LaunchPad programs with the Commonwealth Science and Industrial Research Organisation (CSIRO) aimed at increasing the impact of all the great research that they’re doing. This is part of a series reflecting on what we’ve learned through the course of six programs involving 40 research projects and more than 250 people. The other posts are: