What’s the best way to measure innovation?

Many organisations use only a single measure, like patents, or R&D investment, or revenue from new products/services. The problem with this is that innovation is complex, multi-dimensional, and difficult to capture – especially with only one metric. Take a look at this video from a post about the man who beat Sim City:

Here is how Mike Sterry describes the work that went into the project:

That was the first time I played Sim City. Later, I got my first computer with Sim City 2000. I dicked around with it for about a week before realising I’d rather gun down Nazis in Spear of Destiny, than thanklessly lay down four square miles of sewage piping for silent citizens.

Not Vincent Ocasla though. Vince guy spent four years wallowing in equations and graph paper building a totalitarian Sim City hellscape called Magnasanti, racking up a population of six million and claiming to beat an otherwise unbeatable game.

Ocasla’s objective was to maximise one metric – population. That is the task that all of the graph paper, prototyping and city building went into. He believes that he has ‘beaten’ Sim City by coming up with a configuration that holds the highest possible population – that his 6 million + population is the absolute highest number possible.

He may well be right. However, the interesting part to me is not that he achieved his goal, the interesting part is the price that is paid within the game to pump up the population – here’s how Ocasla describes it:

There are a lot of other problems in the city hidden under the illusion of order and greatness: Suffocating air pollution, high unemployment, no fire stations, schools, or hospitals, a regimented lifestyle – this is the price that these sims pay for living in the city with the highest population. It’s a sick and twisted goal to strive towards. The ironic thing about it is the sims in Magnasanti tolerate it. They don’t rebel, or cause revolutions and social chaos. No one considers challenging the system by physical means since a hyper-efficient police state keeps them in line. They have all been successfully dumbed down, sickened with poor health, enslaved and mind-controlled just enough to keep this system going for thousands of years. 50,000 years to be exact. They are all imprisoned in space and time.

In the interview Sterry points out that none of the sims live past 50. The game is designed so that the sims live lives that should fairly closely resemble those of real people, so this is a fairly significant problem! Ocasla responds:

Health of the sims was not a priority, relative to the main objective. I could have enacted several health ordinances which would have increased the life expectancy, but I decided not to for practical reasons. It shows that by only focusing on one objective, one may end up neglecting, or resorting to sacrificing, other important elements. Similarly, [in the real world] if we make maximizing profits as the absolute objective, we fail to take into consideration the social and environmental consequences.

And that is the innovation level – if we manage the innovation process with only one metric, we will almost certainly perform poorly on other measures that are also important. The best way to manage innovation is to use multiple measures of the process.

Only using one metric means that we will end up like Magnasanti – really good at that one thing, and shockingly bad at everything else – and that’s poor management.