Archive for category complex systems

How to Build Business Metrics – revised

Posted by Tim in complex systems, innovation strategy on 14 December 2011

We’ve written a few posts criticising some of the more common innovation metrics in use, so I thought it would be smart to outline some ways that we can actually develop more effective metrics. Here’s a story that might help:

A while ago I was in charge of managing student recruitment for a tertiary education institution. One of the first things I looked into when I started the job was metrics – how did we measure how well my section was doing? The answer was one number: total number of enrolled students each year. The job that I was given was to increase that number by as much as possible (which begs all kinds of questions about quality, teaching and so on, but let’s set those aside for now…).

The problem was that managing that number as a standalone was hard. Well, impossible, actually. So I looked into what other numbers we had, and I found a that we had measures for total applications received, and total enrolments. I worked with my teams to figure out the path that people took to become students, and we then also figured out a way to measure enquiries. Once we had these numbers, here’s what we did:

We made three metrics: total number of enquiries, the ratio of applications/enquiries, and the ratio of enrolments/applications. Then I made the marketing team responsible for enquiries, the information team responsible for applications/enquiries, and the enrolments team responsible for enrolments/applications.

When my boss told me to increase enrolments as much as possible, he was hoping for a 5% increase. By breaking down the process, developing new metrics, and making people accountable for the measures, we were able to increase enrolments by 12%.

There are several lessons from improving innovation metrics in this:

Innovation is a process not an event: many things that we often think of as an event are actually processes. Enrolments is a good example – previously my institution only considered the end point, enrolled students. By breaking down the process that we went through to actually get an enrolled student, we were able to improve our ability to get enrolled students.
I think of innovation as a process too – this is the diagram that I use to describe it:

To improve our innovation metrics, we need to first think of it as a process, then build metrics to measure the intermediate steps as well as the outcomes.
Use multiple metrics: in the enrolments story, we used three metrics that led to the one that we were most interested in (total enrolments). We can do the same for innovation. Once we think of it as a process, then we need to develop metrics for each of the steps that lead to the outcomes that we are looking for from innovation. Innovation is a complex process, and to manage it we need to use multiple metrics.
Link Your Innovation Metrics to Your Strategy: my tertiary education institution saw increasing enrolments as a central part of its strategy. At the time, the educational sector in New Zealand was fairly turbulent, and there was a strong message from government that it wanted to see the sector consolidated. Increasing enrolments was seen as a way to signal that we were a thriving institution, making it less likely that we’d get absorbed by a larger polytechnic.
We need to do the same thing with innovation – link it to our overall strategy so that it can help drive success. There are a number of broader strategic goals that can be supported by innovation – we just need to be clear about which ones we’re targeting.
Improve the part of the process that is weakest: when we started tracking the enrolments process, we discovered that we were pretty good at generating enquiries, and very good at converting applications into enrolments. The weak link was converting enquiries into applications.
The information team had been given some sales training before I arrived, which they strongly resisted. They saw their role as helping people, not selling them. We implemented a lot of ideas, but the one that had the greatest impact was getting them to ask at the end of each enquiry that they handled “if you’re interested in the course, would you like to put in an application?”

When they started doing that, the applications/enquiries ratio shot up from about 12% to 18% in a couple of weeks. And we weren’t forcing people to apply for courses they didn’t really want to take – the enrolments/application ratio held steady. If the quality of applications had decreased, this metric would have gone down. It turned out that a lot of people really did want to start studying, but they just needed a small nudge to get started.

In looking at our innovation processes, we need to do the same thing: find the weak link, and figure out how to best improve it. As we’ve said many times before, usually the problem in organisations is not that they don’t have enough ideas, but rather that they need to get better at selecting ideas, or at getting them to spread. In any case, once we have identified the part of the process that is most in need of improvement, then we can figure out to best go about making it better.

Getting innovation metrics right is a challenging task. There is no single number that will tell us everything we need to know to manage innovation. I hope these ideas help you figure out how to measure it better in your organisation.

Note: There was also a good series on innovation metrics by Boris Plukowski on Innovation Excellence a while back: Part 1, Part 2 & Part 3.

1 Comment

The Exigency of Extrapolation

Posted by Tim in complex systems, experiments, selection on 12 December 2011

Noun 1. exigency – an unstable situation of extreme danger or difficulty;

I’ve had some jobs in which I’ve performed pretty well, and some where I haven’t been quite so good. Probably the worst job I’ve ever done was part of my portfolio when I was managing sales & marketing for a polytechnic in New Zealand. The specific job was new course evaluation.

We put in a modified stage-gate type process to evaluate potential new courses. It was my job to fill in the numbers. The one number that drove everything else was the expected number of students. If the expected number of students was high, we’d try to run the course. If not, we’d kill the proposal.

I developed a very elaborate model, based on historical data. I knew how many enquiries we could generate from advertising, how many of those we could convert into applications, and how many enrolments we got per application. In fact, figuring out these numbers was one of the biggest innovations that I executed there, and this model was of tremendous use in trying to figure out around November how many total students we could expect at the start of the new school year the next February.

However, the model was terrible at predicting new course enrolments. Why? In large part, because we’re really lousy at figuring out how something new will perform. We rejigged our new course approval process after I pointed out that we hadn’t approved a single new offering in over 6 months – our process was killing everything.

I was originally going to call this post The Perils of Prediction, but Greg Satell beat me to that title. Also, the specific problem that I’m talking about is really extrapolation. You should read all of Greg’s post, but here’s part of what he says:

The problem starts when smart people in nice suits and lab jackets proclaim that “the data says…” In truth, the data never says anything. We interpret it in one way or another and there are lots of ways to interpret it incorrectly.

Data is, after all, messy. It doesn’t spring forth whole, but must be collected in some way. We count, measure, survey, aggregate, slice and dice, picking up errors all the time. We need to make choices about which data we want to focus on and which fades into the background.

How do we deal with this? Usually by finding some numbers from the past and extrapolating them. However, there are a few problems with this approach, including:

We tend to think in straight lines, but there aren’t any straight lines in business: that’s really the point being made by the xkcd cartoon. Taking a straight line and extrapolating it into the future almost never gives us the right answer.
It’s really hard to tell what kind of curved line we’re on: this complicates things too. Even when we have historical data, it is nearly impossible to figure out what kind of engine is generating the output. Take a look at this data from an interesting post on climate change:
Is it likely that the data will progress in a straight line? Or will it level out at some point? Or will it increase exponentially? We don’t know. But when we’re predicting, it pays to consider what circumstances might lead to each of these outcomes.
Even when things are accelerating quickly, they tend to level out: innovations spread through an s-curve, and this is a very common pattern in business.

This is one of the issues with everyone talking about the singularity – it assumes that exponential growth will continue forever. It might, but usually exponential growth levels out, and then it looks like an s-curve.
However, by the far the biggest problem with extrapolation is that if we depend on extrapolation for predicting, we will never anticipate something new happening: extrapolation can only predict that things in the future will be mostly like things in the past. Here’s how Greg puts it:

And that’s what most analysts miss. The future is hard to predict not just because of our cognitive biases or inexplicable natural events, but because we have the power to make our own future.

The first new course that we approved at my Polytechnic after we scrapped the stage-gate process was a program that offered free computer and internet lessons to people in the community, particularly targeted at older adults. And the number of enrolments that we got went so far beyond anything that we had ever seen before that it was almost impossible to believe.

None of my models could have predicted that. When we innovate, our job is to invent the future. The exigency of extrapolation is that if that is the tool we use to predict, we won’t be able to invent anything that doesn’t already exist. And what kind of innovation is that?

5 Comments

The Jenga Theory of Creativity

Posted by Tim in complex systems, experiments on 8 December 2011

I think I actually made yesterday’s post on simplicity too complex. Here’s another try.

Earlier this week I edited two different papers for journals. My main contribution was that I cut 2,000 words out of each. I also wrote about 400 words in each, but it was the cutting that helped the papers.

This reminds me of the drawings by Matisse I talked about yesterday – he was more interested in what he could take out than what he should put in. The key question that he asked was: what is the minimum number of lines that I need to capture the essence of what I’m drawing?

Creativity is often about subtraction as much as it’s about addition – it’s really like playing Jenga. You need to pull out as many pieces as possible while still retaining the shape of the idea that you’re working on.

Or, as Austin Kleon put it his great post How to Steal Like an Artist:

The key issue is how do you know what to take out? That is where experimentation, failure and learning come in. The only way that you identify the essential pieces to keep in is through testing (prototyping). Here is how Joe McCarthy (please go read his blog, it’s awesome) put it in a comment:

And just to bring it full circle, while I agree that getting it right requires learning and skill, I believe that learning and skill often arise primarily through making lots of mistakes (i.e., being wrong alot … but with an open mind).

And that’s what I’m trying here. I gave it a go yesterday, I’m not sure if it worked, so I’m trying a different way today.

If I keep working on it, eventually I’ll get it right.

The way to make something simple is to cut out all the extra bits. But you can only know what to cut when you have a deep understanding of the system in which you’re working. That’s the Jenga Theory of Creativity.

(Jenga picture from flickr/riNux under a Creative Commons License)

5 Comments

Life’s What You Make It

Posted by Tim in complex systems, time on 2 December 2011

Well, we’re all getting older. What do you make of it? I ran across an interesting post by Ben Casnocha, which referenced an article by Benjamin Schwarz which includes this comment on John Updike:

Above all, and most poignantly, this collection highlights Updike’s evaluation of the slackening of his own mental and athletic prowess… A generous and companionable critic and an avowed Christian, Updike met the decline of his powers with courage and good humor, but also with a clear-eyed recognition that the compensations of old age—a hard-won sagacity, a bemused detachment—don’t make up for the irretrievable losses.

Here’s the thing – you have a choice about whether or not the compensations of old age make up for the losses.

I was in the best physical condition of my life when I was 20. And I was a wreck. I was a befuddling mix of arrogant and insecure, I was struggling at university, I was depressed. I had gifts that were not yet developed, and potential that seemed to be fading rather than emerging.

In short, I was an idiot. But probably not that far off the norm for a 20 year old either.

By the time I hit 30, things were a bit better. Nancy and I had just gotten married, which was great. Work was still a bit of a struggle – I still hadn’t figured out how to best use my talent. Physically, I was in ok shape, but nowhere near as fit as I was at 20.

At 40, I was in the middle of making a career change that was the smartest career move I’ve managed to make. There were high levels of uncertainty over whether or not it would work, but life was a lot better than it had been at 30 – even though I was in the worst physical shape of my life.

Now I’m rocketing towards 50 – and things are even better than they had been. Work is good, and some of that potential from when I was 20 is finally turning into something meaningful. Physically, I’m fitter than I was at 40, but I’m starting to lose a few things too, as you do.

I know I’m lucky, but for me, life has just gotten better and better as I’ve aged. Now, Ben Casnocha has been precociously successful, so maybe things will be different for him. I don’t think so though.

Why I am happier now than I was at 20 – despite the irretrievably physical losses? Because of the things I’ve learned, and that I only could have learned through experience. I’m better at executing ideas now because I’ve learned how to do it. This has been the key to developing that long-dormant potential.

This isn’t to say that bad things haven’t happened over the years, or that more won’t happen in the future. Of course they will. But one thing that I’ve learned is that the best way to ensure that your life gets worse as you get older is to convince yourself that life must get worse as you age. It doesn’t have to. The things around you don’t determine how you must live your life – read Man’s Search For Meaning by Victor Frankl for insight into that. The knowledge that you gain as you experience life is unobtainable when you’re young. You’d be smart to place a pretty high value on that.

At all ages, life’s what you make it.

Here’s a song by Talk Talk from my fit dance-club days that I stole the post title from:

5 Comments

The Most Important Innovation of All Time

Posted by Tim in complex systems, experiments, innovation on 23 November 2011

What is the most important innovation ever?

There are plenty of candidates. Fire, the telegraph, electricity, and the internet would all have to be candidates.

There’s another one though, that has had an enormous impact on every single one of us. And surprisingly, it’s not a whiz-bang piece of technology. It’s a simple process innovation.

The most important innovation of all time is: medical practitioners washing their hands before they touch patients.

Hand washing has been an unbelievably important medical breakthrough. It is one of the main reasons that we actually live long enough to retire now.

As with many great innovations, hand washing started with a scientific discovery – the germ theory of disease. And as with some innovations, the theory was driven by beer. Louis Pasteur’s work was motivated by brewers who couldn’t figure out why some batches of beer fermented well, while others failed. So in trying to make better beer, Pasteur made us all healthier.

There are some critical innovation lessons here:

Ideas need to be executed to create value: the germ theory of disease is an important scientific breakthrough, but a theory isn’t an innovation. Theories are often great ideas, but to become an innovation they have to be turned into something that can be executed to create value. Germ theory led to many important innovations: pasteurization, antibiotics, and hand washing. These innovations have had impact on a wide range of industries and activities, and that is where the value has been created.
Innovation isn’t just about new technology: hand washing in hospitals isn’t a sexy new piece of technology (which is maybe part of why it’s still hard to get everyone to do it consistently). Hand washing is a process, and process innovation can be incredibly important. Just think about the assembly line, lean management, or agile software development. All are process innovations, all are important, like hand washing.
Small innovations can have enormous impacts: one feature of complex systems is that small changes can results in gigantic change. That’s what happened with hand washing. This new process has made childbirth much safer, it improved the success rate of all surgeries, and it greatly reduced the chance of secondary infection in medical procedures. And it’s about the simplest thing imaginable!

So the next time you wash your hands before eating, think about what a great breakthrough you’re participating in. And when you’re thinking about innovation, remember that it’s often the smallest ideas that can make the biggest difference.

2 Comments

The Property Ladder Theory of Bubbles

Posted by Tim in business models, complex systems on 1 November 2011

I’ve always thought that the BBC show The Property Ladder provided the perfect illustration of how bubbles worked. The show ran from 2002-2006 and it was hosted by Sarah Beeny. The show profiled aspiring property developers who bought properties, renovated them, then tried to sell them for a profit.

If you’ve never seen it, this will give you a flavour of how it worked:

That show that included Sian Astley – who was atypical in that she went on to become a successful property developer. The normal pattern in the show is that the developer-to-be buys a property, then spends most of the show ignoring the advice given by Beeny (because otherwise it wouldn’t be nearly as interesting).

Inevitably, the projects run way over time, and way over budget. Nevertheless, the properties always ended up selling for a substantial profit, and they nearly always ended with the new developer planning to move on to their next development project.

The reason that it’s the perfect example of a bubble is this: the reason that the properties always made a profit is not that the developers did a great job, it’s that the UK house market was red-hot throughout the time the show was made, so every house went up in value. In fact, running over time helped the developers out, because it gave the properties more time to appreciate.

What happened to all these “property developers” once the GFC hit? There’s a hint in that the revised series is now called Property Snakes & Ladders. I suspect that unlike Astley, nearly all of them went bust.

Here’s the problem: if you start a business in a bubble, it’s easy to make money, but it’s very hard to define the value that you provide. If you fail to provide clear value, whenever the bubble bursts, you’re out of business.

I ran into two examples of this in conversation today. The first was talking with Nancy on our drive in to work. She recently joined the board of directors for an association that for years made money from their conferences. How? By getting lots of sponsorship from drug companies. Now that the pharmaceutical sponsorship bubble has burst, they have no idea how to make up the lost revenue.

But their real problem isn’t how to make up the revenue – it’s that they don’t appear to have any idea what value they actually provide to their members. If they could answer that question, then they could figure out how to make up the money.

The second example came from an energy consultant. He told us about all the companies that formed in New South Wales to take advantage of the government subsidies designed to get people to switch to compact fluorescent lightbulbs. While the subsidy was in place, they all made money. When the subsidy disappeared, so did they.

Again, they didn’t have a clear value proposition.

If you don’t have a clear value proposition, you can’t build an effective business model. Without the value proposition, all the other business model factors are incoherent.

This is one of the reasons that more successful firms are founded during depressions than they are during bubbles. To be successful when times are tough, you have to have a clear value proposition.

Don’t just surf the rising tide. To figure out how to last, figure out how you provide value to people. This is the first, essential step to building a successful business model.

2 Comments

The Fundamental Problem in Management

Posted by Tim in complex systems on 6 October 2011

The fundamental problem in management is that the world is uncertain, and people hate dealing with uncertainty.

The result of this that they go to great lengths to provide themselves with the illusion of certainty. The Bed of Procrustres by Taleb, which I discussed previously, is primarily concerned with the problems caused by false certainty.

The problem with requiring certainty is that when you do, you fail to act. If you have to know in advance whether or not your innovation will succeed, you won’t innovate. If you have to know in advance whether or not your co-workers will perform, you won’t delegate. If you have to know in advance whether or not your idea will be accepted, you won’t put it forward.

All of the bad aspects of bureaucracy come from trying to build systems that provide certainty in a world that is by its very nature uncertain.

The more businesses I work in and talk with, the more convinced I become that the single most important management skill to develop is a tolerance for ambiguity.

7 Comments

Innovation in Aussie Rules

Posted by Tim in complex systems, innovation on 5 October 2011

Collingwood fan and outstanding economist Nick Gruen wrote an interesting piece prior to last weekend’s AFL Grand Final explaining why he thought his team would lose to Geelong. The underlying premise in the piece is that he thought Geelong would win because they were more innovative.

Here are some of the key points:

If I were to set out the way to win the premiership it would be the way Geelong have managed this season. The basic strategy behind the game changes subtly as sides come up with new approaches. But it takes the best part of a year at least to catch up with some new strategy. Thus we’ve seen Sydney get a premiership from flooding, and then they were unpicked. Then we saw the Saints doing something similar but somehow better. In each case both Sydney and the Saints didn’t have a very good bunch of players. They had a new strategy and players who were thoroughly drilled in how to make it work and they became almost impossible to beat.

The reason it takes time to peg such a strategy back is that, apart from figuring out exactly what they’re up to, you then have to figure out what to do about it. Collingwood has had its forward press going sufficiently well to win last year, but being the worrier I am I was always worried about Geelong, not just because they’ve got the fastest, most direct attacking game in the business, but because they added defence – copied from us – to that strategy.

…

More alarming still is that as I read in an article that someone else may remember and link to (I can’t find it) that Geelong’s stats have changed dramatically in the last five or six games. Their average kick length and kick to handball ratio has gone way up. They’ve basically come up with a way of getting the strengths of their attacking game without the downsides of inattention to defence. And they’re tearing other sides apart.

Whether deliberately or not, this new style hasn’t been really shown to the world for long enough for people to figure out how to unpick it, let alone drill the necessary skills and structures into their players to do so. So I reckon we’re in a lot of trouble. Tehy will pick us apart in just the way Hawthorn picked us apart last week – with lots of pressure against us in defence to stop us getting our run out of defence and with lots of long direct kicking zig-zagging down the centre of the ground and leading out from full forward.

This is an almost perfect description of how business competition and innovation works too.

Someone comes up with a great idea and brings it out. This gives them an advantage for a brief period of time. If they have solid management skills and good systems, they can turn it into a sustained advantage, as Geelong has (they’ve won titles now in 2007, 2009 and 2011). But even when the innovator doesn’t have the resources necessar to win for an extended period of time, innovation can still result in short-term advantages, as in Nick’s examples of Sydney and St Kilda.

Finally, innovating doesn’t guarantee winning. If you innovate, and have good processes, your odds of winning increase. But in the end, you need a combination of innovation, good structures, execution and luck. The good news is that the more innovative you are, the luckier you’re likely to be.

No Comments

Carmageddon – Change is Hard, Except When It’s Not

Posted by Tim in complex systems, evolving economic entities on 20 July 2011

The world failed to end over the weekend. In Los Angeles, at least, this was a bit surprising, as there were many dire predictions made about the impact of closing down the 405 Freeway for a weekend of construction.

Many of the key issues are summarised in this remix from Downfall (which is funny, but NSFW, and probably not politically correct). There are also more straightforward discussions of the event dubbed “Carmageddon” on the LA Times website.

Even on weekends, the 405 carries a staggering amount of traffic, so it stands to reason that shutting it completely would have a major cascading effect on traffic throughout the rest of LA.

And yet, it didn’t. Why?

Because people changed their behaviour for the weekend. They stayed home, or walked place, or travelled in the opposite direction of where the traffic was supposed to be.

In the end, Carmageddon was a complete non-event.

This is actually the second time that this has happened in LA. The first time, it was supposed to be even worse. That was during the 1984 Olympic Games. Most of the events took place around the LA Coliseum, which is right next to downtown. Traffic from people going to see the games was supposed to shut down the entire city – this time for two weeks.

The fact that this didn’t happened has been referred to as the “LA Traffic Miracle“.

Why were the Olympic weeks actually one of the smoothest traffic periods in memory? Because people changed their behaviour:

But The Times noted back in 1985 that it wasn’t exactly a miracle: ” [It was] no fluke but resulted to a large degree from employer policies during the Games (23% of major employers surveyed used staggered shifts; 33% permitted flextime).”

In both cases, the prediction that was actually being made was:

This event will be catastrophic, if people act as they normally do.

The problem is that no one ever says the second part out lout – it’s just assumed.

We run into this problem a lot when innovating. We introduce a new idea, and people resist it, because it will only work if they change their behaviour. This is a key innovation challenge – how can we make the behaviour change easier?

Scaring people is one method, as they did with Carmageddon and the 1984 Olympics.

However, this approach often only has short-term success.

The other approach that works is to create clear value for people with your innovation.

Peoples’ capacity to effect large-scale change is often surprising. Major changes in behaviour are often shocking, because we always assume that change is hard. However, if you create value for people with innovation, you can make it easier for them to change.

Once again, change will occur, and it won’t be the end of the world.

(photo from the LA Times)

2 Comments

What’s Your Skype?

Posted by Tim in business models, complex systems, evolving economic entities on 4 July 2011

In his latest book Macrowikinomics, Don Tapscott (along with Anthony Williams) argues that there are two types of business models in existence today: those that have been disrupted by the internet, and those that are waiting to be disrupted by the internet (summarised in an article from the Wall Street Journal).

I’m not 100% convinced that this is true, but I think that there is great value in acting as though it is.

Here’s an example – take a look at this graph (from TeleGeography):

It shows the annual growth in minutes of international phone calls made over regular phone lines, versus those made on Skype.

Skype was founded in 2003, and almost instantly it was heralded as the death of long distance. It hasn’t been yet (although it has been in my household). However, the graph shows that it took close to eight years for Skype to really start biting into the market share of the traditional telcos.

Like I’ve said before, it takes a lot longer for a new idea to spread than we expect. What happens while this delayed diffusion takes place?

First, there are a lot of inflated claims made about the impact of the new idea. Then it takes longer to diffuse than expected, and everyone that should be worried about it is able to decide that it isn’t a threat after all. Eventually, the new idea either dies out (think Friendfeed), or takes off (think Skype) – but usually much later than expected. When these new ideas do take off, they often seem to take by surprise those operating business models that are disrupted by these ideas.

That’s why I think that it is very important for you to think about your business model, and ask: What’s my Skype?

Thinking about this question now will give you a few advantages:

Planning for disruption means it won’t take you by surprise when it arrives: by giving some thought to what might disrupt your current business model, it gives you some time to react to these threats.
Even if you don’t pick the exact idea that disrupts your business model, it’s good practice to think about what could do so: you might not be able to pick the exact idea that will disrupt your current business model. However, giving some thought to what could will help you be flexible in planning for change, and for thinking about scenarios that will help you cope with this change.
Building this flexible thinking will help you maintain a balanced innovation portfolio: to be successful over the long term, you need to divide your innovation efforts across both ideas that will make you better at what you’re currently doing, and ideas that will disrupt whatever you’re currently doing. You don’t need to invest equal amounts into both types of innovation, but you need to be thinking about both.

Your Skype may not be out there yet. If not, great – you can work on inventing it yourself. If so, thinking about it now, and thinking about how you can respond to the changes it will instigate can be extremely valuable.

No Comments