One of the major themes running through the blog is that innovation is a complex process made from different activities. Tim and I have pushed this pretty hard because innovation is often confused with invention (the generation of an idea resulting in a new physical process or thing). There is a lot to be gained from looking at innovation as a value chain because it allows us to see where the weak links are in the innovation process. In nearly all cases, ideas are never the problem. The other advantage of the value chain is that it tells us that we should be measuring different things to get a complete view of performance. Like the story of the blind men and the elephant, measuring only one part of the process will invariably lead us to the wrong conclusion.
Now, a value chain approach to managing and measuring innovation at the firm is relatively straightforward. Surveying people in the business and collecting data about new products takes some time, but it is easy to do. But what happens at the level of industries and nations? How do we measure performance here?
Of course what tends to happen is that the data that are already available and easy to collate get first preference. Things like formally reported R&D expenditure and patents are cheap to access but the obvious downside is that they tell us nothing about how well the innovation process is working. This means that governments who rely on these data to devise policy to support national competitiveness are flying blind.
But it’s actually worse than that. Not only do these measures tell us nothing about the innovation process, they are also an inaccurate measure of innovation!
Now, I have a confession to make. I have used patent counts to measure innovation in my research. In my defense I would like to add that this was in the biotechnology industry where most innovations are patented. However, while writing a review article on innovation networks, I stumbled across a survey that makes me very skeptical about any measure of innovation that relies on patents.
In a survey of 604 large industrial firms in Europe, Arundel and Kabla (1998) found that the average percentage of product innovations that were patented varied from 8% in textiles to 79% in pharmaceuticals, with the average for the entire sample being 36%. With process innovations, the range was 8% in textiles up to 47% in precision instruments. The study also found that larger firms tended to patent more of their innovations and that some firms used secrecy agreements rather than patents.
So, if you are using patents to measure innovation, what does your innovation elephant look like. Well, it’s going to look like the pharmaceutical industry is highly innovative, with precision instruments and machinery a distant third and second. Also, smaller firms won’t look very innovative. On the other hand, the underachievers will be textiles, food and transport. Given this information, it would be tempting to support a pharmaceutical industry and accept that the underachievers will struggle and die.
This is a problem because more detailed surveys of firms in the EU have shown that transport and food are among the fastest growing and most innovative industries. In rich economies like Norway, innovative industries such as fishing, forestry and mining not only produce their own innovations, but they are intense consumers of innovations from other industries. Patents tell us very little about this story.
Little knowledge is a dangerous thing and in measuring and managing innovation, it is deadly.
John,
I agree that measuring innovation by counting patents sounds like a bad idea. I would like to add a couple reasons why I think it is not and suggest a purpose that it can actually serve.
First, a patent count is a metric that poorly reflects innovation in the environment. When I was looking at patents filed by universities in Australia, it turns out that only 4% of patents filed in the last 5 years had a university as the patent filer. It also turns out that the majority of patents filed by universities in Australia are filed by NON-Australian universities. Despite newspaper hoopla, Australian universities file, in fact, a very small number of patents every year. Does this means there is not much innovation going on?
Second, a patent count is -itself- a poor metric. As you rightly point out, It does not cover secrecy or non-disclosure agreements. It also does not cover software, open-access technologies (let’s keep in mind that quite a bit of innovation is funded by government or organisation that require open-access). Moreover, patent count is a quantitative measure that does not (cannot?) reflect the quality of what is being patented. Some organisations patent all their products as a matter of course while others only do so when they think that the product is valuable.
Rather, what counting patents might be useful for, I think, is how successful organisations are in “converting”: pushing their products through the door and in the hands of the community. We are familiar with major products that were never “converted” successfully despite the undeniable innovation involved (e.g. Xerox STAR’s stuff). A patent count might be a way to tell how good organisations are in maximising profit from their innovation profile (be it good or bad.)
Hey,
You can still use patent data to measure innovation, or at least one aspect of innovation. You just have to adjust for industry. The easiest way to do this is to use patent-by-industry data for international comparisons. You could also adjust patent counts for the share of GDP accounted for by ‘big’ patenting industries like ICT and pharma.
It has become very fashionable these days to pooh-pooh patents. We run the risk of throwing the baby out with the bathwater if we aren’t careful.
Suggest people also look into OECD triadic patent families, an attempt to filter out national bias in patenting data.