Tim Kastelle

Category: filter

  • Get Better Ideas by Paying Attention

    Could it be that simply paying attention could make us more innovative?

    Probably.

    The basic premise behind On Looking by Alexandra Horowitz is that we miss a lot of important stuff, all the time.  She opens by saying:

    You missed that.  Right now, you are missing the vast majority of what is happening around you.  You are missing events unfolding in your body, in the distance, right in front of you.

    By marshaling your attention to these words, helpfully framed in a distinct border of white, you are ignoring an uthinkably large amount of information that continues to bombard all of your senses…

    This ignorance is useful: indeed, we compliment it and call it concentration.

    The book, which is wonderful, recounts eleven walks that Horowitz takes with others that allow her to see the world differently, to pay attention to the things that she normally misses.

    She sees her neighbourhood in Manhattan through the eyes of her nineteen month old son, her dog. and entomologist, an artist, a typography expert and others (see Maria Popova’s discussion of the book here).  Through these walks, Horowitz learns a great deal about what she normally misses.

    One of her main points is that even in the seemingly simplest settings, there is far more information available than we are able to process.  If we try to process every piece of data in a forest, or on our street, or in our house, we will end up so overwhelmed that we will be paralyzed.  We have always suffered from information overload – it’s not new at all.

    As it turns out, I was missing pretty much everything. After taking the walks described in this book, I would find myself at once alarmed, delighted, and humbled at the limitations of my ordinary looking. My consolation is that this deficiency of mine is quite human. We see, but we do not see: we use our eyes, but our gaze is glancing, frivolously considering its object. We see the signs, but not their meanings. We are not blinded, but we have blinders. My deficiency is one of attention: I simply was not paying close enough attention. Though paying attention seems simple, there are numerous forms of payment. I reckon that every child has been admonished by teacher or parent to “pay attention.” But no one tells you how to do that.

    Horowitz spends the rest of the book trying to figure how to best pay attention..  By walking with her son and her dog, she has to empathize with their views of the world to learn what is capturing their attention.

    Empathy is one part of the equation, and expertise the second.  To think about this, take a look at this picture of a swee waxbill that Nancy took last year in South Africa:

    Swee Waxbill

    First off, look at the depth of field – the bird is in focus, as is the grass it is eating. Everything else is blurry.  This is what attention looks like – we select a small amount of data to process, ignoring the rest.

    Now, think about what people with different expertise can observe in this relatively simply picture.  Birders  like me see the bird.  A botanist can tell you about the grass, and probably about the plants in the background as well.  In addition to that, by looking at the plants, they can tell you what part of the world that is, what kind of climate is there, the characteristics of the soil, and so on.  An entomologist can tell you about what bugs you could expect to find.

    It’s knowledge that lets you see more than just “hey, a bird!”

    In his terrific book Creative Intelligence, Bruce Nussbaum also uses birding as a way to think about innovation:

    As a birder, I’ve learned to look and listen for what shouldn’t be there. What’s unusual. What goes against popular wisdom. It involves a certain amount of domain expertise—I’m certainly a better birder now than I was when I began fifteen years ago. But even if you’ve yet to amass experience in a particular field, you can still improve your chances of spotting the surprises you may not be expecting. For birders, it can mean going to strange and sometimes unsavory places. When I was in Singapore for a design conference, I went birding at a municipal waste treatment facility and found a number of birds—including one black swan. It was a rarity in Singapore and a good find. I was surprised, but not shocked. I was, after all, looking for what was not supposed to be there. Just as good detectives are trained to hear the dog that did not bark—so too are good scientists trained to look, and listen, for what’s not there.

    We can combine empathy and expertise to find opportunities to innovate.  We can do this by seeing things that others miss – by paying attention.

    Part of this is pattern recognition.  It’s the expertise that helps us recognise patterns that might be unusual and easy to miss for others.  This skill becomes even more powerful when we combine expertise in different domains – it is at the edges and borders that the best ideas often lie.

    Part of this is seeing as others do – empathy.  By seeing from their perspective, we can combine this new view with our own skills and expertise to develop innovative ideas.

    We have to filter to make sense of the world – there’s no way around it.  But when we fall into routines, it means that we are overfiltering – we end up missing important and interesting things.  If we can figure out ways to relax our filters through empathy, or to change them through learning, then we have a chance to see something new.

    That’s paying attention.  The reward we might get for doing this is seeing a great new idea.

     

    Enhanced by Zemanta
  • The Attacker’s Dilemma

    The Attacker’s Dilemma

    Some of you might remember how hard it was to search the internet in the mid-90s. Because search sites were mostly hand-curated, it was often diabolically hard to find even basic information.

    I still remember trying to find the official site for the 1996 Olympics in Atlanta. It took about an hour, and I used at least three different search sites (yahoo, lycos & alta vista). None of them retunred a direct link to the official site, and it took forever to find a site writing about the Olympics that included a link.

    It’s a wonder that we got anything done online back then.

    So when Google arrived with it’s search algorithm, it was able to overtake the most popular site on the internet at the time (yahoo) almost instantly. That’s because if you searched for a phrase like “Olympics official site”, it tended to return the official Olympic site in the top spot.

    That was much more than a 10X performance improvement, and that’s why Google was able to take on the giants of the internet, in their areas of strength, and still win.

    So what will it take to knock off Google?

    That is something that Microsoft has been trying to do with Bing. A couple of weeks ago they launched the Bing It On challenge, which enabled you to enter search terms and compare the results from Google & Bing side by side. Then you picked which results you thought were better.

    Here are the results from mine (sorry for the very egocentric searches! It seemed safest to stick with topics that I know well):

    You can see that I preferred Google’s results to Bing’s in 3 out of the 5 searches. But the big question is: even if I had preferred Bing in 5 out of 5, would I switch?

    The answer is: almost certainly not.

    You probably wouldn’t either.

    There are all kinds of reasons for this, including:

    • The differences are tiny. On most of these searches, 7 or 8 of the top 10 ten sites returned were the same across both. So when I picked one over the other, it tended to be because I liked the 9th result on one search better than on the other. This difference is trivial. Even if I preferred Bing on all 5 searches, a 10% performance improvement usually isn’t enough to justify a change, because:
    • Changing search engines changes your workflow. Google is strongly integrated into my workflow. I use Google Chrome as a browser, I read my RSS feeds on Google Reader, get email through gmail, etc. If I change search engines, I end up needing to change all of these at well. Even if I don’t, they won’t perform as well. So if I’m going to disrupt my entire workflow, I need a lot more than a 10-20% improvement in basic search results.
    • The search problem has been solved. This is the big one. I had a big search problem in 1996. Now I don’t. The search problem has been solved – at least the basic one has. I can see ways in which we can still improve search – but a slightly improved algorithm doesn’t address these ideas.

    All of this adds up to the Attacker’s Dilemma. And that is: unless you bring a major performance improvement, there is no point in directly attacking a strong incumbent in their area of strength.

    You need to find a niche that isn’t being served. You need to find area where you can build a learning advantage. The actions you take when you are entering a market are quite different from those that you take when you are building one.

    The Bing it One challenge would have been great in 1998. This would have been a great promotion back when the algorithmic search market was still being built.

    Now? Not so much.

  • The Problem With Google

    The Problem With Google

    The great thing about Google is that it gives you exactly what you want. The problem with Google is that it gives you exactly what you want.

    That’s what Todd Lohenry said to me a couple days ago when we were talking about managing knowledge, how to build expertise, and how to be recognised as an expert – three things he’s thought about a lot.  And he’s got a point.

    He has developed a method for handling a lot of incoming information that he calls the e1evation workflow.  This is a clever system for finding, processing and sharing high-quality information.

    The issue that he’s grappling with though, is how do you identify the areas that you need to know about?  The problem is that search engines give you precisely what you ask for – and only that.  So how can you tell if there is an important area of knowledge that relates to your specialty about which you’re unaware? A search won’t tell you.

    There is a parallel here with the concept of T-Shaped skills – something that Ralph Ohr brought up in his first guest post here.

    He quoted Nicholas Donofrio, who said:

    The kind of people who will be best able to seize these opportunities are those I call “T-shaped” as opposed to “I-shaped.” I-shaped people have great credentials, great educations, and deep knowledge – deep but narrow. The geniuses who win Nobel prizes are “I-shaped,” as are most of the best engineers and scientists. But the revolutionaries who have driven most recent innovation and who will drive nearly all of it in the future are “T-shaped.” That is, they have their specialties – areas of deep expertise – but on top of that they boast a solid breadth, an umbrella if you will, of wide-ranging knowledge and interests. It is the ability to work in an interdisciplinary fashion and to see how different ideas, sectors, people, and markets connect. Natural-born “T’s are perhaps rare, but I believe people can be trained to be T-shaped. One problem is that our educational system is still intent on training more “I’s. We need to change that.

    Here is the way that Prabhakar Gopalan pictures it:

    Google is an I-Shaped search engine – it goes very deep, but it isn’t broad.  How do we get the breadth we need here?

    This is another people-tech interaction issue.  You can map the history of search using the matrix I’ve talked about before – by analysing where the intelligence sits in the system:

     

    When the web started, there wasn’t much in the way of search.  There was no systematic method for finding things, and no technology to support looking for stuff either – dumb tech and dumb people.

    Then we had the first tools for searching the web – things like yahoo and lycos.  These indices were compiled by hand, and put into categories by people using judgement.  This was smart people with dumb tech.  The problem with this approach is that it works fine when there are 32,000 websites to catalog, but it breaks when there are millions.

    Enter google.  With this, all of the intelligence is placed in the tech – the action is in the algorithm.  The early search sites were hyphens, and the algorithm-driven ones are i-shaped.  It’s possible that algorithms may eventually solve this problem. Bottlenose is a tool that is moving in this direction.  And of course, once we hit the singularity, then it’s not an issue!  Terri Griffith pointed me to Avogadro Corp, a book that outlines how this could happen.  I’m still not sure if the outcome is really cool, or terrifying…

    I think that to get to t-shaped search, we need to combine algorithmic filtering with some form of judgement-based filtering.  That’s what gets us up into the corner with smart people using smart technology.  Right now, we don’t know what will do the job – but solving this will trigger more disruption, and probably will make a bunch of money too.

    Of course, it’s probably not enough to target a big disruption.  If it were me, I’d be looking at problems that require t-shaped data to solve.  Those are the most interesting ones around these days anyway.The solution will most likely come from someone experimenting around the edges.  Larry Page and Sergei Brin weren’t trying to destroy yahoo and lycos when they started out – they just wanted to catalog books in libraries.

    Solve a similar problem now, and you might solve the problem with Google.

     

    Enhanced by Zemanta

  • How to Improve Your Information Diet with Better Filtering

    There’s so much information around these days, how can we possibly deal with it all? Many of us are overwhelmed just by our email, so when you add in everything else (TV, books, newspapers, blogs, twitter, facebook, etc.), it’s just too much.

    And yet, we’ve always been faced with more information than we’re capable of processing ourselves. If we’ve been in a state of information overload for centuries, is that really the problem?

    In his book The Information Diet: A Case for Conscious ConsumptionClay Johnson argues that the answer is “No!”

    The book is very good. Here is how Johnson makes the case on the website to support it:

    The problem of “information overload” isn’t particularly new — it’s a problem older than our nation, and a problem that still has not been solved. You know why? Because the problem isn’t information overload. Trying to solve an overload problem is impossible. Are we obese because we have “food overload”?. Of course not— it is not the poor (mostly) inanimate food’s fault. We’re obese because we have “food overconsumption” and while the abundance may make it easy to be fat, it isn’t as though the food’s mere existence is making us fat. Simply putting Paris Hilton in a room full of Tyson Anytizers won’t make her gain weight.

    The right question to ask is: How do we deal with information overconsumption?

    We’ve gotten dealing with food overconsumption down to a practical science. While there’s 55,000 diet books available to us, they all boil down to the same thing: eat less, exercise more. I think that if you’re interested in improving your focus, productivity, and stress levels, building conscious information consumption and attention fitness into your daily routine seems [eminently] worthwhile.

    To do this, we have to filter.

    I’ve run across two great examples of very different types of filtering recently. The first is my current favourite iPad app – Zite.

    There are at least five forms of filtering, and Zite is a great example of algorithmic filtering. Here is how it works in a picture (made by DDO):

    The explanation in words is pretty interesting, so I recommend reading the full description on the company blog. Here is part of what they say:

    There are tens of billions of web pages out there and more than two million terabytes of text, images and more are created every hour. So, where in this deluge does Zite start looking for what’s interesting to you? Zite observes what’s happening around the social web, because the community, in aggregate, creates a strong signal for what’s interesting. User-generated content, sharing, commenting and bookmarking have overtaken email and web pages in sheer volume of data created and total time spent online – eMarketer expects 115 million people in the U.S. to be creating content by 2013. What’s important is either happening on, or reported through, social media. What’s more, mining the social web makes it possible to personalize content at the moment you start using Zite for the first time .

    The app then tracks what you respond to, and it customises its recommendations so that they are increasingly personalised over time. I’ve been using the app for a month now, and it is incredibly useful.

    It doesn’t replace twitter or my RSS feed, but it is a great addition to them.

    It definitely provides high quality information.

    However, there is a limit to the power of algorithms. There isn’t an algorithm in the world that will tell you first to read Changing the Game by Roger Martin, then A Fine Balance by Rohinton Mistry.

    But that’s exactly what Nilofer Merchant did in the list of books that she put together for the latest TED Conference.

    Maria Popova also put together a great set of books for TED too.

    This is a form of judgement-based filtering – Expert Filtering.

    This form of filtering is based on a few key components including judgement, reputation, and trust. Judgement-based filtering is where you get the out-of-the-blue recommendations – connections that are too obscure or too creative for an algorithm to come up with.

    The upshot for me is that because I trust Nilofer’s judgement, I’ll read Mistry’s book.

    In setting up your information diet, using effective filters is essential. Algorithmic filtering is great. However, to maintain a balanced information diet, you also need to include some judgement-based filtering.

    Howard Rheingold calls building your aggregation and filtering routines Infotention. Since attention is becoming one of the scarcest commodities these days, you need to spend yours wisely.

    If you do, you will consume better information, and your information diet will be a lot healthier.

  • e-Books: Another Innovation Diffusion Problem

    Regular readers of this blog have probably realised that I read a fair number of books. The good news for book publishers is that I buy nearly all of these books (and the others are either legally free, or from the library). The interesting news for book publishers is that the format I use is changing.

    I’m reading a lot more on my iPad. I still prefer real books for things I use in my research – they just fit with my workflow better. But for things that I just read straight through from start to finish, e-books are great.

    I’m not too unusual here either. Check out the great series from Findings called How We Will Read. Reading is changing, and books are changing. And it’s not really one driving the other – they’re co-evolving.

    Which leads to an important question: where does this leave publishers?

    One way or another, they need to adapt to e-books. However, they are having innovation diffusion curve problems, just like Kodak. Frédéric Filloux shows why in an interesting post on e-books:

    I’m an ebook convert. … This leads to this thought about the coming ebook disruption: We’ve seen nothing yet. Eighteen months ago, I was asked to run an ebooks roundtable for the Forum d’Avignon (an ultra-elitist cultural gathering judiciously set in the Palais des Papes). Preparing for the event, I visited most of the French publishers and came to realize how blind they were to the looming earthquake. They viewed their ability to line-up great authors as a seawall against the digital tsunami. In their minds, they might, at some point, have to make a deal with Amazon or Apple in order to channel digital distribution of their oeuvres to geeks like me. But the bulk of their production would sagely remain stacked on bookstores shelves. Too many publishing industry professionals still hope for a soft transition.

    You can see this diagram in Filloux’s post. eBooks have actually been around for quite a while now, and there has been quite a bit of hype about them. The hype leads people to expect their adoption curve to look like curve A.

    But no innovations diffuse like curve A. They follow an S-Curve. And the time it takes to get to the tipping point is much longer than we expect.

    The length of time X lulls incumbents into a false sense of security. They start to believe that the disruptive innovation will diffuse along curve C (I think Jonathan Franzen is probably the only person left who thinks that e-books are a fad that will follow curve B).

    This appears to be exactly how Filloux’s publishers are thinking.

    The result of this is that if you believe that the transition will be slow, you will not experiment enough. To the publishers, this makes good financial sense. Why imperil revenue from physical books if you have time to manage the transition in an orderly manner?

    The problem here is that we need publishers, so it doesn’t do booklovers much good if they put themselves out of business. The excellent author Cat Valente explains all of the services that publishers provide to authors. Importantly, this defense of publishing comes from an author who has actually run a couple of very successful digital experiments – she’s no luddite.

    Craig Mod has written a must-read essay on the future of books. He says:

    In reality, the book worth considering consists only of relationships. Relationships between ideas and recipients. Between writer and reader. Between readers and other readers — all as writ over time.
    …
    For those of us looking to shape the future of books and publishing, where do we begin? Simply, these are our truths:

    The way books are written has changed.
    The canvas for books has changed.
    The post-published life of a book has changed.

    To think about the future of the book is to understand the links between these changes. To think about the future of the book is to think about the future of all content. So intertwined are our words and images and platforms, that to consider individual parts of the publishing process in isolation is to miss transformative connections.

    For me, this gets at the critical issue here. The publishers that Filloux interviewed are thinking of books as physical things. But they are not. As Mod says, they are connections between ideas.

    Thinking about them this way gives some guidance on how to proceed. The primary point of a book has always been to connect people with ideas. Digital distribution gives us opportunities to do this in new ways.

    One of the critical roles of publishers has always been filtering. If you break down the publishing value chain, this is where a huge amount of value is created. We can’t read everything. We can’t even read the merest scintilla of everything.

    So we must filter.

    This is why it is critical to think about aggregating, filtering and connecting as key parts of your business model. These are the things that publishers need to focus on. If they can continue to provide these services, then they will stick around.

    The thing that concerns me is that they’re not doing this, and they’re not experimenting. That leaves it to the authors, if they’re motivated to try new things like Valente has. Or it leaves it to the smaller players in the market, like O’Reilly – who are experimenting like crazy.

    If you misunderstand the nature of the innovation diffusion curve, you will not respond to changes in time. We saw it with Kodak, and we might be seeing it now with book publishers.

    The publishers have to get it figured out, though, because one way or another, I need to keep getting my book fix!

  • There’s No Such Thing as Information Overload

    The size of your inbox or your RSS feed or your twitter stream might all argue otherwise, but there’s no such thing as information overload.

    Or, at least, if there is, it’s not new. Check this out:

    As long as the centuries continue to unfold, the number of books will grow continually, and one can predict that a time will come when it will be almost as difficult to learn anything from books as from the direct study of the whole universe. It will be almost as convenient to search for some bit of truth concealed in nature as it will be to find it hidden away in an immense multitude of bound volumes.

    That was Denis Diderot in “Encyclopedie”, back in 1755. 1755!

    The problems that we have with information isn’t that there’s too much of it – there has always been too much. Rather, there are two related problems with information: how do we filter out information that doesn’t help us, and how do we find information that we need.

    Jorge Luis Borges touches on this in his story The Library of Babel. You should go read it here since everyone should be reading more Borges. The story is short, but packed with ideas. The library has an infinite number of rooms, all filled with books. Each book is the same length, with randomly assembled letters. The Men of the Library spend their lives wandering the shelves, reading the books. Since the library is infinite, it must contain all books ever written (and all that will be written!), but since the library is infinite, the odds of coming across even one sentence that makes sense are exceedingly small.

    It is useless to observe that the best volume of the many hexagons under my administration is entitled The Combed Thunderclap and another The Plaster Cramp and another Axaxaxas mlö. These phrases, at first glance incoherent, can no doubt be justified in a cryptographical or allegorical manner; such a justification is verbal and, ex hypothesi, already figures in the Library. I cannot combine some characters

    dhcmrlchtdj

    which the divine Library has not foreseen and which in one of its secret tongues do not contain a terrible meaning. No one can articulate a syllable which is not filled with tenderness and fear, which is not, in one of these languages, the powerful name of a god. To speak is to fall into tautology. This wordy and useless epistle already exists in one of the thirty volumes of the five shelves of one of the innumerable hexagons — and its refutation as well. (An n number of possible languages use the same vocabulary; in some of them, the symbol library allows the correct definition a ubiquitous and lasting system of hexagonal galleries, but library is bread or pyramid or anything else, and these seven words which define it have another value. You who read me, are You sure of understanding my language?)

    What do you do when you are faced with all of the information in the world? To make any sense of it, you have to find the information that is useful to you. So we filter.

    As Borges suggests, each piece of information means something to someone, even if it’s gibberish to us. We need to knock out the stuff that’s gibberish. So we find ways to ignore information, by saying things like “Twitter is just 100 million people talking about what they ate for lunch, so why would I waste my time with that?” I do this by ignoring TV (unless I can find a hockey game on). Everyone makes choices about what they should be paying attention to.

    The key to dealing with information is to be conscious of the choices that you’re making, and to develop a strategy or a set of routines for handling it. Howard Rheingold has created an outstanding set of resources for his classes on Mind Amplifiers and Infotention. Start with those to develop a filtering strategy.

    We’ve always had too much information to handle, and we’ve always dealt with it by developing routines. The real difference now is not that there’s so much more information, it’s that we don’t have good routines to go with the new channels that the information is taking to get to us.

    The danger in thinking that we have too much information is that we’ll start missing out on innovation opportunities. After all, the creative part of innovation is about making novel connections between ideas. So we actually have to seek out information that is a bit out of the ordinary (see the end of this post for some techniques for doing this).

    If you think that the problem is information overload, then this will seem completely counterintuitive. That’s why it’s a dangerous idea – if you take it seriously, it makes it much harder to innovate.

    That’s why I say that there’s no such thing as information overload. Even if that’s not strictly true, we’re better off acting as though that’s the case.

  • Bad Filtering Kills Businesses

    If your business model is based on information, and whose isn’t these days, then you need to be able to aggregate, filter and connect. While reflecting on the death of Borders Books, I thought of three stories of filtering in retail.

    First Story: Tower Records

    In the mid-80s, I went in to the Tower Records in Tacoma, looking for Stop Pretending, the new record by the Pandoras. I figured my odds of finding it were high, since there was a big promo display for the record up on the wall.

    I went over to the “Rock – Misc P” and flicked through the records. No luck.

    I went up to the counter and asked the clerk if they had it. He said no – they’d gotten one copy, and another guy that worked at Tower had bought it. I asked them why they had the display on the wall, and he told me that the guy that bought the record really liked it, so he made the display.

    Then I asked if another copy was coming in. No. Why? Because for records from independent labels, the buying policy was to send one copy to each store. If they needed more than one copy, then it had to be special ordered.

    There are five forms of filtering, and this is an example heuristic filtering.

    Heuristic filtering is rules-based, and this is a great example of a dumb mechanical process. It’s dumb because there’s no learning (“hey, people in Tacoma seem to like the Pandoras, send them more copies of the record”).

    This approach worked fine as long as Tower was still the biggest aggregator around. The boycott of Tower that I started in response to this didn’t really seem to hurt them, even though I bought a LOT of records back then.

    However, as soon as a bigger aggregator came along – various internet-based options – the Tower business model was toast.

    People say that the internet killed Tower Records, but I think it was killed by bad filtering.

    Second Story: Borders Books

    In the mid-90s, I bought Science as a Process by David Hull, which became one of my all-time favourite non-fiction books. I bought it at the Borders in Westwood, which at the time had a superb science section. Back then, buying was decentralized to each store. So the Westwood Borders, just down the road from UCLA, had a significantly different selection from the Studio City Borders, and every other Borders in LA at the time.

    This was expert filtering. Each buyer knew the kind of people that were shopping in his or her store, and they stocked books appropriate to that market.

    Unlike Barnes & Noble, which appeared to use heuristics to stock their stores, each Borders was unique.

    When Borders came to Australia and New Zealand around 2000, they had individual store buyers then too, so each store was still unique.

    After the chain got sold, the individual buyers disappeared – replaced by a central buyer. This was done in response to the threat of online booksellers. The only way to improve efficiency was to cut down on staffing costs.

    So Borders went to dumb heuristic filtering.

    And now they’re gone too – also killed by bad filtering.

    Third Story: Pulp Fiction Bookshop

    I while ago I was browsing through the shelves at Pulp Fiction Bookshop here in Brisbane. They specialize in Science Fiction, Fantasy and Mysteries. Their selection in these areas is among the best I’ve ever seen.

    A guy walked into the shop and went straight up to the counter. He said “My wife really likes Iain Rankin and Donna Leone. I want to get her a birthday present – is there a similar author that you can recommend?” The owner of the shop said “Yes, there’s a South African author (whose name I didn’t catch) that’s writing really good mysteries, but no one has heard of him (or her) yet. Try that.” The guy bought two books by that author, and left, looking pretty happy.

    That’s expert filtering – both in terms of stocking the store and in terms of helping customers.

    Even though people can buy books on the internet, and the Australian dollar is really strong, and the parallel importing laws here making it nearly impossible to sell books successfully, Pulp Fiction seems to be doing pretty well.

    They’re doing well, because they filter well.

    Conclusions

    Simply calling these filtering problems is probably too simplistic. And yet, bad filtering definitely played a role in the death of Tower and Borders. Both of them were pretty good at aggregating. Borders was pretty good at using expert filtering to connect people with books they might like in-store, while Tower was less consistent in this area. For a while, Borders was pretty good at filtering, and Tower was always fairly bad at it.

    The problems started when the internet killed their aggregation advantages. This caused Borders to do away with the one thing that actually made them distinctive – their expert filtering. Expert filtering is something that Tower never had.

    Neither store ever was able to connect people up with products in the way that Pulp Fiction does. This type of expert filtering & connecting is better even than the algorithmic filtering you get at Amazon or iTunes.

    The problem is that it doesn’t scale. So it’s hard to have a Borders-sized bookshop with great expert filtering. It’s easier if you specialize in something, as Pulp Fiction does.

    To succeed in an information-based business, you must be good at aggregating, filtering and connecting information. And you have to be able to do all three. The stories of Tower and Borders show you how bad filtering can kill a business.

  • The Innovation Filter Bubble

    Here is a must-watch video from Eli Pariser discussing some of the themes from his new book The Filter Bubble (reviewed well here by Cory Doctorow). It’s only 9 minutes, and it is well worth your time:

    Pariser’s main point is that the primary filters on the internet these days are algorithmic, and that these filters have a strong tendency to only expose you to viewpoints that reinforce whatever you currently think.

    This is very important for how we use the internet, but it also has huge implications for innovation as well. I think that many of us work inside of an innovation filter bubble, and that this makes it much harder for us to innovate.

    What is an innovation filter bubble? It is all of the habits and routines that prevent us from being exposed to novel ideas and new points of view. Some of these include:

    • The internet filters that Pariser discusses: much of our information comes from the web these days, and as he shows in the talk, this can lead to only running across viewpoints that reinforce our own.
    • Who we spend time with: do you always eat lunch with the same people? Or alone? Spending time with people that you know well is great (and we often don’t do enough of this), but at the same time, we usually spend time with these people because they think a lot like us.
    • Silos within our organisations: is where you work organised by specialty? Most organisations are. This has benefits in that it makes it easier to find the information that is most relevant to our jobs more easily. Still, this is another form of filtering that reinforces current views.

    The end result of the filtering that occurs through these routines is that the information that we are exposed to can become too restricted. As Pariser argues, these filters make it easy to find information that is relevant to the task at hand – and that is what makes them useful. But does access to information that is highly relevant to the task at hand help innovation? Probably not.

    Innovation is based on connecting ideas in novel and interesting ways. To do this, we need to run across information that is more than just relevant. We also need information that is important, uncomfortable, challenging, and that reflects other points of view.

    We have to make a conscious effort to break out of our innovation filter bubble.

    How can we do this? Here are some ideas:

    • Actively seek out new and different viewpoints: Ethan Zuckerman has some great ideas about how to do this on the internet. But also do it in your day to day activities. Once a week have lunch or a coffee with someone with a completely different background, area of expertise, or view of life. Go out and find those challenging ideas somewhere.
    • Use filters based on expertise instead of algorithms: as I’ve discussed before, there are at least five forms of filtering. The algorithmic filters are more efficient, but they fall prey to the problems outlined by Pariser. Make better use of expertise-based filters. You can do this by accessing people with expertise in different areas, and also by building broad networks and activating them to help you generate new ideas. Algorithms are great, but you still need some people-based filtering as well.
    • Encourage enhanced serendipity: this is an idea from Ross Dawson, and it’s also discussed in The Power of Pull. It involves building your networks (both online and personal) to maximize your exposure to new ideas and novel viewpoints. One of my personal rules in this area is that on twitter I always follow people that follow me if they come from outside of Australia, North America or Europe. And I follow nearly all of the people that run into from Europe too. This is one way to run across new viewpoints.

    In order to innovate we have to generate new connections between ideas. We can’t do this if all of our routines only expose us to viewpoints that are very similar to our own.

    To innovate more effectively, we have to break out of the innovation filter bubble.

  • Should You Only Execute Good Ideas?

    The obvious answer to the question in the title is yes, right?

    But I’m not so sure that this answer is correct.

    I thought of this because of an experiment that Martijn Linssen tried in January – writing one blog post a day for the whole month. In the comments the idea came up that if you set volume goals like this when you blog, then your quality will inevitably suffer.

    Julien Smith made the same point quite forcefully in a discussion of how to increase the impact of your blog:

    If you’re anything like me, you write your posts, and your titles, with yourself as audience. This results in a majority of posts which rank 6, 7, or 8/10 with the outside world.

    Last week, if I didn’t have a 10/10 post, I didn’t publish at all. This resulted in three posts instead of 5-7, and many more subscribers than I’ve gotten in previous weeks combined.

    This is important because we often see the same thing when people talk innovation – a lot of the time it is assumed that every idea that we try should be successful.

    There is a deep flaw in this thinking – it assumes that we know in advance which ideas will work. But it’s impossible to know in advance which ideas will work.

    Sometimes I have a great idea for a post, which I just can’t execute very well. Other times I have a throwaway idea that I execute nicely. The simple fact of the matter is that I don’t know what people are going think are a 10/10 post before I publish it – and no one else does either.

    If every idea that you try is successful, this is a sure sign that you’re not trying enough ideas.

    Check out this video:

    What is being creative? from Kristian Ulrich Larsen on Vimeo.

    Once you get over how cool the phone is, pay attention to the points they make at the end:

    • Stay away from the direct path.
    • Take risks.
    • Don’t be afraid to make mistakes. Because it’s from the mistakes that really interesting things happen.

    Selecting ideas is a critical part of the innovation process. However, it’s only in executing ideas that their true value is discovered.

    Idea selection is important because we all have limited resources. If you write a blog, the limit is usually time. If you run an organisation, the limit can be time as well, or money, or skill.

    Nevertheless, the correct answer to the question of how many ideas you should execute is not: “only the good ones.” It is “as many as you can afford to try.”

    I’m looking forward to finding out if this was a good idea or not…

  • The Problem of Filters and Silos

    Here is a quote from Why The West Rules – For Now by Ian Morris – explaining some of the issues with the inter-disciplinary approach he has taken in writing the book:

    This courts all kinds of dangers (superficiality, disciplinary bias, and just general error). I will never have the same subtle grasp of Chinese culture as someone who has spent a lifetime reading medieval manuscripts, or be as up-to-date on human evolution as a geneticist (I am told that the journal Science updates its website on average every thirteen seconds; while typing this sentence I have probably fallen behind again). But on the otehr hand, those who stay within the boundaries of their own disciplines will never see the big picture.

    And therein lies the problem. Science updates every thirteen seconds – it’s impossible to keep up with that much new knowledge. Our only hope is to filter the flow somehow.

    One way that we do this is by working in silos – our silo becomes the filter. Everything from outside our area of specialty gets ignored.
    silo

    This helps with the information overload problem, but it creates a new one. Big ideas come at the edge of specialisations, and, particularly, at the intersections. To come up with big ideas you need to be outside of the core (see this post for some ideas on how to do this).

    This is another tension in innovation – the need to be both in the core and at the edge. As usual, the best answer is to change this from an either/or into a both/and.

    Both/and solutions are hard to execute. You have to accommodate yourself to conflicting intellectual demands, and you have to be comfortable with a relatively high level of uncertainty. That’s what makes innovation both challenging and rewarding.

    (photo from flickr/contemplative imaging under a Creative Commons License)

  • Succeed by Failing

    “If you want to succeed, double your failure rate.”

    -Thomas Watson, IBM

    That’s a pretty succinct way to say make a point that I was trying to get a couple of weeks ago.

    The key point here is that you can fail at different levels. I’ve talked before about a taxonomy of economic failure. We can actually think of failure as a hierarchy that looks something like this:

    • System failure (the collapse of communism)
    • System component failure (stock market crashes)
    • Major firm failure (Enron going out of business)
    • Start-up failure (pets.com going out of business)
    • Product failure (New Coke tanking)
    • Idea failure (Apple Navigator prototyped but never launched)

    As you go down that list, failure gets less expensive. When I talk about tolerating failure, I’m talking about trying to set up systems that encourage cheap fast failure. This is usually at the level of ideas.

    I think that this is the point that Watson was making as well. He’s not advocating big, expensive, public failure. He was advocating quick, cheap experiments.
    Electronic flashbar prototype

    We need to push our failures down that list, so that we are testing ideas and finding the ones that don’t work when they are still ideas, rather than things. One of the key skills in this is prototyping – figuring out a small-scale way to test your idea.

    As Diego Rodriguez says, anything can be prototyped, and you can prototype with anything.

    (photo from flickr/polapix under a Creative Commons License)

  • Why New Ideas Can be Bad

    Innovation is about more than just having great ideas – a point we’ve made here repeatedly. To innovate, you also have to execute ideas relentlessly. For many people, this is actually the hard part. I’m currently reading Making Ideas Happen by Scott Belsky, and it has some of the most sensible advice on this topic that I’ve run across.

    Here is a talk in which Belsky outlines some of the key points from the first part of the book:

    Scott Belsky: How to Avoid the Idea Generation Trap from 99% on Vimeo.

    The book supports a couple of points that we’ve made here before. One is that idea execution is essential. People are idea-generation machines. Belsky started the 99% Conference based on the old Edison quote – that invention is 1% inspiration and 99% perspiration. The issue is that if you look at the books, tips and consultants that address this topic, it would sure look like the equation is reversed. Given that, it’s great to see someone trying to address the 99%.

    The second issue that he addresses nicely is the idea of constraints – he correctly points out that we’re more creative when we have to deal with constraints. One of the key reasons is that constraints make us focus, which is a critical step in executing ideas. Here’s how Belsky puts it:

    Constraints serve as kindling for execution. When you’re not given constraints, you must seek them. You can start with the resources that are scarce – often time, money and energy (manpower). Also, by further defining the problem you are solving, you will come across certain limitations that are helpful constraints. As you find them, try to better understand them.

    Brilliant creative minds become more focused and actionable when the realm of possibilities is defined and, to some extent, restricted. …

    Despite your natural tendency to thrive on untethered creativity, you must recognize and harness constraints. And it is ultimately your responsibility to seek constraints when they are not given to you.

    These ideas are pulled together with the graphic that shows the project plateau (which he discusses in this post from Smashing Magazine):

    This shows the levels of excitement and energy that we have for ideas over time. When they are new, we have lots of both. However, once we settle into trying to make the idea real, the levels of both excitement and energy go down – it starts to feel more like work. How do we respond to this?

    According to Belsky, the natural response is to look for the excitement of a new idea again – and succumbing to this temptation is deadly. If you do, you’ll end up with a lot of partially-executed ideas, which is functionally equivalent to having, well, no ideas at all.

    The book (and the supporting website) has a lot of ideas for how to work through this. The main idea is to break down ideas (and the projects that result from them) down into action steps, and then focus on getting these done. It is easier to get big projects done when you are able to build momentum by achieving small steps on them on a near-constant basis.

    In some ways this is similar to Dave Allen’s Get Things Done approach, but Belsky’s is more oriented to people doing creative work. Consequently, for me at least, this approach seems more useful. And since innovation is definitely creative, Making Ideas Happen will probably be useful for most people trying to improve innovation.