"Knowledge in the age of abundance" by D. Weinberger

Kindly written by David Weiberger in the run up to his keynote speech at the Online Information conference in London, 29 November 2005.

Knowledge management arose because too much information is the same thing as noise. The aim of KM became to sift through the dross, finding the genuine nuggets of value.
Thus did KM go wrong from the beginning. The main problem with this line of thought is that it fears abundance. It even gets the basic premise wrong: Too much information isn’t noise. It’s possibilities.

It sounds like noise if you take a narrow view of how we come up with ideas and make decisions. That view looks at the last step of the process, notices that only 0.000001% of the available information was useful, and thus assumes what wasn’t useful got in the way. If it’s noise, the thinking goes, then it disrupts signal.

But does our signal get disrupted? When you’re doing a search on Google, all those millions of hits that are of no use to you can count as noise. But if Google (or your search engine of choice) puts what you need in the top ten, those other hits are “silent noise.” So long as we’re finding what we need, the noise doesn’t get in our way. And we are far better at finding what we need than even Information Retrieval experts imagined ten years ago.

Nevertheless, the existence of all that irrelevant information makes problems. It’s not always easy to find what you need. So, KM imagined that we could find a special class of information, a type of “super information,” that would be to information as information is to data. This super information would be a corporate asset of tremendous value.But knowledge generally doesn’t work that way. Knowledge is not an identifiable special class of information that is a corporate asset. Assets can be identified by looking at their properties: If it’s gold, it yields when you bite it. If it’s a diamond, it scratches glass. Super information, on the other hand, only becomes super within difficult-to-predict circumstances. For example, that the game Doom 3 handles lighting better than does Half Life 2 is noise until the moment someone in the organization needs to find a way to mock up an event that depends on lighting effects. There is no way in advance to determine that the IM spat between two engineers over which game’s lighting is better is knowledge. Knowledge isn’t like a diamond. It’s more like a piece of twisted metal in a junkyard that suddenly takes on value because it happens to match the hole in your roof.

The new abundance has its own logic, its own physics, and its own benefits. But it’s hard for KM to shake off its old impulse to manage that which is unmanaged. You can see this attitude in the comforting phrase “Delivering the right information to the right people at the right time,” a line that is used with surprising frequency by people in the KM industry. “Delivering the right pizzas to the right people at the right time” makes total sense for your local Italian restaurant. But it is highly misleading when it comes to knowledge. It actually gets it backward. For the truly difficult decisions - the ones where you need knowledge not just some facts and stats - the information that is delivered to the decision-maker does not determine the decision. Rather, in making the decision, the person decides which information counts, and how much it counts. Should you open up a new office? Should you agree to the terms of a partnership? Should you change your product focus? These are all questions that are not decided by information so much as by the decision about which information to count. A KM system can’t deliver the right information to the right person because the right person’s main value is in deciding which is the right information. The KM system does its job in part by delivering lots and lots of possibilities and open ways to explore the never-ending sea of ideas and information.

This means that in many circumstances we have to give up the idea that there is such a thing as the best information. In an age of abundance, it becomes easier to get good enough information and harder to get the best. But good information is, well, good enough. We don’t have time to get the absolute best information. Besides, with so much information available, how could we ever know that we’re getting the best information? The concept makes less and less sense.
Besides, knowledge is overrated. In a world of information abundance, much knowledge has been commoditized: Anyone can find out anything by using a search engine. Knowledge has been externalized, just as calculators externalized arithmetic.There’s still plenty to do to become smart, though. We just can no longer be smart by stuffing our heads with information, even if it’s the right information at the right time. Now that it’s so easy to lay our hands on the right information, we’re smart only if we can make sense of what we learn.

How do we do this? We can’t do it alone. We never could, and now life is so complex and there’s so much to know, that solitary smartness is quickly becoming a thing of the past. So we do what we always have: We make sense of information by talking with other people. We need rich relationships with diverse people. We need gnarly conversational networks. We need to get smart together. And talk is how we do it. That’s also how we come up with new ideas. At least as important, it’s how we quickly find out that bad ideas are bad.

Organizations succeed at getting smart through conversation not by managing conversation but by facilitating its unruly growth. You never know who is going to talk with whom and what crazy – or insanely profitable – idea is going to grow out of it.

Then the management instinct inevitably re-asserts itself. The little management devil on our shoulder suggests that we set up a system by which we can “harvest” conversations. In fact that system is already in place. Good ideas get passed around in hallways, emails, weblogs even formal memos. We together are not only the best source of ideas, we’re also the best filter for them. We’ll figure out how to surface the ones worth pursuing. If a KM system wants to give us some help publishing what makes it way through our social, conversational filter, great. But Conversation Management would be a truly counter-productive idea.

The traditional tools of KM all have their place, from advanced search engines, to taxonomies and controlled vocabularies, to skill matching systems, to conceptual analysis, and all the rest. But they are without real value if they aim first at individuals, rather than at building a culture abuzz with talk. They are of negative value if they aim at primarily limiting vision rather than opening possibilities.

Knowledge is the loam of possibility.

By D.Weinberger