A New View of Data

Data are now the basic raw material that will drive progress, innovation and profit. We need to thoroughly grasp the end products that will be created from them.

An entire special edition of Newsweek (December 2005/February 2006) was dedicated to the “Knowledge Revolution.” The message is that those with the most knowledge will be the dominate players in business, politics and science in this century. Newsweek proposes that “victory will go to the smartest” and there will no awards for being the dumbest or the less smart.

(Click for larger image)

Several efforts have been made to represent this view of data, showing a path moving data as a raw material, transformed into knowledge and ultimately into wisdom.
While I think this is a noble portrayal, it is limited and perhaps not as well thought out as it might be. The path may be accurate for the data themselves (data to knowledge), but it does not show data getting to the point where there is a level of understanding to make decisions or generate action.

Why? First, the achievement of wisdom is not a trivial undertaking. It takes years, decades or a lifetime to achieve wisdom - and it is a fact that very few of us are wise. So a predictive, repetitive flow cannot be established to produce wisdom. Second, knowledge by itself is not an end - possessing knowledge does not guarantee that you can do anything with it. There are probably some classic examples of professors (maybe you knew some at school) who certainly had the knowledge, but just couldn’t apply it. So we can establish that wisdom is a sometimes thing and knowledge alone is not enough.

Rather than knowledge or wisdom, a better end product would be understanding. Understanding provides the means to act on knowledge and therefore, create, innovate, build or establish.

Because of today’s rate of change and innovation, we all have a pressing need to stay current. In order to remain viable in this new world, we need a consistent, reliable, accurate and rapid process to move data from raw material to understanding and perhaps, occasionally into wisdom.

But even understanding may not be enough. We may need some predictive modeling to help identify options. It is important to have answers to questions such as, “what are the possible scenarios given what I know now?” or “Are there probabilities I can weigh?” Without modeling, data to understanding is not a complete process. All decisions at any level have several possible outcomes and without considering them, understanding is not complete.

Data, or rather the correct data, has become the new crude oil powering the next information age. Certainly, we would all like to have an endless supply of cheap Texas light sweet crude. But as our expectation and demand for higher quality and more precise data rise, the tolerance for anything less will be pretty low. This new intolerance level will apply to decisions and actions as well. It will also have to include appreciation for intuitively knowing when the data is wrong and making decision based on “gut feel.”

It is obvious that understanding based on the wrong core data will produce the wrong information. (The old “garbage in/garbage out” model.) With the wrong knowledge and the wrong understanding, ultimately the wrong decision will be made, and of course, wisdom will never be achieved.

What is certainty is that with the speed of change and innovation increasing, bad decisions will put you out of the race sooner. The new business problem is, with things moving faster, it will now take less time to win and even less time to lose, and the difference may be just a one or a zero.