The Myth of Information Overload

My whole career has been based on managing data and producing information and, as such, I am intrigued with the issue of information overload – or the perception that there is too much information. A former boss of mine referred to me as an information bottom-feeder because I always seemed to have a nugget of information or two that applied to her projects and quests. And I took that as a compliment, no matter how it was actually intended! You see, I’m of the opinion that you can never have enough information – at least regarding those things that you care about.The idea of information overload has been around for a long time, even pre-dating the Internet explosion of the 1990s. Doesn’t it seem somewhat quaint that anyone even imagined the term “information overload” before the advent of the Internet? But here we are in the middle of the second decade of the 21st century – and the concept of information overload seems, perhaps, more relevant than ever. It can be difficult to work your way through the avalanche of information sources that are available to us. And if you couple that with the better technologies at our disposal for managing and sifting through all the data and information, and it can seem overwhelming at times.

But I just cannot bring myself to embrace the idea that there is too much information available. I would rather be overwhelmed with details than have little-to-no data available to use for my decision making and business analyses. Can you imagine ever going back to the pre-Web, pre-Google days of actually having to go to a library or pay large fees to use something like Lexis/Nexus to find relevant information?

Let me put this whole discussion into a different framework. Think about the topic or “thing” about which you are most passionate. It could be a hobby, a sport, or anything, really. Can you ever get enough information about that topic? Does your day brighten when you happen across a tidbit of information you did not know? If you are a fantasy football nut, don’t you dig through every resource available looking at stats and predictions before you pick your team? If you are a collector (coins, stamps, books, etc.), don’t you enjoy scanning through information and tracking down the missing items from your collection? And isn’t it easier than ever with reams and reams of information available on the Internet, loads of items for sale on Amazon and eBay, and great search engines like Google and Bing?

Myself, I am an avid music fan. I have in excess of 6,000 CDs and albums. I scour the web looking for information about my favorites and trying to fill gaps in my collection. There is no such thing as too much information about these areas… to me. Now your hobby, okay, there is too much information out there about that. See what I mean?  It is all a matter of interest and perspective.

Not all information is stored, but that does not make it any less valuable, or indeed, available. Consider the streams of data being emitted by medical devices, stock tickers, weather gauges, and so on. This data is constantly being generated, but is usually not stored. Stream computing is a nascent technology that can be used to battle the ever-growing array of information that is constantly being generated by devices of every type and purpose. Basically, stream computing involves the ingestion of data – structured or unstructured – from arbitrary sources and processing it without necessarily persisting it. Any digitized data is fair game for stream computing. As the data streams it is analyzed and processed in a problem-specific manner. Data that is (1) difficult for humans to interpret easily along with (2) being too voluminous to be stored in a database somewhere, comprises the “sweet spot” for stream computing.

By analyzing large streams of data and looking for trends, patterns, and “interesting” data, stream computing can solve problems that were not practical to solve using traditional computing methods. So we will soon have even more information at our disposal.

The trick is to manage how we deal with the feeling of being overloaded with information? One key is having the ability to discern quickly what data is applicable to your current needs and to rapidly move through that which does not apply. And yet another key is to stop looking when you feel comfortable that you have enough information with which to make an intelligent decision. Richard Saul Wurman introduced the concept of “information anxiety” that somewhat addresses this idea. Information anxiety is “the ever-widening gap between what we understand and what we think we should understand.” I can relate to that… and I’d wager that you can, too.

One important technique you can use to combat information anxiety at your work place is to embrace automated data analytics whenever possible. Relying on software to scan huge amounts of data and identify patterns is easier than trying to do it yourself. Indeed, sometimes you simply cannot do it yourself because of the magnitude of the problem or the volume of the data.

I also recommend that you read a recently published book related to this topic called The Information Diet: A Case for Conscious Consumption by Clay A. Johnson (O’Reilly). This book discusses the impact of the volume of information that is being consumed and its impact on us. It also offers an approach to finding a healthy balance of information consumption – an information diet, if you will.

But the answer is not, and never will be, to eliminate the information. Embrace reality and learn to interact with the information wisely. Indeed, I think that many who claim to suffer from “information overload” do not apply some fundamental methods of streamlining and optimizing their information gathering and analysis processes… or perhaps they just aren’t thinking about it with the proper perspective!