In his recent book, The Wisdom of Crowds, James Surowiecki makes a bold assertion that the collective knowledge of a group is more accurate and more consistent than an expert’s knowledge. He runs through a litany of examples, including:
- 18th century county fairs where the group average was closer than nearly all the local experts (farmers) on guessing the weight of a prize ox
- The “Who Wants to be a Millionaire” show, where the seemingly haphazard “poll the audience” was correct more often than the “phone a friend” (expert) option
- Successful companies such as Procter and Gamble and Netflix posting corporate challenges to the Internet, looking to the browsing public for solutions
These and many other examples demonstrate that under certain situations, the collective knowledge of a crowd can outsmart even the best experts.
In the data management world, we can see examples of this fascinating behavior, as well. Steve Hoberman’s “Design Challenges” present challenging data modeling problems to readers and asks them to respond with solutions. He then aggregates these accounts into an article, built from the knowledge of the group of data professionals. Online forums such as DM Discuss hold threaded discussions where responses build on one another until a collective solution is created. In her “Contentious Issues in Data Management” presentation, Karen Lopez introduces thorny issues in our arena, and then the crowd votes on which approach each individual uses. The result is the group’s wisdom in these hot topics. These are real live data management examples of the “wisdom of a crowd of data professionals.”
Surowiecki lists several mechanisms we can apply in our work to tap into this resource of group intelligence. A simple method at arriving at collective wisdom is to explore the concept that the average of a set of estimates will be closer to the actual number than nearly all the individual guesses. An interesting application of this in the data management arena is project time estimations. How long will it take to design a model for the project? There are several rules of thumb out there to this problematic question, one being a blanket “hour per attribute.” Instead, have each person on the data team estimate the project, and take the average. This is the crowd’s wise estimate of the time required, and it is likely closer than anyone’s individual estimate. Interestingly, Surowiecki posits a corollary: In subsequent exercises, the few individuals that outperform the group will change, so over time, the group is a better than any one of the experts.
Another mechanism to allow the group to guide a decision is a “decision market.” In a decision market, participants speculate on outcomes of a process. Who will win an election or which team will win the World Cup are examples of places where a decision market may be set up. To do this, each participant wagers some money (real or hypothetical) on a specific outcome. Adapting this to a data management problem is straightforward: ask the project team how much of their money, from $0 to $100, they’d be willing to wager that the data model they’ve developed really will meet the requirements defined. If they’re right and it does succeed, they get $100. If they’re wrong, they get nothing. The decision market will fluctuate, as those who believe in the model bid up the market to reflect their confidence at receiving $100 when they’re proven correct and those who see serious flaws pull the market down by bidding low amounts since they’re concerned that they will lose whatever is wagered. When the bidding settles down and a stable price appears, the crowd will have spoken and a relative value of the model will have been created.
A third method that Surowiecki cites to distill group intelligence is deliberation. In this process, when faced with a decision, a group immediately votes on what decision should be made. But that’s not the end of the process. A thorough discussion of the matter follows the initial vote, with everyone having a time to give input to the discussion. Once discussion subsides, a second vote is taken. Differences in the vote count show in what direction the group’s intelligence is moving, indicating how volatile an issue is.
For any of these mechanisms to work, the crowd has to maintain several characteristics. Not just any crowd will work. Specifically, the group needs:
- Diversity of opinion – Each person should have some information
- Independence – People determine their own opinions
- Decentralization – People specialize
- Aggregation – Mechanism for synthesizing collective decision
Data professionals are known for having a diversity of opinions on nearly every subject under the sun. We maintain our independence fiercely, developing our point of view over years of experience. We are certainly decentralized – data professionals are spread across the globe and across private industry, government, and academia. So the first three conditions for being a “wise group” are satisfied.
The final requirement for a group to build group intelligence is that there needs to be a forum to coalesce these different opinions into one voice. A primary place where our ideas are shared and aggregated is at DAMA International. Whether at local chapter meetings or at international conferences, DAMA International exists to build group intelligence in the data community.
DAMA International’s tagline is “The Premier Organization for Data Professionals,” and our vision is to inspire data management excellence. If you are a DAMA member, you’ve likely experienced an “ah-ha” moment at a DAMA event as you discussed a burning issue with your colleagues or applied a portion of the speaker’s presentation to your specific situation. In these cases, you were benefiting from the wisdom of a crowd of data professionals, and you were benefiting from your DAMA membership.
If you are not a DAMA member, please contact your local chapter or DAMA International at www.dama.org and get connected to this nexus of collective knowledge.
