Knowledge Socialization and the Wisdom of Crowds
Don Tapscott, in Wikinomics, uses the term collective intelligence, which he
defines as “the aggregate knowledge that emerges from the decentralized choices and judgments of groups of independent participants” (Tapscott, 2006, p. 41).
Tapscott also discusses James Surowiecki’s provocative book, The Wisdom of Crowds, in which
Surowiecki proposes that a crowd or a group is usually more accurate than an expert alone. The subtitle of his book is Why the Many are Smarter than the Few and How Collective Wisdom Shapes
Business, Economies, Societies and Nations. Four conditions must be met for a crowd’s collective intelligence to produce more accurate outcomes than a small group of experts:
- Diversity of opinion
- Independence of members from one another
- Decentralization
- A good method for aggregating opinions
Tapscott believes that mass collaboration to create collective intelligence in nearly every endeavor of life, ranging from science to business, will be the “next big thing,”
revolutionizing life and business as we know it.
Why Do We Need Experts?
It is obviously easier to refer to one source than have to check multiple sources for every question that might arise. For example, if you need the weather for the day, do you check one source or
multiple ones? Your answer will probably be based on how important the information is to you. For example, if you are in charge of the next space shuttle launch at Cape Canaveral, you will probably
check multiple sources for the weather, and certainly the ones known to be the most accurate. But if you just need to know if you should wear a coat to work that day, probably the local weather
station will do, and one source will suffice.
Surowiecki brings up various problems associated with “expert status”:
- It may be difficult to distinguish between expert knowledge and just pure luck; this is how superstitions start.
- The expert may not even know that he or she is an expert; knowing something and knowing what you know may be two different skills.
It seems that we need the security of consulting an expert, even when he or she may have dubious credentials. Surowiecki uses the term seer sucker to capture our inherent need for an expert: Even
though no expert seems to exist, a sucker will still be found to serve as one.
The credibility of experts is a topic worth consideration. Deborah McGuinness, in her talk entitled “Making Web Applications More Trustable” at SemTech in 2006, presented ideas about
how some sources of knowledge are more credible than others. One way you can measure credibility is to assign weights to people’s opinions. Some websites already do a version of expert
weighting, based on users voting on whether a book review was helpful or whether a person’s list is useful. Amazon and eBay have weighting systems for reviewers and merchants based on the
feedback users provide. Surowiecki says that individual weighting is unnecessary, that the crowd already factors this in naturally by balancing out dummies with experts. The diversity factor counts
for a great deal. Surowiecki also states that experts can be biased and that other opinions can balance out the bias. It is the diverse nature of a crowd, he says, that enables it to be smarter
than any individual expert. A group that is very homogeneous is not as smart as one with diverse points of view.
One reason for the power of the group is the “permission” to dissent from the group’s consensus. The more dissension that is voiced, the more people will not fear expressing
differing opinions, thereby adding to the group’s ability to consider several alternatives and to arrive at a sound conclusion. Therefore, an important characteristic of a “smart”
group is one in which dissension is permitted. This matter touches on the trust issue mentioned earlier concerning the knowledge capture culture.
No matter how smart a group is, however, nothing can take the place of an expert when you have to have brain surgery or are in any other life-threatening situation where only the opinion of an
expert can save you. Obviously, experts have a vital, primary role in certain situations. But even in such dire situations, there is a time for a “second opinion,” and you should
consult with several experts. It might be argued that this presents the best of both worlds: diversity of opinion of experts!
Technology that Fosters Knowledge Socialization
Even though we have a propensity to rely on experts, this does not diminish the power of socialization in the creation and evolution of knowledge, and the importance of collaboration in the
workplace. Today, many advances in technology have provided new ways to socialize knowledge. The Internet has perhaps had the greatest influence on socialization, permitting virtual socialization,
which has created a new reality of social interaction. Is it more personal or less so? Some might argue that this new interaction is less personal because it is not face-to-face. However, because
of the mere fact that it fosters some anonymity and that it creates a vehicle for people to be open in a new way, people feel that it is safe to share things that they might not feel comfortable
doing in a face-to-face venue.
Social Networking
Social networking allows each participant a means to connect to other people with like interests or something in common. For example, Classmates.com was the first such site that enabled people to
find others from their same alma mater (high school or college from which they graduated). Perhaps the most famous is MySpace, a social networking website based in Santa Monica, California, that
offers an interactive network of blogs, user profiles, groups, photos and an internal e-mail system. According to Alexa Internet, as of July 2006, it was the world’s fourth most popular
English-language website and the sixth most popular in any language. It is the most popular site in the United States, accounting for 4.5% of all website visits.
The page shown in Figure 1 came from Wikipedia and belongs to MySpace’s founder.
Tom’s blogs are listed on the right.
Figure 1: MySpace Example
Portals and Collaboration Servers
Collaboration software, otherwise known as groupware, has been around for a while, manifested in portal software and, before that, in environments like Lotus Notes. Among its
features are shared areas, usually project or team focused, where team members can post documents, calendars, have threaded discussions, and so on. Some collaboration environments feature e-mail
archives that are browsable by all team members. In this way, it enables the team to have ready access to the history of e-mails between the group.
Using a collaborative environment can be an effective way to capture knowledge and can allow team members to socialize the knowledge, especially by using the threaded discussions. Figure 2 shows an
early implementation using Lotus Notes and a subcomponent called “Team Room.”
Figure 2: Example of a Collaboration Environment in Lotus Notes’ Team Room
Team Room’s most important feature was its tight integration with the e-mail system. It provided the following benefits:
- The user could easily switch environments with one mouse click, from e-mail to Team Room and vice versa.
- The user could easily add checking Team Room for new postings daily as part of his or her normal workflow because it didn’t require accessing a new tool; it just became part of the
process of getting e-mail. In other words, no new process was required; adapting the old process was sufficient. - E-mails could be sent automatically when documents or discussions were posted or updated.
We used Team Room very successfully on a project and got everyone contributing actively. It also helped that this project was a highly visible one and that there was some rivalry between different
departments and functional categories. Each of the groups didn’t want other groups to get the upper hand, so they were constantly competing to have their voice heard and to have their own
opinions considered on the project. In this case, the rivalry between functional areas fueled sharing rather than stifled it. Team Room provided the groups with a vehicle to be heard; consequently,
they were active users.
Notice that each individual’s contribution is stored separately. There is no implementation of the notion that knowledge is being incrementally built. The threaded discussion tells you who
said what when, but there is no sense of creating a piece of knowledge jointly. The only way to accomplish this is by serially working on a document: each team member downloads the document, checks
it out, works on it a bit, and checks it back in. Then, each team member updates and adds its contribution to the main knowledge artifact.
Most of us today recognize the “twisty” indicator that is commonly used in websites and other hierarchical structures to indicate the presence of entries below the current entry. If you
click on the twisty, it reveals the lower level discussion threads. Notice, too, that the collaborative environment supports commenting on a document (which is what “reference” means on
the right side), as well as a discussion thread that is not related to a document (labeled “Discussion”).
Team Room also had a chronological view indicating which days had entries that the user had not viewed. This enabled users to look at only what was new. See Figure 3.
Figure 3: Team Room’s Chronological View
Some collaborative environments require some monitoring to be effective. The Lotus Notes environment seemed to require a little monitoring, since it was so new. Nowadays, portals are common, so it
is probably less important.
On our project, we provided individualized training to new users by phone. Because users were geographically distributed, phone training was cost-effective and still seemed to get the job done. The
phone training included simple ways that checking Team Room could be integrated into the person’s usual workflow. The technical project lead posted e-mails weekly or bi-weekly with project
status items and what had been newly posted on the Team Room. The e-mails also highlighted special items that needed feedback from users. E-mails occasionally posted controversial issues to make
sure everyone was awake and using Team Room. E-mails also could be automatically generated and could include a link directly to the relevant Team Room entry. This made it even easier to access the
Team Room.
Team Room included what I call the “Big Brother feature,” which enabled the technical lead to monitor usage, showing who, what, when, and whether anything was posted. Most users were
passive, and as is true of other typical groups and communities, you always have a few vocal ones in every bunch. But we found that even the shy members would post a response every now and then. We
found that usage increased when an e-mail blast with a controversial issue had been sent, which is what we expected (and, indeed, this was the purpose of the blast in the first place!).
This particular project’s success was owed in no small part to the Team Room environment. Collaboration environments can make a huge difference with geographically distributed teams. The
secret to our project’s success with collaboration was the periodic checking for new items. It was less onerous for users to read a few postings every day than a massive amount in one
session. Think of how you read e-mails; isn’t it easier to read a few in the morning and some in the afternoon rather than a whole day’s worth at one sitting? Remember what it’s
like when you come back from vacation?
The project’s main focus during the phase when we used Team Room was on business rule gathering and validation. We followed up the discussions with an on-site meeting. The meeting was
extremely productive because everyone, having periodically checked Team Room, was up to speed on all the topics. We were able to go through each business rule and get validation right away, and the
discussions were informed and very meaningful. People were prepared!
The collaboration software caused a huge leap in productivity for this on-site meeting. We actually finished half a day early, which never happens! Everyone was amazed; people from out of town
actually had a little sightseeing time!
Since this project’s conclusion, the collaboration features of Team Room have been absorbed by the Lotus Notes Domino Server, and other vendors have impressive product offerings, such as
Microsoft’s SharePoint. There are many collaboration/groupware products from which to choose.
Wikis and Knowledge Socialization
Introduction to Wikipedia and Wiki Technology
Wikipedia is an open-source encyclopedia on the Internet. It is the “people’s encyclopedia” – anyone can contribute content or update an existing entry. In this way,
everyone can participate in it and own it. Wikipedia is democracy in action: “Of the people, by the people, for the people.”
Wikipedia traditionally has very little governance, its purpose mainly to filter out foul language. Furthermore, it has no verification of facts; it is supposed to govern itself. If someone
contributes something that is incorrect, someone else comes along and edits it. The self-governance seems to work, and it is widely quoted. It is a great source for general information about almost
any subject. (See Figure 4 for the entry on “Data Warehouse.”)
Figure 4: Typical Wikipedia Article
No one is acknowledged as the author of a Wikipedia entry. If you want to know about an article’s origin and history, you can click on the “history” tab to see who edited it
– that is, if the person did not edit anonymously. You can create a username in Wikipedia or you can choose not to do so. Either way, you are allowed to edit anyone’s article. If you
edit without logging in, your IP address will be shown in the history of the page.
Wikipedia is built on top of software called a wiki, which enables the free-flow collaboration discussed here. It is self-organizing; a topic of discussion can be created at any time by anyone. It
does not require fitting a topic into a rigid hierarchy. It builds trust between contributors and a sense of collaboration. It is bottom-up collaboration.
Wikis are an extremely powerful tool for collecting business metadata of all kinds. They allow information exchange on a topic of your own choosing, and they become more useful to everyone the more
people contribute. They are used for all facets of life; some people even use wikis for keeping in touch with family.
The wiki seems to have become the symbol of the new collaboration that Web 2.0 has fostered. This is why Don Tapscott chose to create a name for this new collaboration phenomenon derived from the
word wiki; he calls it Wikinomics [Tapscott, 2007]
The Role of Wiki in Knowledge Capture
As we have seen, the “crowd factor” can be a positive influence on the accumulation of knowledge. Wiki technology is perhaps the crowd at its best. Wikis enable one article or
encyclopedia entry to benefit from a crowd or a multiplicity of authors, each person modifying the same entry.
This has a great psychological payoff: everyone who edits the entry feels that they are contributing to the universal source of knowledge. They, like everyone else, have the power to either create
new knowledge or edit and contribute to existing knowledge.
Difference Between Wikis and Portal Collaboration
You might say, “We are using a portal collaboration area with groupware to encourage people to share insights and questions. What advantage does using a wiki bring?”
Wiki technology offers these advantages:
- A wiki allows entries to stand alone, classified according to their title. Collaborations consist of threads, and an entry is always stuck in the hierarchy of the thread and the message to
which it was responding. - A wiki is self-organizing, meaning that entries do not have to fit into a preexisting taxonomy or classification scheme. People are free to create a page on whatever topic they would
like. - A wiki has a page for threads if they are desired. But the thread is attached to the entry and not the other way around.
- With a wiki, anyone can update another’s entry. This is not so in collaboration software. A message on a thread always stays the same, even if someone wants to edit it later. Some
collaborations allow you to edit your own post, but certainly not someone else’s.
An important motivation for encouraging people to contribute knowledge is the satisfaction that they are contributing to an overall knowledge base – something bigger than themselves. Being
able to edit existing entries provides some of this satisfaction.
Limitations of Wikis
Lately, I have come face to face with a limitation of wikis as a knowledge dissemination mechanism: they are a “roach motel.” It means that you can get data in, but you can’t
easily get data out, in a different form. On one project, we wanted to use a wiki for a project dictionary and then give it to the client when we were finished. The only problem was that it existed
as a wiki only, and there was no way to export it to a text file, Excel file, database, and so on. The ability to export the data is very useful because you can repurpose the dictionary and have it
referenced from different applications by importing it into different tools.
Now, of course, this may not true of all wikis; as you probably know, there are many flavors. You can get a taste of them by visiting www.wikimatrix.org and
comparing them to each other. Unfortunately, however, it is true of the vast majority at the time of this writing, including MediaWiki, the most famous of them all. (Wikipedia is built on this
one.)
This inability to repurpose wiki data can be a major stumbling block for using a wiki to collect business rules, which is not a very useful exercise if you can’t easily repurpose them into
your applications or rule engines.
Wikis and Governance
On the downside, wikis can be very chaotic because they lack governance. Entries change day to day; when you quote an entry as an authoritative source, you should also supply the date that you
accessed it, since it is so subject to change. Some CIOs are afraid to launch wikis in their organizations for fear that they will become a forum for corporate gripes.
Why Some Governance is Usually Necessary
Even Wikipedia has had to face the governance issue. In several high-profile cases, damaging information was published on its site, resulting in legal ramifications. For example, an article that
was allowed to remain up for 132 days said the following:
John Seigenthaler Sr. was the assistant to Attorney General Robert Kennedy in the early 1960s. For a brief time, he was thought to have been directly involved in the Kennedy assassinations of
both John, and his brother, Bobby. Nothing was ever proven.
This information was patently false, posted with malicious intent.
Does this mean that Wikipedia is inherently unreliable? In academia, professors often do not allow it to be used as an authoritative source. Obviously, millions of people rely on Wikipedia for
knowledge all the time (including me). How do you balance your need for truth with the fact that Wikipedia might contain errors?
As a matter of fact, everything could contain errors. Very high-profile mistakes, for example, have been made in the news media in recent years. So what’s a poor person in search of the truth
to do?
It seems it is okay to use Wikipedia; but in order to be wise and judicious, one should, balance it with other sources as well. After reading The Da Vinci Code, I, like many other readers,
went in search of what was fact and what was fiction. The first place I looked was Wikipedia, but I also visited websites like Opus Dei to let them speak for themselves. In addition, I consulted
other sources, followed links, and so forth. At the end of my journey, I was convinced that Wikipedia did a very good job of reporting the facts (and that The Da Vinci Code was excellent
fiction!).
Balancing Out the Need for Governance with the Need for Contributions: “Governance Lite”
If anyone is allowed to create or update a dictionary entry, then some control is needed to reconcile terms cross-functionally with other groups in the organization who may have different usages
for the same term. This creates the need for some level of governance. One such project involved a dictionary, and the question came up about an “authorized” vocabulary. The
organization involved decided that it wanted to have an individual or group to authorize vocabulary, which then brought up questions about who was in charge of the authorization? However, the
requirement for governance had to be balanced against the other side of knowledge capture, which involved encouraging businesspeople – the people who know the vocabulary the best – to
supply the terms and definitions themselves. They know best, because they are on the ground floor. Too much authorization red tape will discourage them from supplying the terms in the first place,
and then authorization will be a moot point.
We have all experienced corporate initiatives that have been burdened with too much governance. The governance gets in the way of flexibility. The business needs to be able to change the definition
of a term when the business itself changes. The resulting situation involves definitions that may have been correct at one point in time, but as things change, they never are in synch with the
business as time goes on because it takes an act of Congress to change the dictionary. “Governance Lite” is a concept that was created (by Bonnie O’Neil) in the context of a
dictionary project in order to create a flexible structure to accommodate both the need for governance and the need to keep constantly in synch with the business. See my article in TDAN entitled
Launching a Corporate Glossary: www.tdan.com/i033ht01.htm, for more information.
Summary
This article has discussed how the socialization process helps to evolve knowledge and also the “wisdom of crowds” effect, which has been shown to be more effective than a small group
of experts in predicting outcomes and answers to various problems. It appears that the advice of experts may not be as effective as the advice of a crowd.
Does this effect always work? Would you trust the advice of a crowd over that of a brain surgeon if you are told you have a tumor in your head? Probably not. But collective intelligence has major
ramifications in how knowledge evolves and is transferred in organizations. We can foster collaboration by putting in place various technologies such as groupware and wikis. However, these
technologies are not perfect, and sometimes you have to work around the shortcomings of these tools. Even so, their benefits drastically outweigh their drawbacks.
