James Surowiecki wrote The Wisdom of Crowds, is talking about the concepts in his book.
Three examples of the phenomenon:
- Race tracks
- Jellybean counting
- Guessing the weight of a cow
In “wise crowds,” there are typically a few experts, a few people in the middle, and lot of people who individually have a high rate of error.
Yet…if you take the average of all the “guesses,” the average is usually a very good approximation…in some cases, within a few percent (3%-5%) of the actual value. Surowiecki: “Results of the market map almost perfectly onto the outcomes of events.”
There are a number of examples where the “collective guess” has provided good results:
- Siemens (what will be the market lifespan of a product?)
- Google (how many users with GMail have after three quarters?)
- Eli Lilly (which drugs will make it through clinical trials?)
Why does this work?
Everyone participating in these markets has “some” idea of what the answers will be, but they also have biases.
However, this doesn’t always work. For this to work, it requires three things:
- Some form of aggregating peoples’ judgements
- Diversity
- Independence
Why is diversity important? Wise crowds need cognitive diversity … difference in frames of reference, tools used to solve problems, etc. It simply expands the range of information that is available, and avoids the hurdles and obstacles that a single individual may run in to. Diversity helps to even out the blind spots and biases in a crowd…and the biases and blind spots of “experts” as well.
The phrase “collective intelligence” is being used extensively to capture this concept.
Diversity also helps in get around peer pressure. Story is told of peer pressure in psychological experients, as well as examples of “groupthink.”
Independence – make decisions on own intuitions, not piggybacking on the statements of others. Instead of aggregating the judgements of independent people, instead we come to watered-down decisions in groups, trying to find the lowest common denominator that makes everyone “happy enough.” The more “independent” the participants in the group are, the more applicable the wisdom of crowds idea is.
Humans are naturally imitative. Example: Put a person on a street corner and have him gaze up at the sky (at nothing). If one person is doing this, some passersby look up. If five people are put on a streetcorner to look up, half the passersby look up. If eight people are placed on a streetcorner and look up, eighty five percent of the people passing by also look up.
One reason independence is hard to come by is that “going with the group” is a way to protect self-reputation. Keynes: “It is better to fail conventionally than to succeed unconventionally.” If you go with the group and are wrong, you can easily say “that’s what everyone was doing” and protect reputation.
Fundamental lessons
The knowledge that we want is often NOT in places that we think to look. We also typically overestimate who “experts” are.
IMPORTANT: There is a difference between “experts” and “expert information.” We overestimate what an “expert” says, but a diverse, independent crowd can derive the “expert information.”
Great point on the “echo chamber” effect — people get locked into small worlds, even on the internet. (Think of the case where one only gets information from a limitied set of sources.)
Final story: What collective intelligence can look like
USS Scorpion, SSN-589, a nuclear submarine, was lost on tour of duty in 1968. The Navy searched fruitlessly, then tried a crazy idea led by John Craven: Craven assembled a diverse crowd of experts, and came up with scenarios (Russian sub had hit it, torpedo went off in tube, etc.). He asked experts to bet on the scenarios (which one do you think is most likely?). He also asked them to bet on direction, velocity, etc. Independent judgements were pulled together, and pointed to a particular place on the ocean floor. No one person knew the facts, for example, how steeply the Scorpion had fallen to the ocean floor, how fast it was going, etc. More importantly, no one individual in the group came up with the location that was predicted by the group.
A few months later, the sub was found 220 yards from the location predicted by the group.
Q&A: “How many people does it take to make a crowd?”
Over 50, it’s pretty certainly a “crowd.” However, even in small groups (6-8 people), even the group’s collective judgement may surpass the results of the group’s smartest member. The challenge with small groups is that it takes a lot of work to ensure diversity and independence.
Q&A: What are some other areas where the wisdom of crowds could work?
Crime solving is one. Juries could be another…although the drive for unanimity is challenging.
