When stupid beats smart: the power of collective ignorance

March 21st, 2012 by

Homer SimpsonThe human brain is an amazing thing. A few pounds of gray matter has taken us from eating bananas in the trees to walking on the moon (it’s also brought us LOLcats, the Car Laptop Tray and Maailmanennätys muurahaispesässä, but still: the moon). With our heads filled with crackling synapses, it’s easy to feel smug about our computational prowess.

But if the history of neurological research has shown anything, it’s that our brains are far less reliable than we may think (one look at Wikipedia’s list of cognitive biases is enough to make anyone want to trade theirs for a Commodore 64). With such clear evidence that individual brains are unreliable, why should we assume that collective reasoning is any better?

Thankfully, researchers Stefan M. Herzog and Ralph Hertwig have investigated this very question, and set out to challenge Thomas Carlyle’s observation that “I do not believe in the collective wisdom of individual ignorance.” We’ve written before about the dangers of overestimating your own ability, but what if our stupidity is our secret weapon?

Stupid is as stupid does?

Herzog and Hertwig designed a novel experiment to compare the ability of experts and uninformed subjects when predicting the results of sporting matches. They created an ‘ignorant crowd’ of people with no special interest in sport, and asked them to predict the outcome of several football matches. Surprisingly, the ignorant crowd was significantly more accurate than the experts. The experiment was repeated, this time with a slightly more knowledgeable crowd of amateur tennis players, who were asked to rate the chances of a list of players. Again, despite each member of the ignorant crowd only recognizing an average of 39% of the named players, the ignorant crowd thrashed the experts.

But before we kick out the professors and start staffing our universities with Fox News ‘reporters’, it’s worth taking a look at Herzog and Hertwig’s conclusions. Despite the individual crowd members’ lack of knowledge or insight, their success was due to a uniquely collective ability to process several factors, including media saturation and word-of-mouth reputation. In short, better football teams or tennis players generate more discussion. In statistics, this is known as the recognition heuristic. Unlike the standard sports-movie plot, where a plucky underdog takes down the arrogant champion, in real life it’s usually the richest, most famous names that win.

Ignorant and proud?

Herzog and Hertwig have named the effect ‘the ignorance of crowds,’ but what do their findings mean for the crowdsourcing industry? Firstly, it’s important to note that the ignorance of crowds effect only applies to predicting things like sporting events. Secondly, the experiment doesn’t show that ignorance is more effective than expertise (Sorry, Rick Santorum, I didn’t mean to get your hopes up). The most important conclusion for our industry is the fact that crowds are capable of incorporating a broad range of factors in their decision-making, whether they are conscious of it or not.

But that’s not all that this experiment has to teach us. The fact that a properly designed crowd of amateurs can make better predictions than experts is highly significant for companies like Microtask, as it demonstrates the importance of organization. For example, the DigiTalkoot project only requires volunteers to be able to read. The more specialized work of compiling the work of crowd members is left to software designed by experts, just as Herzog and Hertwig’s ignorant crowd were able to make accurate predictions thanks to the expert organizational and analytic skills of the researchers.

The combination of a crowd and a well-designed framework for them to operate in is when the magic of crowd labor really occurs. Like Voltron, we’re more powerful when we join together, but someone always needs to be the head.



<<

>>