Designed intelligence: how experts can help the crowd

September 12th, 2012 by

What is wisdom? It seems like a simple question, but as any Dungeons and Dragons player knows, there is a difference between wisdom and intelligence. Legendary D&D creator (and awesomely named dude) Gary Gygax explained the distinction: Intelligence is noticing that it’s raining. Wisdom is putting up an umbrella.

The wisdom of crowds effect has been on a turbulent journey over the last decade, graduating from a statistical curiosity to a key part of a global industry. The concept has been at the heart of some incredible achievements, from the creation of Wikipedia to helping data analysts win chocolate (although we won’t mention the lack of wisdom the crowd has shown in watching, let alone voting in, the X Factor).

Experts witless?

But despite the growing pile of crowd-based success stories, some people contend that the crowd’s wisdom has been overstated. As the CIA turns to crowdsourcing to predict global events, some of its own experts are questioning whether this is a good idea. Mark Lowenthal, a former senior CIA and State Department analyst, provides a snappy argument to me and my fellow crowd-evangelists: “Crowds produce riots. Experts produce wisdom.”

The CIA’s project uses a modified version of the wisdom of crowds effect called Aggregative Contingent Estimation. While this may sound like a classic CIA euphemism the process is actually a carefully designed way to create a crowd of experts and pool a broad range of analysis and opinion.

It’s too early to tell if the CIA’s crowd of experts will be successful in predicting the next major global event. (For the record, I predict that 2013 will be the year of Finnish global supremacy, but I’m not an expert). While we await the results of this experiment, it’s worth considering the important role that individual experts can play in the evolution of crowd wisdom. After all, they are the experts.

A word from the wise

Coming back to the wise words of Gary Gygax, it is experts who can supply the crowd with the intelligence that gives them a better chance of attaining wisdom. So instead of asking them to compete with the crowd, we could use their expert knowledge to provide deeper context and
allow the crowd to make better informed decisions.

The best known recent example of a catastrophic failure of crowd intelligence is the attempt to predict the price of Facebook’s IPO before its disastrous launch (check out on the subject for the grisly details). It seems it was the presence of experts in the crowd that drove the predictions, replacing the crowd’s wisdom with blind optimism.

But what would have happened if the experts had shared their specialist knowledge with the crowd instead of their (mistaken) conclusions? If wisdom lies in forming the correct conclusion, then intelligence lies in realizing the true nature of the problem. In short, if we don’t know it’s raining, why would we put up an umbrella? Traditionally, the point of an expert is to provide the right answers. But, at least when it comes to the wisdom of crowds, what we may actually need from them is the right questions.