The wisdom of crowds is the observation that an aggregate of many independent judgments, such as the average or median of a group's estimates, is often more accurate than the judgment of a typical member of the group and sometimes more accurate than the best individual expert. It applies to estimates of quantities, forecasts of events and answers to factual questions, provided that the people involved bring different information and do not simply copy one another.
The idea underlies much of financial economics, where market prices are seen as a way of pooling the views of many traders, and it is the stated rationale for prediction markets and for averaging analysts' forecasts into a consensus estimate. It is also, in its failures, a central topic of behavioral finance, which studies why crowds sometimes go badly wrong.
History of the idea
The best-known early demonstration came from the statistician Francis Galton, who examined the entries in a weight-judging competition for an ox at a country fair in Plymouth, England, and reported the results in the journal Nature in 1907. The median of roughly 800 guesses was 1,207 pounds, within one per cent of the dressed weight of 1,198 pounds. Earlier, the Marquis de Condorcet had shown in 1785 that a majority vote of many voters, each more likely than not to be right, becomes increasingly likely to reach the correct answer as the group grows, a result known as the Condorcet jury theorem.
The modern phrase comes from the journalist James Surowiecki, whose 2004 book The Wisdom of Crowds argued that well-functioning groups need four conditions: diversity of opinion, independence of judgment, decentralisation, so that people draw on local knowledge, and a mechanism for aggregating their views into one answer.
Why aggregation works
When individual errors are independent and not all in the same direction, they tend to cancel out when averaged, in the same way that the law of large numbers makes a sample average settle near its true value. The point can be stated exactly. For a group's average estimate, the economist Scott Page's diversity prediction theorem shows that:
- is the estimate of person , and is the number of people.
- is the crowd's average estimate.
- is the true value.
In words, the crowd's squared error equals the average individual squared error minus the diversity of the estimates. Suppose three people guess 10, 14 and 18 when the true value is 15. The crowd average is 14, so its squared error is 1. The individual squared errors are 25, 1 and 9, an average of about 11.67. The spread of the guesses around 14 gives squared deviations of 16, 0 and 16, an average of about 10.67. The identity holds: . A crowd is never less accurate than its average member, and it gains most when its members are both reasonably informed and different from one another.
Markets as crowds
The economist Friedrich Hayek argued in 1945 that prices gather knowledge dispersed among many people that no central planner could hold. The efficient market hypothesis builds on this view: if traders with different information compete, the market price should reflect what is collectively known, and price discovery is the process by which that happens.
Prediction markets apply the same principle to events. Traders buy and sell event contracts that pay out if an outcome occurs, and the price is read as the crowd's probability. The Iowa Electronic Markets, run by the University of Iowa since 1988, are an early academic example, and several studies have found such markets to be at least as accurate as polls in forecasting elections. Weather and climate forecasters use a related method, combining many model runs into an ensemble forecast.
When crowds fail
The benefits depend on independence, and in practice it is often missing. Traders watch the same news, the same prices and each other, so their errors become correlated and stop cancelling. Several patterns have been identified:
- Herding. People follow the visible actions of others rather than their own information, which can produce an information cascade in which early choices drive everyone else.
- Speculative feedback. Rising prices attract buyers because prices are rising, a loop seen in many episodes of the market bubble.
- Second-guessing. John Maynard Keynes likened investing to a newspaper beauty contest in which entrants try to pick the faces others will pick, so judgments are about the crowd rather than the underlying value.
- Thin or skewed crowds. A small or homogeneous group, or a market dominated by a few large participants, offers little diversity to average.
These limits mean the wisdom of crowds is a conditional result, not a general rule. Aggregated judgment tends to perform well on well-defined questions with dispersed information, and poorly when participants share the same biases or are influenced by one another.