Dan Gardner on the media's failure to hold pundits accountable, and rush to praise someone who got one prediction right, regardless of how often they were wrong:
Whenever a shocking event occurs, journalists rush to find the wise few who saw it coming, anoint them oracles, and beg them to reveal what will come next. It’s an understandable reaction to surprise and uncertainty. It’s also an embarrassing failure of the elementary skepticism that should be journalism’s foundation.
For big events like presidential elections, terrorist attacks, and stock market crashes, the number of observers making forecasts is always large, with varied forecasts. As a result, every possible outcome will usually have been predicted. In those circumstances, the mere fact that someone correctly predicted something means little. To take it as proof that the forecaster possesses deep insight and knows what’s coming next makes as much sense as asking today’s lottery winner to reveal next week’s winning numbers.
When it comes to the predictions and forecasting, the challenge is to separate the lucky from the skilled. As any baseball fan knows, that requires statistics. One home run or one strikeout says very little. To judge a batter, you need to know his batting average—a performance statistic based on the careful observation and scoring of a large number of at-bats.
Read Gardner's full piece here.