
The frighteningly well-qualified Ian Ayres – a professor at Yale University, both in the Law School and in the School of Management – is boundlessly enthusiastic about the wonders of statistical analysis.
In their book Freakonomics, Ayres' friends Steven D. Levitt and Stephen J. Dubner used statistical analysis of huge databases to reveal "secret levers of causation"; the most notorious was the connection between the abortion rate in 1970 and the crime rate in 1990. As a result of the publication of Freakonomics, statistical analysis has lost at least some of its dry, dusty air.
Super Crunchers is about the impact of number crunching: the extent to which quantitative analysis is impacting real-world decisions. Ayres begins with the impressive example of Princeton economist, Orley Ashenfelter, who had the temerity to apply a formula based on weather data to predicting the price, and quality, of France's Bordeaux wines each year. Traditional wine critics were resentful that Ashenfelter threatened to demystify the world of wine; one called him "an absolute total sham". Nonetheless, his controversial predictions were remarkably accurate.
Ayres introduces the reader to the two fundamental statistical techniques – regressions and randomised trials – and examines how well data-based decision making works versus the traditional tools of intuition and experience. Unfortunately, once human beings form a mistaken belief about something, we tend to cling to it. Bias and overconfidence can distort our predictions.
The case Ayres makes for "evidence-based medicine" versus the traditional model is devastatingly strong. It certainly seems that the technique works, but should we be happy about it? Just how much can the government, and big business, predict about our behaviour from the huge databases that they have amassed?
Ayres is at pains to point out that statistical analysis can be used for good. He writes, "there is this crazy view out there that statistics are rightwing," but he has used statistical analysis in massive class-action litigation against virtually all of the major automotive lenders, to prove that minorities were being discriminated against, and lenders' policies have changed as a result. The Progresa program in Mexico gives poor mothers money if they're willing to invest it in their children's future; the randomised trials which proved its worth are another triumph for statistical analysis.
It seems that, as Ayres says, "You can crunch numbers and still have a passionate and caring soul."
