# Why Economists Will Continue to be Wrong

Carter had initially used arbitrary parameters in his perfect model to generate perfect data, but now, in order to assess his model in a realistic way, he threw those parameters out and used standard calibration techniques to match his perfect model to his perfect data. It was supposed to be a formality--he assumed, reasonably, that the process would simply produce the same parameters that had been used to produce the data in the first place. But it didn't. It turned out that there were many different sets of parameters that seemed to fit the historical data. And that made sense, he realized--given a mathematical expression with many terms and parameters in it, and thus many different ways to add up to the same single result, you'd expect there to be different ways to tweak the parameters so that they can produce similar sets of data over some limited time period.

The problem, of course, is that while these different versions of the model might all match the historical data, they would in general generate different predictions going forward--and sure enough, his calibrated model produced terrible predictions compared to the "reality" originally generated by the perfect model. Calibration--a standard procedure used by all modelers in all fields, including finance--had rendered a perfect model seriously flawed. Though taken aback, he continued his study, and found that having even tiny flaws in the model or the historical data made the situation far worse. "As far as I can tell, you'd have exactly the same situation with any model that has to be calibrated," says Carter.

That financial models are plagued by calibration problems is no surprise to Wilmott--he notes that it has become routine for modelers in finance to simply keep recalibrating their models over and over again as the models continue to turn out bad predictions. "When you have to keep recalibrating a model, something is wrong with it," he says. "If you had to readjust the constant in Newton's law of gravity every time you got out of bed in the morning in order for it to agree with your scale, it wouldn't be much of a law But in finance they just keep on recalibrating and pretending that the models work."

## Notes:

They make models based on past data, and when they fail to predict the future, they adjust them to match the new past data. The problem is that so many models will match the past data, there could be no end to the number of models they throw out.

**Folksonomies:** economics predictions modeling

**Taxonomies:**

/society/work/unemployment (0.482441)

/health and fitness/addiction/substance abuse (0.336402)

/finance (0.318552)

**Keywords:**

perfect model (0.998272 (positive:0.086081)), data (0.961105 (negative:-0.124880)), past data (0.873821 (negative:-0.665426)), historical data (0.856182 (negative:-0.170527)), new past data (0.741217 (neutral:0.000000)), standard calibration techniques (0.725118 (positive:0.686353)), parameters (0.719007 (positive:0.452227)), limited time period (0.709789 (positive:0.225240)), perfect data (0.686949 (positive:0.632703)), different ways (0.680972 (positive:0.225240)), models (0.665054 (negative:-0.618825)), arbitrary parameters (0.627369 (positive:0.579052)), realistic way (0.586345 (positive:0.604067)), single result (0.577062 (neutral:0.000000)), mathematical expression (0.576652 (neutral:0.000000)), terrible predictions (0.576578 (neutral:0.000000)), similar sets (0.568283 (positive:0.225240)), calibration problems (0.561760 (negative:-0.669611)), different versions (0.561706 (neutral:0.000000)), standard procedure (0.561037 (neutral:0.000000)), different predictions (0.558836 (neutral:0.000000)), bad predictions (0.558131 (negative:-0.750799)), different sets (0.557905 (positive:0.318262)), financial models (0.553218 (negative:-0.669611)), finance (0.503788 (negative:-0.633589)), modelers (0.480836 (negative:-0.750799)), Carter (0.479660 (positive:0.305413)), situation (0.471295 (negative:-0.488789)), law (0.462406 (negative:-0.304955)), formality (0.446952 (neutral:0.000000))

**Entities:**

Carter:Person (0.951943 (positive:0.330589)), Wilmott:Person (0.483992 (negative:-0.710205)), mathematical expression:FieldTerminology (0.454034 (neutral:0.000000)), Newton:Person (0.434918 (positive:0.211423))

**Concepts:**

Calibration (0.930081): dbpedia | freebase

Prediction (0.879092): dbpedia | freebase

Future (0.871932): dbpedia | freebase

Economist (0.724638): dbpedia | freebase | opencyc

Standard (0.680638): dbpedia

Futurology (0.675973): dbpedia

Subroutine (0.640607): dbpedia | freebase

Newton's law of universal gravitation (0.632640): dbpedia | freebase | yago

**Why Economic Models Are Always Wrong**

**Electronic/World Wide Web>**

**Internet Article:**Freedman, David H. (October 26, 2011)

*, Why Economic Models Are Always Wrong*, Retrieved on 2013-12-27

**Folksonomies:**economics predictions