How an Excel error impacted the world

In 2010, an important economic science paper was published by Reinhart and Rogoff(pdf here). One of their main points was that countries with a public debt above 90% of GDP had a significant lower economic growth rate. They compared data from various countries since the Second World War.This part, even though it confounded correlation and causatation, was received clearly. Austerity and the reduction of public debt has become a very important (and contested) topic, both in the US and especially here in Europe. Policies should be based upon the best results, and this was certainly an impressive result to work with.

But there was one issue. Researchers trying to replicate did not obtainthe same results. Herndon, a graduate student from the University of Massachusetts tried too. However he tried to tackle a basic question ("is the data correct") and requested a copy of the original spreadsheets (The Independent has an interview with Herndon). To the credit of Reinhart and Rogoff, they were willing to share this. Together with two colleagues, Herndon then published a critical paper.

Business Insider has a nice summary of the various points (read the original paper here), but the most stunning item that created a firestorm on the Internet was a simple Excel coding error. It even got mentioned/slammed in the Colbert Report. Basically, as the picture below shows, a range was wrongly defined, leaving out five countries.

And not just five random countries... It includes my home country, Belgium, which is a country with high debts but also decent economic growth. Based on this, it seems the 90% GDP mark is not that important anymore. Herndon's paper came out two weeks ago, and only a couple of days later Rogoff and Reinhart admitted their error (again, to their credit).

The resulting discussion on the blogosphere however was less than nice (just do a Google Search). It became so bad that yesterday, Rogoff and Reinhart had anop-ed in the New York Times, outlining both their defense of their initial paper (which still kind of holds) but also the fact that they were subject to personal attacks and even hate-mail. As they suggest, they became sort of a handy scapegoat for all the cutbacks and economic woes that have befallen the world. They then go on to defend the nuances of their paper, including various suggestions to handle the economic crisis outside of the pure "austerity" model.

What have we learned? Errors cannot be avoided entirely, and we shouldn't mock the researchers for making them (I have made Excel errors myself, too). However, blogger Lemire indicates that they probably should have shared their data spontaneously when it became one of the basic arguments for the current policies. Opening up your data and coding will help to make scientific conclusions more solid. This is simply how science works, and its a good thing.

Also, it's important to admit error. Reinhart and Rogoff did this, and quickly. The fact that they still stood with other findings of their study might initially seem like weaseling out or trying to save face, but it is beyond that. There is still an (albeit smaller) correlation between higher debt and lower growth, and it should not be overlooked, as this commentary by Aslund in the Financial times (behind a paywall) points out. And as Cox on CNBC pointed out, it's not their fault that one of their arguments was hijacked as the most convincing argument for political action. In any case, science is a complicated matter. It's not because one piece falls, that the entire point is wrong too (an argument reminiscent of creationist tactics). Maybe it is, but that is the continuation of the scientific debate, and will probably not be settled as quickly as finding an Excel error.

Finally, there is the sorry barrage to which the researchers were subjected. Getting slammed on the Colbert Report is one thing, getting hate mail and being attacked on a personal level is entirely different. While one might disagree (and I do, too), we should consider this a learning opportunity to improve our understanding of science and how the world works. The complexities of an economic crisis and attempts to get out if it can never be blamed to two people and an Excel coding error.