Econ Journals Must Demand Data and Code

When a graduate student at the University of Massachusetts Amherst recently found a spreadsheet error in the data provided with an influential 2010 paper on government debt, professor Bruce McCullough wasn’t surprised.

A decade ago, he co-authored a report that was published in the prestigious American Economic Review (AER) about how half of the authors of quantitative papers in a previous issue failed to abide by the journal’s requirement to supply their data and code, when requested, for replication. The report led now-Federal Reserve chairman Ben Bernanke, who was AER’s editor in 2004, to step up enforcement of the journal’s replication policy.

However, beyond that, little else has changed in the world of economics research; AER’s policy is the exception, not the rule.

Data and code archives should be mandatory for all scholarly journals, McCullough recently told The Wall Street Journal, which published an article about his crusade. He says that out of hundreds of economics journals, he’s only aware of five that require authors to provide data and code.

“To rely on the results of a paper when the results cannot even in principle be verified assumes that the authors are infallible,” McCullough told the WSJ. “And we’re all human. So, simply because we’re all human, any article should have its data and code made available … so other people can check that no mistakes were made.”

Bruce McCullough, Ph.D., is a professor of decision sciences at Drexel LeBow.