Why Hiding Computer Code Is Lethal to Climate Science

This post is a follow up to my previous on why Climategate is the real war on science.

My earlier post commented on Willis Eschenbach’s excellent column on Anthony Watts’s blog (WUWT). Here I want to reproduce a comment Eschenbach posted in the back and forth on WUWT. In it, he explains why hiding computer code is lethal in a field, like climate science, where results hinge on statistical methods and mathematical techniques.  Eschenbach’s comment follows.

Willis Eschenbach (17:15:51):

Part of the difficulty with climate science is that, unlike all other physical sciences, it does not study things — instead it studies averages.

This is because climate by definition is the average of weather over a suitably long period of time (typically taken as a minimum of 30 years).

As a result, much of the study that goes on, and the papers that are written, deal almost exclusively with mathematics and statistics. This is the reason that access to the computer codes is so critical.

It’s simple in the physical sciences to describe an experiment, e.g. “I took three grams of carbon and subjected them to a pressure of 50,000KPa and a temperature of 500C. Unfortunately, the experiment did not succeed, I could not replace the diamond I had lost from my wife’s wedding ring.” Anyone can reproduce that experiment (and get the same results).

But when you say “I took the raw temperature data, variance-adjusted it, averaged it, gridded it, area-adjusted it, extrapolated results to data-free areas within 250 km, and made a global temperature record”, that’s far from enough information. In order to determine what was done, we need far more detailed information in climate science, because in general we are describing intricate mathematical operations. These are often very hard to describe clearly in spoken or written language.

And even a crystal-clear description is not enough. Despite what he says he has done, if the scientist has inadvertently used an improper procedure (e.g. the uncentered principal components analysis used in Mann’s Hockeystick), we’ll never be able to determine that the answer is demonstrably wrong unless we have the actual code that he used. Otherwise, we could spend years trying to guess where he went wrong, but we would never be able to show that he went wrong as science demands.

This is why the insistence of scientists that their computer codes are sacrosanct private secret documents best kept under Hermetic seal in a clandestine vault is lethal to good science. Without the codes, we can’t tell if what has been done is correct and free from hidden mathematical error. Of course, this may be unconnected with the reason that Jones et. al are hiding their codes …

Given that climate science is not the study of things but of the averages of things, and that as a result math and statistics are central to climate science, the findings of the Wegman Report are now seen to be even more insightful, trenchant, and valid. They said:

It is important to note the isolation of the paleoclimate community; even though they rely heavily on statistical methods they do not seem to be interacting with the statistical community. Additionally, we judge that the sharing of research materials, data and results was haphazardly and grudgingly done. In this case we judge that there was too much reliance on peer review, which was not necessarily independent. Moreover, the work has been sufficiently politicized that this community can hardly reassess their public positions without losing credibility. Overall, our committee believes that Mann’s assessments that the decade of the 1990s was the hottest decade of the millennium and that 1998 was the hottest year of the millennium cannot be supported by his analysis.

And presciently, that was written thee years ago, well before we got the CRU emails …