In my post on the UK government inquiry that cleared Phil Jones, I noted that the report made a point of mentioning that:
Even if the data that CRU used were not publicly available—which they mostly are—or the methods not published—which they have been—its published results would still be credible: the results from CRU agree with those drawn from other international data sets; in other words, the analyses have been repeated and the conclusions have been verified.
And this highlights an essential point: it is replication (not repetition) that strengthens confidence in scientific conclusions. Repetition tells us very little, replication (especially when approaching the problem from a different angle) gives us confidence in the results. As Robert Grumbine points out:
The most popular vein of controversy is efforts of people to ‘audit’ the work of people who were trying to answer the scientific question. And that mostly baffles me. As a scientist, it’s entirely baffling. As a citizen, I see reasons, some good, some not.
The scientist in me is baffled because “let me have all your data sets and all your programs so that I can see if I get your results” does nothing at all to answer a question about the global mean surface temperatures. No matter what is done, whether the exact same answers are found or not, we know nothing more about what global mean temperatures have been. That makes for a bad scientific question.
In a good scientific question, you learn something regardless of what the result is. The science on global mean temperatures is not in ‘auditing’, but in doing more science. Ask questions like “The prior work did their spatial gridding in a very simple way. Do I still get an answer like theirs if I use better gridding methods?” “The prior work trusted some stations I think are very untrustworthy (insert list of objective reasons here). Do I still get an answer like theirs if I exclude those stations?” “We now have much better data from this new source. Does it show the same trends as the old source?”
Regardless of the outcome we learn something. It may well be that the original result was correct, in which case the confidence we place on it has increased, and the extra science helps reduce the uncertainty surrounding the original result (all science has uncertainty).
But the new science may not match up with the old result. When that happens one needs even more science to determine why such a discrepancy exists.
Repetition or auditing does not advance our knowledge. It doesn’t move us closer to the truth. It nitpicks without ever looking at the big picture. If the temperature record, or hockey stick (actually several hockey sticks now) are wrong, as some self proclaimed auditors claim, then they should start by doing science. Only then can we advance the body of knowledge.
UPDATE: Myles Allen has the same conclusion:
Science generally progresses by taking different approaches to problems and either confirming or refuting published results, not by ‘auditing’ old calculations. There is a danger that if climate science starts to be treated as a bookkeeping exercise, this would actually impede progress in understanding how the real Earth system works.
Leave a Reply