Comments, peer-review and censorship

Remember the study (McLean et al. 2009) that claimed that the recent warming trend was due mostly to el niño? Remember how deniers claimed it it was proof that global warming was bunk? And remember how it was shown that the paper could not say anything about the trend because their methods removed any influence of such trends?

Well a comment to the paper has finally been published, which concludes:

The suggestion in their conclusions that ENSO may be a major contributor to recent trends in global temperature is not supported by their analysis or any physical theory presented in that paper, especially as the analysis method itself eliminates the influence of trends on the purported correlations.

This in it of itself is not surprising (I mentioned that a comment has been submitted in my original post on the matter). What is interesting is that the reply from original authors was not published. This fact has, predictably, stirred up calls of a climate conspiracy keeping dissent out of the peer-reviewed literature. Such claims, as usual, miss the point.

The peer-review system functions as a quality control mechanism, and if your submission doesn’t have sufficient quality it should be rejected and not published. So was the reply of sufficient quality to deserve publication?

No it wasn’t.

Thankfully for us the reply (along with a long whine) was published by the authors so we can determine if the journal editor was correct in not publishing their reply.

From the reply abstract we get this:

Our research did not set out to analyse trends in mean global temperature, but, should any such trend exist, it follows from our analysis that in most part it would be a response to the natural climate mechanisms that underlie the Southern Oscillation.

There are at least two things wrong here. First since the original paper did not analyse trends in mean global temperature it shouldn’t draw conclusions on such trends, which it did saying:

Finally, this study has shown that natural climate forcing associated with ENSO is a major contributor to variability and perhaps recent trends in global temperature, a relationship that is not included in current global climate models

Secondly it doesn’t at all follow from their analysis that such trends would be the result of the natural climate mechanisms that underlie the Southern Oscillation.

As was mentioned in the reply (and elsewhere, see my original post) their analysis method itself eliminates the influence of trends. If the influence of the trend is removed then no conclusion about the trend can be drawn. And that is the bottom line.

And that is reason enough for the journal editor to decide not to publish the reply. It simply doesn’t pass quality control.

As James Annan (one of the authors of the comment) points out they authors of the study even agree:

Their reply first claims that the differencing was only used to establish the lag:

“we used derivatives only to ascertain the existence of the relatively consistent time-lag that exists between changes in the El Niño-Southern Oscillation (ENSO) and later changes in the global average lower tropospheric temperature anomaly (GTTA)”

But then they immediately contradict themselves and admit the obvious, that the statistics they presented were in fact based on the differenced data:

“Our comments about the change in Southern Oscillation Index (SOI) accounting for 72% of the variance in satellite (MSU) GTAA, 68% of variance in the radiosonde (RATPAC-A) GTAA and 81% of variance in the tropospheric temperature in the tropics were made in the context of the discussion of our derivatives based on differentials between 12 month averages, and we stand by them. Contrary to Fea10 claims, those figures do not refer to long-term variations but only to the derivatives that were used.”

Obviously I’m pleased to see them acknowledge the obvious, that their statistics have no relevance to the analysis of any long-term trends in the data. But that leaves their original claim concerning “potential of natural forcing mechanisms to account for most of the temperature variation” to be devoid of any support whatsoever.

In summary, it seems we all agree that ENSO/SOI accounts for a lot of the short-term variability in global temperature (which has been well established for decades) and McLean et al now appear to explicitly agree that their analysis has no bearing on long-term trends. If only they’d said that at the outset then there would have been no need for all of this.

But while their reply eventually states that their original conclusion about warming trends was wrong the authors in their whine still claim otherwise:

Our analysis… indicated that a large portion of the variability in global temperature is explained by ENSO variation, thus leaving little room for a substantial human influence on temperature.

This is wrong. The two are not mutually exclusive. It is indeed possible (and is shown by a large body of research) that el niño (ENSO) has an effect on the variability of global temperature, while at the same time human emissions cause a warming trend in temperature. It is a classic bait and switch. Had they said “thus leaving little room for a substantial human influence on temperature variability ”, that would have been more acceptable, and it would have contradicted the claim in the original study.

Indeed human influence on temperature variability is currently dwarfed by the variability caused by ENSO. The human influence on temperature is primarily the warming trend we have experienced, and thus by not specifying, the authors state something that is demonstrably incorrect:

This is best explained by an analogy involving daily temperature readings between, say, July and December anywhere in Australia. Suppose temperature is recorded twice daily, at midday and at midnight, for those 6 months. It is obvious what we would find: Most days would be hotter than nights and temperature would rise from winter to summer.

Now suppose we change all monthly readings by subtracting them from those of the following month—we subtract July from August, August from September, and so on. This process is called “linear detrending” and it eliminates all equal increments. Days will still be hotter than nights, but the effects of season have been removed. No matter how hot it gets in summer, this detrended analysis would not and could not detect any linear change in monthly temperature.

Astonishingly, McLean and colleagues applied precisely this detrending to their temperature data. Their public statements are thus equivalent to denying the existence of summer and winter because days are hotter than nights.

That they continue to repeat this means that they either do not understand the criticism of their work, or are dishonest.

Their whine makes more claims which James Annan rips to shreds:

In the AGU journals there is no automatic right of reply, there is only the right to submit a reply for the consideration of the editor. In this case, it seems that after peer review McLean et al’s reply was not found to meet the standards for publication…

The complaint that they were “reasonably well known” to one of the more prominent authors is particularly laughable – there could hardly be anyone of similar experience who isn’t. However, the actual choice of reviewers is the responsibility of the editor and I don’t know who was used…

The complaint about “prior publication” due to placing a copy of the submitted manuscript on a personal web-page is just a petty and pathetic attempt at armchair-lawyering. The AGU explicitly endorses publication of manuscripts on a personal web-page, the only minor error was in some carelessness over the formatting which was corrected within a couple of days.

I don’t see on what grounds there could possibly be any criticism of the AGU for using a different editor and reviewers from those who dealt with the original paper. Obviously, the submission of a comment may be considered an implicit criticism of those responsible for the original publication, so it is reasonable to remove this possible source of bias.

The claim that “the practice of editorial rejection of the authors’ response to criticism is unprecedented in our experience” is particularly dishonest. Because most papers never get a comment their experience would be quite limited. It is conceivable that their sample size is limited to this one paper. As a side note James Annan notes that one of his papers received a comment and he wasn’t even made aware of it.

But given the events of recent months there is more to this story than there should be.

Typically the reviewer comments are seen only by the editor of a journal, but due to the CRU email leak they were made public and have been used to bolster the incorrect claim of conspiracy in the peer-review process.

But, while the emails of climategate may seem rude or ‘partisan’ to those not familiar with correspondence between scientists, they are actually the norm:

the personal email messages between senior academics in any field are frequently not very nice. We tend to be very blunt about what appears to us as ignorance, and intolerant of anything that wastes our time, or distracts us from our work. And when we think (rightly or wrongly) that the peer review process has let another crap paper through, we certainly don’t hold back in expressing our opinions to one another…

Don’t be fooled by the more measured politeness in public: when we think an idea is wrong, we’ll tear it to shreds.

Now, in climate science, all our conventions are being broken. Private email exchanges are being made public. People who have no scientific training and/or no prior exposure to the scientific culture are attempting to engage in a discourse with scientists, and neither side understands the other. People are misquoting scientists, and trying to trip them up with loaded questions. And, occasionally, resorting to death threats. Outside of the scientific community, most people just don’t understand how science works, and so don’t know how to make sense of what’s going on…

When scientists are attacked for political reasons, we mistake it for an intellectual discussion over brandy in the senior common room. Scientists have no training for political battles, and so our responses often look rude or dismissive to outsiders. Which in turn gets interpreted as unprofessional behaviour by those who don’t understand how scientists talk. And unlike commercial organisations and politicians, universities don’t engage professional PR firms to make us look good, and we academics would be horrified if they did: horrified at the expense, and horrified by the idea that our research might need to be communicated on anything other than its scientific merits…

The same misunderstandings occur when outsiders look at how we talk about the peer-review process. Outsiders tend to think that all published papers are somehow equal in merit, and that peer-review is a magical process that only lets the truth through (hint: we refer to it more often as a crap-shoot). Scientists know that while some papers are accepted because they are brilliant, others are accepted because its hard to tell whether they are any good, and publication might provoke other scientists to do the necessary followup work. We know some published papers are worth reading, and some should be ignored. So, we’re natural skeptics – we tend to think that most new published results are likely to be wrong, and we tend to accept them only once they’ve been repeatedly tested and refined.

The claims of censorship are without merit. Their reply was of poor quality and did not meet the standards of peer-review.

In a recent ABC interview, Mr McLean claimed that he was “censored” because his reply to this rebuttal was rejected and thus will not appear in the peer-reviewed literature.

The rejection of this reply is a perfect example that peer review exercises quality control and not censorship.

Not getting your work published in the peer-reviewed literature is not censorship, even if people are rude in their private communications about you.