One of the trends over the past year has been increasing concern over the functionality of the peer review system. The system is not broken, but it is creaking under the strain of an ever increasing number of articles and the inability to stop scientific fraud, something it was never designed to do. One of the responses to this has been to start a new journal with an open, interactive peer review process. The new peer review process works as follows:
- The author submits work, which is read by an editor who decides whether the paper will enter into the peer review process.
- The paper is simultaneously sent to reviewers and published on a review section of the journal's website.
- The reviewers respond in the traditional fashion, with a recommendation on publication, comments and suggested changes. These are also published alongside the article. Other scientists can also make comments on the article as well.
- The author responds to the reviewers and other scientists and in light of their comments and recommendations provides a revised edition of the paper.
- Provided that the paper was recommended for publication, the revised paper is published and the discussion closed.
Now this process is not all that different from normal peer review, except that it is all in public and the result is more like a conference presentation, where there is time for discussion at the end of the presentation. Read on for an example of this in action.
Buried in the details of understanding our climate is the interaction between the ice sheets, solar forcing and carbon dioxide. The variation in ice-sheet volume over the last 2.7 million years follows cycles that have periods of approximately 100,000, 41,000, and 23,000 years, which cannot be explained by solar forcing alone. The standard explanation for this is that something within the climate must have driven the expansion or contraction of the ice shelf. The leading candidate for driving the change is carbon dioxide, which provides a nonlinear feedback mechanism, accelerating the formation or destruction of the ice shelf in a self reenforcing mechanism. However, Ruddiman presents an alternative, in which ice shelf formation and destruction drives carbon dioxide concentration in a positive feedback loop. The differences between the two explanations are quantitative, in that the first explanation requires that there is a large lag between the solar forcing peaks and the carbon dioxide peaks, while Ruddiman's theory implies that there is almost no lag between changes in the ice shelf and changes in the carbon dioxide level. Unfortunately, the data are equivocal and cannot at present falsify either approach. However, the mechanisms that would allow the ice shelf to control carbon dioxide levels are not well understood.
Now onto to the referee's comments. The first referee was generally positive about the approach but had concerns with the either/or and cause/effect approach of Ruddiman to what is a closely coupled system. Ruddiman's response was that the either/or false dichotomy was precisely that, however, there is not enough time to explore every option and this approach still provides useful information. Replacing the words cause and effect with "interaction" would not add to the information content of the paper, but would make it less clear. The second referee was also positive about the approach but worried that the paucity of data would make it hard to falsify any particular model. This is due to the fact that the paper focuses on a time period during which only a single 100,000 year cycle occurred. The reviewer also had problems with the apparent lack of analysis of the lead or lag times between ice shelf behavior and carbon dioxide. Ruddiman admitted that in trying to keep the paper focused he had cut much of the analysis and would reinstate that in the revised edition.
There were also comments by two non-referees. One was involved in related climate modeling projects, who happened to be studying deep ocean mixing processes, which are involved in some of Ruddiman's proposed mechanisms. He pointed out several additional possible mechanisms, and more to the point, why they and some of Ruddiman's proposals probably wouldn't work. The final comment was a slightly bizarre and flawed attack on the Milankovitch hypothesis (that orbital variations are the primary driver of ice shelf formation), which form the basis of much basic climatology. All I can say here is that at least Ruddiman was polite about it.
I think the greatest thing about this paper is that it has provided me with the chance of giving non scientists an in sight into peer review and how it works in practice. Having peer review in public is, I think, a good thing. The fact that the reviewers know that their comments are going to be made public makes them pause for thought, frame their arguments more clearly, and generally make a longer, more useful response. That can only be a good thing. Oh, and the paper — interesting, plausible, but lacking convincing mechanisms.