Friday, January 26, 2007

Interesting article on peer review

The CS Monitor had an interesting article on peer review.

My view is that peer review is a highly effective filter. But one should not expect too much from it. While it stops most errors from being published, it cannot catch every problem. Reviewers occasionally fail to notice an obvious mistake, and there are some types of error that reviewers usually cannot catch. They cannot tell if the author misread observations of an instrument, or wrote a number down wrong, or if chemical samples used in an experiment were contaminated. Moreover, peer review often cannot identify clever fraud, such as the rare cases where the scientific work being reported was not really done at all.

But peer review is only the first of many levels of testing and quality control applied to scientific claims. When an important or novel claim is published in a journal, other scientists test the result by trying to replicate it, often using different data sets, experimental designs, or analytic techniques. While one scientist might make a mistake, do a sloppy experiment, or misinterpret their results (and peer reviewers might fail to catch it), it is unlikely that several independent groups will make the same mistake. Consequently, as other scientists repeat an observation, or examine a question using different approaches and get the same answer, the community increasingly comes to accept the claim as correct.

Peer review is also important for evaluating proposals to funding agencies as well as for things like tenure and promotion. It seems difficult to imagine how an alternative system for those things would work. I suspect that changes in the peer review system will eventually occur as our ways of communicating changes, but those changes will be slow.

Tuesday, January 23, 2007

Front page in today's WSJ

A front page story in today's Wall Street Journal (I don't have a link, sorry) begins:
The global warming debate is shifting from science to economics.

For years, the fight over the Earth's rising temperature has been mostly over what's causing it: fossil-fuel emissions or natural factors beyond man's control. Now, some of the country's biggest industrial companies are acknowledging that fossil fuels are a major culprit whose emissions should be cut significantly over time.

...

The broadening, if incomplete, consensus that fossil fuels are at least a big part of the global-warming problem signals real change in the environmental debate. The biggest question going forward no longer is whether fossil fuel emissions should be curbed. It's who will foot the bill for the cleanup --- and that battle is heating up.
I sincerely hope that the WSJ editorial page reads this article.

We are now experiencing a tectonic shift in the political landscape on the issue of climate change. The skeptics are disappearing --- both in numbers and in influence. At the same time, the debate is shifting from "should we do something" to "what should we do?"