“The benefit of published work is that if they have passed the muster of peer review future researchers can have faith in the results”, writes a commenter at The Economist. Such statements are commonplace.
I couldn’t disagree more. Nothing is more fatal to the scientific endeavour than having “faith” in a previously published result — as the string of failed replications in oncology and in social psychology is showing. See also the trivial but crucial spreadsheet error in the economics paper that underlies many austerity policies.
Studies have shown that peer-reviewers on average spend about 2-3 hours in evaluating a paper that’s been sent their way. There is simply no way for even an expert to judge in that time whether a paper is correct: the best they can do is say “this looks legitimate, the authors seem to have gone about things the right way”.
Now that is a useful thing to be able to say, for sure. Peer review is important as a stamp of serious intent. But it’s a long way from a mark of reliability, and enormous damage is done by the widespread assumption that it means more than it does.
Remember: “has passed peer review” only really means “two experts have looked at this for a couple of hours, and didn’t see anything obviously wrong in it”.
Note. I initially wrote this as a comment on a pretty good article about open access at The Economist. That article is not perfect, but it’s essentially correct, and it makes me happy that these issues are now mainstream enough that it’s no longer a surprise when they’re covered by as mainstream an outlet as The Economist.
What specimen is this — the reveal!
May 2, 2013
Yesterday I asked whether anyone could identify this specimen:
There was an interesting range of suggestions, but I suppose no-one will be surprised to hear that Darren Naish was the first to make real progress, saying “Hey, that’s a loooong pelvis… I smell macropod.” From there it was a short leap to William Miller asking “Could it be that wallaby from way back in Things to Make & Do part 3?”
Yes it could, William — you win ten shiny new SV-POW! dollars.
It is indeed Logan the wallaby from waaay back in late 2009. Here’s how I butchered him, and some detail on his feet, and how his skull turned out. Back then I prepped out a forelimb and a hindlimb, the skull and first few cervicals, and the tail (which I don’t think we’ve ever featured here — I should fix that.) When I ran out of time to work on the rest of the specimen, I just dumped it in a plastic tub, added water, and left it for nature to do the work for me. The plan was to fish out the goodies a few months later, but it seems that while my back was turned, three and a half years have passed. I should get on that — if the bones haven’t softened to the point where they’re useless now.
BTW., AnJaCo wins a bonus prize of five SV-POW! dollar for guessing that the specimen was “Sub-adult or juvenile. From the aforementioned disarticulated innominate, and from the dissociated epiphyses of the centra”. Logan was eighteen months old at death, which makes him a sub-adult as Bennett’s wallabies mature at 20-24 months.
What specimen is this?
May 1, 2013
A quiz. What is this?
Here it is in close-up:
(Click through the pictures for full resolution.)
Anyone know?
Sauropod Vertebra Picture of the Week 