Open peer-review at PeerJ

February 14, 2013

There are a lot of things to love about PeerJ, which of course is why we sent our neck-anatomy paper there. I’ll discuss another time how its pricing scheme changes everything for Gold OA in the sciences, and maybe another time write about how well its papers display on mobile devices, or about the quick turnaround or 21st-century graphical design of the PDFs.

But among the most interesting things about PeerJ is its use of open peer review: reviewers are encouraged (though not required) to disclose their identity, and authors are encouraged (but also not required) to make the review history publicly available along with the final papers.

Uptake of open peer-review

Uptake of this option on the initial batch of 30 papers has been OK: 12 papers (40%) have had reviews posted:

# Title Reviews
1 How long is a piece of loop? Reviews
2 Malleable ribonucleoprotein machine: protein intrinsic disorder in the Saccharomyces cerevisiae spliceosome Reviews
3 The roles of STP and LTP in synaptic encoding Reviews
5 Bacterial curli protein promotes the conversion of PAP248-286 into the amyloid SEVI: cross-seeding of dissimilar amyloid sequences Reviews
6 Soil carbon determination by thermogravimetrics
7 Mutations changing tropomodulin affinity for tropomyosin alter neurite formation and extension
8 Dealing with the unexpected: consumer responses to direct-access BRCA mutation testing Reviews
9 The effects of fixation target size and luminance on microsaccades and square-wave jerks
10 Influence of the experimental design of gene expression studies on the inference of gene regulatory networks: environmental factors
11 Assessing insect responses to climate change: What are we testing for? Where should we be heading? Reviews
12 Novel control of lactate dehydrogenase from the freeze tolerant wood frog: role of posttranslational modifications
13 Dissecting the mechanisms of squirrel monkey (Saimiri boliviensis) social learning
14 Simultaneous recordings of ocular microtremor and microsaccades with a piezoelectric sensor and a video-oculography system
15 Fluorescent protein tagging confirms the presence of ribosomal proteins at Drosophila polytene chromosomes Reviews
16 Na+/Ca2+ selectivity in the bacterial voltage-gated sodium channel NavAb Reviews
17 Timing of molt of barn swallows is delayed in a rare Clock genotype
19 Perceptual elements in Penn & Teller’s “Cups and Balls” magic trick Reviews
21 Glucose-6-phosphate dehydrogenase regulation in the hepatopancreas of the anoxia-tolerant marine mollusc, Littorina littorea
22 Poorer verbal working memory for a second language selectively impacts academic achievement in university medical students Reviews
25 Repeated hands-and-knees positioning during labour: a randomized pilot study
26 Organ homologies in orchid flowers re-interpreted using the Musk Orchid as a model
27 Novel enzyme-polymer conjugates for biotechnological applications
28 Reduced expression of glycolate oxidase leads to enhanced disease resistance in rice
29 Anti-apoptotic signaling as a cytoprotective mechanism in mammalian hibernation
30 A practical implementation of de-Pake-ing via weighted Fourier transformation
31 Analysis of innate and acquired resistance to anti-CD20 antibodies in malignant and nonmalignant B cells
33 A perfusion study of the handling of urea and urea analogues by the gills of the dogfish shark (Squalus acanthias)
34 Coronatine inhibits stomatal closure and delays hypersensitive response cell death induced by nonhost bacterial pathogens Reviews
36 Why sauropods had long necks; and why giraffes have short necks Reviews
37 Pain assessment in children undergoing venipuncture: the Wong-Baker faces scale versus skin conductance fluctuations

(Articles 4, 18, 20, 23, 24, 32 and 35 do not exist — presumably they didn’t make it through review, typesetting and proofing in time for the launch. Or maybe they were rejected after having been assigned numbers.)

It’s interesting to see that most of the earliest papers did elect to publish reviews, but few of the later ones. This may reflect that the “early adopters” — the people who were quickest to get their submissions in after PeerJ opened its doors — also tend to be the more open-oriented people in other respects. It would be great if the authors of some of those other 18 papers were to make their reviews open, too: I’m sure it’s not too late.

What’s the value of open peer-review?

First, it improves transparency. In standard peer-review, three people (and editor and two reviewers) make a decision on behalf of the entire community, and no-one else can see what was done or why. In our case, John Hutchinson was our handling editor. We’ve often said on this blog how much we like and respect him, and it would be easy for someone on the outside to suspect that he’d been tempted to give us an easy ride. Anyone who reads the review history can see for themselves that he didn’t.

Second, it gives credit where it’s due. Reviewers who do a good job often plough in many hours of time that they could be spending on their own work, and it’s right that they should be recognised. In this case, Heinrich Mallison did a careful line-by-line critique of the whole 50-page manuscript and sent up a marked-up copy which was invaluable in making revisions. That sort of work should be acknowledged. [At the moment, that marked-up manuscript is not on the PeerJ review-history page. I’ve been told they’re going to fix that.]

Third, it gives blame where it’s due. Some reviewers who are excessively critical, or criticise in a non-constructive way that can’t be addressed in a revision; others are positive about the manuscript but make no real contribution to improve it. It’s right that reviewers who don’t do their job properly should be called out on that. (Of course anonymity can go some way towards shielding bad reviewers, but even then it’s often quite obvious who’s responsible for a given review.)

Fourth, it encourages good behaviour from reviewers. When they know their good work will receive credit and their bad work will reflect on them, they will have more incentive to do their best. Too often, reviews are seen as a tax on researchers’ time. Making them visible helps to bring them into the mainstream.

Fifth, it avoids wasted effort. Sometimes a review is a serious piece of work in its own right — Matt tells me that for one manuscript we was refereeing, he wrote a detailed critical review that was longer than the  manuscript itself. Of course, no-one ever saw that work but the original author and his handling editor, which is a terrible waste. Publishing reviews fixes that.

Sixth, and this is crucial, open peer-review is a fantastic teaching tool. Matt has already explained how showing his Western students real reviews in a real process is going to help them much more than made up ones.

What are the drawbacks of open peer-review?

Search me. I sure as heck can’t think of any.

Changing peer-review culture

PeerJ didn’t invent open peer-review — far from it. It’s been around for a while, practiced by some BMC journals and also adopted more recently by eLIFE — another of the new breed of born-digital open-access journals. Another new publishing initiative, F1000 Research, is built entirely on the concept of open review.

The importance of PeerJ doing the same is that it helps to bring open peer-review into the mainstream. PeerJ’s going to be a big journal — its explicit goal is to be a PLOS ONE-scale megajournal. One of the many things it can achieve is to help shift the default reviewing culture to open.

7 Responses to “Open peer-review at PeerJ

  1. Mike from Ottawa Says:

    Drawbacks of open peer review? There are none as long as nobody in science has power or influence that allows them to exert pressure (before or after) on a reviewer and everyone knows nobody has any such ability.

    If that’s not the case, it doesn’t mean open peer review is A Bad Thing, just that it may not be an unalloyed good and folk should be on the lookout for the effects of such pressures.

    I can assure folk that transparency in government in the form of access to information legislation does not have solely good effects. I makes some folk cagey about what they say and how, and that’s not always a good thing.

  2. […] If you’ll forgive me a rather self-indulgent post, the neck-anatomy paper that I and Matt recently had published in PeerJ is important to me for three reasons beyond the usual satisfaction of getting a piece of work out in a good journal. […]

  3. […] about how peer review works. Not only did it feel fantastic to be able to point the students to a whole bunch of published examples of peer review “in the wild”, but I got some good questions and comments after class. I […]

  4. […] hurt science, like rejecting papers because of anticipated sexiness or for other BS reasons, not publishing peer reviews, etc. Happily, now there are better […]

  5. Mark Costello Says:

    A potential drawback is that oversights and mistakes by authors become public.

  6. Mike Taylor Says:

    Is that a drawback? Sounds to me more like the way science progresses. We can learn from each others’ mistakes instead of repeating them ourselves.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: