Dear PLOS ONE: time to sort out your multiple review tracks
May 17, 2013
Here at SV-POW!, we are an equal-opportunity criticiser of publishers: Springer, PLOS, Elsevier, the Royal Society, Nature, we don’t care. We call problems as we see them, where we see them. Here is one that has lingered for far too long. PLOS ONE’s journal information page says:
Too often a journal’s decision to publish a paper is dominated by what the Editor/s think is interesting and will gain greater readership — both of which are subjective judgments and lead to decisions which are frustrating and delay the publication of your work. PLOS ONE will rigorously peer-review your submissions and publish all papers that are judged to be technically sound.
Which is as we would expect it to be. But their reviewer guidelines page gives more detail as follows (emphasis added):
[Academic Editors] can employ a variety of methods, alone or in combination, to reach a decision in which they are confident:
- They can conduct the peer review themselves, based on their own knowledge and experience
- They can take further advice through discussion with other members of the editorial board
- They can solicit reports from further referees
As has been noted in comments on this blog, this first form, in which the editor makes the decision alone, is “unlike any other first-tier academic journal”. When I submitted my own manuscript to PLOS ONE a few weeks ago, I did it in the expectation that it would be reviewed in the usual way, by two experts chosen by the editor, who would then use those reviews in conjunction with her own expertise to make a decision. I’d hate to think it would go down the easier track, and so not be accorded the recognition that a properly peer-reviewed article gets. (Merely discussing with other editors would also not constitute proper peer-review in many people’s eyes, so only the third track is really the whole deal.)
The problem here is not a widespread one. Back when we first discussed this in any detail, about 13% of PLOS ONE papers slipped through on the editor-only inside lane. But more recent figures (based on the 1,837 manuscripts that received a decision between 1st July and 30th September 2010) say that only 4.2% of articles take this track. Evidently the process was by then in decline; it’s a shame we don’t have more recent numbers.
But the real issue here is lack of transparency. Four and half years ago, Matt said “I really wish they’d just state the review track for each article–i.e., solo editor approved, multiple editor approved, or externally reviewed […] I also hope that authors are allowed to preferentially request ‘tougher’ review tracks”.
It seems that still isn’t done. Looking at this article, which at the time of writing is the most recent one published by PLOS ONE, there is a little “PEER REVIEWED” logo up at the top, but no detail of which track was taken. PLOS themselves evidently take the line that all three tracks constitute peer-review, as “Academic Editors are not employees […] they are external peer reviewers“.
So I call on PLOS ONE to either:
A. eliminate the non-traditional peer-review tracks, or
B1. Allow submitting authors to specify they want the traditional track, and
B2. Specify explicitly on each published paper which track was taken.
Sauropod Vertebra Picture of the Week 
May 17, 2013 at 7:06 am
Actually, as an academic editor of P1, I admit to having peer-reviewed papers myself without reviewers: when I suspected the work to be so weak, my colleagues would hate me for annoying them with weak papers. In those cases where this turned out to be the case, I rejected the papers without outside review. So I’m not sure if that’s the ‘easy track’ lol :-)
That being said, of course the review procedure needs to be transparent upon publication as you demand! As you correctly point out, this has been a request for ages, but never followed through. In fact, there are a number of such requests and suggestions that we, as academic editors, have voiced (some as far back as the inception of P1!), where P1 staff agreed they would implement them, but never followed through. My guess is that they’re so busy keeping pace with grows, that innovation/implementation falls by the way side – but that’s pure speculation.
May 17, 2013 at 7:32 am
I second Björn here. Although I have never accepted a paper without reviewers, I have rejected a very few, or asked for a first round of major revision before I dare bother colleagues. And especially more transparency is needed! In my opinion, this goes through publishing all reviews (http://people.unil.ch/marcrobinsonrechavi/2012/11/my-2-cents-on-open-peer-review/). Then you see whether it was reviewed, how many round, which points were addressed or not, etc. But somehow the big machine which is PLOS One is slow to respond to such initiatives. Also (incomprehensible to me) some AEs are opposed to publishing reviews.
May 17, 2013 at 7:37 am
Interesting to see the distinction between accepting without outside-review involvement (which is what my post was meant to be criticising) and rejecting without outside-reviewer involvement. I hadn’t thought of PLOS ONE doing the latter, but of course it’s essentially the same as any other “reject without review”, which is something we’re all familiar with from mainstream journals. I guess I have no problem with that: the problem is in the other direction, that while the easy track continues to exist for acceptance, it potentially compromises the credibility of all other published PLOS ONE papers that might, for all anyone knows, have gone through this track.
May 17, 2013 at 7:42 am
I am an editor for PLOS ONE and the only manuscripts for which I reach a decision without peer-review is when the manuscripts are so exceedingly poor that I do not want to send them out for review (just as Björn wrote above).
I would really appreciate it if PLOS ONE would publish review reports together with the paper: it would certainly dispell the notion that is held by some that peer-review for PLOS ONE is somehow less strict than for other journals.
May 17, 2013 at 7:46 am
The question is: is it always the ‘easy track’. In order to make that decision, you need not only to look at acceptance but at all handling. If only single digit percentages are published via this route, perhaps manifold that number has been rejected via that route?
Obviously, it may be that only bad papers have been accepted by buddies of the authors via that route!
What I’m trying to say is that it’s not obviously clear that editor accept was an easy route. All of the above, of course, doesn’t affect the need for transparency, I think we all agree on that. Should I ever accept a paper without review (don’t know why I would do that) I will stand by that decision and would want it to be transparent.
P.S.: I have accepted papers without P1 reviewers in cases where the authors have sent me reviews from previous submissions. In these cases I left a comment at the paper noting this information.
May 17, 2013 at 7:51 am
WvSchaik writes:
Agreed. We are big fans of open peer-review, and I see no remotely credible reason other than tradition for any journal not publishing its reviews. We’ve previously discussed six important advantages of publishing reviews, but we missed this one:
Good point. Thinking about it, the availability of PeerJ’s reviews may be an important part of how it’s managed to establish itself so quickly and be so widely respected. It’s shown that editorial standards are high, and made a nonsense of any “peer-review lite” mud that traditionalists might have been preparing to sling. Given that such nonsense still crops up from time to time regarding PLOS ONE, it would be useful to the journal to simply show the world what the reality is.
May 17, 2013 at 8:10 am
As far as I know there is no respectable journal claiming to be peer-reviewed that PASSES manuscripts without external peer review. And REJECTION based on insufficient quality – well, if a paper is not scientific there must be a way of throwing it out without wasting someone’s time.
May 17, 2013 at 8:22 am
“P.S.: I have accepted papers without P1 reviewers in cases where the authors have sent me reviews from previous submissions. In these cases I left a comment at the paper noting this information.”
+1 Bjoern. I wish this was more widespread practice, as it reduces the burden on the community, and speeds up acceptance times. If the only reason for rejection from a previously chosen journal was that the journal likes to be exclusive, but the referees said the was sound, then there is no need to have it checked again.
May 17, 2013 at 8:34 am
Use hashtag #SauropodPress on twitter if you think Mike T should have a go at starting a new science journal. What better way to change practices in the community by taking a lead?
May 17, 2013 at 10:20 am
“…said the paper was sound…”
In fact in mathematics this is even more true, since referees can actually check a proof for correctness. It’s like being able to reproduce the experiment that the author(s) performed and then say to the editor of the journal: yes, this paper does what it says.
May 30, 2017 at 5:03 pm
Why start yet another journal? The more journals there are, the easier it becomes to overlook a paper.
May 30, 2017 at 7:35 pm
It wasn’t a serious suggestion, David. Andrew has a bit of the troll about him at times.