Dear PLOS ONE: time to sort out your multiple review tracks

May 17, 2013

Here at SV-POW!, we are an equal-opportunity criticiser of publishers: SpringerPLOS, Elsevier, the Royal Society, Nature, we don’t care. We call problems as we see them, where we see them. Here is one that has lingered for far too long. PLOS ONE’s journal information page says:

Too often a journal’s decision to publish a paper is dominated by what the Editor/s think is interesting and will gain greater readership — both of which are subjective judgments and lead to decisions which are frustrating and delay the publication of your work. PLOS ONE will rigorously peer-review your submissions and publish all papers that are judged to be technically sound.

Which is as we would expect it to be. But their reviewer guidelines page gives more detail as follows (emphasis added):

[Academic Editors] can employ a variety of methods, alone or in combination, to reach a decision in which they are confident:

  • They can conduct the peer review themselves, based on their own knowledge and experience
  • They can take further advice through discussion with other members of the editorial board
  • They can solicit reports from further referees

As has been noted in comments on this blog, this first form, in which the editor makes the decision alone, is “unlike any other first-tier academic journal”. When I submitted my own manuscript to PLOS ONE a few weeks ago, I did it in the expectation that it would be reviewed in the usual way, by two experts chosen by the editor, who would then use those reviews in conjunction with her own expertise to make a decision. I’d hate to think it would go down the easier track, and so not be accorded the recognition that a properly peer-reviewed article gets. (Merely discussing with other editors would also not constitute proper peer-review in many people’s eyes, so only the third track is really the whole deal.)

The problem here is not a widespread one. Back when we first discussed this in any detail, about 13% of PLOS ONE papers slipped through on the editor-only inside lane. But more recent figures (based on the 1,837 manuscripts that received a decision between 1st July and 30th September 2010) say that only 4.2% of articles take this track. Evidently the process was by then in decline; it’s a shame we don’t have more recent numbers.

But the real issue here is lack of transparency. Four and half years ago, Matt said “I really wish they’d just state the review track for each article–i.e., solo editor approved, multiple editor approved, or externally reviewed […] I also hope that authors are allowed to preferentially request ‘tougher’ review tracks”.

It seems that still isn’t done. Looking at this article, which at the time of writing is the most recent one published by PLOS ONE, there is a little “PEER REVIEWED” logo up at the top, but no detail of which track was taken. PLOS themselves evidently take the line that all three tracks constitute peer-review, as “Academic Editors are not employees […] they are external peer reviewers“.

So I call on PLOS ONE to either:

A. eliminate the non-traditional peer-review tracks, or

B1. Allow submitting authors to specify they want the traditional track, and

B2. Specify explicitly on each published paper which track was taken.


10 Responses to “Dear PLOS ONE: time to sort out your multiple review tracks”

  1. Actually, as an academic editor of P1, I admit to having peer-reviewed papers myself without reviewers: when I suspected the work to be so weak, my colleagues would hate me for annoying them with weak papers. In those cases where this turned out to be the case, I rejected the papers without outside review. So I’m not sure if that’s the ‘easy track’ lol :-)

    That being said, of course the review procedure needs to be transparent upon publication as you demand! As you correctly point out, this has been a request for ages, but never followed through. In fact, there are a number of such requests and suggestions that we, as academic editors, have voiced (some as far back as the inception of P1!), where P1 staff agreed they would implement them, but never followed through. My guess is that they’re so busy keeping pace with grows, that innovation/implementation falls by the way side – but that’s pure speculation.

  2. I second Björn here. Although I have never accepted a paper without reviewers, I have rejected a very few, or asked for a first round of major revision before I dare bother colleagues. And especially more transparency is needed! In my opinion, this goes through publishing all reviews ( Then you see whether it was reviewed, how many round, which points were addressed or not, etc. But somehow the big machine which is PLOS One is slow to respond to such initiatives. Also (incomprehensible to me) some AEs are opposed to publishing reviews.

  3. Mike Taylor Says:

    Interesting to see the distinction between accepting without outside-review involvement (which is what my post was meant to be criticising) and rejecting without outside-reviewer involvement. I hadn’t thought of PLOS ONE doing the latter, but of course it’s essentially the same as any other “reject without review”, which is something we’re all familiar with from mainstream journals. I guess I have no problem with that: the problem is in the other direction, that while the easy track continues to exist for acceptance, it potentially compromises the credibility of all other published PLOS ONE papers that might, for all anyone knows, have gone through this track.

  4. WvSchaik Says:

    I am an editor for PLOS ONE and the only manuscripts for which I reach a decision without peer-review is when the manuscripts are so exceedingly poor that I do not want to send them out for review (just as Björn wrote above).
    I would really appreciate it if PLOS ONE would publish review reports together with the paper: it would certainly dispell the notion that is held by some that peer-review for PLOS ONE is somehow less strict than for other journals.

  5. brembs Says:

    The question is: is it always the ‘easy track’. In order to make that decision, you need not only to look at acceptance but at all handling. If only single digit percentages are published via this route, perhaps manifold that number has been rejected via that route?

    Obviously, it may be that only bad papers have been accepted by buddies of the authors via that route!

    What I’m trying to say is that it’s not obviously clear that editor accept was an easy route. All of the above, of course, doesn’t affect the need for transparency, I think we all agree on that. Should I ever accept a paper without review (don’t know why I would do that) I will stand by that decision and would want it to be transparent.

    P.S.: I have accepted papers without P1 reviewers in cases where the authors have sent me reviews from previous submissions. In these cases I left a comment at the paper noting this information.

  6. Mike Taylor Says:

    WvSchaik writes:

    I would really appreciate it if PLOS ONE would publish review reports together with the paper.

    Agreed. We are big fans of open peer-review, and I see no remotely credible reason other than tradition for any journal not publishing its reviews. We’ve previously discussed six important advantages of publishing reviews, but we missed this one:

    It would certainly dispell the notion that is held by some that peer-review for PLOS ONE is somehow less strict than for other journals.

    Good point. Thinking about it, the availability of PeerJ’s reviews may be an important part of how it’s managed to establish itself so quickly and be so widely respected. It’s shown that editorial standards are high, and made a nonsense of any “peer-review lite” mud that traditionalists might have been preparing to sling. Given that such nonsense still crops up from time to time regarding PLOS ONE, it would be useful to the journal to simply show the world what the reality is.

  7. As far as I know there is no respectable journal claiming to be peer-reviewed that PASSES manuscripts without external peer review. And REJECTION based on insufficient quality – well, if a paper is not scientific there must be a way of throwing it out without wasting someone’s time.

  8. “P.S.: I have accepted papers without P1 reviewers in cases where the authors have sent me reviews from previous submissions. In these cases I left a comment at the paper noting this information.”

    +1 Bjoern. I wish this was more widespread practice, as it reduces the burden on the community, and speeds up acceptance times. If the only reason for rejection from a previously chosen journal was that the journal likes to be exclusive, but the referees said the was sound, then there is no need to have it checked again.

  9. Andrew Miller (Elsevier) Says:

    Use hashtag #SauropodPress on twitter if you think Mike T should have a go at starting a new science journal. What better way to change practices in the community by taking a lead?

  10. “…said the paper was sound…”

    In fact in mathematics this is even more true, since referees can actually check a proof for correctness. It’s like being able to reproduce the experiment that the author(s) performed and then say to the editor of the journal: yes, this paper does what it says.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: