Fumbling towards transparency: the Royal Society’s “reject & resubmit” and submitted/published dates
July 31, 2014
Regulars will remember that nearly two years ago, I reviewed a paper for the Royal Society’s journal Biology Letters, recommended acceptance with only trivial changes (as did both other reviewers) and was astonished to see that it was rejected outright. There was an invitation to resubmit, with wording that made it clear that the resubmission would be treated as a brand new manuscript; but when the “resubmission” was made, it was accepted almost immediately without being sent to reviewers at all — proving that it was in fact a minor revision.
What’s worse, the published version gives the dates “Received August 21, 2012.
Accepted September 13, 2012″, for a submission-to-acceptance time of just 23 days. But my review was done before August 21. This is a clear falsifying of the true time taken to process the manuscript, a misrepresentation unworthy of the Royal Society, and which provoked Matt and me to declare that we would no longer provide peer-review for the Society until they fix this.
By the way, we should be clear that the Royal Society is not the only publisher that does this. For example, one commenter had had the same experience with Molecular Ecology. Misreporting the submission/revision cycle like this works to publishers’ benefit in two ways: it makes them look faster than they really are, and makes the rejection rate look higher (which a lot of people still use as a proxy for prestige).
To the Society’s credit, they were quick to get in touch, and I had what at time seemed like a fruitful conversation with Dr Stuart Taylor, their Commercial Director. The result was that they made some changes:
- Editors now have the additional decision option of ‘revise’. This provides a middle way between ‘reject and resubmit’ and ‘accept with minor revisions’. [It's hard to believe this didn't exist before, but I guess it's so.]
- The Society now publicises ‘first decision’ times rather than ‘first acceptance’ times on their website.
As I noted at the time, while this is definitely progress, it doesn’t (yet) fix the problem.
A few days ago, I checked whether things have improved by looking at a recent article, and was disappointed to see that they had not. I posted two tweets:
e.g. See this very new @royalsociety article rspb.royalsocietypublishing.org/content/281/17… and note that it has Received and Accepted dates but no Revised date.—
Mike Taylor (@MikeTaylor) July 23, 2014
Again, I want to acknowledge that the Royal Society is taking this seriously: less than a week later I heard from Phil Hurst at the Society:
I was rather surprised to read your recent tweets about us not fixing this bug. I thought it was resolved to your satisfaction.
Because newly published articles still only have two dates (submitted and accepted) it’s impossible to tell whether the “submitted” date is that of the original submission (which would be honest) or that of the revision, styled “a new submission” even though it’s not, that follows a “reject and resubmit” verdict.
Also: if the journals are still issuing “reject and resubmit” and then accepting the supposed new submissions without sending them out for peer-review (I can’t tell whether this is the case) then that is also wrong.
Sorry to be so hard to satisfy :-) I hope you will see and agree that it comes from a desire to have the world’s oldest scientific society also be one that leads the way in transparency and honesty.
And Phil’s response (which I quote with his kind permission):
I feel the changes we have made provide transparency.
Now that the Editors have the ‘revise’ option, this revision time is now incorporated in the published acceptance times. If on the other hand the ‘reject and resubmit’ option is selected, the paper has clearly been rejected and the author may or may not re-submit. Clearly if a paper had been rejected from another journal and then submitted to us, we would not include the time spent at that journal, so I feel our position is logical.
We only advertise the average ‘receipt to first decision’ time. As stated previously, we feel this is more meaningful as it gives prospective authors an indication of the time, irrespective of decision.
After all that recapitulation, I am finally in a position to lay out what the problems are, as I perceive them, in how things currently stand.
- Even in recently published articles, only two dates are given: “Received May 13, 2014. Accepted July 8, 2014″. It’s impossible to tell whether the first of those dates is that of the original submission, or the “new submission” that is really a minor revision following a reject-and-resubmit verdict.
- It’s also impossible to tell what “receipt to first decision” time is in the journal’s statistics. Is “receipt” the date of the revision?
- We don’t know what the journals’ rejection rates mean. Do they include the rejections of articles that are in fact published a couple of weeks later?
So we have editorials like this one from 2012 that trumpet a rejection rate of 78% (as though wasting the time of 78% of their authors is something to be proud of), but we have no idea what that number represents. Maybe they reject all articles initially, then accept 44% of them immediately on resubmission, and call that a 22% acceptance rate. We just can’t tell.
All of this uncertainly comes from the same root cause: the use of “reject and resubmit” to mean “accept with minor revisions”.
What can the Royal Society do to fix this? Here is one approach:
- Each article should report three dates instead of two. The date of initial submission, the date of resubmission, and the date of acceptance. Omitting the date of initial submission is actively misleading.
- For each of the statistics they report, add prose that is completely clean on what is being measured. In particular, be clear about what “receipt” means.
But a much better and simpler and more honest approach is just to stop issuing “reject and resubmit” verdicts for minor revisions. All the problems just go away then.
“Minor revisions” should mean “we expect the editor to be able to make a final decision based on the changes you make”.
“Major revisions” should mean “we expect to send the revised manuscript back out to the reviewers, so they can judge whether you’ve made the necessary changes”.
And “reject and resubmit” should mean “this paper is rejected. If you want to completely retool it and resubmit, feel free”. It is completely inappropriate to accept a resubmitted paper without sending it out to peer review: doing so unambiguously gives the lie to the claim in the decision letter that “The resubmission will be treated as a new manuscript”.
Come on, Royal Society. You’ve been publishing science since 1665. Three hundred and forty-nine years should be long enough to figure out what “reject” means. You’re better than this.
And once the Royal Society gets this fixed, it will become much easily to persuade other publishers who’ve been indulging in this shady practice to mend their ways, too.
That last one really hurts. Here’s the original image, which should have gone in the paper with the interpretive trace next to it rather than on top of it:
Papers referenced in these slides:
- Taylor, M.P., and Wedel, M.J. 2013b. The effect of intervertebral cartilage on neutral posture and range of motion in the necks of sauropod dinosaurs. PLOS ONE 8(10): e78214. 17 pages. doi:10.1371/journal.pone.0078214
- Wedel, M.J. 2007a. What pneumaticity tells us about ‘prosauropods’, and vice versa. Special Papers in Palaeontology 77:207-222.
- Wedel, Mathew J., Richard L. Cifelli and R. Kent Sanders. 2000b. Osteology, paleobiology, and relationships of the sauropod dinosaur Sauroposeidon. Acta Palaeontologica Polonica 45(4): 343-388.
September 20, 2013
I was astonished yesterday to read Understanding and addressing research misconduct, written by Linda Lavelle, Elsevier’s General Counsel, and apparently a specialist in publication ethics:
While uncredited text constitutes copyright infringement (plagiarism) in most cases, it is not copyright infringement to use the ideas of another. The amount of text that constitutes plagiarism versus ‘fair use’ is also uncertain — under the copyright law, this is a multi-prong test.
So here (right in the first paragraph of Lavelle’s article) we see copyright infringement equated with plagiarism. And then, for good measure, the confusion is hammered home by the depiction of fair use (a defence against accusations of copyright violation) depicted as a defence against accusations of plagiarism.
This is flatly wrong. Plagiarism and copyright violation are not the same thing. Not even close.
First, plagiarism is a violation of academic norms but not illegal; copyright violation is illegal, but in truth pretty ubiquitous in academia. (Where did you get that PDF?)
Second, plagiarism is an offence against the author, while copyright violation is an offence against the copyright holder. In traditional academic publishing, they are usually not the same person, due to the ubiquity of copyright transfer agreements (CTAs).
Third, plagiarism applies when ideas are copied, whereas copyright violation occurs only when a specific fixed expression (e.g. sequence of words) is copied.
Fourth, avoiding plagiarism is about properly apportioning intellectual credit, whereas copyright is about maintaining revenue streams.
Let’s consider four cases (with good outcomes is green and bad ones in red):
- I copy big chunks of Jeff Wilson’s (2002) sauropod phylogeny paper (which is copyright the Linnean Society of London) and paste it into my own new paper without attribution. This is both plagiarism against Wilson and copyright violation against the Linnean Society.
- I copy big chunks of Wilson’s paper and paste it into mine, attributing it to him. This is not plagiarism, but copyright violation against the Linnean Society.
- I copy big chunks of Rigg’s (1904) Brachiosaurus monograph (which is out of copyright and in the public domain) into my own new paper without attribution. This is plagiarism against Riggs, but not copyright violation.
- I copy big chunks of Rigg’s paper and paste it into mine with attribution. This is neither plagiarism nor copyright violation.
Plagiarism is about the failure to properly attribute the authorship of copied material (whether copies of ideas or of text or images). Copyright violation is about failure to pay for the use of the material.
Which of the two issues you care more about will depend on whether you’re in a situation where intellectual credit or money is more important — in other words, whether you’re an author or a copyright holder. For this reason, researchers tend to care deeply when someone plagiarises their work but to be perfectly happy for people to violate copyright by distributing copies of their papers. Whereas publishers, who have no authorship contribution to defend, care deeply about copyright violation.
One of the great things about the Creative Commons Attribution Licence (CC By) is that it effectively makes plagiarism illegal. It requires that attribution be maintained as a condition of the licence; so if attribution is absent, the licence does not pertain; which means the plagiariser’s use of the work is not covered by it. And that means it’s copyright violation. It’s a neat bit of legal ju-jitsu.
- Riggs, Elmer S. 1904. Structure and relationships of opisthocoelian dinosaurs. Part II, the Brachiosauridae. Field Columbian Museum, Geological Series 2:229-247, plus plates LXXI-LXXV.
- Wilson, Jeffrey A. 2002. Sauropod dinosaur phylogeny: critique and cladistic analysis. Zoological Journal of the Linnean Society 136:217-276.
September 4, 2013
I recently handled the revisions on a paper that hopefully will be in press very soon. One of the review comments was “Be very careful not to make ad hominem attacks”.
I was a bit surprised to see that — I wasn’t aware that I’d made any — so I went back over the manuscript, and sure enough, there were no ad homs in there.
There was criticism, though, and I think that’s what the reviewer meant.
Folks, “ad hominem” has a specific meaning. An “ad hominem attack” doesn’t just mean criticising something strongly, it means criticising the author rather than the work. The phrase is Latin for “to the man”. Here’s a pair of examples:
- “This paper by Wedel is terrible, because the data don’t support the conclusion” — not ad hominem.
- “Wedel is a terrible scientist, so this paper can’t be trusted” – ad hominem.
What’s wrong with ad hominem criticism? Simply, it’s irrelevant to evaluation of the paper being reviewed. It doesn’t matter (to me as a scientist) whether Wedel strangles small defenceless animals for pleasure in his spare time; what matters is the quality of his work.
Note that ad hominems can also be positive — and they are just as useless there. Here’s another pair of examples:
- “I recommend publication of Naish’s paper because his work is explained carefully and in detail” — not ad hominem.
- “I recommend publication of Naish’s paper because he is a careful and detailed worker” — ad hominem.
It makes no difference whether Naish is a careful and detailed worker, or if he always buys his wife flowers on their anniversary, or even if he has a track-record of careful and detailed work. What matters is whether this paper, the one I’m reviewing, is good. That’s all.
As it happens the very first peer-review I ever received — for the paper that eventually became Taylor and Naish (2005) on diplodocoid phylogenetic nomenclature — contained a classic ad hominem, which I’ll go ahead and quote:
It seems to me perfectly reasonable to expect revisers of a major clade to have some prior experience/expertise in the group or in phylogenetic taxonomy before presenting what is intended to be the definitive phylogenetic taxonomy of that group. I do not wish to demean the capabilities of either author – certainly Naish’s “Dinosaurs of the Isle of Wight” is a praiseworthy and useful publication in my opinion – but I question whether he and Taylor can meet their own desiderata of presenting a revised nomenclature that balances elegance, consistency, and stability.
You see what’s happening here? The reviewer was not reviewing the paper, but the authors. There was no need for him or her to question whether we could meet our desiderata: he or she could just have read the manuscript and found out.
(Happy ending: that paper was rejected at the journal we first sent it to, but published at PaleoBios in revised form, and bizarrely is my equal third most-cited paper. I never saw that coming.)
I just got this message from Rana Ashour of Paleontology Journal, an open-access journal published by Hindawi, who are generally felt to be a perfectly legitimate publisher:
Dear Dr. Taylor,
I am writing to invite you to submit an article to Paleontology Journal which is a peer-reviewed open access journal for original research articles as well as review articles in all areas of paleontology.
Paleontology Journal is published using an open access publication model, meaning that all interested readers are able to freely access the journal online without the need for a subscription, and authors retain the copyright of their work. All manuscripts that are submitted to the journal during June 2013 will not be subject to any page charges, color charges, or article processing charges.
(Apart from anything else, the waiving of APCs pretty clearly indicates that this is not a scam journal.)
Hi, Rana. Thanks for this invitation. I am supportive of Hindawi as a good-quality, low-cost open-access publisher. In particular I want Paleontology Journal to do well: it has at least one colleague of mine among its editors. I am particularly pleased to see that no APCs are payable on submissions made during June 2013.
But as a matter of principle I never respond to “academic spam”. Messages sent as bulk mailings to a broad group of potential authors are at best impolite, and at worst actively damage the reputation of the journal and its publisher — see point M on Jefffey Beall’s Criteria for Determining Predatory Open-Access Publishers.
I urge you to use what influence you have to discontinue the use of spam to advertise Paleontology Journal. If the journal is good, it can be advertised by publicising the papers that appear in it.
Dr. Michael P. Taylor
Department of Earth Sciences
University of Bristol
Let’s hope they go with it. I’d love them to build another low-cost, high-quality, journal in the palaeontology OA space, to compete with Acta Palaeontologica Polonica, Palaeontologia Electronica, PalArch and of course PLOS ONE and PeerJ. But they won’t do it by spamming.
March 14, 2013
Whenever I write a complicated document, such as my submission to the Select Committee on open access, I get Matt to do an editing pass before I finalise it. That’s always worthwhile, but I have to be careful not to just blindly hit the Accept All Changes button.
March 5, 2013
Matt and I made a sacred pact not to even think about any new work until we’d got our due-by-the-end-of-March papers done.
But then we got chatting, and accidentally started three new projects. Possibly four. And that’s just today.
Who knows how many of them will ever see the light of day? Realistically, we are surely going to have to kill some of them if we’re ever going to get anything finished. But two of them at least are likely to show up here as the kick-offs of crowdsourced projects. And we have to keep reminding ourselves: NOT TILL APRIL!
As Matt signed off tonight, he wrote:
Matt: Okay, now I gotta go.
Mike: Yeah, like TOO good.
Matt: Or disastrous, from the preventing-new-projects perspective.
We need to get to a point where we can just talk without all this Science spilling out everywhere.
I made you a chat, but I Scienced on it.
But how can we not, when sauropods are so damned fascinating?!
January 17, 2013
My new article is up at the Guardian. This time, I have taken off the Conciliatory Hat, and I’m saying it how I honestly believe it is: publishing your science behind a paywall is immoral. And the reasons we use to persuade ourselves it’s acceptable really don’t hold up.
Because for all that we rightly talk about the financial efficiencies of open access, when it comes right down to it OA is primarily a moral, or if you prefer idealogical, issue. It’s not really about saving money, though that’s a welcome side-effect. It’s about doing what’s right.
I’m expecting some kick-back on this one. Fire away; I’ll enjoy the discussion.
I’ve recently written about my increasing disillusionment with the traditional pre-publication peer-review process [post 1, post 2, post 3]. By coincidence, it was in between writing the second and third in that series of posts that I had another negative peer-review experience — this time from the other side of the fence — which has left me even more ambivalent about the way we do things.
On 17 July I was asked to review a paper for Biology Letters. Having established that it was to be published as open access, I agreed, was sent the manuscript, and two days later sent a response that recommended acceptance after only minor revision. Eleven days later, I was sent a copy of the editor’s decision — a message that included all three reviewers’ comments. I can summarise those reviewers’ comments by directly quoting as follows:
Revewer 1: “It is good to have this data published with good histological images. I have only minor comments – I think the ms should generally be accepted as it is.”
Reviewer 2 (that’s me): “This is a strong paper that brings an important new insight into a long-running palaeobiological issue [...] and should be published in essentially its current form.”
Reviewer 3: “This manuscript reports exciting results regarding sauropod biomechanics [...] The only significant addition I feel necessary is to the concluding paragraph.”
So imagine my surprise when the decision letter said:
I am writing to inform you that your manuscript [...] has been rejected for publication in Biology Letters.
This action has been taken on the advice of referees, who have recommended that substantial revisions are necessary. With this in mind we would like to invite a resubmission, provided the comments of the referees are taken into account. This is not a provisional acceptance.
The resubmission will be treated as a new manuscript.
I can’t begin to imagine how they turned three “accept with very minor revisions” reviews into “your manuscript has been rejected … on the advice of referees, who have recommended that substantial revisions are necessary”.
In fact, let’s dump the “I can’t imagine how” euphemism and say it how it is: “reviewers recommended substantial revisions” is an outright lie. The reviewers recommended no such thing. The rejection can only be because it’s what the editor wanted to do in spite of the reviewers’ comments not because of them. It left me wondering why I bothered to waste my time offering them an opinion that they were only ever going to ignore.
Then six days ago I heard from the lead author, who had just had a revised version of the same manuscript accepted. (It had not come back to me for review, as the editor had said would happen with any resubmission).
The author wrote to me:
The paper will be published (open access) at the 3rd of Octobre. When I had submitted the corrected version of the ms acceptance was only a formality. So [name] was right, they just want to keep time between submission and publishing date short.
Well. We have a word for this. We call it “lying”. When the editor wrote “your manuscript [...] has been rejected for publication in Biology Letters … With this in mind we would like to invite a resubmission … This is not a provisional acceptance. The resubmission will be treated as a new manuscript”, what she really meant was “your manuscript [...] has been provisionally accepted, please sent a revision. The resubmission will not be treated as a new manuscript”.
I find this lack of honesty disturbing.
Because we’re not talking here about some shady, obscure little third-world publisher that no-one’s ever heard of with fictional people on the editorial board. We’re talking about the Royal Freaking Society of London. We’re talking about a journal (Biology Letters) that was calved off a journal (Proceedings B) that emerged from the oldest continuously published academic journal in the world (Philosophical Transactions). We’re talking about nearly three and a half centuries of academic heritage.
And they’re lying to us about their publication process.
When did they get the idea that this was acceptable?
And what else are they lying to us about? Can we trust (for example) that when editors or members submit papers, they are subjected to the same degree of rigorous filtering as every other submission? I would have assumed that, yes, of course they do. But I just don’t know any more.
The paper in question is Klein et al.’s (2012) histological study confirming that the bony cervical ribs of sauropods are, as we suspected, ossified tendons — as we assumed in our recently arXiv’d sauropod-neck paper. I am delighted to be able to say that it is freely available. At the bottom of the first page, it says “Received 21 August 2012; Accepted 13 September 2012″, for a submission-to-acceptance time of 23 days. But I know that the initial submission — and remember, the final published version is essentially identical to that initial submission — was made before 17 July, because that’s when I was asked to provide a peer-review. Honest reporting would give a submission-to-acceptance time of 58 days, which is two and a half times as long as the claimed figure.
Now the only reason for a journal to report dates of submission and acceptance at all is to convey the speed of turnaround, and lying about that turnaround time completely removes any utility those numbers might have. It would be better to not report them at all than to fudge the data.
This is another way that the high-impact fast-turnaround publishing system is so ridiculously gamed that it actually hurts science. We have the journal lying to authors about the status of their manuscripts so that it can then lie to the readers about its turnaround times. That’s deeply screwed up. And it’s hard for authors to blow the whistle — they don’t want to alienate the journals and the editors who have some veto power over their tenure beans, and reviewers don’t usually have all the information. The obvious solution is to make the peer-review process more open, and to make editorial decisions more transparent.
That, really, is only what we’d respect from the Royal Society. Isn’t it?
Note. Nicole Klein did not know I was going to post about this. I want to make that clear so that no-one at the Royal Society thinks that she or any of her co-authors is making trouble. All the trouble is of my making (and, more to the point, the Royal Society’s). Someone really has to shine a light on this misbehaviour.
Update (12 March 2014)
I should have noted this before, but on 10 May 2013, the Royal Society sent me an update, explaining some improvements in their process. But as noted in my write-up, it doesn’t actually solve the problem. Doing so would simply require giving three dates: Received, Revised and Accepted. But as I write this, new Proc. B articles still only show Received and Accepted dates.
- Klein, Nicole, Andreas Christian, and P. Martin Sander. 2012. Histology shows that elongated neck ribs in sauropod dinosaurs are ossiﬁed tendons. Biology Letters, online first. doi:10.1098/rsbl.2012.0778
Subsequent posts discuss how this issue is developing:
- We will no longer provide peer reviews for Royal Society journals until they adopt honest editorial policies
- Biology Letters does trumpet its submission-to-acceptance time
- Lying about submission times at other journals?
- Discussing Biology Letters with the Royal Society
- The Royal Society has taken some steps to improving reporting of submit/resubmit/accept times
- Fumbling towards transparency: the Royal Society’s “reject & resubmit” and submitted/published dates