March 5, 2013
Matt and I made a sacred pact not to even think about any new work until we’d got our due-by-the-end-of-March papers done.
But then we got chatting, and accidentally started three new projects. Possibly four. And that’s just today.
Who knows how many of them will ever see the light of day? Realistically, we are surely going to have to kill some of them if we’re ever going to get anything finished. But two of them at least are likely to show up here as the kick-offs of crowdsourced projects. And we have to keep reminding ourselves: NOT TILL APRIL!
As Matt signed off tonight, he wrote:
Matt: Okay, now I gotta go.
Mike: Yeah, like TOO good.
Matt: Or disastrous, from the preventing-new-projects perspective.
We need to get to a point where we can just talk without all this Science spilling out everywhere.
I made you a chat, but I Scienced on it.
But how can we not, when sauropods are so damned fascinating?!
January 17, 2013
My new article is up at the Guardian. This time, I have taken off the Conciliatory Hat, and I’m saying it how I honestly believe it is: publishing your science behind a paywall is immoral. And the reasons we use to persuade ourselves it’s acceptable really don’t hold up.
Because for all that we rightly talk about the financial efficiencies of open access, when it comes right down to it OA is primarily a moral, or if you prefer idealogical, issue. It’s not really about saving money, though that’s a welcome side-effect. It’s about doing what’s right.
I’m expecting some kick-back on this one. Fire away; I’ll enjoy the discussion.
I’ve recently written about my increasing disillusionment with the traditional pre-publication peer-review process [post 1, post 2, post 3]. By coincidence, it was in between writing the second and third in that series of posts that I had another negative peer-review experience — this time from the other side of the fence — which has left me even more ambivalent about the way we do things.
On 17 July I was asked to review a paper for Biology Letters. Having established that it was to be published as open access, I agreed, was sent the manuscript, and two days later sent a response that recommended acceptance after only minor revision. Eleven days later, I was sent a copy of the editor’s decision — a message that included all three reviewers’ comments. I can summarise those reviewers’ comments by directly quoting as follows:
Revewer 1: “It is good to have this data published with good histological images. I have only minor comments – I think the ms should generally be accepted as it is.”
Reviewer 2 (that’s me): “This is a strong paper that brings an important new insight into a long-running palaeobiological issue […] and should be published in essentially its current form.”
Reviewer 3: “This manuscript reports exciting results regarding sauropod biomechanics […] The only significant addition I feel necessary is to the concluding paragraph.”
So imagine my surprise when the decision letter said:
I am writing to inform you that your manuscript […] has been rejected for publication in Biology Letters.
This action has been taken on the advice of referees, who have recommended that substantial revisions are necessary. With this in mind we would like to invite a resubmission, provided the comments of the referees are taken into account. This is not a provisional acceptance.
The resubmission will be treated as a new manuscript.
I can’t begin to imagine how they turned three “accept with very minor revisions” reviews into “your manuscript has been rejected … on the advice of referees, who have recommended that substantial revisions are necessary”.
In fact, let’s dump the “I can’t imagine how” euphemism and say it how it is: “reviewers recommended substantial revisions” is an outright lie. The reviewers recommended no such thing. The rejection can only be because it’s what the editor wanted to do in spite of the reviewers’ comments not because of them. It left me wondering why I bothered to waste my time offering them an opinion that they were only ever going to ignore.
Then six days ago I heard from the lead author, who had just had a revised version of the same manuscript accepted. (It had not come back to me for review, as the editor had said would happen with any resubmission).
The author wrote to me:
The paper will be published (open access) at the 3rd of Octobre. When I had submitted the corrected version of the ms acceptance was only a formality. So [name] was right, they just want to keep time between submission and publishing date short.
Well. We have a word for this. We call it “lying”. When the editor wrote “your manuscript […] has been rejected for publication in Biology Letters … With this in mind we would like to invite a resubmission … This is not a provisional acceptance. The resubmission will be treated as a new manuscript”, what she really meant was “your manuscript […] has been provisionally accepted, please sent a revision. The resubmission will not be treated as a new manuscript”.
I find this lack of honesty disturbing.
Because we’re not talking here about some shady, obscure little third-world publisher that no-one’s ever heard of with fictional people on the editorial board. We’re talking about the Royal Freaking Society of London. We’re talking about a journal (Biology Letters) that was calved off a journal (Proceedings B) that emerged from the oldest continuously published academic journal in the world (Philosophical Transactions). We’re talking about nearly three and a half centuries of academic heritage.
And they’re lying to us about their publication process.
When did they get the idea that this was acceptable?
And what else are they lying to us about? Can we trust (for example) that when editors or members submit papers, they are subjected to the same degree of rigorous filtering as every other submission? I would have assumed that, yes, of course they do. But I just don’t know any more.
The paper in question is Klein et al.’s (2012) histological study confirming that the bony cervical ribs of sauropods are, as we suspected, ossified tendons — as we assumed in our recently arXiv’d sauropod-neck paper. I am delighted to be able to say that it is freely available. At the bottom of the first page, it says “Received 21 August 2012; Accepted 13 September 2012″, for a submission-to-acceptance time of 23 days. But I know that the initial submission — and remember, the final published version is essentially identical to that initial submission — was made before 17 July, because that’s when I was asked to provide a peer-review. Honest reporting would give a submission-to-acceptance time of 58 days, which is two and a half times as long as the claimed figure.
Now the only reason for a journal to report dates of submission and acceptance at all is to convey the speed of turnaround, and lying about that turnaround time completely removes any utility those numbers might have. It would be better to not report them at all than to fudge the data.
This is another way that the high-impact fast-turnaround publishing system is so ridiculously gamed that it actually hurts science. We have the journal lying to authors about the status of their manuscripts so that it can then lie to the readers about its turnaround times. That’s deeply screwed up. And it’s hard for authors to blow the whistle — they don’t want to alienate the journals and the editors who have some veto power over their tenure beans, and reviewers don’t usually have all the information. The obvious solution is to make the peer-review process more open, and to make editorial decisions more transparent.
That, really, is only what we’d respect from the Royal Society. Isn’t it?
Note. Nicole Klein did not know I was going to post about this. I want to make that clear so that no-one at the Royal Society thinks that she or any of her co-authors is making trouble. All the trouble is of my making (and, more to the point, the Royal Society’s). Someone really has to shine a light on this misbehaviour.
Update (12 March 2014)
I should have noted this before, but on 10 May 2013, the Royal Society sent me an update, explaining some improvements in their process. But as noted in my write-up, it doesn’t actually solve the problem. Doing so would simply require giving three dates: Received, Revised and Accepted. But as I write this, new Proc. B articles still only show Received and Accepted dates.
- Klein, Nicole, Andreas Christian, and P. Martin Sander. 2012. Histology shows that elongated neck ribs in sauropod dinosaurs are ossiﬁed tendons. Biology Letters, online first. doi:10.1098/rsbl.2012.0778
Subsequent posts discuss how this issue is developing:
- We will no longer provide peer reviews for Royal Society journals until they adopt honest editorial policies
- Biology Letters does trumpet its submission-to-acceptance time
- Lying about submission times at other journals?
- Discussing Biology Letters with the Royal Society
- The Royal Society has taken some steps to improving reporting of submit/resubmit/accept times
- Fumbling towards transparency: the Royal Society’s “reject & resubmit” and submitted/published dates
May 8, 2012
Question. I am supposed to be meeting up with Mike Taylor at the conference, but we’ve not met before and I won’t recognise him. Do you know what he looks like?
Candidate Answer #1. He’s a bit overweight and has white hair.
Candidate Answer #2. He exhibits mild to moderate abdominal hypertrophy and accelerated ontogenetic degradation in the pigmentation of the cranial integument.
You wouldn’t use answer #2 in Real Life, so don’t use it in your papers. It’s not big, and it’s not clever.
December 14, 2011
This year, I missed The Paleo Paper Challenge over on Archosaur Musings — it was one of hundreds of blog posts I missed while I was in Cancun with my day-job and then in Bonn for the 2nd International Workshop on Sauropod Biology and Gigantism. That means I missed out on my annual tradition of promising to get the looong-overdue Archbishop description done by the end of the year.
But this year, Matt and I are going to have our own private Palaeo Paper Challenge. And to make sure we heap on maximum pressure to get the work done, we’re announcing it here.
Here’s the deal. We have two manuscripts — one of them Taylor and Wedel, the other Wedel and Taylor — which have been sitting in limbo for a stupidly long time. Both are complete, and have in fact been submitted once and gone through review. We just need to get them sorted out, turned around, and resubmitted.
(The Taylor and Wedel one is on the anatomy of sauropod cervicals and the evolution of their long necks. It’s based on the last remaining unpublished chapter of my dissertation, and turned up in a modified form as my SVPCA 2010 talk, Why Giraffes Have Such Short Necks. The Wedel and Taylor one is on the occurrence and implications of intermittent pneumaticity in the tails of sauropods, and turned up as his SVPCA 2010 talk, Caudal pneumaticity and pneumatic hiatuses in the sauropod dinosaurs Giraffatitan and Apatosaurus.)
We’re going to be realistic: we both have far too much going in (incuding, you know, families) to get these done by the end of 2011. But we have relatively clear Januaries, so our commitment is that we will submit by the end of January 2012. If either of us fails, you all have permission to be ruthlessly derisive of that person.
… and in other news …
Some time while we were all in Bonn, the SV-POW! hit-counter rolled over the One Million mark. Thanks to all of your for reading!
October 17, 2011
Regular readers will know that, as part of a broader strategy favouring open-access publishing, I no longer perform peer-reviews for non-open journals. (I mentioned a recent example in a comment on the last article.) I’ve had support for this stance from some impressive quarters; but also a fair bit of criticism from people who I respect. That includes some strong open-access advocates who agree with me on where we want to land up, but don’t like the tactics I’m using to get there.
The most detailed of those criticisms in an article entitled Should we review for any old journal? by Andy Farke, and I think it deserves a detailed response. Andy’s open-access credentials are impeccable — he writes about the issue in detail on his blog, and is an editor for PLoS ONE, by most metrics the leading open-access journal. So when he has a criticism, it’s worth hearing.
Andy has several concerns. Let’s look at them in turn.
I argue that, unless carefully constructed, such reviewing boycotts may never be noticed by some of the concerned parties. A typical journal editor will think “oh, Reviewer 1 refused to review. . .on to Reviewer 2.” Even if the refusal to review is accompanied by a note explaining the reasoning behind the refusal, only the editor will ever see it (and potentially the publishing admins – who have little vested interest in changing the status quo).
This is an excellent point. A protest that no-one knows about is not going to be an effective protest. From now on, whenever I turn down a non-open review, I will send a message to the editor, the publisher and the authors. (Andy suggests this as one candidate strategy later on in his article.)
Second, when the pool of qualified reviewers is small to begin with, this could have the consequence of letting some really bad stuff slip into publication.
I’m not sure I buy this. If a journal can’t find reviewers for an article, the only honourable thing for them to do is return it to the author and say so, not give it a free pass. At any rate, it looks like a purely hypothetical problem to me. If and when the day comes that a paper comes out that I was asked to review and declined, and I see that it’s bad and should have been blocked in review — on that day I will start to try assessing the damage. At the moment, though, the apparent damage is zero.
I am not — not quite — going to say “never”. For example, suppose someone found a more complete specimen of Xenoposeidon and submitted the description to Cretaceous Research, a non-open Elsevier journal that is actually a good match for the subject matter. That truly is a paper that would benefit most from being reviewed by the person who has spent an order of magnitude more time looking at and thinking about NHM R2095 than anyone else on the planet. In such a situation I might waive my policy.
But I’m hesitant about even admitting that. Once you start to admit that there may be extra-special circumstances, it’s easy to start making more and more exceptions. I’m not going to do that.
Anyway … Back to Andy:
Third, the journals are not the ones hurt most directly by review boycotts; it is the authors. The journal will almost always find someone else to review the paper (with a delay as these reviewers are recruited); and if not, the manuscript will be returned for lack of qualified reviewers (with a delay as the paper is prepared for submission elsewhere). Rightly or wrongly, publications are a primary currency of academia. If getting that publication delayed means my friend or colleague doesn’t get a job, or a grant, or tenure, I have hurt them, not just the profits of the journal.
Here we come to the real issue — the “collateral damage” that Andy mentioned in his title.
First, let’s say that he’s right — there is damage. A reviewing boycott is going to hurt authors. It’s regrettable. If I could hurt the non-open journals without hurting the authors, I surely would. So this is a tough situation. It’s a tough decision.
But as Matt has ably pointed out, we’re in a war. A combination of historical accidents have manoeuvred us into a position where the interests of authors are directly opposite to those of publishers: in short, authors want their papers to be read by everyone with maximum convenience, and publishers want to prevent them from being read except by an elite few who are able and willing to pay. My judgement is that whatever damage I may do to authors through a reviewing boycott is a tiny, tiny proportion of the damage that non-open publishers do to them every time they give away their work to a corporation that hides it away in a walled garden.
In short: there is no wholly good solution here. It’s a matter of finding the least bad solution. In the long term it is, unquestionably, to the advantage of all authors for open access to become ubiquitous. Without a doubt we will need to make sacrifices to reach that future, including passing up opportunities to place our work in higher impact venues. This is one more such sacrifice.
… and at this point, I’m a bit nonplussed. What did we expect? That it would just fall into our laps? That the gigantic multinational corporations that eat our work would happily hand it all back to us? That they would cheerfully give up the anti-science business model that has made them record profits year on year? Did we think there would be no fight? That we wouldn’t have to give anything up along the way?
And so on to Andy’s constructive suggestions.
1) Refuse to review the paper, but fully explain why in a letter submitted directly and separately to the editor, journal, and authors. This way everyone gets the message – not just a select few.
This is definitely the way to go. To be clear: it’s not the only strategy we should be pursuing, but it’s the best way I’ve heard to handle the problem of reviewing.
(Might journals object to an invited reviewer contacting the authors directly? I can’t think of a legitimate reason why they might, but I suppose it’s possible. Anyone have any experience of this?)
2) Review the paper, but include a message with the review (perhaps both in the review text and in a direct letter to the authors) on the shame of the work being locked behind a paywall. Make the authors think twice about whether or not the intended audience will ever see the paper.
This strikes me as weak sauce. I think of it as an emergency backup plan for the very rare cases where there really is a compelling reason to review something in a non-open journal, such as the Xenoposeidon example above.
But even then, aren’t there better alternatives? Like simply contacting the authors directly, and explaining why you think it’s important that they send the work elsewhere? Realistically, no author having gone successfully through peer-review is then going to pull the paper on a reviewer’s recommendation and submit it elsewhere. Better to raise that possibility before the review has happened.
3) Submit your own work to open access journals, cite work in open access journals, and encourage your colleagues to do the same.
Oh! Let me be very clear here: I certainly never meant to suggest a reviewing boycott as a substitute for a submission boycott! No, it’s meant to accompany a proper open-access submission policy.
Again, I am not going to say “never”. There are situations where no doubt I will be more or less forced to allow my work to appear in non-open venues — for example, when I speak at a conference, contribute a paper for the proceedings volume, and find that the volume is going to be non-open. But even then, there are other approaches to be taken. For example, when exactly this happened with my sauropod history paper being published in a non-open and ludicrously expensive Geological Society special volume, I found a way to retain the right to freely redistribute copies of my chapter. (I have not used the SPARC Addendum yet, but may be useful in such situations … even if it does sound like a John Grisham novel.)
OK, last lap. Here we go. Andy says:
I sympathize with the sentiment that we academics shouldn’t be propping up the questionable practices of some publishers, but we also need to avoid shooting ourselves (and our colleagues) in the foot as a result.
I have to disagree. Foot damage is regrettable, but it’s better than slavery. What’s maybe got lost in this pragmatic discussion of ways and means is that the status quo is wrong. Everyone has to make their own moral choices, but for me it would be Just Plain Wrong to perpetuate the corporate incarceration of publicly funded science.
It’s hard to write about these things without coming across as overwrought and hysterical, but let me try an analogy here. The economic sanctions against South Africa in the 1980s, intended to bring about the end of apartheid, most certainly hurt the very citizens that they were intended ultimately to help. But most people would agree that history has vindicated those sanctions. It was a hard decision to make. No doubt there were plenty of anti-apartheid activists who, with the best intentions, opposed the sanctions because of their immediate negative effect on people on the ground. But, happily, longer-term thinking won out. We need to be similarly far-sighted.
Is it hyperbole to compare paywalled research with institutionalised racism? Yes, of course. But maybe not by so much as you think. The developing world is beset by appalling diseases that we in the West don’t even need to think about, and suffers constant famines. Who knows what fruitful research might have been done — both by professional scientists in those countries and by unaffiliated amateurs in the West — if only the foundational research was available to them? Open Access isn’t just a First World Problem: it potentially affects health and access to food and water for millions, or even billions, of people.
So, yeah. I am cool with a bit of collateral damage.
October 15, 2011
I was directed to an article entitled Rookie Review on Nature Jobs by a tweet from Andy Farke (author of the Open Source Palaeontologist and an editor at PLoS ONE). It has a lot of good stuff in it, once you get past the opening section. But getting past that opening was difficult for me, because my blood was boiling by the time I reached the end of the third paragraph.
Here, then, is the opening of the article, with my translation.
Claudio Casola had no idea that journal editors had consistently rated his manuscript reviews highly. Then he received an award from Amsterdam-based publisher Elsevier for his “exceptional contribution to the quality of the journal Gene”.
Translation: Casola has been suckered into investing a huge amount of time and expertise, over and over again, into improving the work of other scientists, funded from the public purse, in order to increase the profits of a foreign-owned corporation that locks away the resulting science from the people who funded it. He has done this so often and so well, that the corporation has very generously given him “an award”. Anyone care to guess the cash value of that award?
(Notice by the way that most reviewers don’t even get the courtesy of feedback from the publisher. Casola is a very rare exception.)
Casola, a postdoc in evolutionary genetics at Indiana University in Bloomington, says that his first review, in 2006, was typical of rookie referees. He spent more than 10 hours on the manuscript, poring over the details and asking faculty members for advice. After reviewing more than two dozen papers in the past five years, he has been able to cut the process down to three hours, quickly assessing the originality and merit of a paper. “Reviewing manuscripts makes me feel like I’m a fully fledged member of the scientific community,” says Casola.
There is it, folks. The actual reason he gives all this work to a profiteering corporation? They’ve managed to persuade him that they are the Scientific Community rather than a parasite that clings to it. Fished in.
“Young scientists should get involved in the process as they start building their careers, particularly since reviewers are harder and harder to find,” says Bart Wacek, an executive publisher in charge of Elsevier’s genetics portfolio based in Cambridge, Massachusetts.
Translation: “Young scientists should give us free professional work, and establish the habit early in their careers”, says Bart Wacek, an executive publisher at Elsevier. “Only by getting started young can researchers hope to develop fully-fledged Stockholm Syndrome. We need them to put in enough effort early on that the sunk cost fallacy begins to pervade their thinking: then they will invent reasons to justify to themselves why it’s a good thing to give this work to profiteers instead of to the wider scientific community. Better still, in some cases they will even evangelise on our behalf!”
Young reviewers are certainly sought after. “The best referees are postdocs,” says Leslie Sage, a senior physical-sciences editor at Nature in Washington DC. “They are at the top of their game, well versed in the literature and politically naive enough to tell the truth.”
“… and sufficiently in awe of Real, Grown-Up Journals that they will do whatever we tell them in exchange for the oxygen of acceptance. Catch ‘em while they’re young!”
Despite all this, the Nature Jobs article is worth reading because it does contain some useful hints about what makes a good review — especially this nugget: “Reviewers should avoid concentrating on what the study could show in principle. The focus should be on what it actually shows.” A big amen on that. Nothing is more frustrating than getting back a review that says “Well, you should have written this other paper instead.”
Still and all, folks. You can choose where to direct your reviewing effort. You can give it to open-access journals that let the whole world benefit from your work (and more important, from the author’s work and the nation’s funding). Or you can give it to Elsevier shareholders.
It’s on the record that I think it’s stupid to do the latter — immoral, even, in an “all that is necessary is for good men to do nothing” way, because donating to paywalled journals helps to prop up their corrupt business and therefore keeps research out of the hands of the people who pay for it. (Who are these people who want access but can’t get it? See Peter Murray Rust’s excellent ongoing series on “The Scholarly Poor” — Dentists, Industry, The Climate Code Foundation, Patient groups, so many different types.)
So again I urge you — join me in refusing to do free work for paywalled journals.
Update (Sunday 16th October 2011)
Andy Farke offers a counterpoint over on the Open Source Paleontologist. He raises important points that deserve to be properly addressed: I’ll probably do that in a new post here rather than as a comment. Stay tuned!