September 24, 2013
I woke up this morning to find its third substantial review waiting for me.
That means that this paper has now accumulated as much useful feedback in the twenty-seven hours since I submitted it as any previous submission I’ve ever made.
It’s worth reviewing the timeline here:
- Monday 23rd September, 1:19 am: I completed the submission process.
- 7:03 am: the preprint was published. It took less than six hours.
- 10:52 am: received a careful, detailed review from Emanuel Tschopp. It took less than four hours from publication, and so of course less than ten from submission.
- About 5:00 pm: received a second review, this one from Mark Robinson. (I don’t know the exact time because PeerJ’s page doesn’t show an actual timestamp, just “21 hours ago”.)
- Tuesday 24th September, about 4:00 am: received a third review, this from ceratopsian-jockey and open-science guru Andy Farke.
Total time from submission to receiving three substantial reviews: about 27 hours.
It’s worth contrasting that with the times taken to get from submission to the receipt of reviews — usually only two of them — when going through the traditional journal route. Here are a few of mine:
- Diplodocoid phylogenetic nomenclature at the Journal of Paleontology, 2004-5 (the first reviews I ever received): three months and 14 days.
- Revised version of the same paper at PaleoBios, 2005 (my first published paper): one month and 10 days.
- Xenoposeidon description at Palaeontology, 2006: three months and 19 days, although that included a delay as the handling editor sent it to a third, tie-breaking, reviewer.
- Brachiosaurus revision at the Journal of Vertebrate Paleontology, 2008: one month and 11 days.
- Sauropod neck anatomy (eventually to be published in a very different form in PeerJ) at Paleobiology: five months and two days.
- Trivial correction to the Brachiosaurus revision at the Journal of Vertebrate Paleontology, 2010: five months and 11 days, bizarrely for a half-page paper.
Despite the wide variations in submission-to-review time at these journals, it’s clear that you can expect to wait at least a month before getting any feedback at all on your submission at traditional journals. Even PeerJ took 19 days to get the reviews of our neck-anatomy paper back to us.
So I am now pretty such sold on the pre-printing route. As well as getting this early version of the paper out there early so that other palaeontologists can benefit from it (and so that we can’t be pre-emptively plagiarised), issuing a preprint has meant that we’ve got really useful feedback very quickly.
I highly recommend this route.
By the way, in case anyone’s wondering, PeerJ Preprints is not only for manuscripts that are destined for PeerJ proper. They’re perfectly happy for you to use their service as a place to gather feedback for your work before submitting it elsewhere. So even if your work is destined for, say, JVP, there’s a lot to be gained by preprinting it first.
September 4, 2013
I recently handled the revisions on a paper that hopefully will be in press very soon. One of the review comments was “Be very careful not to make ad hominem attacks”.
I was a bit surprised to see that — I wasn’t aware that I’d made any — so I went back over the manuscript, and sure enough, there were no ad homs in there.
There was criticism, though, and I think that’s what the reviewer meant.
Folks, “ad hominem” has a specific meaning. An “ad hominem attack” doesn’t just mean criticising something strongly, it means criticising the author rather than the work. The phrase is Latin for “to the man”. Here’s a pair of examples:
- “This paper by Wedel is terrible, because the data don’t support the conclusion” — not ad hominem.
- “Wedel is a terrible scientist, so this paper can’t be trusted” – ad hominem.
What’s wrong with ad hominem criticism? Simply, it’s irrelevant to evaluation of the paper being reviewed. It doesn’t matter (to me as a scientist) whether Wedel strangles small defenceless animals for pleasure in his spare time; what matters is the quality of his work.
Note that ad hominems can also be positive — and they are just as useless there. Here’s another pair of examples:
- “I recommend publication of Naish’s paper because his work is explained carefully and in detail” — not ad hominem.
- “I recommend publication of Naish’s paper because he is a careful and detailed worker” — ad hominem.
It makes no difference whether Naish is a careful and detailed worker, or if he always buys his wife flowers on their anniversary, or even if he has a track-record of careful and detailed work. What matters is whether this paper, the one I’m reviewing, is good. That’s all.
As it happens the very first peer-review I ever received — for the paper that eventually became Taylor and Naish (2005) on diplodocoid phylogenetic nomenclature — contained a classic ad hominem, which I’ll go ahead and quote:
It seems to me perfectly reasonable to expect revisers of a major clade to have some prior experience/expertise in the group or in phylogenetic taxonomy before presenting what is intended to be the definitive phylogenetic taxonomy of that group. I do not wish to demean the capabilities of either author – certainly Naish’s “Dinosaurs of the Isle of Wight” is a praiseworthy and useful publication in my opinion – but I question whether he and Taylor can meet their own desiderata of presenting a revised nomenclature that balances elegance, consistency, and stability.
You see what’s happening here? The reviewer was not reviewing the paper, but the authors. There was no need for him or her to question whether we could meet our desiderata: he or she could just have read the manuscript and found out.
(Happy ending: that paper was rejected at the journal we first sent it to, but published at PaleoBios in revised form, and bizarrely is my equal third most-cited paper. I never saw that coming.)
September 26, 2012
Today sees the publication, on arXiv (more on that choice in a separate post), of Mike and Matt’s new paper on sauropod neck anatomy. In this paper, we try to figure out why it is that sauropods evolved necks six times longer than that of the world-record giraffe — as shown in Figure 3 from the paper (with a small version of Figure 1 included as a cameo to the same scale):
This paper started life as a late-night discussion over a couple of beers, while Matt was over in England for SVPCA back in (I think) 2008. It was originally going to be a short note in PaleoBios, just noting some of the oddities of sauropod cervical architecture — such as the way that cervical ribs, ventral to the centra, elongate posteriorly but their dorsal counterparts the epipophyses do not.
As so often, the tale grew in the telling, so that a paper we’d initially imagined as a two-or-three-page note became Chapter 5 of my dissertation under the sober title of “Vertebral morphology and the evolution of long necks in sauropod dinosaurs”, weighing in at 41 1.5-spaced pages. By now the manuscript had metastatised into a comparison between the necks of sauropods and other animals and an analysis of the factors that enabled sauropods to achieve so much more than mammals, birds, other theropods and pterosaurs.
(At this point we had one of our less satisfactory reviewing experiences. We sent the manuscript to a respected journal, where it wasn’t even sent out to reviewers until more than a month had passed. We then had to repeatedly prod the editor before anything else happened. Eventually, two reviews came back: one of them careful and detailed; but the other, which we’d waited five months for, dismissed our 53-page manuscript in 108 words. So two words per page, or about 2/3 of a word per day of review time. But let’s not dwell on that.)
This work made its next appearance as my talk at SVPCA 2010 in Cambridge, under the title Why giraffes have such short necks. For the talk, I radically restructured the material into a form that had a stronger narrative – a process that involved a lot of back and forth with Matt, dry-running the talk, and workshopping the order in which ideas were presented. The talk seemed to go down well, and we liked the new structure so much more than the old that we reworked the manuscript into a form that more closely resembled the talk.
That’s the version of the manuscript that we perfected in New York when we should have been at all-you-can-eat sushi places. It’s the version that we submitted on the train from New York to New Haven as we went to visit the collections of the Yale Peabody Museum. And it’s the version that was cursorily rejected from mid-to-low ranked palaeo journal because a reviewer said “The manuscript reads as a long “story” instead of a scientific manuscript” — which was of course precisely what we’d intended.
Needless to say, it was deeply disheartening to have had what we were convinced was a good paper rejected twice from journals, at a cost of three years’ delay, on the basis of these reviews. One option would have been to put the manuscript back into the conventional “scientific paper” straitjacket for the second journal’s benefit. But no. We were not going to invest more work to make the paper less good. We decided to keep it in its current, more readable, form and to find a journal that likes it on that basis.
At the moment, the plan is to send it to PeerJ when that opens to submissions. (Both Matt and I are already members.) But that three-years-and-rolling delay really rankles, and we both felt that it wasn’t serving science to keep the paper locked up until it finally makes it into a journal — hence the deposition in arXiv which we plan to talk about more next time.
In the paper, we review seven characteristics of sauropod anatomy that facilitated the evolution of long necks: absolutely large body size; quadrupedal stance; proportionally small, light head; large number of cervical vertebrae; elongation of cervical vertebrae; air-sac system; and vertebral pneumaticity. And we show that giraffes have only two of these seven features. (Ostriches do the next best, with five, but they are defeated by their feeble absolute size.)
The paper incorporates some material from SV-POW! posts, including Sauropods were corn-on-the-cob, not shish kebabs. In fact, come to think of it, we should have cited that post as a source. Oh well. We do cite one SV-POW! post: Darren’s Invading the postzyg, which at the time of writing is the only published-in-any-sense source for pneumaticity invading cervical postzygapogyses from the medial surface.
As for the non-extended epipophyses that kicked the whole project off: we did illustrate how they could look, and discussed why they would seem to make mechanical sense:
But we found and explained some good reasons why this apparently appealing arrangement would not work. You’ll need to read the paper for details.
Sadly, we were not able to include this slide from the talk illustrating the consequences:
Anyway, go and read the paper! It’s freely available, of course, like all arXiv depositions, and in particular uses the permissive Creative Commons Attribution (CC BY) licence. We have assembled related information over on this page, including full-resolution versions of all the figures.
In the fields of maths, physics and computer science, where deposition in arXiv is ubiquitous, standard practice is to go right ahead and cite works in arXiv as soon as they’re available, rather than waiting for them to appear in journals. We will be happy for the same to happen with our paper: if it contains information that’s of value to you, then feel free to cite the arXiv version.
May 10, 2012
How things have always been
Traditional scientific journals ask peer-reviewers to do two things: assess whether a manuscript is scientifically sound, and judge whether it’s sufficiently important to appear in the particular journal it’s been submitted to.
So I could have sent my 2009 paper on Brachiosaurus to Nature, and the reviewers would (presumably) have said “this is good science, but not exciting or sexy enough for Nature“. My article would have been filtered out of Nature, which after all is very limited for space. Instead, I sent it to the Journal of Vertebrate Paleontology, where the exciting-and-sexy bar is calibrated differently, and it passed both halves of the peer-review test.
Unlike many journals which attempt to use the peer review process to determine whether or not an article reaches the level of ‘importance’ required by a given journal, PLoS ONE uses peer review to determine whether a paper is technically sound and worthy of inclusion in the published scientific record. Once the work is published in PLoS ONE, the broader community is then able to discuss and evaluate the significance of the article (through the number of citations it attracts; the downloads it achieves; the media and blog coverage it receives; and the post-publication Notes, Comments and Ratings that it receives on PLoS ONE etc).
I like to think of this as “kill ‘em all and let God sort it out”, but a less colourful description that has caught on is “publish then filter“. This name contrasts nicely with the traditional model, which can be called “filter then publish“.
The models compared
On the whole, traditionalists prefer the older model, because when filtering is done in advance by professionals it saves them from having to do their own filtering.
Or does it?
There was a time when it probably did: when to keep up with a field, it would be sufficient to read (or at least scan) the articles in a handful of the discipline’s top journals. But those days are long gone. I took a random selection of ten PDFs from my own library, and checked what journals they were in. In that sample, only a single journal came up more than once: there were two papers from Acta Paleontologica Polonica. The others were from Acta Geologica Sinica, the Anatomical Record, Animal Behaviour, Comparative Biochemistry and Physiology, Herpetological Conservation and Biology, the Journal of Vertebrate Paleontology and the Quarterly Journal of the Geological Society, plus a dissertation from the University of Flensburg. (And I am one of the most narrowly focussed researchers you could meet.)
In the face of such a flood of information, no-one can read everything that’s made it through the filters into all their favourite journals. So in practice what actually happens is that each of us filters again – finding relevant publications in a huge range of journals by the social web we’re in: mailing lists, blogs, Twitter, and so on. I believe some people even use FaceBook.
A tentative conclusion
So the real choice is between publish-then-filter or filter-then-publish-then-filter.
Put that way, I’m not sure I see very much value in that first filtering phase. I know it’s going to let through a ton of stuff that I don’t care about — all the palaeobotany papers in Palaeontologia Electronica, for example. But that pre-filter is also bound to stop a lot of stuff that I would care about if it were published. If JVP rejects someone’s unexciting paper on a partial Brachiosaurus specimen because it’s not sufficiently exciting, that may be good for the journal’s “prestige” (whatever that means) but it certainly doesn’t serve me as a researcher: I want all known specimens to be published.
So I am coming round to thinking that the PLoS way is best: if a paper is good science, then why even bother thinking about its likely impact? It’s not like that’s something we can expect to guess accurately, anyway. Just publish it and let the ashes fall where they may. The world will figure out for itself whether it’s worth reading and citing.
February 15, 2012
August 6, 2011
Matt just wrote this, in an email exchange. It struck a chord in me, and I thought it deserved a wider audience:
I hate to admit it, but those two papers (i.e., Taylor et al. 2009 and 2011) that had particularly protracted gestations and lots of review time are among the ones I am most proud of. There might be a lesson there — but if so, I’d rather not learn it.