How can we get post-publication peer-review to happen?
February 20, 2021
Today marks the one-month anniversary of my and Matt’s paper in Qeios about why vertebral pneumaticity in sauropods is so variable. (Taylor and Wedel 2021). We were intrigued to publish on this new platform that supports post-publication peer-review, partly just to see what happened.

Taylor and Wedel (2021: figure 3). Brontosaurus excelsus holotype YPM 1980, caudal vertebrae 7 and 8 in right lateral view. Caudal 7, like most of the sequence, has a single vascular foramen on the right side of its centrum, but caudal 8 has two; others, including caudal 1, have none.
So what has happened? Well, as I write this, the paper has been viewed 842 times, downloaded a healthy 739 times, and acquired an altmetric score 21, based rather incestuously on two SV-POW! blog-posts, 14 tweets and a single Mendeley reader.
What hasn’t happened is even a single comment on the paper. Nothing that could be remotely construed as a post-publication peer-review. And therefore no progress towards our being able to count this as a peer-reviewed publication rather than a preprint — which is how I am currently classifying it in my publications list.
This, despite our having actively solicited reviews both here on SV-POW!, in the original blog-post, and in a Facebook post by Matt. (Ironically, the former got seven comments and the latter got 20, but the actual paper none.)
I’m not here to complain; I’m here to try to understand.
On one level, of course, this is easy to understand: writing a more-than-trivial comment on a scholarly article is work, and it garners very little of the kind of credit academics care about. Reputation on the Qeios site is nice, in a that-and-two-bucks-will-buy-me-a-coffee kind of way, but it’s not going to make a difference to people’s CVs when they apply for jobs and grants — not even in the way that “Reviewed for JVP” might. I completely understand why already overworked researchers don’t elect to invest a significant chunk of time in voluntarily writing a reasoned critique of someone else’s work when they could be putting that time into their own projects. It’s why so very few PLOS articles have comments.
On the other hand, isn’t this what we always do when we write a solicited peer-review for a regular journal?
So as I grope my way through this half-understood brave new world that we’re creating together, I am starting to come to the conclusion that — with some delightful exceptions — peer-review is generally only going to happen when it’s explicitly solicited by a handling editor, or someone with an analogous role. No-one’s to blame for this: it’s just reality that people need a degree of moral coercion to devote that kind of effort to other people’s project. (I’m the same; I’ve left almost no comments on PLOS articles.)
Am I right? Am I unduly pessimistic? Is there some other reason why this paper is not attracting comments when the Barosaurus preprint did? Teach me.
References
Name the journal. Shame the publisher.
September 11, 2020
Here’s an odd thing. Over and over again, when a researcher is mistreated by a journal or publisher, we see them telling their story but redacting the name of the journal or publisher involved. Here are a couple of recent examples.
First, Daniel A. González-Padilla’s experience with a journal engaging in flagrant citation-pumping, but which he declines to name:

Interesting highlight after rejecting a paper I submitted.
Is this even legal/ethical?
EDITOR-IN-CHIEF’S COMMENT REGARDING THE INCLUSION OF REFERENCES TO ARTICLES IN [REDACTED]
Please note that if you wish to submit a manuscript to [REDACTED] in future, we would prefer that you cite at least TWO articles published in our journal WITHIN THE LAST TWO YEARS. This is a polict adopted by several journals in the urology field. Your current article contains only ONE reference to recent articles in [REDACTED].
And here is Waheed Imran’s experience of editorial dereliction:

I submitted my manuscript to a journal back in September 2017, and it is rejected by the journal on September 6, 2020. The reason of rejection is “reviewers declined to review”, they just told me this after 3 years, this is how we live with rejections. @AcademicChatter
@PhDForum
My, my question is, why in such situations do we protect the journals in question? In this case, I wrote to Waheed urging him to name the journal, and he replied saying that he will do so once an investigation is complete. But I find myself wondering why we have this tendency to protect guilty journals in the first place?
Thing is, I’ve done this myself. For example, back in 2012, I wrote about having a paper rejected from “a mid-to-low ranked palaeo journal” for what I considered (and still consider) spurious reasons. Why didn’t I name the journal? I’m not really sure. (It was Palaeontologia Electronica, BTW.)
In cases like my unhelpful peer-review, it’s not really a big deal either way. In cases like those mentioned in the tweets above, it’s a much bigger issue, because those (unlike PE) are journals to avoid. Whichever journal sat on a submission for three years before rejecting it because it couldn’t find reviewers is not one that other researchers should waste their time on in the future — but how can they avoid it if they don’t know what journal it is?
So what’s going on? Why do we have this widespread tendency to protect the guilty?
Update (13 September 2021)
One year later, Waheed confirms that the journal in question not only did not satisfactorily resolve his complaint, it didn’t even respond to his message. At this stage, there really is no point in protecting the journal that has behaved so badly, so Waheed outed it: it’s Scientia Iranica. Avoid.
What is an ad-hominem attack?
September 4, 2013
I recently handled the revisions on a paper that hopefully will be in press very soon. One of the review comments was “Be very careful not to make ad hominem attacks”.
I was a bit surprised to see that — I wasn’t aware that I’d made any — so I went back over the manuscript, and sure enough, there were no ad homs in there.
There was criticism, though, and I think that’s what the reviewer meant.
Folks, “ad hominem” has a specific meaning. An “ad hominem attack” doesn’t just mean criticising something strongly, it means criticising the author rather than the work. The phrase is Latin for “to the man”. Here’s a pair of examples:
- “This paper by Wedel is terrible, because the data don’t support the conclusion” — not ad hominem.
- “Wedel is a terrible scientist, so this paper can’t be trusted” — ad hominem.
What’s wrong with ad hominem criticism? Simply, it’s irrelevant to evaluation of the paper being reviewed. It doesn’t matter (to me as a scientist) whether Wedel strangles small defenceless animals for pleasure in his spare time; what matters is the quality of his work.
Note that ad hominems can also be positive — and they are just as useless there. Here’s another pair of examples:
- “I recommend publication of Naish’s paper because his work is explained carefully and in detail” — not ad hominem.
- “I recommend publication of Naish’s paper because he is a careful and detailed worker” — ad hominem.
It makes no difference whether Naish is a careful and detailed worker, or if he always buys his wife flowers on their anniversary, or even if he has a track-record of careful and detailed work. What matters is whether this paper, the one I’m reviewing, is good. That’s all.
As it happens the very first peer-review I ever received — for the paper that eventually became Taylor and Naish (2005) on diplodocoid phylogenetic nomenclature — contained a classic ad hominem, which I’ll go ahead and quote:
It seems to me perfectly reasonable to expect revisers of a major clade to have some prior experience/expertise in the group or in phylogenetic taxonomy before presenting what is intended to be the definitive phylogenetic taxonomy of that group. I do not wish to demean the capabilities of either author – certainly Naish’s “Dinosaurs of the Isle of Wight” is a praiseworthy and useful publication in my opinion – but I question whether he and Taylor can meet their own desiderata of presenting a revised nomenclature that balances elegance, consistency, and stability.
You see what’s happening here? The reviewer was not reviewing the paper, but the authors. There was no need for him or her to question whether we could meet our desiderata: he or she could just have read the manuscript and found out.
(Happy ending: that paper was rejected at the journal we first sent it to, but published at PaleoBios in revised form, and bizarrely is my equal third most-cited paper. I never saw that coming.)
Why giraffes have short necks
September 26, 2012
Today sees the publication, on arXiv (more on that choice in a separate post), of Mike and Matt’s new paper on sauropod neck anatomy. In this paper, we try to figure out why it is that sauropods evolved necks six times longer than that of the world-record giraffe — as shown in Figure 3 from the paper (with a small version of Figure 1 included as a cameo to the same scale):

Figure 3. Necks of long-necked sauropods, to the same scale. Diplodocus, modified from elements in Hatcher (1901, plate 3), represents a “typical” long-necked sauropod, familiar from many mounted skeletons in museums. Puertasaurus modified from Wedel (2007a, figure 4-1). Sauroposeidon scaled from Brachiosaurus artwork by Dmitry Bogdanov, via commons.wikimedia.org (CC-BY-SA). Mamenchisaurus modified from Young and Zhao (1972, figure 4). Supersaurus scaled from Diplodocus, as above. Alternating pink and blue bars are one meter in width. Inset shows Figure 1 to the same scale.
This paper started life as a late-night discussion over a couple of beers, while Matt was over in England for SVPCA back in (I think) 2008. It was originally going to be a short note in PaleoBios, just noting some of the oddities of sauropod cervical architecture — such as the way that cervical ribs, ventral to the centra, elongate posteriorly but their dorsal counterparts the epipophyses do not.
As so often, the tale grew in the telling, so that a paper we’d initially imagined as a two-or-three-page note became Chapter 5 of my dissertation under the sober title of “Vertebral morphology and the evolution of long necks in sauropod dinosaurs”, weighing in at 41 1.5-spaced pages. By now the manuscript had metastatised into a comparison between the necks of sauropods and other animals and an analysis of the factors that enabled sauropods to achieve so much more than mammals, birds, other theropods and pterosaurs.
(At this point we had one of our less satisfactory reviewing experiences. We sent the manuscript to a respected journal, where it wasn’t even sent out to reviewers until more than a month had passed. We then had to repeatedly prod the editor before anything else happened. Eventually, two reviews came back: one of them careful and detailed; but the other, which we’d waited five months for, dismissed our 53-page manuscript in 108 words. So two words per page, or about 2/3 of a word per day of review time. But let’s not dwell on that.)

Figure 6. Basic cervical vertebral architecture in archosaurs, in posterior and lateral views. 1, seventh cervical vertebra of a turkey, Meleagris gallopavo Linnaeus, 1758, traced from photographs by MPT. 2, fifth cervical vertebra of the abelisaurid theropod Majungasaurus crenatissimus Depéret, 1896,UA 8678, traced from O’Connor (2007, figures 8 and 20). In these taxa, the epipophyses and cervical ribs are aligned with the expected vectors of muscular forces. The epipophyses are both larger and taller than the neural spine, as expected based on their mechanical importance. The posterior surface of the neurapophysis is covered by a large rugosity, which is interpreted as an interspinous ligament scar like that of birds (O’Connor, 2007). Because this scar covers the entire posterior surface of the neurapophysis, it leaves little room for muscle attachments to the spine. 3, fifth cervical vertebra of Alligator mississippiensis Daudin, 1801, MCZ 81457, traced from 3D scans by Leon Claessens, courtesy of MCZ. Epipophyses are absent. 4, eighth cervical vertebra of Giraffatitan brancai (Janensch, 1914) paralectotype HMN SII, traced from Janensch (1950, figures 43 and 46). Abbreviations: cr, cervical rib; e, epipophysis; ns, neural spine; poz, postzygapophysis.
This work made its next appearance as my talk at SVPCA 2010 in Cambridge, under the title Why giraffes have such short necks. For the talk, I radically restructured the material into a form that had a stronger narrative — a process that involved a lot of back and forth with Matt, dry-running the talk, and workshopping the order in which ideas were presented. The talk seemed to go down well, and we liked the new structure so much more than the old that we reworked the manuscript into a form that more closely resembled the talk.
That’s the version of the manuscript that we perfected in New York when we should have been at all-you-can-eat sushi places. It’s the version that we submitted on the train from New York to New Haven as we went to visit the collections of the Yale Peabody Museum. And it’s the version that was cursorily rejected from mid-to-low ranked palaeo journal because a reviewer said “The manuscript reads as a long “story” instead of a scientific manuscript” — which was of course precisely what we’d intended.
Needless to say, it was deeply disheartening to have had what we were convinced was a good paper rejected twice from journals, at a cost of three years’ delay, on the basis of these reviews. One option would have been to put the manuscript back into the conventional “scientific paper” straitjacket for the second journal’s benefit. But no. We were not going to invest more work to make the paper less good. We decided to keep it in its current, more readable, form and to find a journal that likes it on that basis.
At the moment, the plan is to send it to PeerJ when that opens to submissions. (Both Matt and I are already members.) But that three-years-and-rolling delay really rankles, and we both felt that it wasn’t serving science to keep the paper locked up until it finally makes it into a journal — hence the deposition in arXiv which we plan to talk about more next time.
In the paper, we review seven characteristics of sauropod anatomy that facilitated the evolution of long necks: absolutely large body size; quadrupedal stance; proportionally small, light head; large number of cervical vertebrae; elongation of cervical vertebrae; air-sac system; and vertebral pneumaticity. And we show that giraffes have only two of these seven features. (Ostriches do the next best, with five, but they are defeated by their feeble absolute size.)
The paper incorporates some material from SV-POW! posts, including Sauropods were corn-on-the-cob, not shish kebabs. In fact, come to think of it, we should have cited that post as a source. Oh well. We do cite one SV-POW! post: Darren’s Invading the postzyg, which at the time of writing is the only published-in-any-sense source for pneumaticity invading cervical postzygapogyses from the medial surface.
As for the non-extended epipophyses that kicked the whole project off: we did illustrate how they could look, and discussed why they would seem to make mechanical sense:

Figure 10. Real and speculative muscle attachments in sauropod cervical vertebrae. 1, the second through seventeenth cervical vertebrae of Euhelopus zdanskyi Wiman, 1929 cotype specimen PMU R233a-δ(“Exemplar a”). 2, cervical 14 as it actually exists, with prominent but very short epipophyses and long cervical ribs. 3, cervical 14 as it would appear with short cervical ribs. The long ventral neck muscles would have to attach close to the centrum. 4, speculative version of cervical 14 with the epipophyses extended posteriorly as long bony processes. Such processes would allow the bulk of both the dorsal and ventral neck muscles to be located more posteriorly in the neck, but they are not present in any known sauropod or other non-avian dinosaur. Modified from Wiman (1929, plate 3).
But we found and explained some good reasons why this apparently appealing arrangement would not work. You’ll need to read the paper for details.
Sadly, we were not able to include this slide from the talk illustrating the consequences:
Anyway, go and read the paper! It’s freely available, of course, like all arXiv depositions, and in particular uses the permissive Creative Commons Attribution (CC BY) licence. We have assembled related information over on this page, including full-resolution versions of all the figures.
In the fields of maths, physics and computer science, where deposition in arXiv is ubiquitous, standard practice is to go right ahead and cite works in arXiv as soon as they’re available, rather than waiting for them to appear in journals. We will be happy for the same to happen with our paper: if it contains information that’s of value to you, then feel free to cite the arXiv version.
Reference
- Taylor, Michael P., and Mathew J. Wedel. 2012. Why sauropods had long necks; and why giraffes have short necks. arXiv:1209.5439. 39 pages, 11 figures, 3 tables. [Full-resolution figures]
Filter-then-publish vs. publish-then-filter
May 10, 2012
How things have always been
Traditional scientific journals ask peer-reviewers to do two things: assess whether a manuscript is scientifically sound, and judge whether it’s sufficiently important to appear in the particular journal it’s been submitted to.
So I could have sent my 2009 paper on Brachiosaurus to Nature, and the reviewers would (presumably) have said “this is good science, but not exciting or sexy enough for Nature“. My article would have been filtered out of Nature, which after all is very limited for space. Instead, I sent it to the Journal of Vertebrate Paleontology, where the exciting-and-sexy bar is calibrated differently, and it passed both halves of the peer-review test.
Enter PLoS
The great insight of PLoS ONE was to recognise the two-pronged nature of peer-review, and to tease them apart by discarding the second prong completely. Its guidelines for reviewers are clear:
Unlike many journals which attempt to use the peer review process to determine whether or not an article reaches the level of ‘importance’ required by a given journal, PLoS ONE uses peer review to determine whether a paper is technically sound and worthy of inclusion in the published scientific record. Once the work is published in PLoS ONE, the broader community is then able to discuss and evaluate the significance of the article (through the number of citations it attracts; the downloads it achieves; the media and blog coverage it receives; and the post-publication Notes, Comments and Ratings that it receives on PLoS ONE etc).
I like to think of this as “kill ’em all and let God sort it out”, but a less colourful description that has caught on is “publish then filter“. This name contrasts nicely with the traditional model, which can be called “filter then publish“.
The models compared
On the whole, traditionalists prefer the older model, because when filtering is done in advance by professionals it saves them from having to do their own filtering.
Or does it?
No.
There was a time when it probably did: when to keep up with a field, it would be sufficient to read (or at least scan) the articles in a handful of the discipline’s top journals. But those days are long gone. I took a random selection of ten PDFs from my own library, and checked what journals they were in. In that sample, only a single journal came up more than once: there were two papers from Acta Paleontologica Polonica. The others were from Acta Geologica Sinica, the Anatomical Record, Animal Behaviour, Comparative Biochemistry and Physiology, Herpetological Conservation and Biology, the Journal of Vertebrate Paleontology and the Quarterly Journal of the Geological Society, plus a dissertation from the University of Flensburg. (And I am one of the most narrowly focussed researchers you could meet.)
In the face of such a flood of information, no-one can read everything that’s made it through the filters into all their favourite journals. So in practice what actually happens is that each of us filters again — finding relevant publications in a huge range of journals by the social web we’re in: mailing lists, blogs, Twitter, and so on. I believe some people even use FaceBook.
A tentative conclusion
So the real choice is between publish-then-filter or filter-then-publish-then-filter.
Put that way, I’m not sure I see very much value in that first filtering phase. I know it’s going to let through a ton of stuff that I don’t care about — all the palaeobotany papers in Palaeontologia Electronica, for example. But that pre-filter is also bound to stop a lot of stuff that I would care about if it were published. If JVP rejects someone’s unexciting paper on a partial Brachiosaurus specimen because it’s not sufficiently exciting, that may be good for the journal’s “prestige” (whatever that means) but it certainly doesn’t serve me as a researcher: I want all known specimens to be published.
So I am coming round to thinking that the PLoS way is best: if a paper is good science, then why even bother thinking about its likely impact? It’s not like that’s something we can expect to guess accurately, anyway. Just publish it and let the ashes fall where they may. The world will figure out for itself whether it’s worth reading and citing.