Two days ago, I wrote about what seemed to be an instance of peer review gone very wrong. I’ve now heard from two of the four authors of the paper and from the reviewer in question — both by email, and in comments on the original post — and it’s apparent that I misinterpreted the situation. When the lead author’s tweet mentioned “pushing it through eight rounds of review”, I took this at face value as meaning eight rounds at the same journal with the same reviewers — whereas in fact the reviewer in question reviewed only four drafts. (That still seems like too many to me, but clearly it’s not as ludicrous as the situation as I misread it.) In this light, my assumption that the reviewer was being obstructive was not warranted.

I have decided to retract that article and I offer my apologies to the reviewer, Dave Grossnickle, who approached me very politely off-list to offer the corrections that you can now read in his comment.

THIS POST IS RETRACTED. The reasons are explained in the next post. I wish I had never posted this, but you can’t undo what is done, especially on the Internet, so I am not deleting it but marking it as retracted. I suggest you don’t bother reading on, but it’s here if you want to.

 


Neil Brocklehurst, Elsa Panciroli, Gemma Louise Benevento and Roger Benson have a new paper out (Brocklehurst et al. 2021, natch), showing that the post-Cretaceous radiation of modern mammals was not primarily due to the removal of dinosaurs, as everyone assumed, but of more primitive mammal-relatives. Interesting stuff, and it’s open access. Congratulations to everyone involved!

Neil Brocklehurt’s “poster” explaining the new paper in broad detail. From the tweet linked below.

Neil summarised the new paper in a thread of twelve tweets, but it was the last one in the thread that caught my eye:

Thanks to all my co-authors for their tireless work on this, pushing it through eight rounds of review (my personal best)

I’m impressed that Neil has maintained his equanimity about this — in public at least — but if he is not going to be furious about it then we, the community, need to be furious on his behalf. Pushed to explain, Neil laid it out in a further tweet:

Was just one reviewer who really didn’t seem to like certain aspects, esp the use of discrete character matrices. Fair enough, can’t please everyone, but the editor just kept sending it back even when two others said our responses to this reviewer should be fine.

Again, somehow this tweet is free of cursing. He is a better man than I would be in that situation. He also doesn’t call out the reviewer by name, nor the spineless handling editor, which again shows great restraint — though I am not at all sure it’s the right way to go.

There is so, so much to hate about this story:

  • The obstructive peer reviewer, who seems to have to got away with his reputation unblemished by these repeated acts of vandalism. (I’m assuming he was one of the two anonymous reviewers, not the one who identified himself.)
  • The handling editor who had half a dozen opportunities to put an end to the round-and-round, and passed on at least five of them. Do your job! Handle the manuscript! Don’t just keep kicking it back to a reviewer who you know by this stage is not acting in good faith.
  • The failure of the rest of the journal’s editorial board to step in and bring some sanity to the situation.
  • The normalization of this kind of thing — arguably not helped by Neil’s level-headed recounting of the story as though it’s basically reasonable — as someting authors should expect, and just have to put up with.
  • The time wasted: the other research not done while the authors were pithering around back and forth with the hostile reviewer.

It’s the last of these that pains me the most. Of all the comforting lies we tell ourselves about conventionl peer review, the worst is that it’s worth all the extra time and effort because it makes the paper better.

It’s not worth it, is it?

Maybe Brocklehurst et al. 2021 is a bit better for having gone through the 3rd, 4th, 5th, 6th, 7th and 8th rounds of peer review. But if it is, then it’s a marginal difference, and my guess is that in fact it’s no better and no worse that what they submitted after the second round. All that time, they could have been looking at specimens, generating hypotheses, writing descriptions, gathering data, plotting graphs, writing blogs, drafting papers — instead they have been frittering away their time in a pointless and destructive conflict with someone whose only goal was to prevent the advancement of science because an aspect of the paper happened to conflict with a bee he had in his bonnet. We have to stop this waste.

This incident has reinforced my growing conviction that venues like Qeios, Peer Community in Paleontology and BiorXiv (now that it’s moving towards support for reviewing) are the way to go. Our own experience at Qeios has been very good — if it works this well the next time we use it, I think think it’s a keeper. Crucially, I don’t believe our paper (Taylor and Wedel 2021) would have been stronger if it had gone through the traditional peer-review gauntlet; instead, I think it’s stronger than it would have been, because it’s received reviews from more pairs of eyes, and each of them with a constructive approach. Quicker publication, less work for everyone involved, more collegial process, better final result — what’s not to like?

References

A month after I and Matt published our paper “Why is vertebral pneumaticity in sauropod dinosaurs so variable?” at Qeios, we were bemoaning how difficult it was to get anyone to review it. But what a difference the last nineteen days have made!

In that time, we’ve had five reviews, and posted three revisions: revision 2 in response to a review by Mark McMenamin, version 3 in response to a review by Ferdinand Novas, and version 4 in response to reviews by Leonardo Cotts, by Alberto Collareta, and by Eduardo Jiménez-Hidalgo.

Taylor and Wedel (2021: Figure 2). Proximal tail skeleton (first 13 caudal vertebrate) of LACM Herpetology 166483, a juvenile specimen of the false gharial Tomistoma schlegelii. A: close-up of caudal vertebrae 4–6 in right lateral view, red circles highlighting vascular foramina: none in Ca4, two in Ca5 and one in Ca6. B: right lateral view. C: left lateral view (reversed). D: close-up of caudal vertebrae 4–6 in left lateral view (reversed), red circles highlighting vascular foramina: one each in Ca4, Ca5 and Ca6. In right lateral view, vascular foramina are apparent in the centra of caudal vertebrae 5–7 and 9–11; they are absent or too small to make out in vertebrae 1–4, 8 and 12–13. In left lateral view (reversed), vascular foramina are apparent in the centra of caudal vertebrae 4–7 and 9; they are absent or too small to make out in vertebrae 1–3, 8, and 10–13. Caudal centra 5–7 and 9 are therefore vascularised from both sides; 4 and 10–11 from one side only; and 1–3, 8 and 12–13 not at all.

There are a few things to say about this.

First, this is now among our most reviewed papers. Thinking back across all my publications, most have been reviewed by two people; the original Xenoposeidon description was reviewed by three; the same was true of my reassessment of Xenoposeidon as a rebbachisaur, and there may have been one or two more that escape me at the moment. But I definitely can’t think of any papers that have been under five sets of eyes apart from this one in Qeios.

Now I am not at all saying that all five of the reviews on this paper are as comprehensive and detailed as a typical solicited peer review at a traditional journal. Some of them have detailed observations; others are much more cursory. But they all have things to say — which I will return to in my third point.

Second, Qeios has further decoupled the functions of peer review. Traditional peer review combines three rather separate functions: A, Checking that the science is sound before publishing it; B, assessing whether it’s a good fit for the journal (often meaning whether it’s sexy enough); and C, helping the authors to improve the work. When PLOS ONE introduced correctness-only peer-review, they discarded B entirely, reasoning correctly that no-one knows which papers will prove influential[1]. Qeios goes further by also inverting A. By publishing before the peer reviews are in (or indeed solicited), it takes away the gatekeeper role of the reviewers, leaving them with only function C, helping the authors to improve the work. Which means it’s no surprise that …

Third, all five reviews have been constructive. As Matt has written elsewhere, “There’s no way to sugar-coat this: getting reviews back usually feels like getting kicked in the gut”. This is true, and we both have a disgraceful record of allowing harshly-reviewed projects to sit fallow for far too long before doing the hard work of addressing the points made by the reviewers and resubmitting[2].

The contrast with the reviews from Qeios has been striking. Each one has sent me scampering back to the manuscript, keen to make (most of) the suggested changes — hence the three revised versions that I’ve posted in the last fortnight. I think there are at least two reasons for this, a big one and a small one.

  • The big reason, I think, is that the reviewers know their only role is to improve the paper. Well, that’s not quite true: they also have some influence over its evaluation, both in what they write and in assigning a 1-to-5 star score. But they know when they’re writing their reviews that whatever happens, they won’t block publication. This means, firstly, that there is no point in their writing something like “This paper should not be published until the authors do X”; but equally importantly, I think it puts reviewers in a different and more constructive mindset. They feel themselves to be allies of the authors rather than (as can happen) adversaries.
  • The smaller reason is it’s easier to deal with one review at a time. I understand why journals solicit multiple reviews: so the handling editor can consider them all in reaching a decision. I understand why the authors get all the reviews back at once. But that process can’t help but be discouraging: because, once the decision has been made, they’re all on hand and there’s no point in stringing them out. One at a time may not be better, exactly; but it’s emotionally easier.

Is this all upside? Well, it’s too early to say. We’ve only done this once. The experience has certainly been more pleasant — and, crucially, much more efficient — than the traditional publishing lifecycle. But I’m aware of at least two potential drawbacks:

First, the publish-first lifecycle could be exploited by cranks. If the willingness to undergo peer-review is the mark of seriousness in a researcher — and if non-serious researchers are unwilling to face that gauntlet — then a venue that lets you make an end-run around peer-review is an obvious loophole. How serious a danger is this? Only time will tell, but I am inclined to think maybe not too serious. Bad papers on a site like Qeios will attract negative reviews and low scores, especially if they start to get noticed in the mainsteam media. They won’t be seen as having the stamp of having passed peer-review; rather, they will be branded with having publicly failed peer-review.

Second, it’s still not clear where reviewers will come from. We wrote about this problem in some detail last month, and although it’s worked out really well for our present paper, that’s no guarantee that it will always work out this well. We know that Qeios itself approached at least one reviewer to solicit their comments: that’s great, and if they can keep doing this then it will certainly help. But it probably won’t scale, so either a different reviewing culture will need to develop, or we will need people who — perhaps only on an informal basis — take it on themselves to solicit reviews from others. We’re interested to see how this develops.

Anyway, Matt and I have found our first Qeios experience really positive. We’ve come out of it with what I think is a good paper, relatively painlessly, and with much less friction than the usual process. I hope that some of you will try it, too. To help get the process rolling, I personally undertake to review any Qeios article posted by an SV-POW! reader. Just leave a comment here to let me know about your article when it’s up.

 

Notes

[1] “No-one knows which papers will prove influential”. As purely anecdotal evidence for this claim: when I wrote “Sauropod dinosaur research: a historical review” for the Geological Society volume Dinosaurs: A Historical Perspective, I thought it might become a citation monster. It’s done OK, but only OK. Conversely, it never occurred to me that “Head and neck posture in sauropod dinosaurs inferred from extant animals” would be of more than specialist interest, but it’s turned out to be my most cited paper. I bet most researchers can tell similar stories.

[2] One example: my 2015 preprint on the incompleteness of sauropod necks was submitted for publication in October 2015, and the reviews[3] came back that same month. Five and a half years later, I am only now working on the revision and resubmission. If you want other examples, we got ’em. I am not proud of this.

[3] I referred above to “harsh reviews” but in fact the reviews for this paper were not harsh; they were hard, but 100% fair, and I found myself agreeing with about 90% of the criticisms. That has certainly not been true of all the reviews I have found disheartening!

 

Today marks the one-month anniversary of my and Matt’s paper in Qeios about why vertebral pneumaticity in sauropods is so variable. (Taylor and Wedel 2021). We were intrigued to publish on this new platform that supports post-publication peer-review, partly just to see what happened.

Taylor and Wedel (2021: figure 3). Brontosaurus excelsus holotype YPM 1980, caudal vertebrae 7 and 8 in right lateral view. Caudal 7, like most of the sequence, has a single vascular foramen on the right side of its centrum, but caudal 8 has two; others, including caudal 1, have none.

So what has happened? Well, as I write this, the paper has been viewed 842 times, downloaded a healthy 739 times, and acquired an altmetric score 21, based rather incestuously on two SV-POW! blog-posts, 14 tweets and a single Mendeley reader.

What hasn’t happened is even a single comment on the paper. Nothing that could be remotely construed as a post-publication peer-review. And therefore no progress towards our being able to count this as a peer-reviewed publication rather than a preprint — which is how I am currently classifying it in my publications list.

This, despite our having actively solicited reviews both here on SV-POW!, in the original blog-post, and in a Facebook post by Matt. (Ironically, the former got seven comments and the latter got 20, but the actual paper none.)

I’m not here to complain; I’m here to try to understand.

On one level, of course, this is easy to understand: writing a more-than-trivial comment on a scholarly article is work, and it garners very little of the kind of credit academics care about. Reputation on the Qeios site is nice, in a that-and-two-bucks-will-buy-me-a-coffee kind of way, but it’s not going to make a difference to people’s CVs when they apply for jobs and grants — not even in the way that “Reviewed for JVP” might. I completely understand why already overworked researchers don’t elect to invest a significant chunk of time in voluntarily writing a reasoned critique of someone else’s work when they could be putting that time into their own projects. It’s why so very few PLOS articles have comments.

On the other hand, isn’t this what we always do when we write a solicited peer-review for a regular journal?

So as I grope my way through this half-understood brave new world that we’re creating together, I am starting to come to the conclusion that — with some delightful exceptions — peer-review is generally only going to happen when it’s explicitly solicited by a handling editor, or someone with an analogous role. No-one’s to blame for this: it’s just reality that people need a degree of moral coercion to devote that kind of effort to other people’s project. (I’m the same; I’ve left almost no comments on PLOS articles.)

Am I right? Am I unduly pessimistic? Is there some other reason why this paper is not attracting comments when the Barosaurus preprint did? Teach me.

References

 

Picture is unrelated. Seriously. I’m just allergic to posts with no visuals. Stand by for more random brachiosaurs.

Here’s something I’ve been meaning to post for a while, about my changing ideas about scholarly publishing. On one hand, it’s hard to believe now that the Academic Spring was almost a decade ago. On the other, it’s hard for me to accept that PeerJ will be only 8 years old next week–it has loomed so large in my thinking that it feels like it has been around much longer. The very first PeerJ Preprints went up on April 4, 2013, just about a month and a half after the first papers in PeerJ. At that time it felt like things were moving very quickly, and that the landscape of scholarly publishing might be totally different in just a few years. Looking back now, it’s disappointing how little has changed. Oh, sure, there are more OA options now — even more kinds of OA options, and things like PCI Paleo and Qeios feel genuinely envelope-pushing — but the big barrier-based publishers are still dug in like ticks, and very few journals have fled from those publishers to re-establish themselves elsewhere. APCs are ubiquitous now, and mostly unjustified and ruinously expensive. Honestly, the biggest changes in my practice are that I use preprint servers to make my conference talks available, and I use SciHub instead of interlibrary loan.

But I didn’t sit down to write this post so I could grumble about the system like an old hippie. I’ve learned some things in the past few years, about what actually works in scholarly publishing (at least for me), and about my preferences in some areas, which turn out to be not what I expected. I’ll focus on just two areas today, peer review, and preprints.

How I Stopped Worrying and Learned to Love Peer Review

Surprise #1: I’m not totally against peer review. I realize that the way it is implemented in many places is deeply flawed, and that it’s no guarantee of the quality of a paper, but I also recognize its value. This is not where I was 8 years ago; at the time, I was pretty much in agreement with Mike’s post from November, 2012, “Well, that about wraps it up for peer-review”. But then in 2014 I became an academic editor at PeerJ. And as I gained first-hand experience from the other side of the editorial desk, I realized a few things:

  • Editors have broad remits in terms of subject areas, and without the benefit of peer reviews by people who specialize in areas other than my own, I’m not fit to handle papers on topics other than Early Cretaceous North American sauropods, skeletal pneumaticity, and human lower extremity anatomy.
  • Even at PeerJ, which only judges papers based on scientific soundness, not on perceived importance, it can be hard to tell where the boundary is. I’ve had to reject a few manuscripts at PeerJ, and I would not have felt confident about doing that without the advice of peer reviewers. Even with no perceived importance criterion, there is definitely a lower bound on what counts as a publishable observation. If you find a mammoth toe bone in Nebraska, or a tyrannosaur tooth in Montana, there should probably be something more interesting to say about it, beyond the bare fact of its existence, if it’s going to be the subject of a whole paper.
  • In contentious fields, it can be valuable to get a diversity of opinions. And sometimes, frankly, I need to figure out if the author is a loony, or if it’s actually Reviewer #2 that’s off the rails. Although I think PeerJ generally attracts fairly serious authors, a handful of things that get submitted are just garbage. From what I hear, that’s the case at almost every journal. But it’s not always obvious what’s garbage, what’s unexciting but methodologically sound, and what’s seemingly daring but also methodologically sound. Feedback from reviewers helps me make those calls. Bottom line, I do think the community benefits from having pre-publication filters in place.
  • Finally, I think editors have a responsibility to help authors improve their work, and reviewers catch a lot of stuff that I would miss. And occasionally I catch something that the reviewers missed. We are collectively smarter and more helpful than any of us would be in isolation, and it’s hard to see that as anything other than a good thing.

The moral here probably boils down to, “white guy stops bloviating about Topic X when he gains actual experience”, which doesn’t look super-flattering for me, but that’s okay.

You may have noticed that my pro-peer-review comments are rather navel-gaze-ly focused on the needs of editors. But who needs editors? Why not chuck the whole system? Set up an outlet called Just Publish Everything, and let fly? My answer is that my time in the editorial trenches has convinced me that such a system will silt up with garbage papers, and as a researcher I already have a hard enough time keeping up with all of the emerging science that I need to. From both perspectives, I want there to be some kind of net to keep out the trash. It doesn’t have to be a tall net, or strung very tight, but I’d rather have something than nothing.

What would I change about peer review? Since it launched, PeerJ has let reviewers either review anonymously, or sign their reviews, and it has let authors decide whether or not to publish the reviews alongside the paper. Those were both pretty daring steps at the time, but if I could I’d turn both of those into mandates rather than options. Sunlight is the best disinfectant, and I think almost all of the abuses of the peer review system would evaporate if reviewers had to sign their reviews, and all reviews were published alongside the papers. There will always be a-holes in the world, and some of them are so pathological that they can’t rein in their bad behavior, but if the system forced them to do the bad stuff in the open, we’d all know who they are and we could avoid them.

Femur of Apatosaurus and right humerus Brachiosaurus altithorax holotype on wooden pedestal (exhibit) with labels and 6 foot ruler for scale, Geology specimen, Field Columbian Museum, 1905. (Photo by Charles Carpenter/Field Museum Library/Getty Images)

Quo Vadis, Preprints?

Maybe the advent of preprints was more drawn out than I know, but to me it felt like preprints went from being Not a Thing, Really, in 2012, to being ubiquitous in 2013. And, I thought at the time, possibly transformative. They felt like something genuinely new, and when Mike and I posted our Barosaurus preprint and got substantive, unsolicited review comments in just a day or two, that was pretty awesome. Which is why I did not expect…

Surprise #2: I don’t have much use for preprints, at least as they were originally intended. When I first confessed this to Mike, in a Gchat, he wrote, “You don’t have a distaste for preprints. You love them.” And if you just looked at the number of preprints I’ve created, you might get that impression. But the vast majority of my preprints are conference talks, and using a preprint server was just the simplest way to the get the abstract and the slide deck up where people could find them. In terms of preprints as early versions of papers that I expect to submit soon, only two really count, neither more recent than 2015. (I’m not counting Mike’s preprint of our vertebral orientation paper from 2019; he’s first author, and I didn’t mind that he posted a preprint, but neither is it something I’d have done if the manuscript was mine alone.)

My thoughts here are almost entirely shaped by what happened with our Barosaurus preprint. We put it up on PeerJ Preprints back in 2013, we got some useful feedback right away, and…we did nothing for a long time. Finally in 2016 we revised the manuscript and got it formally submitted. I think we both expected that since the preprint had already been “reviewed” by commenters, and we’d revised it accordingly, that formal peer review would be very smooth. It was not. And the upshot is that only now, in 2021, are we finally talking about dealing with those reviews and getting the manuscript resubmitted. We haven’t actually done this, mind, we’re just talking about planning to make a start on it. (Non-committal enough for ya?)

Why has it taken us so long to deal with this one paper? We’re certainly capable — the two of us got four papers out in 2013, each of them on a different topic and each of them substantial. So why can’t we climb Mount Barosaurus? I think a big part of it is that we know the world is not waiting for our results, because our results are already out in the world. We’re the only ones being hurt by our inaction — we’re denying ourselves the credit and the respect that go along with having a paper finally and formally published in a peer-reviewed journal. But we can comfort ourselves with the thought that if someone needs our observations to make progress on their own project, we’re not holding them up. Just having the preprint out there has stolen some of our motivation to the get the paper done and out, apparently enough to keep us from doing it at all.

Mike pointed out that according to Google Scholar, our Barosaurus preprint has been cited five times to date, once in its original version and four times in its revised version. But to me, the fact that the Baro manuscript has been cited five times is a fail. Because all of my peer-reviewed papers from 2014-2016, which have been out for less long, have been cited more. So I read that as people not wanting to cite it. And who can blame them? Even I thought it would be supplanted by the formally-published, peer-reviewed paper within a few weeks or months.

Mike then pointed me to his 2015 post, “Four different reasons to post preprints”, and asked how many of those arguments still worked for me now. Number 2 is good, posting material that would otherwise never see the light of day — it’s basically what I did when I put my dissertation on arXiv. Ditto for 4, which is posting conference presentations. I’m not moved by either 1 or 3. Number 3 is getting something out to the community as quickly as possible, just because you want to, and number 1 is getting feedback as quickly as possible. The reason that neither of those move me is that they’re solved to my satisfaction by existing peer-reviewed outlets. I don’t know of any journals that let reviewers take 2-4 months to review a paper anymore. I don’t know how much credit for the acceleration should go to PeerJ, which asks for reviews in 10 to 14 days, but surely some. And I don’t usually have a high enough opinion of my own work to think that the community will suffer if it takes a few months for a paper to come out through the traditional process.

(If it seems like I’m painting Mike as relentlessly pro-preprint, it’s not my intent. Rather, I’d dropped a surprising piece of news on him, and he was strategically probing to determine the contours of my new and unexpected stance. Then I left the conversation to come write this post while the ideas were all fresh in my head. I hope to find out what he thinks about this stuff in the comments, or ideally in a follow-up post.)

Back to task: at least for me, a preprint of a manuscript I’m going to submit anyway is a mechanism to get extra reviews I don’t want*, and to lull myself into feeling like the work is done when it’s not. I don’t anticipate that I will ever again put up a preprint for one of my own manuscripts if there’s a plausible path to traditional publication.

* That sounds awful. To people who have left helpful comments on my preprints: I’m grateful, sincerely. But not so grateful that I want to do the peer review process a second time for zero credit. I didn’t know that when I used to file preprints of manuscripts, but I know it now, and the easiest way for me to not make more work for both of us is to not file preprints of things I’m planning to submit somewhere anyway.

So much for my preprints; what about those of other people? Time for another not-super-flattering confession: I don’t read other people’s preprints. Heck, I don’t have time to keep up with the peer-reviewed literature, and I have always been convinced by Mike’s dictum, “The real value of peer-review is not as a mark of correctness, but of seriousness” (from this 2014 post). If other people want me to part with my precious time to engage with their work, they can darn well get it through peer review. And — boomerang thought — that attitude degrades my respect for my own preprint manuscripts. I wouldn’t pay attention to them if someone else had written them, so I don’t really expect anyone else to pay attention to the ones that I’ve posted. In fact, it’s extremely flattering that they get read and cited at all, because by my own criteria, they don’t deserve it.

I have to stress how surprising I find this conclusion, that I regard my own preprints as useless at best, and simultaneously extra-work-making and motivation-eroding at worst, for me, and insufficiently serious to be worthy of other people’s time, for everyone else. It’s certainly not where I expected to end up in the heady days of 2013. But back then I had opinions, and now I have experience, and that has made all the difference.

The comment thread is open. What do you think? Better still, what’s your experience?

We’ve noted many times over the years how inconsistent pneumatic features are in sauropod vertebra. Fossae and formamina vary between individuals of the same species, and along the spinal column, and even between the sides of individual vertebrae. Here’s an example that we touched on in Wedel and Taylor (2013), but which is seen in all its glory here:

Taylor and Wedel (2021: Figure 5). Giraffatitan brancai tail MB.R.5000, part of the mounted skeleton at the Museum für Naturkunde Berlin. Caudal vertebrae 24–26 in left lateral view. While caudal 26 has no pneumatic features, caudal 25 has two distinct pneumatic fossae, likely excavated around two distinct vascular foramina carrying an artery and a vein. Caudal 24 is more shallowly excavated than 25, but may also exhibit two separate fossae.

But bone is usually the least variable material in the vertebrate body. Muscles vary more, nerves more again, and blood vessels most of all. So why are the vertebrae of sauropods so much more variable than other bones?

Our new paper, published today (Taylor and Wedel 2021) proposes an answer! Please read it for the details, but here’s the summary:

  • Early in ontogenly, the blood supply to vertebrae comes from arteries that initially served the spinal cord, penetrating the bone of the neural canal.
  • Later in ontegeny, additional arteries penetrate the centra, leaving vascular foramina (small holes carrying blood vessels).
  • This hand-off does not always run to completion, due to the variability of blood vessels.
  • In extant birds, when pneumatic diverticula enter the bone they do so via vascular foramina, alongside blood vessels.
  • The same was probaby true in sauropods.
  • So in vertebrae that got all their blood supply from vascular foramina in the neural canal, diverticula were unable to enter the centra from the outside.
  • So those centra were never pneumatized from the outside, and no externally visible pneumatic cavities were formed.

Somehow that pretty straightforward argument ended up running to eleven pages. I guess that’s what you get when you reference your thoughts thoroughly, illustrate them in detail, and discuss the implications. But the heart of the paper is that little bullet-list.

Taylor and Wedel (2021: Figure 6). Domestic duck Anas platyrhynchos, dorsal vertebrae 2–7 in left lateral view. Note that the two anteriormost vertebrae (D2 and D3) each have a shallow pneumatic fossa penetrated by numerous small foramina.

(What is the relevance of these duck dorsals? You will need to read the discussion in the paper to find out!)

Our choice of publication venue

The world moves fast. It’s strange to think that only eleven years ago my Brachiosaurus revision (Taylor 2009) was in the Journal of Vertebrate Palaeontology, a journal that now feels very retro. Since then, Matt and I have both published several times in PeerJ, which we love. More recently, we’ve been posting preprints of our papers — and indeed I have three papers stalled in peer-review revisions that are all available as preprints (two Taylor and Wedels and a single sole-authored one). But this time we’re pushing on even further into the Shiny Digital Future.

We’ve published at Qeios. (It’s pronounced “chaos”, but the site doesn’t tell you that; I discovered it on Twitter.) If you’ve not heard of it — I was only very vaguely aware of it myself until this evening — it runs on the same model as the better known F1000 Research, with this very important difference: it’s free. Also, it looks rather slicker.

That model is: publish first, then filter. This is the opposite of the traditional scholarly publishing flow where you filter first — by peer reviewers erecting a series of obstacles to getting your work out — and only after negotiating that course to do get to see your work published. At Qeios, you go right ahead and publish: it’s available right off the bat, but clearly marked as awaiting peer-review:

And then it undergoes review. Who reviews it? Anyone! Ideally, of course, people with some expertise in the relevant fields. We can then post any number of revised versions in response to the reviews — each revision having its own DOI and being fixed and permanent.

How will this work out? We don’t know. It is, in part, an experiment. What will make it work — what will impute credibility to our paper — is good, solid reviews. So if you have any relevant expertise, we do invite you to get over there and write a review.

And finally …

Matt noted that I first sent him the link to the Qeios site at 7:44 pm my time. I think that was the first time he’d heard of it. He and I had plenty of back and forth on where to publish this paper before I pushed on and did it at Qeios. And I tweeted that our paper was available for review at 8:44 — one hour exactly after Matt learned that the venue existed. Now here we are at 12:04 my time, three hours and 20 minutes later, and it’s already been viewed 126 times and downloaded 60 times. I think that’s pretty awesome.

References

  • Taylor, Michael P. 2009. A re-evaluation of Brachiosaurus altithorax Riggs 1903 (Dinosauria, Sauropoda) and its generic separation from Giraffatitan brancai (Janensch 1914). Journal of Vertebrate Paleontology 29(3):787-806. [PDF]
  • Taylor, Michael P., and Mathew J. Wedel. 2021. Why is vertebral pneumaticity in sauropod dinosaurs so variable? Qeios 1G6J3Q. doi: 10.32388/1G6J3Q [PDF]
  • Wedel, Mathew J., and Michael P. Taylor 2013b. Caudal pneumaticity and pneumatic hiatuses in the sauropod dinosaurs Giraffatitan and Apatosaurus. PLOS ONE 8(10):e78213. 14 pages. doi: 10.1371/journal.pone.0078213 [PDF]

Here’s an odd thing. Over and over again, when a researcher is mistreated by a journal or publisher, we see them telling their story but redacting the name of the journal or publisher involved. Here are a couple of recent examples.

First, Daniel A. González-Padilla’s experience with a journal engaging in flagrant citation-pumping, but which he declines to name:

Interesting highlight after rejecting a paper I submitted.
Is this even legal/ethical?
EDITOR-IN-CHIEF’S COMMENT REGARDING THE INCLUSION OF REFERENCES TO ARTICLES IN [REDACTED]
Please note that if you wish to submit a manuscript to [REDACTED] in future, we would prefer that you cite at least TWO articles published in our journal WITHIN THE LAST TWO YEARS. This is a polict adopted by several journals in the urology field. Your current article contains only ONE reference to recent articles in [REDACTED].

We know from a subsequent tweet that the journal is published by Springer Nature, but we don’t know the name of the journal itself.

And here is Waheed Imran’s experience of editorial dereliction:

I submitted my manuscript to a journal back in September 2017, and it is rejected by the journal on September 6, 2020. The reason of rejection is “reviewers declined to review”, they just told me this after 3 years, this is how we live with rejections. @AcademicChatter
@PhDForum

My, my question is, why in such situations do we protect the journals in question? In this case, I wrote to Waheed urging him to name the journal, and he replied saying that he will do so once an investigation is complete. But I find myself wondering why we have this tendency to protect guilty journals in the first place?

Thing is, I’ve done this myself. For example, back in 2012, I wrote about having a paper rejected from “a mid-to-low ranked palaeo journal” for what I considered (and still consider) spurious reasons. Why didn’t I name the journal? I’m not really sure. (It was Palaeontologia Electronica, BTW.)

In cases like my unhelpful peer-review, it’s not really a big deal either way. In cases like those mentioned in the tweets above, it’s a much bigger issue, because those (unlike PE) are journals to avoid. Whichever journal sat on a submission for three years before rejecting it because it couldn’t find reviewers is not one that other researchers should waste their time on in the future — but how can they avoid it if they don’t know what journal it is?

So what’s going on? Why do we have this widespread tendency to protect the guilty?

It’s common to come across abstracts like this one, from an interesting paper on how a paper’s revision history influences how often it gets cited (Rigby, Cox and Julian 2018):

Journal peer review lies at the heart of academic quality control. This article explores the journal peer review process and seeks to examine how the reviewing process might itself contribute to papers, leading them to be more highly cited and to achieve greater recognition. Our work builds on previous observations and views expressed in the literature about (a) the role of actors involved in the research and publication process that suggest that peer review is inherent in the research process and (b) on the contribution reviewers themselves might make to the content and increased citation of papers. Using data from the journal peer review process of a single journal in the Social Sciences field (Business, Management and Accounting), we examine the effects of peer review on papers submitted to that journal including the effect upon citation, a novel step in the study of the outcome of peer review. Our detailed analysis suggests, contrary to initial assumptions, that it is not the time taken to revise papers but the actual number of revisions that leads to greater recognition for papers in terms of citation impact. Our study provides evidence, albeit limited to the case of a single journal, that the peer review process may constitute a form of knowledge production and is not the simple correction of errors contained in submitted papers.

This tells us that a larger number of revisions leads to (or at least is correlated with) an increased citation-count. Interesting!

Immediately, I have two questions, and I bet you do, too:

1. What is the size of the effect?
2. How robust is it?

If their evidence says that each additional round of peer-review yields an dozen additional citations, I might be prepared to revise my growing conviction that multiple rounds of peer review are essentially a waste of time. If it says that each round yields 0.0001 additional citations, I won’t. And if the effect is statistically insignificant, I’ll ignore it completely.

But the abstract doesn’t tell me those simple and fundamental facts, which means the abstract is essentially useless. Unless the authors’ goal for the abstract was for it to be an advertisement for the paper — but that’s not what an abstract is for.

In the old days, authors didn’t write abstracts for their own papers. These were provided after the event — sometimes after publication — by third parties, as a service for those who did not have time to read the whole paper but were interested in its findings. The goal of an abstract is to act as a summary of the paper, a surrogate that a reader can absorb instead of the whole paper, and which summarises the main findings. (I find it interesting that in some fields, the term “précis” or “synopsis” is used: both are more explicit.)

Please, let’s all recognise the painful truth that most people who read abstracts of our papers will not go on to read the full manuscripts. Let’s write our abstracts for those short-on-time people, so they go away with a clear and correct understanding of what our findings were and how strongly they are supported.

References

Rigby, J., D. Cox and K. Julian. 2018. Journal peer review: a bar or bridge? An analysis of a paper’s revision history and turnaround time, and the effect on citation. Scientometrics 114:1087–1105. doi:10.1007/s11192-017-2630-5

 

Lots of discussion online lately about unpaid peer reviews and whether this indicates a “degraded sense of community” in academia, improper commoditization of the unwritten responsibilities of academics, or a sign that we should rethink incentives in academia. (NB: that’s my galloping sound-bite-ization of those three posts, which you should go read in full.)

Part of this “reviewers don’t get paid” thing is good, because it indicates that academics broadly are waking up to how badly they’ve been had by commercial publishers. It’s part of that necessary anger that Scott Aaronson wrote about back when. But I can also understand why people are pushing back and saying, “Oh, if you don’t review you’re not supporting the academic community that (in part) makes your career possible. We should all pitch in and do the work.” Until recently, there was no way to separate those two strands: in doing peer reviews (and editing, etc.), one was both supporting the community as a good citizen, and also, unavoidably, helping commercial publishers line their pockets. But now that previously single path has bifurcated (no, not that way). Now it’s possible to be a good citizen for the community by editing and reviewing for OA journals, and stick it to the barrier-based publishers by not editing and reviewing for them (here’s how to politely decline, and see more discussion here).

Here’s how jacked the situation is: if you edit or review for a barrier-based publisher whose journals you also subscribe to or otherwise pay for, then in effect you are paying them for the privilege of reviewing. Put like that, it sounds insane. In any normal transaction, I give you X and you give me Y in return, because we’ve jointly agreed that these things are of roughly equal worth. In barrier-based publishing, academics give publishers (1) their papers, which publishers then exert copyright over, (2) their effort as editors and reviewers, and (3) their money, in subscriptions or other access fees, individually or collectively as institutions. And publishers sell the work back to us, retaining the copyright, and reap massive profits. There is no part of that sequence where academics – and indeed humanity at large – are getting the upside of the deal. The publishers are running the table on us, because for a long time, there were no other options. That’s not true anymore.

In his post on community, Zen Faulkes wrote, “I think people are refusing to do reviews in part because they don’t feel connected to the academic community.” Possibly. But maybe people are refusing to do reviews because they’re tired of being had. Has anyone done any work that would allow us to test those hypotheses? If so, I’d love to hear about it in the comments.

TL;DR: The separation of community goals and corporate profits shouldn’t be a fine theoretical point of discussion. It should be what we lead with. Yes, I will support the academic community. No, I won’t donate my time and effort to rapacious barrier-based publishers. It’s possible to achieve both of those things at once. And we should.

This morning, I was invited to review a paper — one very relevant to my interests — for a non-open-access journal owned by one of the large commercial barrier-based publishers. This has happened to me several times now; and I declined, as I have done ever since 2011.

I know this path is not for everyone. But for anybody who feels similarly to how I do but can’t quite think what to say to the handling editor and corresponding author, here are the messages that I sent to both.

First, to the handling editor (who in this case also happened to be the Editor-in-Chief):

Dear EDITOR NAME,

I’m writing to apologise for turning down your request that I review NAME OF PAPER. The reason is that I am wholly committed to the free availability of all scholarly research to everyone, and I cannot in good conscience give my time and expertise to a paper that is destined to end up behind PUBLISHER‘s paywall.

I know this can sound very self-righteous — I am sorry if it appears that way. I also recognise that there is serious collateral damage from limiting my reviewing efforts to open-access journals. My judgement is that, in the long term, that regrettable damage is a price worth paying, and I laid out my reasons a few years ago in this blog post: https://svpow.com/2011/10/17/collateral-damage-of-the-non-open-reviewing-boycott/

I hope you will understand my reasons for pushing hard towards an open-access future for all our scholarship; and I even hope that you might reconsider the time you yourself dedicate to PUBLISHER‘s journal, and wonder whether it might be more fruitfully spent in helping an open-access palaeontology journal to improve its profile and reputation.

Yours, with best wishes,

Mike.

Then, to the corresponding author, a similar message:

Dear AUTHOR NAME,

I was invited by JOURNAL to review your new manuscript NAME OF PAPER. I’m writing to apologise for turning down that request, and to explain why I did so.

The reason is that I am wholly committed to the free availability of all scholarly research to everyone, and I cannot in good conscience give my time and expertise to a paper that is destined to end up behind PUBLISHER‘s paywall.

I know this can sound very self-righteous — I am sorry if it appears that way. I also recognise that there is serious collateral damage from limiting my reviewing efforts to open-access journals. My judgement is that, in the long term, that regrettable damage is a price worth paying, and I laid out my reasons a few years ago in this blog post: https://svpow.com/2011/10/17/collateral-damage-of-the-non-open-reviewing-boycott/

I hope you will understand my reasons for pushing hard towards an open-access future for all our scholarship; and I even hope that you might consider withdrawing your work from JOURNAL, and instead submitting to one of the many fine open-access journals in our field. (Examples: Palaeontologia Electronica, Acta Palaeontologica Polonica, PLOS ONE, PeerJ, PalArch’s Journal of Vertebrate Paleontology, Royal Society Open Science.)

Yours, with apologies for the inconvenience and my best wishes,

Mike.

Anyone is welcome to use these messages as templates or inspiration if they are useful. Absolutely no rights reserved.