It’s common to come across abstracts like this one, from an interesting paper on how a paper’s revision history influences how often it gets cited (Rigby, Cox and Julian 2018):

Journal peer review lies at the heart of academic quality control. This article explores the journal peer review process and seeks to examine how the reviewing process might itself contribute to papers, leading them to be more highly cited and to achieve greater recognition. Our work builds on previous observations and views expressed in the literature about (a) the role of actors involved in the research and publication process that suggest that peer review is inherent in the research process and (b) on the contribution reviewers themselves might make to the content and increased citation of papers. Using data from the journal peer review process of a single journal in the Social Sciences field (Business, Management and Accounting), we examine the effects of peer review on papers submitted to that journal including the effect upon citation, a novel step in the study of the outcome of peer review. Our detailed analysis suggests, contrary to initial assumptions, that it is not the time taken to revise papers but the actual number of revisions that leads to greater recognition for papers in terms of citation impact. Our study provides evidence, albeit limited to the case of a single journal, that the peer review process may constitute a form of knowledge production and is not the simple correction of errors contained in submitted papers.

This tells us that a larger number of revisions leads to (or at least is correlated with) an increased citation-count. Interesting!

Immediately, I have two questions, and I bet you do, too:

1. What is the size of the effect?
2. How robust is it?

If their evidence says that each additional round of peer-review yields an dozen additional citations, I might be prepared to revise my growing conviction that multiple rounds of peer review are essentially a waste of time. If it says that each round yields 0.0001 additional citations, I won’t. And if the effect is statistically insignificant, I’ll ignore it completely.

But the abstract doesn’t tell me those simple and fundamental facts, which means the abstract is essentially useless. Unless the authors’ goal for the abstract was for it to be an advertisement for the paper — but that’s not what an abstract is for.

In the old days, authors didn’t write abstracts for their own papers. These were provided after the event — sometimes after publication — by third parties, as a service for those who did not have time to read the whole paper but were interesting in its findings. The goal of an abstract is to act as a summary of the paper, a surrogate that a reader can absorb instead of the whole paper, and which summarises the main findings. (I find it interesting that in some fields, the term “précis” or “synopsis” is used: both are more explicit.)

Please, let’s all recognise the painful truth that most people who read abstracts of our papers will not go on to read the full manuscripts. Let’s write our abstracts for those short-on-time people, so they go away with a clear and correct understanding of what our findings were and how strongly they are supported.

References

Rigby, J., D. Cox and K. Julian. 2018. Journal peer review: a bar or bridge? An analysis of a paper’s revision history and turnaround time, and the effect on citation. Scientometrics 114:1087–1105. doi:10.1007/s11192-017-2630-5

 

Advertisements

Lots of discussion online lately about unpaid peer reviews and whether this indicates a “degraded sense of community” in academia, improper commoditization of the unwritten responsibilities of academics, or a sign that we should rethink incentives in academia. (NB: that’s my galloping sound-bite-ization of those three posts, which you should go read in full.)

Part of this “reviewers don’t get paid” thing is good, because it indicates that academics broadly are waking up to how badly they’ve been had by commercial publishers. It’s part of that necessary anger that Scott Aaronson wrote about back when. But I can also understand why people are pushing back and saying, “Oh, if you don’t review you’re not supporting the academic community that (in part) makes your career possible. We should all pitch in and do the work.” Until recently, there was no way to separate those two strands: in doing peer reviews (and editing, etc.), one was both supporting the community as a good citizen, and also, unavoidably, helping commercial publishers line their pockets. But now that previously single path has bifurcated (no, not that way). Now it’s possible to be a good citizen for the community by editing and reviewing for OA journals, and stick it to the barrier-based publishers by not editing and reviewing for them (here’s how to politely decline, and see more discussion here).

Here’s how jacked the situation is: if you edit or review for a barrier-based publisher whose journals you also subscribe to or otherwise pay for, then in effect you are paying them for the privilege of reviewing. Put like that, it sounds insane. In any normal transaction, I give you X and you give me Y in return, because we’ve jointly agreed that these things are of roughly equal worth. In barrier-based publishing, academics give publishers (1) their papers, which publishers then exert copyright over, (2) their effort as editors and reviewers, and (3) their money, in subscriptions or other access fees, individually or collectively as institutions. And publishers sell the work back to us, retaining the copyright, and reap massive profits. There is no part of that sequence where academics – and indeed humanity at large – are getting the upside of the deal. The publishers are running the table on us, because for a long time, there were no other options. That’s not true anymore.

In his post on community, Zen Faulkes wrote, “I think people are refusing to do reviews in part because they don’t feel connected to the academic community.” Possibly. But maybe people are refusing to do reviews because they’re tired of being had. Has anyone done any work that would allow us to test those hypotheses? If so, I’d love to hear about it in the comments.

TL;DR: The separation of community goals and corporate profits shouldn’t be a fine theoretical point of discussion. It should be what we lead with. Yes, I will support the academic community. No, I won’t donate my time and effort to rapacious barrier-based publishers. It’s possible to achieve both of those things at once. And we should.

This morning, I was invited to review a paper — one very relevant to my interests — for a non-open-access journal owned by one of the large commercial barrier-based publishers. This has happened to me several times now; and I declined, as I have done ever since 2011.

I know this path is not for everyone. But for anybody who feels similarly to how I do but can’t quite think what to say to the handling editor and corresponding author, here are the messages that I sent to both.

First, to the handling editor (who in this case also happened to be the Editor-in-Chief):

Dear EDITOR NAME,

I’m writing to apologise for turning down your request that I review NAME OF PAPER. The reason is that I am wholly committed to the free availability of all scholarly research to everyone, and I cannot in good conscience give my time and expertise to a paper that is destined to end up behind PUBLISHER‘s paywall.

I know this can sound very self-righteous — I am sorry if it appears that way. I also recognise that there is serious collateral damage from limiting my reviewing efforts to open-access journals. My judgement is that, in the long term, that regrettable damage is a price worth paying, and I laid out my reasons a few years ago in this blog post: https://svpow.com/2011/10/17/collateral-damage-of-the-non-open-reviewing-boycott/

I hope you will understand my reasons for pushing hard towards an open-access future for all our scholarship; and I even hope that you might reconsider the time you yourself dedicate to PUBLISHER‘s journal, and wonder whether it might be more fruitfully spent in helping an open-access palaeontology journal to improve its profile and reputation.

Yours, with best wishes,

Mike.

Then, to the corresponding author, a similar message:

Dear AUTHOR NAME,

I was invited by JOURNAL to review your new manuscript NAME OF PAPER. I’m writing to apologise for turning down that request, and to explain why I did so.

The reason is that I am wholly committed to the free availability of all scholarly research to everyone, and I cannot in good conscience give my time and expertise to a paper that is destined to end up behind PUBLISHER‘s paywall.

I know this can sound very self-righteous — I am sorry if it appears that way. I also recognise that there is serious collateral damage from limiting my reviewing efforts to open-access journals. My judgement is that, in the long term, that regrettable damage is a price worth paying, and I laid out my reasons a few years ago in this blog post: https://svpow.com/2011/10/17/collateral-damage-of-the-non-open-reviewing-boycott/

I hope you will understand my reasons for pushing hard towards an open-access future for all our scholarship; and I even hope that you might consider withdrawing your work from JOURNAL, and instead submitting to one of the many fine open-access journals in our field. (Examples: Palaeontologia Electronica, Acta Palaeontologica Polonica, PLOS ONE, PeerJ, PalArch’s Journal of Vertebrate Paleontology, Royal Society Open Science.)

Yours, with apologies for the inconvenience and my best wishes,

Mike.

Anyone is welcome to use these messages as templates or inspiration if they are useful. Absolutely no rights reserved.

As explained in careful detail over at Stupid Patent of the Month, Elsevier has applied for, and been granted, a patent for online peer-review. The special sauce that persuaded the US Patent Office that this is a new invention is cascading peer review — an idea so obvious and so well-established that even The Scholarly Kitchen was writing about it as a commonplace in 2010.

Apparently this is from the actual patent. I can't verify that at the moment, as the site hosting it seems to be down.

Apparently this is from the actual patent. I can’t verify that at the moment, as the site hosting it seems to be down.

Well. What can this mean?

A cynic might think that this is the first step an untrustworthy company would take preparatory to filing a lot of time-wasting and resource-sapping nuisance lawsuits on its smaller, faster-moving competitors. They certainly have previous in the courts: remember that they have brought legal action their own customers as well as threatening Academia.edu and of course trying to take Sci-Hub down.

Elsevier representatives are talking this down: Tom Reller has tweeted “There is no need for concern regarding the patent. It’s simply meant to protect our own proprietary waterfall system from being copied” — which would be fine, had their proprietary waterfall system not been itself copied from the ample prior art. Similarly, Alicia Wise has said on a public mailing list “People appear to be suggesting that we patented online peer review in an attempt to own it.  No, we just patented our own novel systems.” Well. Let’s hope.

But Cathy Wojewodzki, on the same list, asked the key question:

I guess our real question is Why did you patent this? What is it you hope to market or control?

We await a meaningful answer.

When a paper goes for peer-review at PLOS ONE, the reviewers are told not to make any judgement about how important or sexy or “impacty” the paper is — to judge it only on methodical soundness. All papers that are judged sound are to be published without making guesses about which will and won’t improve the journal’s reputation through being influential down the line. (Such guesses are hopelessly inaccurate anyway.)

When PLOS ONE was new, this approach drew scorn from established publishers, but now those publishers all have their own journals that use similar editorial criteria (Nature’s Scientific Reports, AAAS‘s Science Advances, Elsevier’s first attempt, Elsevier’s second attempt, the Royal Society’s Royal Society Open Science). Those editorial criteria have proved their worth.

But what are we going to call this style of peer-review?

It’s not a new problem. I discussed it with with David Crotty three years ago without reaching any very satisfactory conclusion. But three years have not really helped us much as we try to agree on a term for this increasingly important and prevalent model.

What are the options on the table?

PLOS ONE-style peer-review. It’s a cumbersome term, and it privileges PLOS ONE when that is now far from the only journal to use this approach to peer-review (and may not even have been first).

Peer-review Lite. A snide term coined by people who wanted PLOS ONE to fail. It’s not a good description, and it carries baggage.

Scientific peer-review. This one came up in the discussion with David Crotty, but it’s not really acceptable because it would leave us still needing a term for what the Open Library of Humanities does.

Objective peer-review. This is the term that was used at the Royal Society meeting at the start of this week — the idea being that you review objectively for the quality of the research, but don’t make a subjective judgement of its importance. Several people didn’t like this on the grounds that even the “objective” half is inevitably subjective.

Any others that I missed?

I don’t have a good solution to propose to this problem; but I think it’s getting more urgent that we do solve it. We have to have a simple, unambiguous, universally understood term to understand a model of peer-review that is becoming increasingly pervasive and may well end up as the dominant form of peer-review.

Plough in — comments are open!

Update, 6pm

Liz Wager asked a very similar question four years ago, over on the BMJ blog: what to call the journals that use this approach to peer-review. Terms that she mentions include:

  • “bias to publish” (from BioMed Central)
  • “non-selective” (her own coinage, which she doesn’t like)
  • “bumboat” (I can’t explain this one, you’ll have to read the article)
  • “author-driver” or “author-focused” publication (AFP for short)
  • “search-located” (which she coins, the dismisses as tautologous)
  • “unconventional” or “non-traditional” (discarded as disparaging)
  • “non-discriminatory”, “impartial” or “unprejudiced”
  • “general” (dismissed as a non-starter)
  • “broad-spectrum” (inapplicable to specialised journals)

And then in the comments various people proposed:

  • “below the fold” journals
  • “omnivorous” (I quite like that one)
  • “alternative”
  • “Voldermortian journals”, which I don’t understand at all.
  • “Unfiltered”, contrasted with “filtered”
  • “inclusive”, contrasted with “exclusive” (I quite like this, too)
  • “high volume low hassle”

But there’s no conclusion or preferred term.

There’s been some concern over Scientific Reports‘ new scheme whereby authors submitting manuscripts can pay $750 to have them peer-reviewed more quickly. Some members of the editorial board have quit over this development, feeling that it’s unfair to authors who can’t pay. Myself, I feel it at least shows admirable audacity — NPG has found a way to monetise its own lethargy, which is surely what capitalism is all about.

The real problem with this scheme is that $750 is an awful lot to gamble, as a sort of “pre-APC”, at a point when you don’t know whether your article is actually going to be published or not. If the peer-review returns an unfavourable verdict it’s just money down the drain.

So I welcome today’s announcement that, for only a slightly higher payment of a round $1000, it’s now possible to bypass peer-review completely, and move directly to publication. This seems like a much fairer deal for authors, and of course it streamlines the publication process yet further. Now authors can obtain the prestigious Nature Publishing Group imprint in a matter of a couple of days.

Onward and upward!

Arriving as an early Christmas present, and coming in just a week before the end of what would otherwise have been a barren 2014, my paper Quantifying the effect of intervertebral cartilage on neutral posture in the necks of sauropod dinosaurs is out! You can read it on PeerJ (or download the PDF).

Figure 4. Effect of adding cartilage to the neutral pose of the neck of Diplodocus carnegii CM 84. Images of vertebra from Hatcher (1901:plate III). At the bottom, the vertebrae are composed in a horizontal posture. Superimposed, the same vertebrae are shown inclined by the additional extension angles indicated in Table 2.

Figure 4: Effect of adding cartilage to the neutral pose of the neck of Diplodocus carnegii CM 84. Images of vertebra from Hatcher (1901:plate III). At the bottom, the vertebrae are composed in a horizontal posture. Superimposed, the same vertebrae are shown inclined by the additional extension angles indicated in Table 2.

Yes, that posture is ludicrous — but the best data we currently have says that something like this would have been neutral for Diplodocus once cartilage is taken into account. (Remember of course that animals do not hold their necks in neutral posture.)

The great news here is that PeerJ moved quickly. In fact here’s how the time breaks down since I submitted the manuscript (and made it available as a preprint) on 4 November:

28 days from submission to first decision
3 days to revise and resubmit
3 days to accept
15 days to publication

TOTAL 49 days

Which of course is how it ought to be! Great work here from handling editor Chris Noto and all three reviewers: Matt Bonnan, Heinrich Mallison and Eric Snively. They all elected not to be anonymous, and all gave really useful feedback — as you can see for yourself in the published peer-review history. When editors and reviewers do a job this good, they deserve credit, and it’s great that PeerJ’s (optional) open review lets the world see what they contributed. Note that you can cite, or link to, individual reviews. The reviews themselves are now first-class objects, as they should be.

At the time of writing, my paper is top of the PeerJ home-page — presumably just because it’s the most recent published paper, but it’s a nice feeling anyway!

Screenshot from 2014-12-23 10:39:34

 

A little further down the front-page there’s some great stuff about limb function in ratites — a whole slew of papers.

Well, I’m off to relax over Christmas. Have a good one, y’all!