The only winning move is not to play

March 19, 2013

The problem

I find myself reading a lot recently about “portable peer-review” — posts like Take me as I am, and my paper as it is? by scicurious at Neurotic Physiology, which excellently diagnoses a terrible, wasteful problem in scientific publishing:

My papers don’t often get in with minor revisions. Often I’ve got a ridiculously puffed head about my own work (apparently), and send them to places which reject them out of hand, or suggest major revisions and piles of new experiments which we just cannot do for various reasons. Then the paper ends up shuttled around. Send it in, wait 3 months, get rejected. Reformat (+2 mo or even more depending on collaborators and how much other crap you’ve got on your plate at the time) and send it out again. Years go by. In the meantime, suggested reviewers begin to hate me and I run out of new ones (only so many people in the field!).

I really wish there was a way to get out of this. This sort of thing contributes to the long lag times and slowness of scientific advance.

What a waste! What a drag on the progress of science! What a ridiculous situation we’re got ourselves into, with our chasing-after-prestigious-journals games.

An inadequate solution

The solution proposed by scicurious is:

You submit a paper to a large umbrella of journals of several “tiers”. It goes out for review. The reviewers make their criticisms. Then they say “this paper is fine, but it’s not impactful enough for journal X unless major experiments A, B, and C are done. However, it could fit into journal Y with only experiment A, or into journal Z with only minor revisions”.

As an incremental improvement on the current system, this is good, if rather impractical to implement.

But it doesn’t go nearly far enough. It still wastes time by going to multiple journals, probably with different formatting requirements, requiring assessment (albeit more lightweight) by several editors. And it does all that in the name of getting a designer label onto the paper by placing it in a “good” journal.

What are we, fourteen?

High-school kids are dumb enough to judge other kids by how fashionable their clothes are, by the labels on them, by whether they’re the clothes other kids think are cool.

Have we really not got beyond that?

The ugly truth

Trying to get into “good” journals is an idiot game. (Notice I don’t say “an idiot’s game” — more on this distinction below.) Although the political and bean-counting value of getting into Nature is huge, the scientific value of getting into Nature is zero. A paper in Nature is literally no better at all than the same paper would be in PLOS ONE. (In fact, it’s probably less good, because it will be butchered to fit the draconian space requirements.) Spending time and effort in trying to get a given piece of research into Nature is just about the least useful thing that can be done for that research.

I think deep down everyone knows this. But of course scientists still waste innumerable hours formatting their work first for Nature, then for Science when it gets rejected, then for PNAS when it gets rejected again, and so on “down the ladder”. But that direction is only “down” by agreement. And the reason of course is because it’s widely (though not universally) believed that wearing these designer clothes is the way to get jobs and grants. That’s why people who are not idiots play this idiot game.

(Thanks heavens for funders and assessors who explicitly state that the journal a work is published in has no effect on how it’s evaluated. You can find such statements from The Wellcome Trust, and regarding the Research Excellence Framework (REF). I want to see more granting and evaluation bodies make similar statements, and I look forward to seeing a university hiring policy that says the same.)

A better way

Happily for me, I don’t need a job or a grant, so I have the luxury of standing on the sidelines, shaking my head sagely yet smugly at the ridiculous manouevres happening on the pitch.

I admit to my shame that I have played the getting-into-a-good-journal game in the past, just because I blindly copied what I saw my colleagues doing without really thinking about it. One result is that our neck-anatomy paper was needlessly held up for more than four yearsNo-one benefits from these delays. They are a completely avoidable net loss for science.

No more. I am done with having my work rejected for spurious (i.e. non-scientific reasons). I’m only planning on submitting to journals that don’t do that. I reject the idiot notion that the natural lifecycle of a piece of work involves multiple submissions-review-reject cycles. From now on, my cycle is: do some work, write it up, submit it, see it published, move on to the next thing.

And note that “move on to the next thing” is a crucial step here. What really burns me is not the four-year delays on the papers I mentioned above, but all the other work that I’ve not done because I’ve been buggering about, excuse my French, with the corpses of these long-dead projects instead of getting the next thing done. And if that’s true for me, I bet it’s true for you, as well. Yes, you, reading this!

As of now, except in exceptional circumstances, my plan is only to submit to venues where I know scientifically sound work will be accepted. That means “megajournals” like PLOS ONE, PeerJ and (I don’t know, I will look into it) maybe some or all BMC journals. It also means edited volumes that I’m invited to contribute to (though they have their own issues). It probably also means certain other journals, such as PalArch, though they don’t make it explicit (and it would be good if they did).

Three clarifications

First clarification: to be clear, I am not arrogant enough to think this means I will never again have a paper rejected. No doubt there will be occasions where I’ve made significant scientific errors, and reviewers will have to point those out and recommend rejection. I don’t mind that: it’s peer-review actually doing its job, and I’d rather fix those mistakes before publication. What I’m done with is rejections on the basis of “not impacty enough for this journal”, or the often equally specious “not a good fit”.

Second clarification: I don’t absolutely rule out exceptions. There might be occasions where, say, an impact-selective journal announces plans to put out a special volume that I want to be part of. I might submit to that; then again, I might not. I’ll judge it as it comes. But the point is, any exceptions will be exceptions. When I start thinking “where shall I send this?”, my list won’t start with Palaeontology and JVP. I’m glad to have got those notches on my bedpost, but I don’t feel any great need to go back to them.

Third clarification: I do understand that others might not be in a position to make the same leap. I am 99.7% certain that Darren won’t, for example, as he is convinced of the absolute necessity of Science‘n’Nature papers to advance his career. Matt, on the other hand, can and I think will — he’s got a tenure-track job at a university that he likes, has no plans to move on, and doesn’t need “prestigious” papers for his tenure case, only good ones.

(It pained me to have to make that distinction. What a stupid world, where “prestigious papers” and “good papers” are not synonymous, and don’t even overlap that much.)

But for people who, like me, don’t need to have an eye on the possible job-power of “prestige”, it seems obviously better to do what advances science best and fastest. And what a tragedy that advancing science isn’t what gets jobs.

10 Responses to “The only winning move is not to play”

  1. protohedgehog Says:

    Neat article as usual, Mike, and I agree on all counts with the 0 overlap between quality and label. It goes back to the points made on one of your earlier posts, however, that in the current academic system, we are (especially younger researchers) judged by our peers on which journals we publish articles in, irrespective of the wealth of evidence that this is a meaningless measure of quality. Until that changes, most trying to climb up the academic ladder won’t be able to adopt this, arguably much more logical, stance. Pity eh.

  2. Mike Taylor Says:

    Well, this is where funders, evaluators and job-search committees have so much power to change things. It’s why I am mightily impressed by Wellcome, and by (this aspect of) the REF.

    In fact, it would be great to put together a list of funders and evaluators who similarly make an explicit statement that they don’t care what journal work is in, but about the quality of the work. Does anyone know of others?

  3. Heteromeles Says:

    I happen to agree, and I happen to be really glad I’m no longer in academia where I would have had to play that game.

    The really scary part (for academicians and others) is that it’s possible for any group of like-minded people to throw together a journal, review articles, and publish online for fairly little money. It would cost rather more to archive stuff in perpetuity, but the point is that the current journals are both expensive and highly politicized entirely by custom and history, not by need. Academic publishing uses the same model as vanity presses, and it appears to be mostly by, for, and about vanity, to the profit of the publishers and the impoverishment of academic scientists (and the public whose taxes support their grants) who pay to play and pay to read the results.

    The odd thing is that the rest of the publishing world apparently expected academic publishing to lead the way down the steep slope of digital media plowed by music and videos. Instead, academic publishing has been amazingly reactionary in clinging to the vanity publishing model. Now, in an age when ebooks are killing publishers left and right and anyone can publish their scientific paper on Amazon and sell it for a penny, scientific publishing remains this, well, game.

  4. Hello, my name is Marius Buliga and I used to be a serial submitter.

  5. […] PS2. See also the very interesting post by Mike Taylor “The only winning move is not to play“. […]

  6. Renate Wesselingh Says:

    An interesting initiative I would like to mention in this context is Peerage of Science, which turns the world of peer review upside down: you submit your paper not to a specific journal, but to be reviewed by your peers, double-blind. Members in your field are notified when your manuscript is submitted, and they can choose themselves to review or not. The deadlines are strict, but a reviewer who initially signed up to do a review (here called “essay”) can always opt out before the deadline. After the deadline, the reviewers are invited to judge each other’s essays. The author then receive the essays and their quality scores, and can submit a revised version (or retract the manuscript), after which the final score is determined for the paper by the original reviewers, weighed by their essay quality score. So as a member, you have an article quality score (for the papers you submitted), an essay quality score (for the reviews you wrote) and a review balance, which prevents peers from submitting too many papers without reviewing any.

    Journals subscribing to Peerage of Science can track papers throughout the process and make an offer for publication at any time, until a participating journal has acquired it or the authors have taken it to a journal outside Peerage of Science, where they can also offer access to the reviews in Peerage.

    There are now over 1300 peers in this system, mainly from the fields of Ecology, Evolutionary Biology and Conservation Biology, but numbers keep growing. You can become a peer after having published at least one paper in a peer-reviewed journal. It’s invitation only, so you will have to find someone willing to invite you.

  7. […] But I hear you on the message sent by acquiescing to a $1600-APC journal, even if that APC is waived. We both want to shift towards a world where there are no journals that charge that kind of money — or at least, that if they do, it’s because they’re the kind of “selective” journal that thinks there’s something praiseworthy about rejecting most scientifically sound submissions. Journals of that kind don’t concern me one way or another, because I just don’t play that game. […]

  8. […] was my first encounter with “not significant enough for our journal” — a game that I no longer play. It was to be very far from my last experience of Wrong Mike Taylor […]

  9. […] peer-review. The special sauce that persuaded the US Patent Office that this is a new invention is cascading peer review — an idea so obvious and so well-established that even The Scholarly Kitchen was writing […]

  10. Tim Says:

    Sadly, for those of us just starting in research we simply have no choice but to play the idiot game. I am currently, on a PhD and my supervisor (ego level is incredible) ALWAYS plays the game.
    Result? Reformatting and resubmitting ad infinitum this is AFTER every sentence in my drafts have been needlessly rephrased multiple times by my supervisor before we even start submitting.
    I am seriously considering whether I can stay with the team after the programme given that I don’t agree with this manner of thinking among others.
    But then the research fellowships are not exactly plentiful so when the time comes I wonder if I would be able to leave the team. In fact I might just be glad for a postdoc with them or perhaps go into industry.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: