Scholarly Stockholm Syndrome in full gallop
February 4, 2023
I was a bit shaken to read this short article, Submit It Again! Learning From Rejected Manuscripts (Campbell et al. 2022), recently posted on Mastodon by open-access legend Peter Suber.
For example:
Journals may reject manuscripts because the paper is not in the scope of the journal, because they recently published a similar article, because the formatting of the article is incorrect, or because the paper is not noteworthy. In addition, editors may reject a paper expecting authors to make their work more compelling.
Let’s pick this apart a bit.
“Because they recently published a similar article”? What is this nonsense. Does the Journal of Vertebrate Paleontology reject a paper on, say, ornithopod ontogeny because “we published something on ornithopod ontogeny a few months ago”? No, it doesn’t because it’s a serious journal.
“Because the formatting of the article is incorrect”? What is this idiocy? If the formatting is incorrect, the job of the publisher is to correct it. That’s literally what they’re there for.
“Expecting authors to make their work more compelling”. This is code for sexing up the results, maybe dropping that inconvenient outlier, getting p below 0.05 … in short, fraud. The very last thing we need more of.
Elsewhere this paper suggests:
… adjusting an original research paper to a letter to the editor or shifting the focus to make the same content into a commentary or narrative essay.
Needless to say, this is putting the cart before the horse. Once we start prioritising what kind of content a journal would like to have ahead of what our work actually tells us, we’re not scientists any more.
Then there is this:
Most manuscripts can eventually Ynd a home in a PubMed-indexed journal if the authors continually modify the manuscript to the specifications of the editors.
I’m not saying this is incorrect. I’m not even saying it’s not good advice. But I worry about the attitude that it communicates — that editors are capricious gods whose whims are to be satisfied. Editors should be, and good editors are, partners in the process of bringing a work to publication, not barriers.
Next up:
Studies confirming something already well known and supported might not be suitable for publication, but looking for a different perspective or a new angle to make it a new contribution to the literature may be useful.
In other words, if you run an experiment, however well you do the work and however well you write the paper, you should expect to have it rejected if the result doesn’t excite the editor. But if you can twist it into something that does excite the editor, you might be OK. Is this really how we want to encourage researchers to behave?
I’ve seen studies like this. I have seen projects that set out to determine how tibia shape correlates with lifestyle in felids, find out the rather important fact that there is no correlation, and instead report the Principle Component 1, which explains 4.2% of the morphological difference, sort of shows a slight grouping if you squint hard and don’t mind all your groups overlapping. (Note: all details changed to protect the guilty. I know nothing of felid tibiae.) I don’t wish to see more such reporting. I want to know what a study actually showed, not what an editor thought might be exciting.
But here is why I am so unhappy about this paper.
It’s that the authors seem so cheerful about all this. That they serenely accept it as a law of the universe that perfectly good papers can be rejected for the most spurious of reasons, and that the proper thing to do is smile broadly and take your ass to the next ass-kicking station.
It doesn’t seem to occur to them that there are other ways of doing scientific communication: ways that are constructive rather than adversarial, ways the aim to get at the truth rather than aiming at being discussed in a Malcolm Gladwell book[1], ways that make the best use of researchers’ work instead of discarding what is inconvenient.
Folks, we have to do better. Those of us in senior positions have to make sure we’re not teaching out students that the psychopathic systems we had to negotiate are a law of the universe.
References
Notes
- I offer the observation that any finding reported and discussed in a Malcolm Gladwell book seems to have about an 80% chance of being shown to be incorrect some time in the next ten years. In the social sciences, particularly, a good heuristic for guessing whether or not a given result is going to replicate is to ask: has it been in a Gladwell book?
Filed in Peer review, rants, stinkin' authors, stinkin' editors, stinkin' publishers, stinkin' reviewers
Why publishers cannot be around the table when the scholarly community discusses its communication options
June 29, 2017
Publishers provide certain services (peer-review management, typesetting, brand badges, sometimes proof-reading or copy-editing, archiving, indexing) to the scholarly community.
Those services are of greater and lesser value, provided at higher and lower levels of quality, and cost greater and lesser amounts. Of course, we in the scholarly community want high-value, high-quality low-cost services. This is true whether the publisher in question is a multinational corporation with a multi-billion-dollar turnover, or a tiny boutique press run on a non-profit basis for the sheer love of the process.
Since the scholarly community (researchers, authors, peer-reviewers, academic editors, etc.) is spending money in exchange for publication services, and since publishers are providing publication services in exchange for money, it is clear that the goals of these two groups cannot be aligned. Any money that the scholarly community can save on publication costs is income lost to publishers; and any additional money that publishers can charge for their services is money lost to the scholarly community. I hope that so far, this is uncontroversial.
In the same way, if you sell me a second-hand car, then however well you and I might get on in civilian life — we might support the same football team, drink the same beer, discuss the same novelists, watch the same films — then for the purposes of that transaction, what is good for you (a high price) is bad for me; and vice versa. Note that in saying this I am not condemning or even criticising you. I am just stating a fact about transactions.
Now, suppose my wife and I sit down and decide that we need to buy a new car. We consider Hondas, Fords and Fiats. We weigh up various models on their merits, compare their prices with their features, and reach a decision on what we want to buy and how much we’re prepared to spend. We then approach the various Honda dealers (or, as we may have decided, Ford dealers or Fiat dealers). We negotiate with them to agree a price that we are happy with for a model that is in good enough condition. Different dealers compete with each other to win our custom by offering good cars at a low price. This is a functioning market.
What we don’t do is invite all the dealers to come and join us in our initial conversation. When my wife and I are discussing how important it is to us that our new car has variable-speed intermittent windscreen-wipers, we have that discussion in an environment quite free of car dealers telling us how great Fiat’s intermittent-wipe feature is. How could we possibly reach a coherent decision on what our own requirements are if we’re bombarded by the claims — some competing, some in collusion — of all the car dealers? And how can we think sensibly about what we’re prepared to spend if we’re surrounded by the dealers’ defences of the various financing arrangements they offer?
So in the same way, I feel that the scholarly community needs to figure out what publication services it needs, free of the influence of publishers who (and again this is not a criticism) have their own agenda. Then, when we know what we want, we can go to the publishers who offer the kinds of services we’re interested in, and invite them compete for our business on the basis of features and price.
But involving them in the initial what-we-want discussion can only lead to confusion, and a compromised outcome. Which is what we’ve seen for the last 50 years. This was the fatal flaw that led to the deeply flawed Finch Report and to the erosion of the RCUK’s initially very progressive OA policy.
As a side-note: my wife and I may end up deciding we don’t need a car at all: we might decide we can walk, or cycle, or take public transport. Car dealers would hate that: they would advocate against such an outcome with all their might if they were involved in that discussion. Which is why they can’t be.
Note. This post is adapted from a message to the Open Scholarship Initiative mailing list.
Filed in stinkin' authors, stinkin' editors, stinkin' publishers, stinkin' reviewers
Interview with Open Access Nigeria
October 22, 2014
Last night, I did a Twitter interview with Open Access Nigeria (@OpenAccessNG). To make it easy to follow in real time, I created a list whose only members were me and OA Nigeria. But because Twitter lists posts in reverse order, and because each individual tweet is encumbered with so much chrome, it’s rather an awkward way to read a sustained argument.
So here is a transcript of those tweets, only lightly edited. They are in bold; I am in regular font. Enjoy!
—
So @MikeTaylor Good evening and welcome. Twitterville wants to meet you briefly. Who is Mike Taylor?
In real life, I’m a computer programmer with Index Data, a tiny software house that does a lot of open-source programming. But I’m also a researching scientist — a vertebrate palaeontologist, working on sauropods: the biggest and best of the dinosaurs. Somehow I fit that second career into my evenings and weekends, thanks to a very understanding wife (Hi, Fiona!) …
As of a few years ago, I publish all my dinosaur research open access, and I regret ever having let any of my work go behind paywalls. You can find all my papers online, and read much more about them on the blog that I co-write with Matt Wedel. That blog is called Sauropod Vertebra Picture of the Week, or SV-POW! for short, and it is itself open access (CC By)
Sorry for the long answer, I will try to be more concise with the next question!
Ok @MikeTaylor That’s just great! There’s been so much noise around twitter, the orange colour featuring prominently. What’s that about?
Actually, to be honest, I’m not really up to speed with open-access week (which I think is what the orange is all about). I found a while back that I just can’t be properly on Twitter, otherwise it eats all my time. So these days, rather selfishly, I mostly only use Twitter to say things and get into conversations, rather than to monitor the zeitgeist.
That said, orange got established as the colour of open access a long time ago, and is enshrined in the logo:
In the end I suppose open-access week doesn’t hit my buttons too strongly because I am trying to lead a whole open-access life.
… uh, but thanks for inviting me to do this interview, anyway! :-)
You’re welcome @MikeTaylor. So what is open access?
Open Access, or OA, is the term describing a concept so simple and obvious and naturally right that you’d hardly think it needs a name. It just means making the results of research freely available on the Internet for anyone to read, remix and otherwise use.
You might reasonably ask, why is there any other kind of published research other than open access? And the only answer is, historical inertia. For reasons that seemed to make some kind of sense at the time, the whole research ecosystem has got itself locked into this crazy equilibrium where most published research is locked up where almost no-one can see it, and where even the tiny proportion of people who can read published works aren’t allowed to make much use of them.
So to answer the question: the open-access movement is an attempt to undo this damage, and to make the research world sane.
Are there factors perpetuating this inertia you talked about?
Oh, so many factors perpetuting the inertia. Let me list a few …
- Old-school researchers who grew up when it was hard to find papers, and don’t see why young whippersnappers should have it easier
- Old-school publishers who have got used to making profits of 30-40% turnover (they get content donated to them, then charge subscriptions)
- University administrators who make hiring/promotion/tenure decisions based on which old-school journals a researcher’s papers are in.
- Feeble politicians who think it’s important to keep the publishing sector profitable, even at the expense of crippling research.
I’m sure there are plenty of others who I’ve overlooked for the moment. I always say regarding this that there’s plenty of blame to go round.
(This, by the way, is why I called the current situation an equilibrium. It’s stable. Won’t fix itself, and needs to be disturbed.)
So these publishers who put scholarly articles behind paywalls online, do they pay the researchers for publishing their work?
HAHAHAHAHAHAHAHAHAHA!
Oh, sorry, please excuse me while I wipe the tears of mirth from my eyes. An academic publisher? Paying an author? Hahahahaha! No.
Not only do academic publishers never pay authors, in many cases they also levy page charges — that is, they charge the authors. So they get paid once by the author, in page-charges, then again by all the libraries that subscribe to read the paywalled papers. Which of course is why, even with their gross inefficiencies, they’re able to make these 30-40% profit margins.
So @MikeTaylor why do many researchers continue to take their work to these restricted access publishers and what can we do about it?
There are a few reasons that play into this together …
Part of it is just habit, especially among more senior researchers who’ve been using the same journals for 20 or 30 years.
But what’s more pernicious is the tendency of academics — and even worse, academic administrators — to evaluate research not by its inherent quality, but by the prestige of the journal that publishes it. It’s just horrifyingly easy for administrators to say “He got three papers out that year, but they were in journals with low Impact Factors.”
Which is wrong-headed on so many levels.
First of all, they should be looking at the work itself, and making an assessment of how well it was done: rigour, clarity, reproducibility. But it’s much easier just to count citations, and say “Oh, this has been cited 50 times, it must be good!” But of course papers are not always cited because they’re good. Sometimes they’re cited precisely because they’re so bad! For example, no doubt the profoundly flawed Arsenic Life paper has been cited many times — by people pointing out its numerous problems.
But wait, it’s much worse than that! Lazy or impatient administrators won’t count how many times a paper has been cited. Instead they will use a surrogate: the Impact Factor (IF), which is a measure not of papers but of journals.
Roughly, the IF measures the average number of citations received by papers that are published in the journal. So at best it’s a measure of journal quality (and a terrible measure of that, too, but let’s not get into that). The real damage is done when the IF is used to evaluate not journals, but the papers that appear in them.
And because that’s so widespread, researchers are often desperate to get their work into journals that have high IFs, even if they’re not OA. So we have an idiot situation where a selfish, rational researcher is best able to advance her career by doing the worst thing for science.
(And BTW, counter-intuitively, the number of citations an individual paper receives is NOT correlated significantly with the journal’s IF. Bjorn Brembs has discussed this extensively, and also shows that IF is correlated with retraction rate. So in many respects the high-IF journals are actually the worst ones you can possibly publish your work in. Yet people feel obliged to.)
*pant* *pant* *pant* OK, I had better stop answering this question, and move on to the next. Sorry to go on so long. (But really! :-) )
This is actually all so enlightening. You just criticised Citation Index along with Impact Factor but OA advocates tend to hold up a higher Citation Index as a reason to publish Open Access. What do you think regarding this?
I think that’s realpolitik. To be honest, I am also kind of pleased that the PLOS journals have pretty good Impact Factors: not because I think the IFs mean anything, but because they make those journals attractive to old-school researchers.
In the same way, it is a well-established fact that open-access articles tend to be cited more than paywalled ones — a lot more, in fact. So in trying to bring people across into the OA world, it makes sense to use helpful facts like these. But they’re not where the focus is.
But the last thing to say about this is that even though raw citation-count is a bad measure of a paper’s quality, it is at least badly measuring the right thing. Evaluating a paper by its journal’s IF is like judging someone by the label of their clothes
So @MikeTaylor Institutions need to stop evaluating research papers based on where they are published? Do you know of any doing it right?
I’m afraid I really don’t know. I’m not privy to how individual institution do things.
All I know is, in some countries (e.g. France) abuse of IF is much more strongly institutionalised. It’s tough for French researchers
What are the various ways researchers can make their work available for free online?
Brilliant, very practical question! There are three main answers. (Sorry, this might go on a bit …)
First, you can post your papers on preprint servers. The best known one is arXiv, which now accepts papers from quite a broad subject range. For example, a preprint of one of the papers I co-wrote with Matt Wedel is freely available on arXiv. There are various preprint servers, including arXiv for physical sciences, bioRxiv, PeerJ Preprints, and SSRN (Social Science Research Network).
You can put your work on a preprint server whatever your subsequent plans are for it — even if (for some reason) it’s going to a paywall. There are only a very few journals left that follow the “Ingelfinger rule” and refuse to publish papers that have been preprinted.
So preprints are option #1. Number 2 is Gold Open Access: publishing in an open-access journal such as PLOS ONE, a BMC journal or eLife. As a matter of principle, I now publish all my own work in open-access journals, and I know lots of other people who do the same — ranging from amateurs like me, via early-career researchers like Erin McKiernan, to lab-leading senior researchers like Michael Eisen.
There are two potential downsides to publishing in an OA journal. One, we already discussed: the OA journals in your field may not be be the most prestigious, so depending on how stupid your administrators are you could be penalised for using an OA journal, even though your work gets cited more than it would have done in a paywalled journal.
The other potential reason some people might want to avoid using an OA journal is because of Article Processing Charges (APC). Because OA publishers have no subscription revenue, one common business model is to charge authors an APC for publishing services instead. APCs can vary wildly, from $0 up to $5000 in the most extreme case (a not-very-open journal run by the AAAS), so they can be offputting.
There are three things to say about APCs.
First, remember that lots of paywalled journals demand page charges, which can cost more!
But second, please know that more than half of all OA journals actually charge no APC at all. They run on different models. For example in my own field, Acta Palaeontologica Polonica and Palaeontologia Electronica are well respected OA journals that charge no APC.
And the third thing is APC waivers. These are very common. Most OA publishers have it as a stated goal that no-one should be prevented from publishing with them by lack of funds for APCs. So for example PLOS will nearly always give a waiver when requested. Likewise Ubiquity, and others.
So there are lots of ways to have your work appear in an OA journal without paying for it to be there.
Anyway, all that was about the second way to make your work open access. #1 was preprints, #2 is “Gold OA” in OA journals …
And #3 is “Green OA”, which means publishing in a paywalled journal, but depositing a copy of the paper in an open repository. The details of how this works can be a bit complicated: different paywall-based publishers allow you to do different things, e.g. it’s common to say “you can deposit your peer-reviewed, accepted but unformatted manuscript, but only after 12 months“.
Opinions vary as to how fair or enforceable such rules are. Some OA advocates prefer Green. Others (including me) prefer Gold. Both are good.
See this SV-POW! post on the practicalities of negotiating Green OA if you’re publishing behind a paywall.
So to summarise:
- Deposit preprints
- Publish in an OA journal (getting a fee waiver if needed)
- Deposit postprints
I’ve written absolutely shedloads on these subjects over the last few years, including this introductory batch. If you only read one of my pieces about OA, make it this one: The parable of the farmers & the Teleporting Duplicator.
Last question – Do restricted access publishers pay remuneration to peer reviewers?
I know of no publisher that pays peer reviewers. But actually I am happy with that. Peer-review is a service to the community. As soon as you encumber it with direct financial incentives, things get more complicated and there’s more potential for Conflict of interest. What I do is, I only perform peer-reviews for open-access journals. And I am happy to put that time/effort in knowing the world will benefit.
And so we bring this edition to a close. We say a big thanks to our special guest @MikeTaylor who’s been totally awesome and instructive.
Thanks, it’s been a privilege.
Filed in arXiv, Ask SV-POW!, craven administrators, Green open access, Look, this isn't complicated, open access, rants, recycled, Shiny digital future, stinkin' academics, stinkin' authors, stinkin' publishers
Researchers: do your damned job
August 6, 2014
I am just about out of patience with academic departments putting up endless idiot arguments about open access.
Bottom line: we pay you good money out of the public purse to do a highly desirable job where you get to work on what you love — jobs that have tens or dozens of candidates for every post. That job is: make new knowledge for the world. Not just for you and a few of your mates: for the world. If you’re not prepared to do that, then get the heck out of the job, and vacate a position for someone who will actually do what we pay them for.
Sheesh. I try to be understanding, I really do. But all this “Oh, oh, it’s not like it used to be in the old days” whining has worn me down. No, it’s not like it was in the old days, when you got paid to play, with nothing expected in return. Earn your damned keep, or get out of the road.
(And, yes, this is a toned down version of the comment I originally composed in my head.)
[Originally posted as a comment at The Guardian.]
Filed in craven administrators, open access, Shiny digital future, stinkin' authors
How untrue “facts” propagate in the scientific literature
August 5, 2014
Short post today. Go and read this paper: Academic urban legends (Rekdal 2014). It’s open access, and an easy and fascinating read. It unfolds a tale of good intentions gone wrong, a chain of failure, illustrating an important single crucial point of academic behaviour: read what you cite.
References
Filed in credit where it's due, paleontologists behaving badly, stinkin' authors