April 20, 2015
I’m at the Royal Society today and tomorrow as part of the Future of Scholarly Scientific Communication conference. Here’s the programme.
I’m making some notes for my own benefit, and I thought I might as well do them in the form of a blog-post, which I will continuously update, in case anyone else is interested.
I stupidly didn’t make notes on the first two speakers, but let’s pick up from the third:
Deborah Shorley, ex-librarian of Imperial College London
Started out by saying that she feels her opinion, as a librarian, is irrelevant, because librarians are becoming irrelevant. A pretty incendiary opening!
“Scientific communication in itself doesn’t matter; what matters is that good science be communicated well.”
And regarding the model of giving papers to publishers gratis, then paying them for the privilege of reading them:
“I can’t think of any other area where such a dopey business model pertains.”
(On which, see Scott Aaronson’s brilliant take on this in his review of The Access Principle — the article that first woke me up to the importance of open access.)
Shorey wants to bring publishing skills back in-house, to the universities and their libraries, and do it all themselves. As far as I can make out, she simply sees no need for specialist publishers. (Note: I do not necessarily endorse all these views.)
“If we don’t seize the opportunity, market forces will prevail. And market forces in this case are not pretty.”
Robert Parker, ex-head of publishing, Royal Society of Chemistry
Feels that society publishers allowed themselves to be overtaken by commercial publishers. Notes that when he started working for the RSC’s publishing arm, it was “positively dickensian”, using technology that would mostly have been familiar to Gutenberg. Failure to engage with authors and with technology allowed the commercial publishers to get ahead — something that is only now being redressed.
He’s talking an awful lot about the impact factors of their various journals.
My overall impression is that his perspective is much less radical than that of Deborah Shorley, wanting learned-society publishers to be better able to compete with the commercial publishers.
Gary Evoniuk, policy director at Glaxo Smith Klein
GSK submits 300-400 scientific studies for publication each year.
Although the rise of online-only journals means there is no good reason to not publish any finding, they still find that negative results are harder to get published.
“The paper journal, and the paper article, will soon be dead. This makes me a little bit sad.”
He goes further and wonders whether we need journal articles at all? When actual results are often available long before the article, is the context and interpretation that it provides valuable enough to be worth all the effort that’s expended on it? [My answer: yes — Ed.]
Discussion now follows. I probably won’t attempt to blog it (not least because I will want to participate). Better check out the twitter stream.
Nigel Shadbolt, Open Data Institute
Begin by reflecting on a meeting ten years ago, convened at Southampton by Stevan Harnad, on … the future of scholarly scientific communication.
Still optimistic about the Semantic Web, as I guess we more or less have to be. [At least, about many separate small-sw semantic webs — Ed.] We’re starting to see regular search-engines like Google taking advantage of available machine-readable data to return better results.
Archiving data is important, of course; but it’s also going to be increasingly important to archive algorithms. github is a useful prototype of this.
David Lambert, president/CEO, internet2
Given how the digital revolution has transformed so many fields (shopping, auctions, newspapers, movies) why has scholarly communication been so slow to follow? [Because the incumbents with a vested interesting in keeping things as they are have disproportionate influence due to their monopoly ownership of content and brands — Ed.]
Current publication models are not good at handling data. So we have to build a new model to handle data. In which case, why not build a new model to handle everything?
New “born-digital” researchers are influenced by the models of social networks: that is going to push them towards SN-like approaches of communicating more stuff, more often, in smaller unit. This is going to affect how scholarly communication is done.
Along with this goes an increasing level of comfort with collaboration. [I’m not sure I see that — Ed.]
Bonus section: tweets from Stephen Curry
He posted these during the previous talk. Very important:
Ritu Dhand, Nature
[A disappointing and unconvincing apologia for the continuing existence and importance of traditional publishers, and especially Nature. You would think that they, and they alone, guard the gates of academia from the barbarians. *sigh*. — Ed.]
Georgina Mace, UCL
[A defence of classical peer-review. Largely an overview of how peer-review is supposed to work.]
“It’s not perfect, it has its challenges, but it’s not broken yet.”
Richard Smith, ex-editor of BMJ
[An attack on classical peer-review.]
“Peer review is faith-, not evidence-based; ineffective; a lottery; slow; expensive; wasteful; ineffective; easily abused; biased; doesn’t detect fraud; irrelevant.
Apart from that, it’s perfect.”
He doesn’t want to reform peer-review, he wants to get rid of it. Publish, let the world decide. That’s the real peer-review.
He cites studies supporting his assertions. Cochrane review concluded there is no evidence that peer-review is effective. The Ioannidis paper shows that most published findings are false.
Someone should be recording this talk. It’s solid gold.
Annual cost of peer-review is $1.9 billion.
[There is much, much more. I can’t get it down quickly enough.]
Georgina Mace’s rebuttal
… amounts to contradicting Richard Smith’s evidence-supported statements, but she provides no evidence in support of her position.
Richard Smith’s counter-counter rebuttal
… cites a bunch more studies. This is solid. Solid.
For those who missed out, see Smith’s equally brutal paper Classical peer review: an empty gun. I find his conclusion (that we should just dump peer-review) emotionally hard to accept, but extremely compelling based on actual, you know, evidence.
Fascinating to hear the level of denial in the room. People really, really want to keep believing in peer-review, in spite of evidence. I understand that impulse, but I think it’s unbecoming in scientists.
The challenge for peer-review advocates is: produce evidence that it has value. No-one has responded to that.
Richard Sever, Cold Spring Harbour Press
Richard presents the BiorXive preprint server. Turns out it’s pronounced “bio-archive”, not “bye-orx-ive”.
Nothing in this talk will be new to regular SV-POW! readers, but he makes good, compelling points in favour of preprinting (which we of course agree with!)
Elizabeth Marincola, CEO, PLOS
PLOS is taking steps towards improving peer-review:
- Use of article-level metrics
- Moves towards open review
- Move toward papers evolving over time, not being frozen at the point of publication
- Better recognition of different kinds of contribution to papers
- Intention to make submitted paper available to view before peer-review has been carried out, subject only to checks on ethical and technical standard: they aim to make papers available in “a matter of days”.
She notes that much of this is not original: elements of these approaches are in F1000 Research, BiorXiv, etc.
Jan Velterop, science publisher with everyone at some point.
“I’m basically with Richard Smith when it comes to abolishing peer review, but I have a feeling it won’t happen in the next few weeks.”
The situation of publishers:
“Academia throws money at you. What do you do? You pick it up.”
Velterop gets a BIG laugh for this:
“Does peer-review benefit science? I think it does; and it also benefits many other journals.”
He quotes a Scholarly Kitchen blog-post as saying that the cost of technical preparation at PubMed Central — translating from an MS-Word manuscript to valid JATS XML — at $47. So why do we pay $3000 APCs? Surely the peer-review phase doesn’t cost $2953?
Update: here is that Scholarly Kitchen article.
Velterop’s plan is to streamline the review-and-publish process as follows:
- Author writes manuscript.
- She solicits reviews from two experts, using her own knowledge of the field to determine who is suitably skilled.
- They eventually sign off (perhaps after multiple rounds of revisions)
- The author submits the manuscript, along with the endorsements.
- The editor checks with the endorsers that they really have given endorsement.
- The article is posted.
And at that point in the proceedings, my battery was running dangerously low. I typed a tweet: “low battery may finally force me to shut up! #RSSC”, but literally between typing at and hitting the Tweet button, my laptop shut down. So that’s it for day 1. I’ll do a separate post for the second and final day.
There’s been some concern over Scientific Reports‘ new scheme whereby authors submitting manuscripts can pay $750 to have them peer-reviewed more quickly. Some members of the editorial board have quit over this development, feeling that it’s unfair to authors who can’t pay. Myself, I feel it at least shows admirable audacity — NPG has found a way to monetise its own lethargy, which is surely what capitalism is all about.
The real problem with this scheme is that $750 is an awful lot to gamble, as a sort of “pre-APC”, at a point when you don’t know whether your article is actually going to be published or not. If the peer-review returns an unfavourable verdict it’s just money down the drain.
So I welcome today’s announcement that, for only a slightly higher payment of a round $1000, it’s now possible to bypass peer-review completely, and move directly to publication. This seems like a much fairer deal for authors, and of course it streamlines the publication process yet further. Now authors can obtain the prestigious Nature Publishing Group imprint in a matter of a couple of days.
Onward and upward!
December 23, 2014
Arriving as an early Christmas present, and coming in just a week before the end of what would otherwise have been a barren 2014, my paper Quantifying the effect of intervertebral cartilage on neutral posture in the necks of sauropod dinosaurs is out! You can read it on PeerJ (or download the PDF).
Yes, that posture is ludicrous — but the best data we currently have says that something like this would have been neutral for Diplodocus once cartilage is taken into account. (Remember of course that animals do not hold their necks in neutral posture.)
The great news here is that PeerJ moved quickly. In fact here’s how the time breaks down since I submitted the manuscript (and made it available as a preprint) on 4 November:
28 days from submission to first decision
3 days to revise and resubmit
3 days to accept
15 days to publication
TOTAL 49 days
Which of course is how it ought to be! Great work here from handling editor Chris Noto and all three reviewers: Matt Bonnan, Heinrich Mallison and Eric Snively. They all elected not to be anonymous, and all gave really useful feedback — as you can see for yourself in the published peer-review history. When editors and reviewers do a job this good, they deserve credit, and it’s great that PeerJ’s (optional) open review lets the world see what they contributed. Note that you can cite, or link to, individual reviews. The reviews themselves are now first-class objects, as they should be.
At the time of writing, my paper is top of the PeerJ home-page — presumably just because it’s the most recent published paper, but it’s a nice feeling anyway!
A little further down the front-page there’s some great stuff about limb function in ratites — a whole slew of papers.
Well, I’m off to relax over Christmas. Have a good one, y’all!
November 27, 2014
Despite the flagrant trolling of its title, Nature‘s recent opinion-piece Open access is tiring out peer reviewers is mostly pretty good. But the implication that the rise of open-access journals has increased the aggregate burden of peer-review is flatly wrong, so I felt obliged to leave a comment explaining why. Here is that comment, promoted to a post of its own (with minor edits for clarity):
Much of what is said here is correct and important. Although it would be nice if Nature could make a bit more of an effort to avoid the obvious conflict-of-interest issues that lead it to title the piece so misleadingly as an attack on open access. I am glad that so many of the other commenters on this piece saw straight through that rather snide piece of propaganda.
Only one important error of interpretation here, I think. I quote:
The rise of the open-access (OA) movement compounds this effect [i.e. the increasing number of articles needing peer-review.] The business case for online OA journals, to which authors pay submission fees, works best at high volume. And for many of these journals, submitted work is published as long as it is methodologically sound. It does not have to demonstrate, for example, the novelty or societal relevance that some traditional journals demand.
The implication is that journals of this kind (PLOS ONE, PeerJ, the various Frontiers journals) increase the total peer-review burden. In fact, the exact opposite is the case. They greatly reduce the the total amount of peer reviewing.
It’s an open secret that nearly every paper eventually gets published somewhere. Under the old regime, the usual approach is to “work down the ladder”, submitting the same paper repeatedly to progressively less prestigious journals until it reached one that was prepared to publish work of the supplied level of sexiness. As a result, many papers go through four, five or more rounds of peer-review before finally finding a home. Instead, such papers when submitted to a review-for-soundness-only venue such as PLOS ONE require only a single round of review. (Assuming of course that they are indeed methodologically sound!)
The rise of review-for-soundness-only journals (“megajournals”) is an unequivocal improvement in the scientific publishing landscape, and should be welcomed by all parties: authors, who no longer have to submit to the monumental waste of time and effort that is the work-down-the-ladder system; readers, who get access to new research much more quickly; and editors and reviewers who no longer have to burn hours re-reviewing and re-re-reviewing perfectly good papers that have already been repeatedly rejected for a perceived lack of glamour.
November 22, 2014
Matt’s post yesterday was one of several posts on this blog that have alluded to Clay Shirky’s now-classic article How We Will Read [archived copy]. Here is the key passage that we keep coming back to:
Publishing is not evolving. Publishing is going away. Because the word “publishing” means a cadre of professionals who are taking on the incredible difficulty and complexity and expense of making something public. That’s not a job anymore. That’s a button. There’s a button that says “publish,” and when you press it, it’s done.
In ye olden times of 1997, it was difficult and expensive to make things public, and it was easy and cheap to keep things private. Privacy was the default setting. We had a class of people called publishers because it took special professional skill to make words and images visible to the public. Now it doesn’t take professional skills. It doesn’t take any skills. It takes a WordPress install.
… and of course as SV-POW! itself demonstrates, it doesn’t even need a WordPress install — you can just use the free online service.
This passage has made a lot of people very excited; and a lot other people very unhappy and even angry. There are several reasons for the widely differing responses, but I think one of the important ones is a pun on the word “publish”.
When Shirky uses the word, he is talking about making something public, available to the world. Which after all is its actual meaning.
But when academics use the word “publish” they usually mean something quite different — they mean the whole process that a research paper goes through between submission and a PDF appearing in a stable location (and in some cases, copies being printed). That process involves many other aspects besides actual publishing — something that in fact Shirky goes straight on to acknowledge:
The question isn’t what happens to publishing — the entire category has been evacuated. The question is, what are the parent professions needed around writing? Publishing isn’t one of them. Editing, we need, desperately. Fact-checking, we need. For some kinds of long-form texts, we need designers.
And this is dead on target. Many writers need editors[*], to varying degrees. Fact-checking could be equated with peer-review, which we pretty much all agree is still very important. Most academic publishers do a certain amount of design (although I suspect that in the great majority of cases this is 99% automatic, and probably involves human judgement only in respect of where to position the illustrations).
But due to the historical accident that it used to be difficult and costly to make and distribute copies, all those other tasks — relatively inexpensive ones, back in the days when distribution was the expensive thing — have become bundled with the actual publishing. With hilarious consequences, as they say. You know, “hilarious” in the sense of “tragic, and breathtakingly frustrating”.
That’s why we’re stuck in an idiot world where, when we need someone to peer-review our manuscript, we usually trade away our copyright in exchange (and not even to the people who provide the expert review). If you stop and think about that for a moment, it makes absolutely no sense. When I recently wrote a book about Doctor Who, I had several people proofread it, but I didn’t hand over copyright to any of them. My ability to distribute copies was not hobbled by having had independent eyes look it over. There is no reason why it should have, and there is no reason why our ability to distribute copies of our academic works should be limited, either.
What we need is the ability to pay a reasonable fee for the services we need — peer-review, layout design, reference linking — and have the work published freely.
Well, wouldja lookit that. Looks like I just invented Gold Open Access.
Is publishing just a button? Yes. Making things public is now trivial to do, and in fact much of what so-called publishers now do is labouring to prevent things from being public. But we do need other things apart from actual publishing — things that publishers have historically provided, for reasons that used to make sense but no longer do.
Exactly what those things are, and how extensive and important they are, is a discussion for another day, but they do exist.
[*] Note: the whole issue of academic publishing is further confused by another pun, this one on the word “editor”. When Shirky refers to editors, he means people who sharpen up an author’s prose — cutting passages, changing word choices, etc. Academic editors very rarely do that, and would be resented if they did. In our world, an “editor” is usually the nominally independent third party who solicits and evaluates peer-reviews, and makes the accept/reject decision. Do we need editors, in this academic sense? We’ll discuss that properly another time, but I’ll say now that I am inclined to think we do.
November 21, 2014
…is not actually about scholarly publication. It’s Steve Albini’s keynote address at Melbourne’s Face the Music conference. It’s about the music industry, and how the internet transformed it from a restrictive, top-down oligarchy that mostly benefited middlemen into a more open, level, vibrant ecosystem where artists can get worldwide exposure for free, and yet are often compensated better than they were under the old system. Go read it, and then think about this:
Once the music world met the internet, the problem of getting information from musicians (authors) to listeners (readers) didn’t require any central planning to solve. What little building needed to happen was taken care of by people who were just happy to let the internet work the way it was designed to, and the way it works the most naturally: it makes sharing information almost effortless. Publishers (record labels) still exist, because they offer certain conveniences, but few people are under the delusion that they are necessary.
Over here in academia, we’ve already spent more than a decade wringing our hands over how to manage the shift from a barrier-based publishing world to one based on OA. We’ve put so much time and effort and thought into the problem of how to “save” or “transform” scholarly publishing. Why do we do that? Why not just walk away? Publishing is a button, and anything that we do to lend it any more importance–anything we feed it, in terms of time, effort, energy, or regard–is wasted. Wasted because we deliberately ignore the new reality in favor of propping up a system that performed a job that no-one needs done anymore. I keep wondering when the hell we’re all going to wake up, and start sharing our work the way that musicians and listeners share digital music.
And yet even out here on the crazy-eyed, axe-wielding fringe of the OA movement, we are still conservative. Zen Faulkes published a paper on his blog, and he did it 26 months ago, which is a near eternity in the Shiny Digital Future (it’s 13.4% of the lifespan to date of Google). Mike and I have admired that move, and talk about it, but we haven’t done it. Why not? We could even solicit peer reviews from people we know to be tough but fair reviewers. We all do unpaid editorial and review work for publishers, why not for each other directly? It’s like we’re thinking, “Okay, okay, I’ll review this paper, but only if there’s a publisher somewhere that will benefit from my unpaid labor!”
I suppose that for us, one answer is that PeerJ has given us other options that are just as easy as blogging, like posting preprints. So I am a bit torn: I like PeerJ, I support it, I have several papers in the pipeline that I’m planning on sending there. It offers certain conveniences, like sticking DOIs on everything for us, and tracking all of our metrics. But do we need PeerJ? I wonder if it is just the methadone that will help ease us out of our sad addiction to publishers.
Bonus observation: don’t just translate Albini’s thoughts on music to scholarly publishing, also try doing the reverse. It becomes pretty clear that the central theme of The Scholarly Kitchen is, “How will poor, helpless music listeners survive without all the middlemen to tell them what to listen to? They’ll be so lost.” Keep polishing that brass, guys, and thanks for the patronization!
The photos are of the dodo skeleton in the Yorkshire Museum, which I saw at SVPCA back in September. If you’re a dodo-phile like me, you should consider supporting Leon Claessens’s, Kenneth Rijsdijk’s, and Hanneke Meijer’s quest to better understand the skull and feeding mechanics of dodos. Their crowdfunding campaign runs through the end of the year–please go check it out.
November 10, 2014
When Susie Maidment presented her in-progress research at SVP in Berlin last week, someone came in late, missed her “no tweeting, please” request, and posted a screenshot of the new work (since deleted).
On the back of that, Susie started an interesting thread in which it became apparent that people have very different assumptions. She, and Marc Jones, and others, were assuming that if you don’t tell people not to tweet, then they’ll know not to. Meanwhile, I, and Björn Brembs, and others were assuming the opposite: unless someone says not to tweet, you’re good to go.
Obviously this state of affairs is a recipe for disaster. We’re all going to find ourselves giving presentations where we assume the audience will be doing one thing, but at least some audience members are assuming the other.
So the first thing to say is that we should be explicit about our expectations. My talk at SVPCA this year contained this slide:
I’m going to get into the habit of including something like this every time. Similarly, people who don’t want material from their talks appearing on Twitter should say so.
The second thing is that conferences should state their default policies (always of course allowing individual authors to override them). Someone at, say, SVP, should know from the registration material either that it’s OK to tweet unless told not to, or that it’s not OK to tweet unless told that it is. I think it’s reasonable that different conferences would lean in different directions on this.
The third thing is in the absence of other guidance, it’s better not to tweet. I feel a bit uncomfortable about this because it goes against my pro-open tendencies, but it’s a matter of failing safe. If I want you to tweet my talk but but I forget to say so and there is no conference-wide policy (or the conference policy is No Tweeting), then you won’t tweet it, and that is a missed opportunity –but I’ll live. But if Susie doesn’t want you to tweet but forgets to say so, and you do, then she will be unhappier. (For example, in the present case, Susie is hoping for a media splash, which could be diluted if knowledge of the new finding is already leaking out.
- Individual presenters should say what they want.
- The conference should provide a default policy
- If the absence of both, fail safe by not tweeting.
That’s what I think, anyway. What do you think?