Live-blog: the Future of Scholarly Scientific Communication
April 20, 2015
I’m at the Royal Society today and tomorrow as part of the Future of Scholarly Scientific Communication conference. Here’s the programme.
I’m making some notes for my own benefit, and I thought I might as well do them in the form of a blog-post, which I will continuously update, in case anyone else is interested.
I stupidly didn’t make notes on the first two speakers, but let’s pick up from the third:
Deborah Shorley, ex-librarian of Imperial College London
Started out by saying that she feels her opinion, as a librarian, is irrelevant, because librarians are becoming irrelevant. A pretty incendiary opening!
Important observations:
“Scientific communication in itself doesn’t matter; what matters is that good science be communicated well.”
And regarding the model of giving papers to publishers gratis, then paying them for the privilege of reading them:
“I can’t think of any other area where such a dopey business model pertains.”
(On which, see Scott Aaronson’s brilliant take on this in his review of The Access Principle — the article that first woke me up to the importance of open access.)
Shorey wants to bring publishing skills back in-house, to the universities and their libraries, and do it all themselves. As far as I can make out, she simply sees no need for specialist publishers. (Note: I do not necessarily endorse all these views.)
“If we don’t seize the opportunity, market forces will prevail. And market forces in this case are not pretty.”
Robert Parker, ex-head of publishing, Royal Society of Chemistry
Feels that society publishers allowed themselves to be overtaken by commercial publishers. Notes that when he started working for the RSC’s publishing arm, it was “positively dickensian”, using technology that would mostly have been familiar to Gutenberg. Failure to engage with authors and with technology allowed the commercial publishers to get ahead — something that is only now being redressed.
He’s talking an awful lot about the impact factors of their various journals.
My overall impression is that his perspective is much less radical than that of Deborah Shorley, wanting learned-society publishers to be better able to compete with the commercial publishers.
Gary Evoniuk, policy director at Glaxo Smith Klein
GSK submits 300-400 scientific studies for publication each year.
Although the rise of online-only journals means there is no good reason to not publish any finding, they still find that negative results are harder to get published.
“The paper journal, and the paper article, will soon be dead. This makes me a little bit sad.”
He goes further and wonders whether we need journal articles at all? When actual results are often available long before the article, is the context and interpretation that it provides valuable enough to be worth all the effort that’s expended on it? [My answer: yes — Ed.]
—
Discussion now follows. I probably won’t attempt to blog it (not least because I will want to participate). Better check out the twitter stream.
—
Nigel Shadbolt, Open Data Institute
Begin by reflecting on a meeting ten years ago, convened at Southampton by Stevan Harnad, on … the future of scholarly scientific communication.
Still optimistic about the Semantic Web, as I guess we more or less have to be. [At least, about many separate small-sw semantic webs — Ed.] We’re starting to see regular search-engines like Google taking advantage of available machine-readable data to return better results.
Archiving data is important, of course; but it’s also going to be increasingly important to archive algorithms. github is a useful prototype of this.
David Lambert, president/CEO, internet2
Given how the digital revolution has transformed so many fields (shopping, auctions, newspapers, movies) why has scholarly communication been so slow to follow? [Because the incumbents with a vested interesting in keeping things as they are have disproportionate influence due to their monopoly ownership of content and brands — Ed.]
Current publication models are not good at handling data. So we have to build a new model to handle data. In which case, why not build a new model to handle everything?
New “born-digital” researchers are influenced by the models of social networks: that is going to push them towards SN-like approaches of communicating more stuff, more often, in smaller unit. This is going to affect how scholarly communication is done.
Along with this goes an increasing level of comfort with collaboration. [I’m not sure I see that — Ed.]
Bonus section: tweets from Stephen Curry
He posted these during the previous talk. Very important:
Ritu Dhand, Nature
[A disappointing and unconvincing apologia for the continuing existence and importance of traditional publishers, and especially Nature. You would think that they, and they alone, guard the gates of academia from the barbarians. *sigh*. — Ed.]
Lunch
Georgina Mace, UCL
[A defence of classical peer-review. Largely an overview of how peer-review is supposed to work.]
“It’s not perfect, it has its challenges, but it’s not broken yet.”
Richard Smith, ex-editor of BMJ
[An attack on classical peer-review.]
“Peer review is faith-, not evidence-based; ineffective; a lottery; slow; expensive; wasteful; ineffective; easily abused; biased; doesn’t detect fraud; irrelevant.
Apart from that, it’s perfect.”
He doesn’t want to reform peer-review, he wants to get rid of it. Publish, let the world decide. That’s the real peer-review.
He cites studies supporting his assertions. Cochrane review concluded there is no evidence that peer-review is effective. The Ioannidis paper shows that most published findings are false.
Someone should be recording this talk. It’s solid gold.
Annual cost of peer-review is $1.9 billion.
[There is much, much more. I can’t get it down quickly enough.]
Georgina Mace’s rebuttal
… amounts to contradicting Richard Smith’s evidence-supported statements, but she provides no evidence in support of her position.
Richard Smith’s counter-counter rebuttal
… cites a bunch more studies. This is solid. Solid.
For those who missed out, see Smith’s equally brutal paper Classical peer review: an empty gun. I find his conclusion (that we should just dump peer-review) emotionally hard to accept, but extremely compelling based on actual, you know, evidence.
Fascinating to hear the level of denial in the room. People really, really want to keep believing in peer-review, in spite of evidence. I understand that impulse, but I think it’s unbecoming in scientists.
The challenge for peer-review advocates is: produce evidence that it has value. No-one has responded to that.
Richard Sever, Cold Spring Harbour Press
Richard presents the BiorXive preprint server. Turns out it’s pronounced “bio-archive”, not “bye-orx-ive”.
Nothing in this talk will be new to regular SV-POW! readers, but he makes good, compelling points in favour of preprinting (which we of course agree with!)
Elizabeth Marincola, CEO, PLOS
PLOS is taking steps towards improving peer-review:
- Use of article-level metrics
- Moves towards open review
- Move toward papers evolving over time, not being frozen at the point of publication
- Better recognition of different kinds of contribution to papers
- Intention to make submitted paper available to view before peer-review has been carried out, subject only to checks on ethical and technical standard: they aim to make papers available in “a matter of days”.
She notes that much of this is not original: elements of these approaches are in F1000 Research, BiorXiv, etc.
Jan Velterop, science publisher with everyone at some point.
“I’m basically with Richard Smith when it comes to abolishing peer review, but I have a feeling it won’t happen in the next few weeks.”
The situation of publishers:
“Academia throws money at you. What do you do? You pick it up.”
Velterop gets a BIG laugh for this:
“Does peer-review benefit science? I think it does; and it also benefits many other journals.”
He quotes a Scholarly Kitchen blog-post[citation needed] as saying that the cost of technical preparation at PubMed Central — translating from an MS-Word manuscript to valid JATS XML — at $47. So why do we pay $3000 APCs? Surely the peer-review phase doesn’t cost $2953?
Update: here is that Scholarly Kitchen article.
Velterop’s plan is to streamline the review-and-publish process as follows:
- Author writes manuscript.
- She solicits reviews from two experts, using her own knowledge of the field to determine who is suitably skilled.
- They eventually sign off (perhaps after multiple rounds of revisions)
- The author submits the manuscript, along with the endorsements.
- The editor checks with the endorsers that they really have given endorsement.
- The article is posted.
Bam, done!
…
And at that point in the proceedings, my battery was running dangerously low. I typed a tweet: “low battery may finally force me to shut up! #RSSC”, but literally between typing at and hitting the Tweet button, my laptop shut down. So that’s it for day 1. I’ll do a separate post for the second and final day.
Sauropod Vertebra Picture of the Week 
April 20, 2015 at 2:16 pm
So, if peer review does no good and should be abolished, shouldn’t there be some good non-peer-reviewed journals, full of good papers?
Likewise, some of the advocates of that view should be publishing their papers in those journals, right? Has Smith been doing so?
April 20, 2015 at 2:24 pm
Good questions. I think that lack of good non-reviewed journals is because we all have this lingering reverence for peer-review, so it’s hard to be the first mover away from it.
April 20, 2015 at 4:09 pm
Love Jan Velterop’s plan. I’ve known about it for a couple of years but he didn’t go public until last year. It was initially going to be called One Science then JoNaS (has a Greek God meaning, but was also having a dig at the Glam Magz as it also stood for Journal of Nature and Science).
April 20, 2015 at 5:04 pm
I am for peer-review, but I will go over the benefits and drawbacks of either method:
The value of peer-review is that other researchers can set new eyes to the paper and give their take. There are a few issues with peer-review, namely:
1. The researchers reviewing the paper, essay, etc. may be biased. This rarely happens because in science you aren’t supposed to be biased.
2. There are credit issues (as pointed out by people who you have included the tweets of in this article).
3. Some scientists may nitpick the paper, essay, etc. ‘s argument to the point where the paper essentially becomes a summary or a review.
Now there are the issues with non-peer-reviewed, namely:
1. There is a lack of data checked by multiple 3rd party sources.
2. Less discussion among reviewers and writers can lead to less discoveries or ideas.
3. Sometimes reviewers catch issues which cannot be caught by basic editors which can seriously screw the paper up (see “Rayosaurus tessonei” in Mr. Don Henderson’s and Mr. Bob Nicholl’s paper on theropod biomechanics).
April 20, 2015 at 5:05 pm
EDIT: Biasing doesn’t often happen. Rarely is an understatement.
April 20, 2015 at 6:06 pm
Velterop’s plan is terrifyingly efficient. I find myself drawn to it.
April 20, 2015 at 6:23 pm
It seems to me that even if peer-review itself is ineffective, abolishing it might encourage a lot more crummy papers to be published, as the fear of peer-review forces people to self-correct. So, maybe the solution is to find a more effective way to do peer review, rather than stop doing it altogether.
Also, letting authors choose their own reviewers seems ripe for abuse.
April 20, 2015 at 7:36 pm
Reblogged this on O mundo de Maria and commented:
Para discutir dilemas da ciência
April 20, 2015 at 11:33 pm
“People really, really want to keep believing in peer-review”
…because it’s the only thing that journals can honestly say they can add to the process of publishing that academics can’t do themselves using modern technology.
Oh wait, what about all those diamond OA journals run by academics….
April 21, 2015 at 7:23 am
As a diligent reviewer, I believe fresh eyes DO add a perspective to a paper (see one of the papers I reviewed: http://link.springer.com/article/10.1007%2Fs11738-015-1777-z#page-1; “Acknowledgments (…)We are grateful to two anonymous reviewers for their highly constructive comments.”).
On the flipside, I was subject to abuse by a reviewer, nitpicking my paper in PLoS One, possibly just slowing it down, to publish his own, lesser quality findings, proudly yet unjustly labeling it “first report of”.
April 21, 2015 at 3:43 pm
I’m kind of disappointed by Deborah Shorley’s comment that librarians are becoming irrelevant! Hopefully I’m just missing the point of it as an opening? And perhaps she means *academic* librarians are becoming irrelevant?
Full disclosure: I’m married to a librarian who works in a public library. From where I’m sitting, librarians are more important than ever as access to information becomes more important for getting by in your day-to-day life. And what I see is that public librarians are there not for the privileged people who already know how to get information, and can, but for the people without various kinds of privilege who cannot access information as easily – immigrants, non-English speakers, prisoners, economically disadvantaged groups, etc.
Anyway, I know that’s not the main gist of this blog post, but I hate seeing libraries and librarians dismissed as irrelevant, and I think it behooves us as scientists to remember that not everyone has the same information literacy that we might. Librarians are free information literacy teachers.
April 21, 2015 at 5:41 pm
Hi, Victoria. Yes, she was talking about academic librarians.
April 24, 2015 at 12:07 pm
[…] For a more detailed transcription of the day’s events by someone who was actually in attendance, go here. […]
April 24, 2015 at 1:11 pm
[…] peer-review. This is the term that was used at the Royal Society meeting at the start of this week — the idea being that you review objectively for the quality of the […]
April 25, 2015 at 2:07 pm
[…] in chief (and current Center for Scientific Integrity board of directors member) Richard Smith said earlier this week. “Apart from that, it’s […]
April 29, 2015 at 8:15 am
[…] discussion on the first day – vividly live-blogged by Mike Taylor – was an attempt to define the challenges facing scholarly publishing in the 21st […]
May 5, 2015 at 9:11 am
[…] 2 of the Royal Society’s Future of Scholarly Scientific Communication meeting, as I did for the first day of part 1. We’ll see how it […]
May 5, 2015 at 3:49 pm
I have experienced situations both as an author and a reviewer where the review has been highly beneficial, there are also plenty of cases where the reviews have been of little use to the final product. The problem isn’t peer review as such, but how easy it is to get away with sloppy reviews or sloppy responses to reviewers. One of the most problematic aspects of peer review in terms of inefficiency, however, is the fact that so many papers go through many rounds of peer review not because of technical problems but because of hype-based selective journals. Jan Velterop’s proposal may be an elegant solution for uncoupling review from publishing, or at least loosening the connection.
May 5, 2015 at 5:16 pm
I would summarise my own experience, too, as: peer-review has improved my papers, but not by enough to justify how much extra work handling reviews has cost me, let alone what it’s cost the reviewers. My time would have been better spent doing new work.
May 6, 2015 at 9:54 pm
I think it is important to see what a world might look like past the current peer review model. Imagine drafts of papers held on servers that editorial boards would scan, comment on, and then republish as appropriate. I like to think of this new style of journal as a ‘overlay’ journal. It could have an edition of papers that all analyse one big dataset, or theme themselves drawing on a list of papers over time, not just the most recent. Or it could be a journal that only publishes the most outrageously new research. Just becasue a paper is published in one overlay journal wouldn’t mean it couldn’t be published in another, given the right licensing framework.
Reviewers would scramble to find the best content, rather than grumble about being asked. (Well, I can hope).
Oh, and the servers with the drafts on them: that would be repositories (run by librarians in the main…)
Its a big step into the future, and requires changes in licensing, data publishing, storage, and discoverability. Oh, Iibrarians can help with all that. We do it all already.
May 6, 2015 at 10:17 pm
Anton,
Yes, overlay journals have been around as a concept for some time now. I know there are a few of them out there, but they’ve yet to catch on properly. That doesn’t mean they won’t, of course!
As it turned out, overlay journals didn’t get discussed at all at the FSSC meeting. That’s because the tendency was towards more radical approaches: several groups independently came up with a design for a system where preprints are posted after only a cursory editorial check, then reviewed after the event, with the value of each review weighted by the reputation of the reviewer (which is in turn weighted by others’ reviews of his or her work and perhaps other activities). There are actually several publishers already offering systems along these lines, including F1000 Research and Science Open.
May 26, 2015 at 4:52 pm
[…] of the striking things about the Royal Society’s recent Future of Scholarly Scientific Communication meetings was that during the day-two breakout session, so many of the groups independently came up with more […]
June 3, 2015 at 7:06 pm
[…] מאמרי דעה על ביקורת עמיתים: 1, 2, 3, 4, […]
June 19, 2015 at 10:14 am
[…] event (click through for audio recordings and a detailed report, see also, Mike Taylor’s live blog of the event). This was a workshop spread over four days to mark the 350th anniversary of the Philosophical […]