An extraordinary study has come to light today, showing just how shoddy peer-review standards are at some journals.

Evidently fascinated by Science‘s eagerness to publish the fatally flawed Arsenic Life paper, John Bohannon conceived the idea of constructing a study so incredibly flawed that it didn’t even include a control. His plan was to see whether he could get it past the notoriously lax Science peer-review provided it appealed strongly enough to that journal’s desire for “impact” (designed as the ability to generate headlines) and pandered to its preconceptions (that its own publication model is the best one).

So Bohannon carried out the most flawed study he could imagine: submitting fake papers to open-access journals selected in part from Jeffrey Beall’s list of predatory publishers without sending any of his fake papers to subscription journals, noting that many of the journals accepted the papers, and drawing the flagrantly unsupported conclusion that open-access publishing is flawed.

Incredibly, Science not only published this study, but made it the lead story of today’s issue.

It’s hard to know where Science can go from here. Having fallen for Bohannon’s sting, its credibility is shot to pieces. We can only assume that the AAAS will now be added to Beall’s list of predatory publishers.

Rolling updates

Here are some other responses to the Science story:

• Michael Eisen: I confess, I wrote the Arsenic DNA paper to expose flaws in peer-review at subscription based journals
• Martin Eve: Flawed sting operation singles out open access journals (and his longer original version)
• Peter Suber: New “sting” of weak open-access journals
• The Library Loon: Which is it?
• Björn Brembs: Science Magazine Rejects Data, Publishes Anecdote
• Kausik Datta at SciLogs: What Science’s “Sting Operation” Reveals: Open Access Fiasco or Peer Review Hellhole?
• John Hawks: “Open access spam” and how journals sell scientific reputation
• Retraction Watch:

  Science reporter spoofs hundreds of open access journals with fake papers

• OASPA: response to the recent article in Science entitled “Who’s Afraid of Peer Review?”
• Jeroen Bosman: Science Mag sting of OA journals: is it about Open Access or about peer review?
• Curt Rice: What Science — and the Gonzo Scientist — got wrong: open access will make research better (now also appearing at the Guardian)
• Michelle N. Meyer: The troubled peer-review system, the open-access wars, and the blurry line between human subjects research and investigative journalism
• Ernesto Priego: Who’s Afraid of Open Access?
• Marius Buliga: On John Bohannon article in Science
• DOAJ: response to the recent article in Science entitled “Who’s Afraid of Peer Review?
• Guardian:
• Zen Faulkes: Open access or vanity press, the Science “sting” edition
• Graham Steel: Glam Mag fucks up, news at eleven
• Heather Joseph (SPARC): Science Magazine’s Open Access “Sting”
• Lenny Teytelman: What hurts science – rejection of good or acceptance of bad?
• Fabiana Kubke: Science gone bad; or, or the day after the sting
• Gunther Eysenbach: Unscientific spoof paper accepted by 157 “black sheep” open access journals – but the Bohannon study has severe flaws itself
• Jon Brock: This study lacked an appropriate control group: Two stars
• Me again, this time with the gloves off: Anti-tutorial: how to design and execute a really bad study
• Paul Basken (Chronicle of Higher Education): Critics Say Sting on Open-Access Journals Misses Larger Point
• Neurobonkers: Science’s Straw Man Sting
• The Winnower: The Real Peer Review: Post-Publication
• Sal Robinson: John Bohannon’s Open Access sting paper annoys many, scares the easily scared, accomplishes relatively little
• Peerage of Science: It’s gotta sting
• Peter Murray-Rust: The Bohannon “Sting”; Can we trust AAAS/Science or is this PRISM reemerging from the grave?
• Heather Morrison: Bohannon and Science: bogus articles and PR spin instead of peer review
• Barbara Fister (Inside Higher Ed): The Sting
• Jon Tennant (guesting at SciLogs): Peer Review Quality is Independent of Open Access
• Stuart Shieber: Lessons from the faux journal investigation
• DOAJ: Second response to the Bohannon article 2013-10-18
• Andreas Thoss: Peer review: how to distinguish the good from the bad?