Blogs, papers, and the brave new digital world: Matt’s thoughts

June 11, 2009

First off, thanks to everyone for reading, commenting on, and discussing the previous post. Seeing the diversity of opinions expressed has been interesting and gratifying for us, and we’ve learned a lot from you about how the blogosphere is changing science already. My own thoughts follow, Mike chimes in at the end, and Darren will probably have something to add soon, too.

The Intolerable Problem

Sometimes people push back on posts of mine they don’t like by telling me I’m out of bounds. Somehow, they say, I’ve crossed the boundary of what I’m allowed to write about. They are angry that I’m now writing about something outside my defined area.

I’m usually taken aback by this, because I didn’t realize I’d actually agreed to any boundaries.

Seth Godin, 2009, “Out of Bounds”

Several commenters have brought up what I call the Intolerable Problem, which is that people online can critique papers and present new evidence and arguments in a format that is impermanent and not peer-reviewed. It’s intolerable because on one hand such material is not currently (operative word) citable in most outlets, and on the other hand repeating it sans citation in peer-reviewed literature smacks of plagiarism (to some, but not to all). Although this material is potentially valuable it “doesn’t count” professionally (see exceptions below), which some professionals (not necessarily those who have commented here) regard as a fatal argument against posting it in the first place. But–and this is crucial–it’s only a problem for the tiny fraction of the audience who might want to cite the freely exchanged material. If you’re in that fraction, we value your attention and comments, but don’t assume we’re writing only for you, or to further our professional standing. We blog because we love this stuff, and even at a technical niche blog like SV-POW! the majority of readers probably don’t care at all whether the information is peer-reviewed or “counts” for professionals; they mostly care whether it’s right or not.

One obvious solution to the Intolerable Problem is to simply let people cite anything they want, including blog posts and DML posts. This is already starting to be implemented–see examples here and here and more discussion here. This runs into two problems: one is permanence (there is no guarantee that the cited post will be up forever, or that the author won’t revise it later in response to criticism [as I have done with this very post!]), which can already be solved using tools such as WebCite (thanks to Cameron Neylon for bringing this to our attention in a comment on the previous post).

The other problem is that citations serve two functions, which are to establish priority and to lend authority to an argument. Citing a blog post may establish priority, but some researchers will cavil at the idea that a blog post is an authoritative source (for varying combinations of researchers and blog posts). Whether they would be right to cavil I don’t know; in the end the market will decide. The market–that is, the desire to attain professional respect and avoid censure–will also dissuade authors from larding up their papers with citations to trivial or worthless online sources.

Those who are troubled by the free discussion of papers, evidence, and hypotheses online need to realize that:

  • it’s been going on for a long time (15 years for the Dinosaur Mailing List);
  • it’s only going to accelerate in the future;
  • it’s not a problem for the vast majority of people participating in the discussions;
  • any solution must involve accommodation to the reality of how people exchange information online (immediately, freely, globally, without prior filtering).

These discussions are not going to stop, and ignoring the output of such discussions (because they “don’t count”) will eventually become prohibitively expensive as those workers who insist on playing only by the old rules are outmaneuvered by others who find ways to use all available information regardless of its provenience or “respectability”.

Paper journals will die when online journals stop sucking

Most online publications are hampered by having to be identical to the dead-tree versions (no links, no embedded video, no rotating 3D PDF images, etc.). Eventually people will realize that it is counterproductive to keep hobbling the new medium to make it as slow, flat, and inefficient as the old medium. Once one journal takes the hobbles off, others will do the same rather than lose contributors to cutting-edge outlets. A few boutique journals may still produce flattened, gutted versions of the online publications on paper. People still fly biplanes, too. Paper-based journals will never be popular again and their existence will not stop people from doing whatever technology allows them to in the online venues.

Note that this does not even refer to the economic argument against dead-tree publishing, which has already relocated encyclopedias and newspapers from ubiquity to marginality or extinction.

I’m surprised that the revolution isn’t farther along already. The cage is open.

Whither peer review and editing?

This is all part of the Big Flip in publishing generally, where the old notion of “filter, then publish” is giving way to “publish, then filter.” There is no need for Slashdot’s or Kuro5hin’s owners to sort the good posts from the bad in advance, no need for Blogdex or Daypop to pressure people not to post drivel, because lightweight filters applied after the fact work better at large scale than paying editors to enforce minimum quality in advance.

Clay Shirky, 2003, “The Music Business and the Big Flip”

PLoS ONE is already going gangbusters, without peer-review prior to publication in many cases. The only holdup there is that the post-hoc review by commenters is not working out quite like they’d hoped, because few people are commenting. Not everyone agrees that there is a dearth of commenting at PLoS ONE; the larger point is that people publish there a lot and the community treats those pubs like they count, even though in many cases they are essentially un-reviewed.

[Update: I misunderstood peer review at PLoS ONE. Papers may be reviewed externally by people unconnected to PLoS, or by one or more unpaid Academic Editors, or by a combination. I had thought of the review by Academic Editors only, which accounts for 13% of papers, as a form of internal review, but according to Bora (down in the comments) it should count as external review. If you’re happy with that–and the system is not without its critics–then all papers at PLoS ONE are externally reviewed prior to publication; even if you’re not, pre-publication review by someone is still in place across the board at PLoS ONE, and 87% of papers are externally reviewed by people unaffiliated with PLoS. Post-publication commenting supplements rather than replaces pre-publication review.]

People do comment on blogs, all the time. Post-hoc review will work, in fact already does work, just fine on blogs. I predict that PLoS ONE clones of the future (PLoS TWO?) will emulate whatever features of blogs make people willing to comment on them but not on PLoS ONE v1.0.

Alternatively, the paucity of post-hoc commenting at PLoS ONE could be taken as further evidence that journal-mediated peer review, whether before or after publication, is dying just off to a slow start. I think that editorial control is not far behind. Both are locally extinct in some parts of the science publishing ecosystem, since people are already citing blogs.

Q: But–but–but? What about protecting the sanctity of the process? What about about guaranteeing respectability? What about prestige?

A: Hey, those questions would make a terrific opinion piece for your local newspaper–oops, too late.

I don’t deny that editors and peer reviewers often make significant contributions to the quality of published work. I just think that people will learn to get along without them if doing so allows faster and easier exchange of information. That was never possible on paper; it’s long been possible here.

A priori peer review and editorial control were invented because publications were scarce (in the Econ 101 sense of being limited) and there needed to be a barrier to entry. Now publication is instant, free, and global. Error correction and the assignment of value will still happen, but they’ll happen after publication rather than before, and they’ll be distributed rather than centralized.

Creeping blogification

Clay Shirky described the problem for newspapers and the recording industry as the existence of “cheap perfect copies”. An expanded but by no means exhaustive list for science publication includes:

  • cheap perfect copies
  • editable (but also archivable)
  • sharable
  • linkable (both incoming and outgoing)
  • globally distributed
  • instantly
  • for free
  • without pre-publication filtering
  • with multimedia embeds (as opposed to including video etc. separately in the suppl. info.)

Online open-access journals currently take advantage of all of those capabilities except the last two. Newsgroup posts cover all the bases except the last one (so do tweets, despite the severe length limitations).

What covers everything? Blog posts. Which have the added advantage that people will comment on them without being asked.

But that’s not the whole simple story.

The center cannot hold–or can it?

So we’re looking at total chaos, right–a world where anyone posts anything they want, no one has any control, and no one knows how to find the good stuff? Well, two out of three, at least. I’m not worried about that last point, for two reasons.

First, thanks to search engines, aggregators, tags, tweets, links, etc., we already have pretty good tools for finding the good stuff. Those direction finders will get better even as the map gets more complicated.

Second, prestige will always be a motivator, so people will always compete to get into exclusive venues. Nature is not going away, although I think that in the near future they will decouple their online and print publications so that the former can take advantage of all the possibilities the web offers.

If I have a really good idea backed up with lots of data, I’ll keep trying to get it into the most prestigious outlet I can. I won’t put my best stuff on a blog just because it’s faster and less encumbered. Blogs probably won’t replace journals, at least not anytime soon. Rather, the spectrum of publishing possibilities will expand; below the category of Least Publishable Unit we’ll add Most Bloggable Unit and so on down to Least Tweetable Unit, and the new categories will interpenetrate with the old over time.

How nice for me

Well, what a striking coincidence that Mr. Paleo Blogger looks into the ole digital crystal ball and sees “bloggy with a 90% chance of exactly-what-he’s-already-doing”.

I can’t claim to be either uninterested or unbiased in all of this. But I am new to actually thinking about the implications. I hadn’t been to most of the above links or had any of these thoughts as of a week ago. When Casey first e-mailed me six days ago, I replied:

If you’re curious, here’s the short short version of my thoughts: science bloggers critique published papers and blog about unpublished observations all the time. Our post-paper run of posts might be an extreme or even vulgar example, and it might fire more discussion about “what counts?”, but I don’t see it as being different in kind from what many science bloggers do. Papers are papers and blogs are blogs, and I never intended to blur the lines. If people feel that all the blog posts only count as “crap some guys wrote on the internet” and that they can be safely ignored, that’s fine with me. If they think the blog posts deserve some higher level of recognition a la “what counts?”, then I’m honored, but that’s extra value that others are investing in our blog, and not anything that we’ve knowingly sought. I suppose you could turn around and say that I’m trying to have my cake and eat it, too, first with all the pro-paper blogging and now with this “I’m innocent” schtick. I don’t know what the answer is, but I know that I’m too tired to figure it out tonight. All the more reason to have an open conversation about this stuff.

Now I realize that the lines between papers and blog posts are blurring, and whether we mean to or not, we SV-POW!sketeers are contributing (Darren’s doing double duty thanks to Tet Zoo). I still think that the investment of blog posts with respectability, value, citability, or whatever rests entirely with readers, and always will. Options range from treating posts like papers to treating them like bar conversations to treating them like spam. You decide.

Also, I tried to keep the writing above value-neutral but probably failed. It’s hard not to get a bit evangelistic about the potential advantages of online publication and online everything else, a tendency I call DISSUADE: Da Internet Shall Save Us All Dead-trees Excepted. Getting published in science hasn’t always been easy up until now, but the process has been relatively clear and familiar. And stable, on decadal and even centennial timescales. Everything about scientific publication is about to get much more fluid and much less clear, and it will probably stay that way for a long time, and it may stay that way forever. Not all of the changes will be for the better, and it may be hard to decide what’s better and what’s worse until we look back with some perspective. Mechanical looms were bad for weavers but good for everyone else. I think many of the changes discussed in this post and the previous comment thread are likely, and some are inevitable.

Set against the shiny digital future is the inertia of the academy and those of us who roost there. I’m not going to stop publishing papers in dead-tree journals (although I will never publish in a journal that doesn’t provide PDFs to authors). Heck, I’m not even going to stop publishing in closed-access journals, some of which are run by societies I admire and want to participate in (after all, everything is open anyway). At the same time I will keep blogging, and while I will frequently bring up technical stuff I don’t want to publish more formally (at least not yet), I will try not to deliberately blur the lines any more than I already have. I don’t need to; the web is already blurring them faster than most of us can keep up.

Hang on.

Oh, about that mystery vert…

Metapophyses, I haz them

Metapophyses, I haz them

…at the end of the post Necks Lie. Nima called it–good spot on the split neural spine. It’s a mid-cervical of Barosaurus, AMNH 6341, in the big bone room (well, one of many big bone rooms) at the American Museum of  Natural History in New York. A cast of this vertebra makes up part of the neck in the awesome mounted skeleton in the museum rotunda. Here’s that skeleton, with Mike for scale.

Mike with Baro 480

Thanks for slogging through all this. We’ll get back to perforated postcentrodiapophyseal laminae, sacralized caudal transverse processes, and the air space proportions of pneumatic vertebrae soon.

Addendum

Matt is much more ready than I am to throw away peer-review, editorial control, and journals in general.  Sometimes, the reasons that things are the way they are, are good ones; it’s not in the interests of professional iconoclasts like Clay Shirky and Cory Doctorow to point that out or to discuss the strengths of how things are today, but that doesn’t mean we have to accept their arguments as uncritically as (say, to pick a name out of the air completely at random) Matt.

Anyway, happily, G. K. Chesterton foresaw the abolition of journals in favour of blogs, and commented thus:

Suppose that a great commotion arises in the street about something, let us say a lamp-post, which many influential persons desire to pull down. A grey-clad monk, who is the spirit of the Middle Ages, is approached upon the matter, and begins to say, in the arid manner of the Schoolmen, “Let us first of all consider, my brethren, the value of Light. If Light be in itself good–” At this point he is somewhat excusably knocked down. All the people make a rush for the lamp-post, the lamp-post is down in ten minutes, and they go about congratulating each other on their unmediaeval practicality. But as things go on they do not work out so easily. Some people have pulled the lamp-post down because they wanted the electric light; some because they wanted old iron; some because they wanted darkness, because their deeds were evil. Some thought it not enough of a lamp-post, some too much; some acted because they wanted to smash municipal machinery; some because they wanted to smash something. And there is war in the night, no man knowing whom he strikes. So, gradually and inevitably, to-day, to-morrow, or the next day, there comes back the conviction that the monk was right after all, and that all depends on what is the philosophy of Light. Only what we might have discussed under the gas-lamp, we now must discuss in the dark.

Heretics (1905).


Advertisements

24 Responses to “Blogs, papers, and the brave new digital world: Matt’s thoughts”

  1. Philip Kahn Says:

    Oh, this post stirs up the “I want to make something!” impulse in me. I think that this has a great deal of merit to it. I can see an initial journal being something like this:

    Have a radio box for pre- or post- publishing commentary and review. This provides the flexibility of both methods.
    Accept HTML, LaTeX, and PDFs for initial submission. Again, this provides the flexibility of old and new methods, permanent archival, yet allowing rich media should someone choose.
    Allow “attachments” to publications, such as coding scripts or source, high-resolution photographs.
    Since there would be a very low cost overhead, require some nominal (perhaps $5-$20) fee to publish to cover web costs and initial editorial screen
    Wide range of available “copyleft” options. The point of this would be to provide a site of formalization and publication, not to retain copyright
    In addition to formal “publication”, also have a “cited discussion” format, which would be something like “a formal, technical blog post”. This would enable writers to “pre-publish” and get their ideas out in the open, and accept a wide variety of commentary, with the acknowledgement that it is not final, but they have a “claim” to this formalization.
    Free and open PDF repository of publications

    I kind of want to tinker around and see how close I can get to something like this, now … though, of course, the trick would be getting people to go along with this!
    Yeah, I’m with you Matt … I’m a bit of an ideologue for the free-and-open connected world.

  2. Matt Wedel Says:

    Oops. My bad. I had been under the impression that there was a track for papers to be published without prior review. I will revise the post accordingly.

  3. Philip Kahn Says:

    Gah, just noticed WP ate my HTML. Argh. Apparently <ul> isn’t valid …


  4. Matt says that The Intolerable Problem is “only a problem for the tiny fraction of the audience who might want to cite the freely exchanged material.” But that tiny fraction of the audience is, without question, the most important fraction. I write on SV-POW! for many reasons, including sauropod-vertebra evangelism for the as-yet unconverted — and I do love you guys — but I realistically, the opinions of other publishing scientists are the ones that matter most to me.

  5. Matt Wedel Says:

    Okay, having now looked carefully at PLoS ONE’s review policy, I see that, “Around 13% of articles are peer reviewed by the Academic Editors themselves”, which is fine, but that’s usually referred to as internal review and not conflated with external peer review. In my experience, when people say that something is “peer reviewed”, they mean externally. I was still massively overstating the decline of peer review at PLoS ONE, since 87% of papers are externally reviewed.

    BTW, I’ll reiterate my request that PLoS ONE “state the review track for each article–i.e., solo editor approved, multiple editor approved, or externally reviewed.”

  6. Matt Wedel Says:

    The Intolerable Problem is “only a problem for the tiny fraction of the audience who might want to cite the freely exchanged material.” But that tiny fraction of the audience is, without question, the most important fraction.

    Sure, that may be the case here, at least most of the time (not all posts have to be pitched to the same segment of the audience). But it’s not always the case on blogs, and it’s probably never the case on the DML. The problem of setting a boundary on “what counts” is still a problem for the professional community, and not for the vastly larger online community.

  7. Nathan Myers Says:

    I admit that I would hate to find my remarks on late-surviving giant aquatic boneless pterosaurs cited in a reputable journal, even if to contradict them. Ah, who am I kidding? I’d love it.


  8. Hi Matt, this is a great post (right after the other one), particularly now that the mention of PLoS ONE has been put straight. One point I liked that is relevant to my research but rarely comes up in these discussions is the embedding of supplementary materials (e.g. videos) that are all too often relegated to dark corners of the publisher’s server and can frequently not be found at all after some years. And there is at two other breads that “cover everything” – wikis and social aggregators of the FriendFeed type. As for permanence, there are services like Webcitation.org, to which I have just submitted this post (will put the citable metadata into a separate comment).


  9. Here is the Webcitation record (did not show up after first submit, so I replaced http by xttp):

    Thank you for your submission. Your request to archive the content of

    xttp://svpow.wordpress.com/2009/06/11/blogs-papers-and-the-brave-new-digital-world-matts-thoughts/

    has been entered into the archival queue. An archive of this page should shortly be available at

    xttp://www.webcitation.org/5hSmMI5k0

    If the archiving process has been successful, you can cite this work as follows:

    Wedel, Matt. Blogs, papers, and the brave new digital world: Matt’s thoughts. SV-POW blog. 2009-06-11. URL:xttp://svpow.wordpress.com/2009/06/11/blogs-papers-and-the-brave-new-digital-world-matts-thoughts/. Accessed: 2009-06-11. (Archived by WebCite® at xttp://www.webcitation.org/5hSmMI5k0)

    Please note that the short (“opaque”) form of the WebCite® URL should be used only in addition to citing the original URL in your bibliographic reference.

    Alternatively, please use the “transparent” (but very long!) WebCite® URL:

    xttp://www.webcitation.org/query?url=http%3A%2F%2Fsvpow.wordpress.com%2F2009%2F06%2F11%2Fblogs-papers-and-the-brave-new-digital-world-matts-thoughts%2F&date=2009-06-11

  10. Coturnix Says:

    Academic Editors are not employees, not in the offices, and though they are an “inner circle” of our community, they are external peer reviewers for all practical purposes and they certainly act that way: they are independent actors.

  11. Matt Wedel Says:

    Daniel, thanks for the further thoughts and webcitation demo.

    Bora, thanks for the further clarification. I stand corrected. Between the copyright confusion in last week’s open access post and the peer review confusion in this one, Mike and I should probably run all PLoS-related material past Darren first. I will diminish, and go into the West.

  12. Mike Taylor Says:

    Matt says: “People do comment on blogs, all the time. Post-hoc review will work, in fact already does work, just fine on blogs.”

    I beg to differ: when a blog post has a dozen or a hundred comments, I as a casual reader really don’t want to have to plough through them all just to determine whether the article was worth reading in the first place. By contrast, anything that gets published in JVP or Palaeontology, I know is not nonsense. It’s worth some of my attention.

    I would be more amenable to the idea of “post-hoc review” on blog posts if commenters also gave a numeric score which could be aggregated. That way there would be an at-a-glance indication of whether something is worth reading.

    But wait — does “the wisdom of crowds” even work on science? I would remind you that by every metric, far and way the most popular site on “Science Blogs” is Pharyngula, a blog in which maybe one post in ten is actually about science. I am not honestly sure I want to let my science reading be guided by a mass of pharynguloids. Reactionary that I am, I somehow prefer the idea of sauropod science that is reviewed by Upchurch and Wilson rather than by Rev. bigDumbChimp, He11razor and Bride of Shrek OM.

  13. Daniel Mietchen Says:

    Mike, you bring up a very good point with the “a numeric score which could be aggregated”. They key for me is to couple the rating of individual comments with the expertise of the commentators, which would keep many pharynguloids (as well as He11razorout & his friends) out, though I wouldn’t object to them being able to give a popularity vote (some call this the “Blog Impact Factor”, after a similarly named popularity vote that underlies many a decision in current scientific affairs). I have put some thoughts in this for a recent blog post — “reactionary” that you are, I’m sure you will find it stimulating.

  14. Scott Hartman Says:

    As someone planning to start a blog soon, my interest in this post was more than academic (double entendre intended). Still, I find myself in greater agreement with Mike at this junction; which is not to say that Matt is wrong, but I too am concerned about the “wisdom of crowds” impact on post hoc reviews with scientific subjects.

    I still have no intention (at this point in time) of citing a blog in a technical paper, but that has more to do with the “value added” I perceive in “personal communication” references. Personal references still serve to preserve priority, and can adapt as quickly as blogs and other digital media, while encouraging direct communication between researchers (always a good thing). And written communications (including email) are harder to modify after the fact.

    Still, I expect the future will be closer to Matt’s description than to my preference, but time will tell. Thanks for the thoughts guys!


  15. […] Blogs, papers, and the brave new digital world: Matt’s thoughts « Sauropod Vertebra Picture of th… […]


  16. […] a little in the context of the debates / random thoughts going on at SV-POW! right now (try, here, here and here for […]


  17. […] Wedel (2009) has raised some fascinating issues in relation to the links between published articles in peer […]


  18. […] is an interesting position for me to come to, given what I’ve said here in the past about filters. It was easier to deal with the thought of completely open publication […]


  19. […] scientific communication with the realities of the newly wired world, in which everything is open, amateurs can have public, automatically archived high-level technical conversations about published … (that the authors probably can’t afford to ignore), and nobody knows what the landscape will […]


  20. […] historically have a habit of leaving things no better than they found them — to be fair, a point that I have also made at times. I was pleasantly surprised by how much of his statement I agreed with, and look forward to seeing […]


  21. […] problem here is not a widespread one. Back when we first discussed this in any detail, about 13% of PLOS ONE papers slipped through on the editor-only inside lane. But more recent […]


  22. […] and it Just Works without any tinkering or subjective decisions on the part of the user (other than What Counts – but that affects all metrics dealing with publications, so no one metric is better off than any […]


  23. […] As we noted in our abstract, its total length of 1370 mm is exactly twice that of the C9 in AMNH 6341, which suggests its neck was twice as long over all — not 8.5 m but 17 […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: