Is my new sauropod-neck cartilage paper “published”?
November 6, 2014
In a comment on the last post, Mark Robinson asked an important question:
You linked to the preprint of your The neck of Barosaurus was not only longer but also wider than those of Diplodocus and other diplodocines submission – does this mean that it has not yet been formally published?
As so often in these discussions, it depends what we mean by our terms. The Barosaurus paper, like this one on neck cartilage, is “published” in the sense that it’s been released to the public, and has a stable home at a well known location maintained by a reputable journal. It’s open for public comment, and can be cited in other publications. (I notice that it’s been cited in Wikipedia). It’s been made public, which after all is the root meaning of the term “publish”.
On the other hand, it’s not yet “published” in the sense of having been through a pre-publication peer-review process, and perhaps more importantly it’s not yet been made available via other channels such as PubMed Central — so, unlike say our previous PeerJ paper on sauropod neck anatomy, it would in some sense go away if PeerJ folded or were acquired by a hostile entity. But then the practical truth is of course that we’d just make it directly available here on SV-POW!, where any search would find it.
In short, the definition of what it means for a paper to be “published” is rather fluid, and is presently in the process of drifting. More than that, conventions vary hugely between fields. In maths and astronomy, posting a preprint on arXiv (their equivalent of PeerJ Preprints, roughly) pretty much is publication. No-one in those fields would dream of not citing a paper that had been published in that way, and reputations in those fields are made on the basis of arXiv preprints. [Note: I was mistaken about this, or at least oversimplified. See David Roberts’ and Michael Richmond’s comments below.]
Maybe the most practical question to ask about the published-ness or otherwise of a paper is, how does it affect the author’s job prospects? When it comes to evaluation by a job-search panel, or a promotion committee, or a tenure board, what counts? And that is a very hard question to answer, as it depends largely on the institution in question, the individuals on the committee, and the particular academic field. My gut feeling is that if I were looking for a job in palaeo, the Barosaurus preprint and this cartilage paper would both count for very little, if anything. But, candidly, I consider that a bug in evaluation methods, not a problem with pre-printing per se. But then again, it’s very easy for me to say that, as I’m in the privileged position of not needing to look for a job in palaeo.
For Matt and me, at least as things stand right now, we do feel that we have unfinished business with these papers. In their present state, they represent real work and a real (if small) advance in the field; but we don’t feel that our work here is done. That’s why I submitted the cartilage paper for peer-review at the same time as posting it as a preprint (it’s great that PeerJ lets you do both together); and it’s why one of Matt’s jobs in the very near future will be getting the Barosaurus revised in accordance with the very helpful reviews that we received, and then also submitted for peer-review. We do still want that “we went through review” badge on our work (without believing it means more than it really does) and the archiving in PubMed Central and CLOCKSS, and the removal of any reason for anyone to be unsure whether those papers “really count”.
But I don’t know whether in ten years, or even five, our attitude will be the same. After all, it changed long ago in maths and astronomy, where — glory be! — papers are judged primarily on their content rather than on where they end up published.
Sauropod Vertebra Picture of the Week 
November 6, 2014 at 8:33 am
Well, it’s hard to get a job in maths on the basis of preprints, unless they are obviously stellar. By which time you’ve already given seminars at Princeton etc, and people have already formed an opinion on your work. For run-of-the-mill people (e.g. myself), getting something accepted by a journal means that someone cares enough about it to think it worth publishing. Otherwise, if its not from your own field, it can be hard to judge how big a deal a paper is. I mean, say I solved the Doolittle conjecture about the nonexistence of flanged widgets. Is that good? Or was it obvious, and just needed some hard work. Was it open for 50 years or six months, and Doolittle didn’t see the obvious proof?
A paper in the Annals of Mathematics is worth much more than a preprint. As an example, Vladimir Voevodsky proved the Bloch-Kato conjecture (one special case of which earned him a Fields medal – it really was a big deal, not like my imaginary Doolittle example). It was dependent on a preprint, and wasn’t published for years, because people found out the preprint had some subtle errors, and so the Bloch-Kato proof wasn’t watertight, and so couldn’t be published. People had to work hard, get the fixes done, get *them* peer reviewed and published, and *then* Voevodsky’s work was known to be correct and could be accepted. His reputation was secure, but still couldn’t rely on preprints…
That said, I know one mathematician in Romania who is treating stuff on the arXiv as ‘published’, and foregoing journals out of annoyance at the system. Likewise, another mathematician, now deceased, who treated the Bonn preprint server as ‘publishing’ (he was a well-established and famous guy too).
And on top of that, putting something on the arXiv is not a final act: you can revise. And if some problem turned up in a proof, you’d better revise that preprint. And not too many times, else people think you’re patching a leaky dyke with sticky tape.
November 6, 2014 at 9:14 am
Thanks, David, that’s an important correction. I’ll add a note to the main article, to the effect that I misunderstood the way things work in maths. Would it be fair to say that the field progresses almost entirely on preprints, but that careers progress on journal-certified versions?
November 6, 2014 at 9:27 am
That’s pretty accurate. People don’t have to wait to read a paper in a journal, that would be wasting six months to two years of time. If a paper is useful, you can check the veracity of the proofs of the bits you need yourself. Some people in fact only trust stuff they’ve checked themselves this way, *despite* publication, so it’s no skin off their nose to do so for a preprint. (Terry Tao was once lamenting he had to use a theorem as a black box, as he didn’t have the subject expertise to work over all the proofs leading up to it. Brian Conrad can confidently say that the proof of Fermat’s Last Theorem doesn’t use a certain technical crutch that is all over the literature in that area, because he has *read all the relevant proofs back to basics*(!)).
There are of course outliers. Jacob Lurie springs to mind: some papers as a student on assorted topics, one book, and some conference proceedings papers – and he’s a professor at Harvard. A lot of unpublished stuff is just available from his website. His major work started based on an unpublished manuscript by another mathematician.
November 6, 2014 at 1:42 pm
While I am totally on board your open access crusade I am of two minds about “publishing” without “peer review.” I say this as someone who is not looking for an academic position in any field (although I must say if out-the-blue a funded chair were offered to me, I would not reject it out of hand, heh-heh). My ambivalence has nothing to do with either the paper’s influence or the author’s employment prospects, but rather its use in public policy discussions.
Here in the U.S. we are experiencing another of our periodic spates of anti-intellectualism, this version being particularly anti-scientific. There is one party (I’ll not name it on your non-partisan site) that is particularly apt to believe that a theory is a wish that the heart makes. They cite anything or nothing to make up education policy, environmental policy even research funding. One of the fire-walls that the rationally-inclined minority clings to is the dismissal of hokum on the ground it is not “peer reviewed.” I realize that that might seem lazy or elitist but this particularly culture war that is being waged is not conducted on a high intellectual plane and the audience is not receptive to a lot of subtlety.
This may not be directly related to paleo, but even paleo comes up in education policy debates and a lot of fraudulent “research” is cited to “disprove” natural selection. Since the stakes are so high (climate policy, environmental regulation, education policy, research funding decisions), I would hate to let the members of this unnamed party to be left to their own devising on what is or is not “science.”
November 6, 2014 at 1:58 pm
It’s a good point, DK, and an important one. As I’ve noted before, I do think that the peer-review system has the real value that undergoing peer-review shows a willingness to undergo peer-review. Even as a purely arbitrary hoop to jump through (and we can hope it’s more than that), it does at the very least provide a test of seriousness.
But I don’t think the solution is to prohibit publication without peer-review. I think it’s just to be clear about what is and isn’t peer-reviewed. Within a field, readers can make their own judgement — other sauropod workers, or indeed animal anatomists — can form their own opinions about my new paper. They can use it or not as they judge best, but the key thing here is that they are the constituency for whom its timeliness is most important. For people distant from the field, and unable to judge the work for themselves, it makes sense to wait for a peer-reviewed version; but the need of those people to wait isn’t a good reason to make other specialists wait, too.
Hence the path this paper is taking: immediately publication without peer-review, for those who want to be up to date; followed by (hopefully not too) delayed publication with peer-review, for those who need that stamp of approval. (Though let’s hope they don’t take that stamp of approval too seriously — after all, peer-review evidently didn’t catch this important oversight in the original Stevens and Parrish (1999) paper, but did give it a stamp of approval that’s allowed it to have an arguably damaging degree of influence.
November 6, 2014 at 2:02 pm
Responding to DK Fennell’s comment: what you’re saying is of course a legitimate concern to be had. One must remember though that peer-reviewed or preprint, it still doesn’t guarantee that it is real science – there are plenty of “peer-reviewed” journals of ill-repute that will publish anything.
On the specific topic of preprints in PeerJ Preprints there is a human editor looking at each submission to avoid “crank” papers, pseudoscience or ones violating any ethics principles, etc from being published. This is done to maintain credibility and build trust, which will pay higher dividends in the long-run than simply publishing anything. arXiv is a little different, whereby one must be recommended before being allowed to publish – although the quality checks are less stringent than at PeerJ Preprints. And of course, in the peer-reviewed portion of PeerJ the standards are much higher and the papers go through peer-review as expected.
November 6, 2014 at 2:08 pm
In the world of astronomy, I can’t say exactly where reputations are made; some of the biggest results are first released in press conferences at national meetings, for example, and occasionally, one of those results may indeed give a major boost to its author. I can say that when one is evaluating a job applicant at my institution, one gives much greater weight to papers which have been published in peer-reviewed journals than to preprints. In addition, when one applies for tenure or promotion at my institution, it is again the peer-reviewed papers which are considered.
But, of course, ordinary astronomers read the arXiv postings each day to keep up with the latest results, so in that sense, one’s reputation among one’s peers may indeed be influenced heavily by preprints.
I guess it depends on what is important: the informal opinion of one’s peers, the ability to get a job, one’s appearance in a promotion review. If one is working outside the traditional workplace — say, one earns a salary at a patent office while doing theoretical physics as a hobby — then some of these concerns might not matter.
However, I do share some of the concerns echoed above that it is possible that a changing landscape in scientific publishing might endanger the peer-review process. Peer review has its faults, I admit, but it also serves some important purposes.
November 6, 2014 at 2:54 pm
Is published all right, in the sense you write, that it can be cited. It is up to public scrutiny, which is excellent and rarely happening in math, unfortunately (will come back to this, there are nuances). If it pass the review process then the readers who need to use your result without reading the paper will be happier. Others will find the paper and the reviews together a more valuable resource than the regular one published in the old way. If will be errors discovered in the paper after the “publication” then you shall be publicly accountable (not responsible) for them, and the anonymous reviewers will be more careful in the future.
I think David Roberts mentioned me in his comment, so let me write that arXiv is the main place to look for math articles for most of the research fields (but not all), that the arXiv eprints are citable and mathematicians slowly learn this from their much more advanced (in this matter) physics researchers. Statistically about a third of the arXiv eprints are published later in regular ways. In my case I have about 60 articles, most on them on arXiv, published something like 25 because I got in love with the Net since 1994. More recently I don’t even think that articles are good dissemination tools, in the form they are now. However, I would publish in PeerJ tomorrow (new articles, not the old ones which are already on arXiv) if PeerJ would have this extension towards maths. Is sad to know that mathematicians have been very slow adopters of arXiv, which is already an old publication venue, and they are still again adopters of newer and better ways.
As for the career side of the publication business, as David mentions, I have only one word for those who look into the future: DORA.
November 7, 2014 at 5:04 pm
I recall discussing this on this blog or perhaps on mine at some point with you. My final point was, I think, that the paper so published must have a “final form” and as such not be subject to further editing. Even typographic errors must be preserved (so pedantics can use “[sic],” I guess). The purpose of this is to show that when you place this document to the public and subject it to public *scientific* scrutiny, you’re being serious about it. And that’s the big distinction. “Publishing” papers is different from “publishing” blogs, otherwise you could just plop all your work here and never touch PeerJ — or any other journal, book, or venue ever again. So why do the latter at all? “Papers published” should have a permanent final condition of their content that secures them from the access to content revision any other format entails. Locked PDFs achieve this. It is also important to consider your peers, and why PeerJ works: they recognize it as “published.”
November 7, 2014 at 5:09 pm
Well. That is certainly what we have done in the past — not least because in the not-so-long-ago days of ink-on-paper, there was no other option. I don’t see it follows this is must always be so.
I do think it’s important to be able to refer to a specific version of a paper — as you can on arXiv, PeerJ Preprints and indeed Wikipedia. But I don’t see why we need ever say “Right, you can’t make any more new versions now.”
November 8, 2014 at 8:35 am
I’m not sure that protecting the validity of peer-review for “policy” purposes is particularly useful. In my experience, most people whom one might want to use that argument against fundamentally distrust “science” as a human process/social milieu – IE they would (in my understanding) say that peer review is not useful since the “community” pretty much all shares the same biases.