My Thoughts on the Peer Review Process (Part 2) - Including Dissenting Opinions

[see Part 1 here]

The peer review process is time consuming and thus inconvenient, and as stated several times during this debate, doesn’t always work properly. But I would argue that good peer review is absolutely crucial to not only the publication process, but in also ensuring that the author is putting forth the finest written, best supported and documented, piece of work that they can with the data available at that time. To do any less will only open yourself and your work up to criticism and ensure that your work will not have lasting effect on the field of paleontology. I consider one of the most important aspects of peer review is that of seeking how the specialists (i.e., persons actively researching and publishing in a certain aspect of the science) in your field regard your work, after all they will be the persons reading it more in depth and hopefully drawing upon it in future work. Often this may even entail having your work reviewed by a rival. After all isn’t it this person or group’s opinion that you want up front to know how your published paper will be regarded? If you include them in the process and address their concerns, you all may still not agree, but at least you are showing professional courtesy.

Recently on this blog I criticized a recent paper on aetosaurs (the group I study the most), where I expressed disappointment in the work because I felt that it was not well-documented (e.g., description and figures) or supported, and thus would not be of much use to myself or other colleagues conducting work on the Aetosauria. Besides these authors, there are two persons in North America who are actively studying and publishing on this group at this time. Neither was considered as a reviewer for this manuscript. The two reviewers who were utilized are highly competent, respected scientists who have published extensively on the Triassic, however, neither are experts in the taxonomic group, or the genus Typothorax, that is the focus of the paper. I will assume that the reason that the two North American aetosaur workers (including myself) were not asked is because the authors specifically requested that they not be used as reviewers. Presumably this would be because of past history (i. e. “aetogate” ) between these individuals and the authors, and that the authors of the Typothorax paper felt they would not get a fair review. If correct, this is extremely unfortunate, because that assumes that I would not objectively review the manuscript and instead would be petty and vindictive. This would not have been the case at all. In fact, having read the published manuscript (and seen first hand the specimens that were described) I would not have rejected the manuscript. I would have suggested that the description be fleshed out, incorporating details of the skull and providing comparisons with other Typothorax and aetosaur specimens; I would have asked that the figures be better labeled so that important details would be more clear; I would have asked for new data regarding the taxonomic points being made, and pointed out a few places were relevant literature has not been cited or had been misrepresented. Had all of this been done, I would have had very little criticism of the paper and indeed probably would have found it very useful (and thus positively citable) in my future work. It would have been of little use for me, if requested as a reviewer, to be petty and vindictive as this would have been seen through by the editor and would have done nobody any good [Before anyone says it, I’ll admit that my blog post review of the paper was a bit harsh in spots, but this was based mainly on frustration with what could have been, and of course my comments were made on a personal blog (everyone should understand that by their nature blog posts are prone to outbursts) and does not constitute a professional review for the purposes of journal peer review].

Interestingly, I have a paper in press (also on aetosaurs) where I did specifically request a “rival” as a reviewer. Of course I knew based on past circumstances that the person might be highly critical, proclaim the work unworthy of such a prominent journal, and reject it outright; and sure enough (and sadly as I was hoping to be wrong) a multiple page, scathing (and often very snarky) review came back completely rejecting the paper (the other two reviews were positive). In the end, however, the paper was accepted. Why? I’m presuming because of the other two positive reviews, and because I warned the editor ahead of time that I was requesting a potentially hostile reviewer and that a scathing rejection (but with useful comments) could be expected. This was seriously risky; however, in between the scathing, snarky comments, and the unsupported criticisms, were very useful comments that positively increased the quality of the final paper. These types of comments can often only be received from someone who does not like or agree with your work, and often allows you to see what their concerns are and to help identify the holes in your own argument in contrast to theirs. This resulted in a much better paper, containing portions that would have been lacking if I had not made this reviewer selection.

Thus, this will hopefully alleviate some of the future criticism of this paper from this reviewer as I was professionally courteous in including them in the process (instead of the paper being a surprise) and did cover many of their concerns. Overall this will hopefully result in a much more useful paper for all involved. Of course, this reviewer requested to remain anonymous and they would probably be stunned to know that I am clearly aware of their identity simply because I am the one who requested them as a reviewer. They may even be stunned that I even made such a request.  Peer review is not about simply arranging for a close colleague to read through your work, nod approvingly, and then hand it back.  You want someone who will critically, but fairly, tear the paper apart, and seeking out weaknesses in reasoning or support.  Proper review, although inconvenient, almost always results in a much superior final product.

I think that we owe it to ourselves to make sure that our published work as paleontologists is of the highest standard methodologically, thus well thought out, well supported, and repeatable. Of course we will all never agree with the various hypotheses and conclusions, but we can all agree when a study is well done regardless of its results. Peer review is a crucial step in this process. We need the comments of our peers, including those who disagree with us, to ensure that we are meeting these scientific criteria and to improve our work. To do any less may be easier, and more convenient, but in the end it only lessens the influence of our work in the eyes of the scientific community.

One final note.  E-mailing a peer who does not personally know you well and asking them to look something over for you does not constitute proper peer review.  Often you will get back a reply that is simply based on pleasant correspondence etiquette, and the same detail will not go into the review as when the paper is presented by an editor for the person to judge if worthy of publication.