My Thoughts on the Peer Review Process (Part 1) - Self-Publication

The recent publication by Robert Gay of a self-printed set of theropod dinosaur papers using the printing service lulu.com has caused a rash of debate on blogs, the Dinosaur Mailing List, and even Facebook. Much of this debate regards whether the new taxonomic name, Kayentavenator elysiae, is valid under the ICZN. From what I can tell it is, furthermore the booklet has an ISSN number making it an official book or publication and thus the name is valid.

My issue with these papers (and I did purchase and read them) has been more regarding the issue of how they were peer reviewed. Robert claims they were both originally submitted to journals, accepted, but never actually published, thus he simply addressed the concerns of the reviewers and published them together in his own publically available booklet. I have discussed this with several of the reviewers listed and they vaguely remember reading the manuscripts, one even thought that the ms he read had been rejected. Nonetheless, my point (and one on currently concerned with regarding other potential publications discussed at the SVPOW blog) is that regardless of the outcome of the peer-review process conducted back in 2004, given numerous additional studies published in the six years since, these reviews are technically out of date and I argue no longer valid. Robert Gay should have rewritten his manuscripts, incorporating all pertinent recent literature, and then sent them back out for review. This surely would have lessened some of the firestorm.

Another concern would be editing. As Gay, through his actions, acted as both author and editor, where is the quality control to ensure he had incorporated the concerns of his reviewers accurately? Maybe he should have enlisted the aid of an outside editor? This is especially relevant as he introduced a new taxonomic name and thus would want to whole process to be as unambiguous as possible. A careful update, a few more reviews, and invited some editorial help may have lessened the controversy surrounding these papers and the new taxonomic name.

Why do I care? Primarily I am concerned with people who misrepresent themselves to the scientific community. All the time we see reference made to professionals vs. amateurs in regards to paleontological study and publishing, but what does this really mean? Does having a job (or a degree) where you get paid to do paleontology make you a better scientist than someone who does it as a side interest? Certainly not, and I think the history of the science will clearly support this. What matters is if good science is being done by the “professional” or “advocational” paleontologist. Is there a clear question being addressed, what is the quality of the data collected, are the hypotheses well thought out, supported, and testable? Does the paleontologist have a firm understanding of pertinent literature and is it properly cited? Has the study undergone an unambiguous peer review process to ensure that they have covered all of their bases and the work is considered sound? I've argued in the past that everyne has great hypotheses worthy of publication. Thus, it does not matter what your title is, all that matters is that if you want to publish your ideas, make sure you are producing good well-supported and repeatable science.  That is what makes you a scientist, not a job or a degree.  To do any less, yet still demand respect from peers, is misrepresenting yourself.

Second, I think that lulu.com can be a very valuable tool when used properly as a printing service. The SVP abstract books are now being offered in print through this venue, and it is an excellent choice for other types of publications including museum bulletin series, proceedings volumes, etc… As long as what is being printed has undergone proper peer review and editing. I’d hate to see this resource smeared in the scientific community because of mis-use by others in the field. As I stated elsewhere, lulu.com is a printing service, NOT a scientific journal.

Some have claimed that it is unfair to single out Robert Gay in these debates, as this type of publishing actually is more common than it should be, sometimes masquerading under the guise of a legitimate institution publication series, but his case is the most recent and the first to use lulu.com, thus certainly worthy of discussion.  As scientists we should expect that any publishing we do will be scrutinized by the rest of the community.

[Part 2, will be posted Tuesday morning]

3 comments:

  1. Interesting how you seem to work to implicitly discredit Dr Gay. But from what I can see here, there's no suggestion that Dr Gay is actually wrong. He has transgressed the norms of science, however, by publishing under his own steam, rather than in an expensive and inaccessible specialist journal. It's not clear how he has 'misrepresented' himself. Actually - and I'm no paleantologist - it looks to me as though he has simply acted independently of the main organs of his discipline. And he's only done this - by your account - because others made it necessary for him to do so. What's illegitimate about this? Surely the focus of everyone's annoyed attention should be the journals that accepted these paper but then did not publish them? Since precedence is important in science it looks to me as though Robert Gay has merely done what he had to do to establish his and assign the name. Good for him!

    ReplyDelete
  2. Mr. May,

    I hope that you are not entirely serious or that whomever had you write this is paying you well. Am I trying to implicitly discredit MR. Gay? No, should I be? As a researcher am I thrilled with the choices he made with this publication. No not one bit, as should be obvious.

    It is also obvious from your comment that for some reason you are not a fan of technical paleontology journals and applaud Mr. Gay for bypassing the system. In fact, you claim that he was almost forced to do this. Let's reexamine this:

    Sometime around 2004 Mr. Gay submitted two papers, one to the Journal of Vertebrate Paleontology (according to you and expensive and inaccessible specialist journal...but from an organization to which Mr. Gay belongs), and the other the in-house published Bulletin of the New Mexico Museum of Natural History and Science.

    The first article presumably was sent out for review and then returned to Mr. Gay for revisions. Revisions which he never made (per Mr. Gay), thus the article was not accepted.

    The second article was also returned with revisions, which Mr. Gay never made (per the editor), thus this paper was also not published.

    This is standard for most scientific journals. If you do not make requested revisions (for any reason) your article is not accepted. This is not the fault of the journals, it is the fault of the author.

    How did he misrepresent himself in my eyes? Let's see, he self published six year old, non-re-reviewed or updated data and conclusions completely ignoring more recent work by colleagues.

    He erected a new taxonomic name without recent review and without entirely following the rules of zoological nomenclature.

    He boldly announces this work, boasting that it "will be carried in stores..far surpassing the availablity of other high profile journals anywhere on the globe". Then he has the audacity to fix it with an IBSN number and charge people for paper copies and downloads.

    I cannot understand at all why he, considering himself a scientist, would even begin to think that this would be acceptable, never mind a good idea. The very fact that he had to include a page on "notes on publication" demonstrates that he knew it was a bad idea and not how scientific publication is regularly done.

    I repeat, Mr. Gay should have brought his articles up to date, and re-submitted them to proper journals rather than circumvent the system (and affix blame to the system). The fact that he has taken the route he has demonstrates in my opinion that he was more interested in getting his name on some publications and a new dinosaur name out than in actually making a sound scientific contribution. It shouldn't take a paleontologist to realize that there is something wrong with this. Shame on you for suggesting that precedence is all that matters in science and supporting this type of behavior.

    ReplyDelete
  3. Interesting that I immediately noticed the manuscript didn't include citations of Nesbitt et. al., 2006, where they discussed the prey being underneath the Coelophysis and also not being a juvenile Coelophysis.

    Nesbitt, Sterling J., Alan Turner, Gregory Erickson, and Mark Norrell.
    2006. Prey choice and cannibalistic behaviour in the theropod Coelophysis. Royal Society Biology Letters

    ReplyDelete

Markup Key:
- <b>bold</b> = bold
- <i>italic</i> = italic
- <a href="http://www.fieldofscience.com/">FoS</a> = FoS