Chickens and Eggs

Yesterday two posts appeared in my feed both challenging the requirement of glamor mag (Nature, Science, Cell) level publications for career advancement.  Michael Eisen (@mbeisen) wrote a post in response to this idea suggesting that this is not a criteria for hiring in his experience (he is referring to job applications where he is, at UC Berkeley as well experiences with his  trainees).  Key point:

My own lab provides several examples that demonstrate this reality. My graduate students have gone on to great postdocs and many have landed prestigious fellowships “despite” having only published in open access journals. More curiously, I have had four postdoctoral fellows go out onto the academic job market, who  all got great jobs: at Wash U., Wisconsin, Idaho and Harvard Medical School. Not only did none of them have glamour mag publications from my lab. None of them had yet published the work on the basis of which they were hired! They got their interviews on the basis of my letters and their research statements, and got the jobs because they are great scientists who had done outstanding, as of yet unpublished, work. If anything demonstrates the fallacy of the glamour mag or bust mentality this is it.

In fact, as a co-founder of PLOS and a strong, vocal advocate of open science his group primarily publishes in PLOS journals.  He hasn't been the last author on a non-PLOS paper since a PNAs paper in 1998, so he is certainly putting his science where his mouth is. Earlier in the day, William Gunn (@mrgunn) made a similar argument:

I'm starting to think that the plodding careerists who always raise the "but I have to publish in X journal for my career" criticism just need to be routed around. You shouldn't be in science because you want a stable career, you should be here because you can't fathom doing anything else.

Now these are both admirable positions to take.  But as the title alludes to, this is a chicken and egg problem.  If a postdoc decides to only publish in open access journals then he hopes that prospective departments and grant committees agree with his stance.  If a faculty member takes this stance he hopes that grants and tenure/promotion committees agree.  If tenure and promotion committees agree, then they hope granting agencies agree.  If granting agencies agree then they hope that the public (or their foundations or government agencies agree).  If any link breaks, then its a risk.  As someone who agrees with this, I might find a department happy with this policy, but if my NIH study section isn't on board then I am in some trouble.

Taking this further, I thought why should this even only apply to open access and open review.  Lets say I do all my research totally in the open, self-publishing it online either on my own site or on a pre-print server like ArXiv or the newer Faculty of 1000 Research and engaging in discussion on these forums.  If I completely ignored journals entirely, would anyone accept this as being ok?  I posted it on twitter, and there were positive responses, but that is really hard to imagine.

Without anonymous peer-review how could I (or the reader) be assured that controls were done properly and the context of the work was appropriately stated.  If that is done via peer review posted with the article anonymously how could the reader be sure I didn't just delete the bad reviews or comments.  If I post some data on a blog or pre-print server and some other person finds it, expands on it and publishes it in Science then do I have any right to feel aggrieved?  Who should get the credit?

In an ideal world, things might work analogously to how Rosie Redfield (@rosieredfield) has addressed the arsenic life question.  After posting an initial rebuttal online, Dr. Redfield did some experiments, engaged with the community about the data and put it all together.  This was (to me at least) the first archetypical evidence of the open evaluation of a research claim.  It was done in the open, public suggestions were incorporated, the work was posted to a preprint archive and then in the end.... it was submitted to Science.

Now this is not entirely fair to Dr. Redfield, Science was chosen as that was where the first arsenic paper was published and where her (and other) critiques were published.  But if this, most open and public scientific re-evaluation, still needs glamour mag validation what hope does the rest of research have?

So who should mandate this?  Various policies of public access have helped make publicly funded research open access and a new generation of scientists has shown more proclivity towards goal but who needs to take the first steps.  Dr. Eisen suggests it has to be everyone.  For this change to happen, primary researchers, group leaders, departments, granting agencies and the public all need to take that step, and leave those who are betrothed to impact factor chasing looking like relics of the past.


Chickens and Eggs by Dave Bridges is licensed under a Creative Commons Attribution 3.0 Unported License.