The publication of scientific papers is slowly becoming more “open,” meaning not only that in some fields (like math or physics) many of them don’t undergo the usual review process, but also that journals are increasingly adopting a policy of having free and open public access (the PLoS journals are an example), a feature subsidized by charging rather high fees to the scientists who submit papers. And scientists now have their very own Facebook equivalent, ResearchGate, where you can link to your papers, discuss those papers or other scientific questions, and find colleagues or collaborators.
In the January 16 issue of the New York Times, Thomas Lin discusses the rapidly changing face of scientific publication in a piece called “Cracking open the scientific process.”
Some of this change is facilitated, of course, by bloggers, who demand immediate access to everything, and the ability to discuss results as soon as they’re printed:
On [last] Thursday, 450 bloggers, journalists, students, scientists, librarians and programmers will converge on North Carolina State University (and thousands more will join in online) for the sixth annual ScienceOnline conference. Science is moving to a collaborative model, said Bora Zivkovic, a chronobiology blogger who is a founder of the conference, “because it works better in the current ecosystem, in the Web-connected world.”
Indeed, he said, scientists who attend the conference should not be seen as competing with one another. “Lindsay Lohan is our competitor,” he continued. “We have to get her off the screen and get science there instead.”
Well, no, Lindsay Lohan is not our competitor: those who follow that sort of tabloid journalism simply won’t be following scientific advances. We have to realize that despite all of our efforts, a large fraction of the American public simply can’t be induced to follow science. That doesn’t mean, of course, that we shouldn’t increase our efforts to popularize our work, simply that you can’t interest everyone all of the time.
But I do approve of much of the open-access movement. There are, however, problems with it, some of them highlighted in the Times piece.
- If there is no peer review of published papers, then there is no quality control, at least not beyond that made on posts following online publication. I myself have benefitted tremendously from the comments of reviewers, and also vet my papers to my colleagues before submitting them to journals.
- Peer review, however flawed, is a sign of professional acceptance and recognition, and peer-reviewed papers (like grants) are appropriate measures of professional success for promotion, tenure, and other ways to climb the scientific ladder.
- Open publication is expensive for the scientist, often costing several thousand dollars.
But there are upsides, too:
- Much scientific research in the U.S. and other countries is funded by government agencies (the NSF, NIH, and USDA in our country), and that money comes from taxpayers. Why should they have to pay again to get access to scientific articles reporting research funded by the taxpayers? Publicly funded research should be publicly available.
- The peer review system is slow: in the bad old days, for example, it took a year from when a paper was submitted to some evolution-related journals to when that paper appeared. That’s not a terrible problem in my field, since evolutionary research doesn’t become obsolete quickly, but it is a problem in faster-moving fields like physics and molecular biology.
- If there are problems with a paper, those problems used to take a long time to become public: other scientists would write a note of critique or response, which then had to be reviewed, another process that could take months. By that time people had largely forgotten the original paper. This was compounded by the policy of many journals (I think Science and Nature are among these) to publish the original articles in the paper journals but the responses only online. This problem has become somewhat alleviated by the instant reaction of bloggers to research that seems dubious, such as the Darwinius fossil paper and the dubious “arsenic-based life” paper.
My own views on this are the following:
- Retain peer-reviewed papers, at least in biology, as a way to ensure quality and establish criteria for professional advancement.
- Papers should be published online as soon as they are accepted in final form; eventually, all publication should be online only as a way to save trees, energy, and other resources. (This does create some problems for me because I simply can’t read articles on a computer screen and must print them out. However, I don’t print out every article in a journal, so there’s still a net saving of paper.)
- If formal critiques are accepted by a paper journal, they should be published in that paper journal instead of simply online. Nobody looks online for critiques (and hence will miss them) if they subscribe to the paper journal. If everything is online, critiques should be published following the paper as soon as they’re reviewed and accepted.
- There should be a comment section (moderated) following online papers for scientists and others to weigh in on the research.
“This does not solve all the problems, for there’s still the issue of expense. Online journals will still have to make money to support their efforts and to moderate any comments if they allow comments.”
Could the journal apply for a grant for the peer review, publication, and comment moderation costs of publicly funded research?
Or should the journals be taken into public ownership?! (Half-joking.)
More seriously, in the long run won’t costs even out? Institutions have to pay for their research to be published, but they don’t have to pay for access to others’ published research.
/@
I’m a fan of peer review. A post-doc in my wife’s old lab has written a fairly bad paper. It was rejected by Cell.
He took the paper and resubmitted it, with the peer review notes (from Cell) because that’s how it’s done in Biology. It was rejected, again, because he still hasn’t addressed the problems from the first peer review.
Without peer review, this turkey of a paper could be published by throwing money at it. Possibly clogging the arteries of developmental biology and leading to research/funding heart attacks as people chase bad work down a rabbit hole.
I think peer-review – and knowing what has been peer reviewed and what not — is essential.
But is a two-tier system feasible? Have papers published as “public drafts” or “requests for comments” – clearly labelled as such — in the first instance, publish and link to all peer review (solicited or not), and then a final version, revised in the light of peer review and accepted by an editor. As well as formal critiques of the final version. Would this be unwieldy?
/@
It’s not a bad idea. As long as it was clear to what extent critical eyes have had their view on the work it could keep the scientific community fairly informed.
Of course, even with the peer-review process there is a lot of bad stuff that gets through. I’ve read that some people in my wife’s field (and she concurs) that probably half of the papers out there are in the range that goes from ‘overstated’ to ‘just plain wrong.’
No need to abandon peer review. Keep peer review but have everything posted publicly before it is submitted like the math/physics people do with the arXiv (OK, it’s not everything but it’s a lot). and leave it open to the community for comments. You still have the 3 reviewers examining the in detail but you also have everyone interested looking at it at the same time. The reviewers and the editors will know what the response from the community is and will be influenced by it. Everyone wins:
1. It is out there immediately for everyone to look at it. Science moves faster as a result.
2. If it is a good paper but happens to be sent to reviewers with an axe to grind, it will have more chance of getting published.
3. If it is a bad paper but happens to be sent to reviewers who only look at it superficially, fail to see the flaws and accept it, the issues will be more likely to be found out and the paper rejected.
4. If it is a good paper and it still doesn’t get accepted, it is still out there for people to read. Even if it is a bad paper but has some truth in it, those ideas/results are still out there to be seen and not hidden. We may actually start seeing more negative results being published that way which is a well known problem
And… I should have read #3 before replying to #2.
I like your analysis.
/@
I’m in agreement with GM. The web is about hte most flexible medium there is, so there are no technical issues in a journal site implementing all of these:
1) Quick publication.
2) A clear status for the paper: New, Reviewed, etc. Anyone interested in only peer reviwed can search by the status.
3) Publication on a journal site would still need to be authorized for publication. There’s no reason why a journal body can’t have a secondary site listing rejected papers, with links to an author’s site, so there can still be public scrutiny that might challenge unfair or biased rejection.
4) Discussion can be broken down too, so that professional registered members only can write to the peer forum, and the public can write to a public forum but can also read the member forum. Any non-registered or rejected peer response to the paper could still be put on the public forum.
This is easy functional software design. It just needs a good designer to make it work well.
And this is all software. Cost are low. If WordPress and Google can maintain so much free web space it’s not beyond a journal to fund the their own sites, or even subcontract the site out to someone bigger. Membership fees, for authors and member peer readers, can be reduced dramaitically by avoiding paper publishing.
I like you ideas too.
One thing though – we should keep actually printing journals simply for the sake of preserving the knowledge.
It is by no means guaranteed that there will even be internet 100 years from now. It may well have disappeared, for various reasons that it is not the time and place to go into right now. And if things exist only in electronic format, they will be lost forever. Obviously, you can’t store things forever in paper form either because paper decays (and, in the worst case scenario, it burns) with time, but it is a better backup compared to something that only exists in electronic form.
It does not consume that much paper as it may seem – the total circulation of scientific journals pales into insignificance compared to all the glossy tabloid magazines that get published.
I absolutely agree with the value of a printed record. Imagine a complete collapse of scientific/technological culture. Scholars in some future re-enlightenment would have a far better chance of deciphering our written records (think cuneiform and hieroglyphics)than knowing what to do with hard drives recovered from our land fills.
Keep the print. I know of ten year old online papers I can no longer find. Yet the library has the Journal of Ornithology going back to the beginning of time (which may be real useful to track climate change).
Indeed. Keep the printed version of journals.
but do it this way:
make a master printed version of the journals ONLY FOR LIBRARIES, and then have the primary distribution of the journal to private users be electronic.
solves all needs: Archiving, availability, ease of distribution, and even cuts way back on how many trees are needed.
Of course, people like Jerry could pay a bit extra to have a copy of the print version made, covering any additional costs of materials and distribution.
but the default would be electronic: private; paper: public institutions.
I know a lot of journals already have this system in place, but formalizing it couldn’t hurt.
I like #3 re unfair or biased rejection.
And #4.
Ah, yes, the software. It seems to me like it should be within the functionality of a COTS document-management system. But maybe an OSS system would be more appealing to the scientific community.
And host in the cloud — Amazon S3, Rackspace. Very feasible — and much better for availability and resilience than private hosting.
/@
GM: Best post in this thread so far, including every reply. Agree with all.
That’s not bad. But math is paper and biology is experimental. Sometimes you must do the experiment to see if it’s true. And that can take a long time.
This comment applies generally to the topic more open publishing practices, especially the intersection of GM’s & Ron Murphy’s comments. I’ve attached it to GM’s first comment just because I can’t attach it to the thread as whole:
I think these are admirable goals and I can’t fault anything (nor would I want to). I rather agree. However, I think the comments of this flavor tend to remain silent on the source of some of the problems of peer review: incentives. There are currently insufficient incentives to promote a more thorough peer review system in general (though there are specific exceptions — See Rosie Redfield’s yeoman’s work on the Arsenic kerfuffle.)
To chime in with my $0.02, in the same vein as the others here, I think it would be awesome to have:
public/informal peer review before publication (like allowing comments on an arXiv paper);
formal peer review leading to publication (traditional style);
public/informal peer review following publication (like PLoS does in allowing comments on its papers).
However, of the three above, only #2 is incentivized, and poorly at that. Number 2 is voluntary and is pretty low on the ladder of hiring/promotion decisions — a lot of the incentive to participate is tied up in altruism (this is generally good), but a lot is also tied to desire to nudge the field to a researcher’s own vision by molding the literature (this is generally bad).
I predict that the uptake of #1 will be about as prevalent as the uptake of #3 (which is to say, not a lot based on current evidence) without a concerted effort by the funding and academic communities to incentivize these options.
Apparently <ol> tags aren’t permitted. Here is how it should have looked:
1) public/informal peer review before publication (like allowing comments on an arXiv paper);
2) formal peer review leading to publication (traditional style);
3) public/informal peer review following publication (like PLoS does in allowing comments on its papers).
Could academic reviews be incentivised by scoring them in a similar way to scoring publications? (Within our firm, our peer reviews of each others’ work count towards our performance appraisal along with the research we publish.)
/@
They could, but you have to make them public for that to happen. And this creates a lot of other issues
Or anonymise them in some way?
/@
I agree with everything you say. And it is not a new concern – for a long time there has been discussion on the need to create some sort of credit-earning system for reviewing papers. Easier said than done though because pretty much all conceivable ways of doing so would mean that peer review has to stop being anonymous and this opens a whole can of worms.
One obvious way is to make reviews public, by extending the current practice of publishing one or two-page previews of published papers in the same journal issue as the paper itself. It is done only for the hottest research and not in all fields but those are PubMed indexed and they do count for something even if not on the same level as actual publications. So one can imagine publishing the reviews together with the paper in a similar manner after the paper has been accepted. This will mean that the reviews will have to be well written and substantial and will mean that disasters such as the arsenic paper will be less likely. And since it’s going to be PubMed indexed, the reviewer will get some minor brownie points for it.
The obvious objection applying to all schemes involving doing away with peer review anonymity is that young researchers will be very afraid to say anything negative about the work of the big shots in their field. Most papers are not that obviously flawed as the arsenic study, the one that Lynn Margulis got into PNAS, or some of the other famous examples. I, and probably other people too, know of horror stories of that kind, involving those short preview articles sometimes written – the PI of the paper, already published, doesn’t like something in the preview and retaliates against the person who wrote it. I don’t see a way around it, other than the fact that both the paper and the reviews are out there in the open for people to judge, but that’s not much.
I see no reason to abandon peer review, but we could use this open process as an impetus to improve it. My biggest complaint against peer review is how trivial most of the reviews are—precisely because there rarely is ANY accountability for the quality of the reviews. Routinely, I find I spend weeks on a review—reviewing the literature, especially the papers and chapters referenced in the article of which I had been unaware, recomputing the statistics (where possible: checking the reported degrees of freedom usually reveals all), and so on. And then, in the cc’d letter to the authors, I find that the other reviewers wrote a paragraph to my 10 page, single-spaced treatise.
So, a simple solution: all reviews are published with the article. Two things should immediately happen: 1) reviews will finally receive some academic credit, and 2) the quality of the reviews will improve dramatically.
In mathematics at least, I am not aware of any movement away from peer review. Posting on the arxiv is not publishing, although there is pressure to make your post polished. The same is true for blogging about your research or even doing collaborative research via blog–you still have to get peer review for publication.
One other possibility you might have in mind is conference proceedings. Some of these are refereed, but there is a long tradition of discounting them as less than truly reviewed papers.
This.
Also, this is relevant to the discussion: http://gowers.wordpress.com/2011/10/31/how-might-we-get-to-a-new-model-of-mathematical-publishing/#more-3645
I at least one branch of Mathematics (theoretical computer science) conference proceedings are preferred over journal publications (at least currently).
But I think the post inadvertently seems to say that there is some movement away from peer-review in mathematics and physics. This is certainly not true. All the open-access movements I know of in Math/ Statistics/ CS basically ask for two things: 1) a peer-review process (unlike the arxiv) and 2) a freely accessible place for archiving the said peer-reviewed papers (like the arxiv). The arxiv is seen as more of a “use at user’s risk” repository of new results to facilitate quick dissemination of ideas, based on the premise that most mathematicians self-select enough that most results, when they have been written up, are correct and already usable by others, peer-review notwithstanding.
I fear bogus science papers will flood the non peer reviews and will be used by religious radicals to confuse the public’s concept of real and phony. We need to reform the peer review process and make peer review papers easier and cheaper to publish.
I think that is a very realistic danger.
That’s what charged into my mind first. Perhaps each page on a given journal site could have some code embedded in it such that copying from it would watermark ‘this non peer-reviewed talking paper has likely been quote-mined by an unscrupulous person; please consult someone with more integrity for information on science’ or something.
Well, physicists routinely use arXiv as a way to get ideas circulating and to stake credit. Even if a preprint has the same status as a personal blog post, it seems to work.
Why don’t other sciences use arXiv or similar preprint archives? If there’s a good reason, why doesn’t that reason apply to physics?
And… I should’ve read this too.
So, a two-tier system is already in place, in a sense.
Can you follow an arXiv post forwards? That is, does it provide links to peer review and other comments and the final published version?
I’d forgotten for the moment, but we certainly used to distribute paper preprints before publication when I was doing my physics Ph.D. I didn’t realise it was peculiar to physics!
/@
The reason is that physics and mathematics are a harder wall to penetrate by most of the unscrupulous types out there because of the formal notation. It’s very much harder to quotemine that than perhaps other topics which aren’t as mathematically dense and rarefied in that way. That isn’t a dig at any of the other sciences around, but it seems to me to be why I very rarely hear or see the creationist types out there trying to argue mathematical points. To the extent that one does see that, it’s generally quite a trivial matter to conclusively demonstrate their error.
Like, say, William Lane Craig and his naive attempts at arguing about transfinite arithmetic, statements about certain classes of functions and curious abuse of statistics. He doesn’t get very far with those with anyone who can read the maths. It’s a bit different with English sentences though.
That doesn’t fit in with the oceans of green ink to be found on arXiv, let alone viXra. Physics cranks are not in the least bit rare. (True story: physics cranks were the reason Wikipedia put in its “no original research” rule.)
Peer review is the barrier walling off cranks and Creationists. If that wall crumbles, so does so much more, and the cost will be very, very high.
The arXiv operates a moderation system that seems to be quite good at filtering those out.
An this is in fields where it is actually much easier to be a crank because you don’t need to generate any data if you are a theorist.
Your typical biology paper costs tens or hundreds of thousands, sometimes millions of dollars in experimental expenses. That alone filters out the cranks
Well, if it were a crank biology paper, they might only claim to have done $10^5-6’s worth of experiments! 😉
/@
So they might only claim to do $1/10 worth of experiments? Sounds fairly accurate to me!
I would be worried about this too, but there might be a balancing effect in the ability to see what actual scientists publish compared to creationists. Right now, save for a few havens of reason, Google belongs to the creationists (they actually pay for adspace and featured links!) and that is mostly what the public is going to see when they search for Evolution. I just had a discussion with a non-religious chap who was repeating creationist garbage because that is what he found and he wasn’t knowledgeable enough to avoid the trap of sites like darwinismrefuted.com that don’t explicitly state their intentions other than some small discussions regarding “materialism”. When most of the research is behind a wall, it’s easy to buy into the idea of an “elitist scientist conspiracy”. Opening this process and being able to watch scientists comment on other scientists and show that they are actually doing science will do a lot to improve the public appeal of science. Much like that Baylor paper that was recently discussed, when you compare that to a real scientific paper it’s a complete joke. Sure, a creationist might slip in a crank paper, but with a comment process and reviews along with mandatory links to each review and refutation, it won’t survive and will probably do more to diminish creationism’s hold on US society. I personally love where this is going, sure there will be bumps along the way, but, just like science, the methods will improve and the process will ultimately invite more public interest in the scientific process. No, that’s not going to convert die-hard Lohan fans, but it will help turn science from something only scientists and college students have access to into something anyone can access and I would be surprised if these journals could mitigate some of the costs through more traditional means of online publication revenue.
“If there is no peer review of published papers, then there is no quality control, at least not beyond that made on posts following online publication.”
“Open access” does NOT mean “not peer reviewed.”
There are a couple of open access journals (I use the term loosely) that are experimenting with alternate forms of peer review, but they are a tiny minority that have not gained any significant traction, as far as I have seen.
Excellent point. Open Access and Peer Review are by no means mutually exclusive.
John Stuart Mill showed 160 years ago that ownership, production and distribution were not necessarily linked.
High time to address the question in the scientific production, too.
Plus, with the advent of instantaneous communication, scientists no longer belong to a restricted club. The choice is only between:
A. communication with the public at large by leak and hearsay, with those interested but not involved hitting a prohibitive paywall,
and
B. taking the initiative, i.e., Open Access wherever possible.
The Peer Review process must also be rendered transparent, and its value exemplified to the public. Like trains running on time, it is hardly acknowledged where it works, but publicly derided where it fails. Example: the El Naschie debacle @ “Chaos, Solitons & Fractals”.
I really like the new model of publishing that’s used by PloS ONE: rapid technical review followed by prompt, open access, online publishing.
The fact that PloS ONE only peer-reviews the scientific rigour of the experimental work and analysis, and does not judge the perceived importance of the work is an excellent move too.
There’s a huge amount of data out there that is either significantly delayed in getting published, or will simply never be published at all, just because it doesn’t fit into a nice “story” for the journal.
Hopefully all scientific publishing will start to move towards this model, with smaller papers being published more quickly and more frequently, and remove the problem of people sitting on valuable data for years whilst they try to build a story to get into a good journal.
Keep the peer-review process BUT give free, unlimited access to articles on the web once they have gone through the process. One of the main reasons religion has a hold on society is the fact that information is provided for FREE, community “leaders” are easily accessible, public forums are frequent. I can call ANY denomination within my community and ask for free access to their literature and probably wouldn’t be denied. Though I may be asked for a small donation or to ponder membership, it’s not something I must provide to get the information. I can walk into (almost) any religious gathering and have access to their teachings. Science has been an elitist endeavour. Scientific disciplines need to be accessible to the public in order to combat religious influences.
I totally agree. I am so depressed to arrive at paywalls every time I try to pursue some particularly compelling piece of research. Science bemoans scientific illiteracy at the same time it prevents public access to its most basic educational documents. If more journals were accessible we wouldn’t be so dependent on much-maligned science journalism.
Ben Goldacre has harped on this theme frequently. Elsevier are basically vampire ticks on science itself. Considering most of this research is paid for by public money, there’s gotta be a better way.
Good read, thanks! From the article:
“…people who’ve wandered away from institutional academia.”
Does that ever describe me. Kinda nice to reflect on the fact that I’m no doubt part of a larger community. Ironically I have some very close (husband, son) channels who do have institutional access that I’ve been known to exploit when desperate. It still frosts me that I can’t do it by myself, tho, esp. as I do most of my “research” in the middle of the night…
The first time I heard the “we’ve already paid for it via our taxes” argument for free access, I thought it would only be a matter of a few years before the irrefutable logic of that view brought about the needed change. Unfortunately, that was back in the ’70’s…
I’m glad to see people arguing for peer review. I don’t think of myself as old (who does?) but in discussion with some younger post docs and new appointee academic staff I have noticed a worrying trend to review manuscripts sent to them in a rather cursory way. Their argument has been – I’m not joking – is that it may not be particularly good paper but it should get out there and be judged by others. I take reviewing my peers seriously and won’t take on a review unless I am sure I can give it the attention it deserves. Similarly I have a had a few papers of mine that have been recently reviewed coming back with a non-committal review of a paragraph; totally useless for improving my ms. and so wishy washy it’s impossible to judge it was positive or negative. Anyone else get this impression?
“This does create some problems for me because I simply can’t read articles on a computer screen and must print them out.”
I’m glad I’m not the only one.
It’s true for everyone. Holding something physical in your hands and not having the distraction of the e-mail and the web at a one-click distance helps you focus on the subject, and the experience of reading something physical leaves much longer lasting impressions of what has been read in one’s mind.
I find it hard to read anything of any significant length at my desk on my screen. Namely because I’m usually sitting at my desk for the purpose of doing something other than reading a lot of text. I use a tablet as a reader for longer texts I want to give a great deal of focus to (such as WEIT on Kindle 🙂 ) If it’s the screen itself that creates problems, the eInk readers such as the Nook and Kindle are very kind on the eyes, and a Kindle subscription to a science journal would be awesome.
The other week I read some research on how elders perceive text better from tablets than paper. (Even if they themselves don’t feel it that way.)
Ehm. Not implying anything on age here. I t is just interesting that those who may struggle hardest with reading adapt to tablets the best.
LOL! Even the young can have bad eyes 🙂 Though my parents recently bought Nook Color readers and absolutely love them. I find the tablet easier to read on, namely because the screen is already lit and I don’t have to have a reading light in bed. Most eInk readers still require a book light (I mean, seriously?) Plus I can highlight words I don’t know and have them defined for me or search the term on google. Many ebooks have embedded links as well, WEIT is full of them (though you can tell the Kindle version is an OCR compilation because there are mistakes such as http://www.tallcorigins.org that an OCR algorithm would have made by converting the letter “k” to and “lc”.) and I can click on the referenced YouTube videos and watch them. And, since my progress is synched with the Amazon cloud, I can open the reader on my phone or browser as well to keep up on my reading when I ‘m out and about or sitting idle at work.
It is hard though to get your Kindle signed at an event though.
I’m not sure I can read a science paper without a pen/highlighter in my hands…
Hold on there! Open access is NOT the same as no peer review. You’ve fallen for “a lie invented by a PR hack who was explicitly commissioned to come up with soundbites to undermine open access”, as Mike Taylor put it here- http://svpow.wordpress.com/2012/01/23/publishers-do-not-provide-peer-review-we-do/ .
Also, open access publication is NOT always expensive for the scientist. PLOS ONE offers a no-questions-asked publication fee waiver to anyone without the finances necessary.
So there go all of your problems with open access. With publishers like Elsevier trying to implement horrible policies like the Research Work Act (see http://svpow.wordpress.com/2012/01/09/do-your-bit-to-oppose-the-evil-research-works-act/) despite providing extremely little service themselves, this is not the time to spread incorrect information about how open access works.
…And, speaking of the egregious Elsevier, they have not exactly covered themselves in shining glory where their internal review processes were concerned, see the “Chaos, Solitons & Fractals” affair (& my comment @ #8.)
I don’t think anyone has ever argued for the end of peer review.
The argument is that the enforcement of review *pre-publication* should go. You can argue about the details of an alternative system, but that seems obvious–pre-publication review became the standard so that journals could decide what to put in their limited page space, and there’s plenty of space on the internet. Maybe journals could pick up papers from something arXiv-like that they think deserve wider recognition.
Also arxiv is not all pre-peer-review stuff. For example, in the Earth and Planetary Astrophysics section ( http://arxiv.org/list/astro-ph.EP/recent ), right now 7 of the 10 most recent papers have already been accepted to high-quality peer-reviewed journals.
An excellent piece and comment thread, but a few objections:
As many others have said, confusing open access with no peer review is nonsense.
This may stem from confusing green open access (e.g. sharing preprints, like Arxiv, or Nature’s own Preceedings) with gold open access (like PLoS, or Nature’s Scientific Reports), which is peer reviewed.
The idea that (gold) open access is unreasonably expensive is also silly. In either case, tax-payer funded grants cover subscription costs. If the person selecting what journal to publish in has to pay the cost, then better quaility journals will successfully charge more. In a subscription based system, journals don’t have to compete on quality for cost, but can charge whatever they like for the subscription. Gold open access regulates prices the way any capitalist market does, subcription based systems do not, and it is they who have the ridiculous publishing costs. Competition would drive prices down.
The idea of having comments online on papers has been tried with very little success by Nature, PLoS and others.
A yes or no vote on peer review is over-simplifying things way too much. With the exception of PLoS one and its ~ 7 clones, reviewers are asked to evaluate not only scientific validity, but predict scientific impact. Most rejections are made on grounds of this second criteria “cannot compete for our limited space.” Most defense of peer review is made on the basis of the first — conclusions are supported by methods, etc. The first is a scientifically objective question, the second is largely a question of predicting future social phenomena, something that no-one has a good record on. Why must these two roles be coupled into one step involving the same 2-3 reviewers?
There’s various efforts to popularize open access. Many universities are starting their own digital repositories to accommodate this.
There’s the Budapest Open Access Initiative (http://www.soros.org/openaccess).
Peter Suber has written extensively on it (http://www.earlham.edu/~peters/fos/overview.htm)
The one thing I dislike about peer review (and I’ve done my fair share of that) is the privacy. I would prefer a system that isn’t blind and where the paper, and the reviews are all publicly viewable.
In cases where papers include significant IP, there should probably be a way to limit that visibility.
I agree that peer-review is essential. The trick is to get enough peers to care enough to do their due diligence. Lots of us do. But not all.
The problem there is that IP is the opposite of science.
As pretty much all the comments have said, this is not the end. Merely an addendum.
Open publication will no more reduce the market for exclusive, hard-to-publish-in journals, than standard housing will reduce the market for mansions.
Prof. Coyne:
I think you will find the following story in Science relevant:
Online Social Network Seeks to Overhaul Peer Review in Scientific Publishing
http://news.sciencemag.org/scienceinsider/2012/01/online-social-network-seeks-to.html?ref=hp
I’m glad to hear I’m not the only person who has problems reading articles on a computer screen. This is why I don’t do kindle.
I have a very utopical opinion… maybe somebody said that already… didn’t have the time to read every comment right now, but wanted to leave this opinion
RATING REVIEW!
I think It would be preety nice in fact to stop peer review and start a more open publication type.
I think the main issue that concerns people about eliminating peer-review is the quality of the published papers.
I think there can be some sort of “rating”… I think we haver a LOT of digital capacity in order to essentially publish anything. But how can we make sure “bad science” isn’t out there… You can’t, but you sure would be able to “unlike” or Zero-star it! That way you would still be able to see the best quality papers at the top… but still if in the need… you can use the crappy papers maybe in case you can get an idea or somthing… Also I think everything could be more dynamic, corrections of crappy papers could be made faster.
Think of it like facebook, if you post awful photos of you or incriminating photos of you… that’s the image evrybody is going to see… so there is also incentives to avoid publishing crap.
Also people can be rated… something compared as “levels” or “ranks” in video game… It wouldn’t be fair that a vote from… let’s say a nobel-prize scientist… to weigh the same as my vote 😛 a noob grad-student…
In theory… this would still be peer reviewed, but a gigantic scale… 🙂
🙁 i think nobody reads long posts…
TL,DR …
Not true; rated 3.1 on a scale of 5 🙂
“But I do approve of much of the open-access movement. There are, however, problems with it, some of them highlighted in the Times piece. If there is no peer review of published papers, then there is no quality control.”
May I remind you that openness or otherwise of papers has literally nothing to do with whether they are peer-reviewed? There are reviewed open journals and unreviewed closed jourmals.
Publishers do not provide peer review. We do.
Yeah, I mistakenly conflated open access with lack of peer review; thanks to you and other readers for pointing that out. I am aware that they’re not concordant, but wrote too fast. I am in favor of peer review and also open access if the latter is financially viable.
In the vernacular of OA, “OA gold” is OA applied to journals, which are (usually) peer-reviewed. “OA green” on the other hand, applies to repositories, and these are not peer-reviewed.
“In the vernacular of OA, “OA gold” is OA applied to journals, which are (usually) peer-reviewed. “OA green” on the other hand, applies to repositories, and these are not peer-reviewed.”
Well. A repository is not itself peer-reviewed, sure. (I’m not even clear what that would mean.) But the manuscripts deposited in most “Green OA” repositories are peer-reviewed: what gets deposited is usually the very final manuscript, having been revised in response to peer-review, as accepted by the journal.
I have a question, why is it so expensive open access publishing?
“Yeah, I mistakenly conflated open access with lack of peer review”
An interesting lapsus which coincides with the false equivalence “Open Source = Free Software” in the public perception of the software industry.
Open Source being the closest equivalent to Peer Review the industry has.
I wonder why this confusion still occurs so frequently, in both domains.
I published a paper in an open access journal last year. I felt the review/acceptance process was a tad hasty. It would have made me feel guilty except for an extremely helpful review I got, from a top scientist in the specific area, on a previous submission of the paper to a journal that rejected it. So I felt like the paper at least got a quality review, but I would urge careful selection of the journal before submission. There ARE indeed some good ones out there. I have been pretty impressed with Ecology and Evolution (http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)2045-7758) which I first found out about at the Evolution meetings in Norman.
p.s., I had to pay US$500 to publish it.
PLoS’ submissions and review process is absolutely awful. You would think that they would be doing a better job given the purpose they were set up with, but no, that’s not the case…
“PLoS’ submissions and review process is absolutely awful.”
Do you really mean their process? Or the treatment you got from specific editors or reviewers? PLoS ONE obviously uses a lot of different editors since it covers such a wide area (“science”), so it’s probably inevitable that they will vary in quality. For what it’s worth, my impression is that its handling of manuscripts in my field (vertebrate palaeontology) is very good — but that’s only second-hand information, as I’ve not yet submitted there myself.
There are some open access publishers who are basically crooks, e.g. Bentham Science Publishers.
There’s Beall’s List of Predatory, Open-Access Publishers, which I understand is useful.
Ecology & Evolution looks great–thanks!
There is another upside – sheer expense can be a barrier to good science journalism.
A lot of non-specialist media do not have subscriptions to the major science journals, and thus it can be difficult for reporters or even sub-editors (Who are supposed to check facts) to get ahold of the original paper.
Which means you get the odd thing that should be exciting either not reported, or misreported.
It is certainly a bizarre system at present, where taxpayer-funded academics do the work, do the refereeing and editing, and (in subjects which use LaTex) do the typesetting as well, then a publishing company dives in at the last minute and steals the copyright.
Ideally learned societies should be taking the lead in producing web-based, open-access, peer-reviewed publications. Unfortunately, since the learned societies often get a lot of their cash from journals published under the existing system, this seems unlikely to happen soon.
In my area, Statistics, we still suffer from this system, but on the other hand nearly all new statistical methods are coded up and made available to everyone via the freeware package R, which is now industry standard in academic statistics. So, if someone develops a wonderful method which is just right for your data then you can freely download software written by the person who developed it and apply it to your data, but you might not be able to read the paper where they present and explain the method.
Topical update:
Neue Zuercher Zeitung, one of the leading German language newspapers, today carries a feature on the attacks of scientific publishers, specifically Elsevier, Springer and Thieme, on Open Access and academic libraries.
In one particularly repugnant instance of short-sighted greed, Elsevier, Springer and Thieme have sued the library of the Swiss Federal Polytechnic in Zurich (the country’s largest and most prestigious scientific institution), seeking to stop the internal distribution of digital copies of papers from subscribed (!) publications. They want to replace this service with their own paid offer of €29-32 per paper instead.
http://www.nzz.ch/nachrichten/hintergrund/wissenschaft/ein_baerendienst_an_der_forschung_1.14511447.html
I won’t add much here since this has been discussed in much more detail on many physics and maths blogs.
While I agree more or less with your bullet points, the claim that in maths and physics many papers don’t undergo peer review is dubious at best.
One needs to keep several issues separate, even though there is some connection between them: a) peer review, b) public access at little or no charge, c) ability of a scientist to publish a paper at little or no charge, d) online or paper, e) profits made by publishers. It is important to keep it separate since it is more effective to concentrate on the more important points, which vary from field to field. Also, many proposals for reform throw the baby out with the bathwater.
With respect to computers, someone once said that there are lies, damn lies, and open systems. “Open access” most usually means that it is free to the reader, but the author pays a hefty fee.
In my field (astronomy, astrophysics and cosmology) the situation is much better than in some other fields. With reference to the points above: a) Scientific achievement is judged, correctly in my view, almost exclusively on the basis of peer-reviewed publications. The internet has sped up peer reviewing somewhat. b) Almost everyone puts papers on arXiv, and almost all journals allow this, including, if not the final version, a version identical in content to the final published version. Anyone can download these papers at no charge. c) While some journals have page charges, many leading journals don’t, so one can be poor and still publish. (Some journals are financed primarily by page charges and not by subscriptions. Some online-only open-access journals are financed exclusively by page charges. However, since there is arXiv, one doesn’t have to pay for this privilege.) With arXiv, everything is online. d) Paper journals still exist, usually with an online addition. It is nice if this can continue but not essential. However, I think it is important than traditional refereed journals continue, even if they aren’t on paper, since peer review both separates the wheat from the chaff and the comments can also improve a paper. e) Subscription prices to many traditional journals are OK, without the problems associated with Elsevier or Springer.
Are you tired by unfair peer review process? Read this article:
http://www.strategy-of-innovation.com/article-open-peer-review-is-finally-available-for-scientific-publications-84928130.html
Researchers paying higher fees for publishing is not something necessarily bad. Grants should include this cost when awarded.
George,
Depending on where in the world you are, it may or may not be possible to add publication costs to your grant proposals. Yet the policies of particular publishers in this regard are generally universal.
This makes the cost of publication onerous for some but not others. That sounds like a classic definition of “unfair,” does it not?