It’s easy to find flaws in the peer review process. It’s slow, for highly-competitive journals or conferences, the cutoff between acceptance and rejection is often almost aritrary, and it doesn’t catch many cases of bad science. But the question is, what is the alternative?

As a person who was a reviewer of many papers, I admit that there are indeed things that peer review doesn’t filter out, but know what kind of crap (sorry for the language) it does filter out. It filters out bad and even patently ignorant ideas, badly written papers, old ideas that everyone (but the author) already knows about. Basically, it leaves only *interesting* papers. If I read a peer-reviewed journal or attend a peer-reviewed conference, I know I’m not wasting my time because each one of the papers was judged by my peers to be *interesting*, so I would with high degree of probability find it interesting too. I’d hate to lose this advantage of peer review.

Another advantage of peer review is that authors make a bigger effort because of it (and I know this from my experience as an author). On your website/blog/whatever, you can send any sort of half-baked idea. If this was the accepted practice, most researchers would stop here – and not go the extra mile to better research their idea and better document it. But with peer review, authors are encouraged to improve the quality of their writing and the quality of their arguments (and tests, or whatever, depending on field), so as to convince their peers. Moreover, they know they need to work harder for more competitive journals/conferences, to be even more convincing. And authors want to reach the more competitive journals because they have more readers, and because people still use this as promotion criteria.

How can we keep these advantages without the current peer review process?

One possibility is to have a (sorry for the blashpemy) “like”-oriented, or in other words large-scale peer review process: Everybody can post *any* paper, without any process whatsoever. Readers who come across this paper can “review” this paper – grading it on quality, interestingness, novelty, truth, writing quality, and so on, and also leaving a textual review if their wish. One who wishes to find interesting papers can then find them by limiting himself by these grades.

Google Scholar’s ability to find highly-cited papers is a good start, because citations act as a good sign of someone thought this paper was important. I often use “Google Scholar” to find interesting papers, without caring which journal the result came from (or what kind of peer review process this journal used). But citation is a very crude instrument – and very slow to adapt (typically taking several years to get the first estimate on the interestingness of a paper). It would have been nice to have a much faster estimate of reader-judged quality of papers.

Nadav Har'El	May 3, 2012 at 12:24 pm