Much like what happens in the Twilight Zone, we often hear that peer review before publication is an index of quality, that it is there to serve Science. I would argue it's a cookbook.
Of course Bob, Arian and anonymous all have valid points about SL0 but the real issue is that in the pre-publication peer review model, the interesting discussion happens between the authors, some random limited number of reviewers at some random paper and most importantly, it remains secret. The pre-publication peer review process has robbed us all from the only thing we really care about: Insight, the one provided by post publication peer reviewers such as Bob, Arian and anonymous.
As I have mentioned earlier, in the post peer review process, reproducible research is a requisite whereas in a pre-publication peer review process, it is only a "nice to have" feature, something that only virtuous people will embrace.
In the prepublication peer review process, publishers are all too happy to create new journals and provide some sort of market segmentation that yields journals of varying quality bundled in one offering to universities at a steep price. In sum, prepublication peer review is the culprit and for some it's a cookbook.
Let us now imagine a new iteration of this peer review process that now happens post publication. Let us imagine that there is something on top of Arxiv. Clearly, we need something like StackOverFlow where each question is replaced by an article featured on Arxiv. The Q&A sites generally make a difference between providing an answer and providing a comment, much in the same way one would expect a difference between substantive cristicism of a paper and lighter ones. The StackOverFlow model provides a mechanism for recognizing members of that community who can then be seen as experts. Some amount of change could be made so that people, members of the site would be provided with updates on new versions of papers, anonymity for reviewers while preserving their scores, etc....
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.