Blog Archeology: No Decision Letter, No Peer Reviewing
Brad Delong calls this “hoisted from the archives,” which is clearly a better term for what I’m doing. But, as that’s taken and I’m not as smart as the great economics professor, I guess I’ll just have to stick with this alternative.
Peer reviewing: a call to arms (updated)
From: 22 April 2009
I just turned down a request that I review for a journal because, in part, they failed to send me an anonymized copy of the decision letter the last time I reviewed for them. And this despite the journal using an electronic review system that automates the process.
I can think of a number of reasons why all peer-reviewed journals should be required to supply reviewers with copies of their decision letters. In no particular order:
(1) It provides closure to the reviewer.
If I invested–at minimum–a few days in carefully reading an article and writing a review of anywhere from two to six pages, it seems like basic courtesy to let me know what the editors decided to do with the manuscript.
(2) It helps improve the quality of reviews.
I find reading other reviews helpful in assessing my own. Did I miss something important? How much of my opinion was shaped by my prior commitments? Did I otherwise do an adequate job of providing feedback? Was my review helpful to the editors? If I split with the other reviewers, was I able to swing the editors around to my point of view or not?
(3) It helps me with my own work.
About 50-60% of the reviews I do involve papers that intersect in some non-trivial way with my own research and writing (this is how peer-review is supposed to function). This means that I have some interest in gauging how reviewers will react to certain kinds of arguments and warrants for them. Reading the other peer reviews helps with this. And even if the manuscript isn’t related to my own areas of research, I find I still learn things about the process that can be quite helpful down the road.
UPDATE: a reader emails me a fourth reason:
(4) It keeps editors honest.
One other important reason why reviewers should see the other reviews: it keeps the editors honest. Some journals never communicates with their reviewers about the fate of manuscripts, and certainly never send around the other reviews because, if they did, then reviewers might more openly question the decision-making of the journal. Don’t want to be circulating positive reviews when a manuscript was rejected for other reasons [I've edited the email to eliminate references to a specific journal as an exemplar of these practices].
I think that’s right; for some journal editors, the arguments I made above amount bugs, not features, of providing reviewers with decision reports.
Almost all of the major North American journals in political science provide decision letters to reviewers.
The sociology journals I’ve reviewed for do as well, but, somewhat puzzlingly, send the letters via snail mail.
The European journals are much spottier in this respect. Some (*cough* Millennium *cough*) won’t even send these materials–unless requested to do so–when asking for a second-round review!
But, regardless, given the almost universal use of electronic systems for submission and review, there is simply no excuse for not providing anonymized decision letters to peer reviewers.
It seems to me that there’s only one way to ensure that journals “do their duty” on this front: refuse to review for them unless they do.
So I’m calling–right here, right now–for reviewers to boycott the holdouts.
Peer-reviewers of the world unite! You have nothing to lose but lack of closure!