When Natalie Tindall posted a link to Eric Schneiderhan's Chronicle article on mean journal reviewers, I immediately knew I had to blog about it. As a grad student, I thought all reviewers were mean, just for pointing out flaws in my work. Today I know better, and in my own work and as editor of the Journal of Public Relations Research I've now seen reviews that run the gamut from too nice to too mean.
First of all, I have to ask what constitutes "mean." Schneiderhan points to several mean comments on one of his own papers as an example:
My prose was "passable." I bordered on being "uninformed." The reviewer, in response to the question of "significance of content," checked "low" (the worst one).
Is he suggesting reviewers should never check "low" because it's "the worst one"? If a reviewer thinks a paper's contribution is not highly signficant, an editor needs to know that -- we don't need journal-reviewing grade inflation. The whole reason we use multiple reviewers is so that the editor gets multiple perspectives, and one "low" rating alone wouldn't be enough to reject a paper. I'm not convinced that "passable" is mean, either.
At any rate, we all know that there ARE mean reviewers who insult, name-call, or trivialize another scholar's work instead of offering genuinely constructive criticism that can help improve the body of knowledge. I've had only a few instances of this type of review out of the hundreds of manuscripts each reviewed by 2-3 scholars since 2009. The vast, vast majority of JPRR reviewers make a genuine effort to offer honest feedback and to provide advice on revisions even for a manuscript that they believe has little to offer. And the reviewers are cognizant of authors' feelings, sometimes even telling me in the "comments to the editor" box to edit their remarks if they seem too harsh. Typically these reviews are quite negative, but I don't see that as inherently mean.
Schneiderhan suggests that reviewers are mean because it's payback time, anonymity breeds trolls, or editors are too timid. Perhaps there is an element of payback, but I can't know that as editor. As for anonymity, Schneiderhan forgets that there is one person who knows who wrote a review: the editor. If anyone submits truly mean reviews, not just negative ones, with no redeeming qualities, I won't invite them to review again. Maybe not all fields are this way, but I think most people in public relations see it as an honor to serve on the editorial board or review for JPRR (and if they don't, they won't accept the invitation to review). As for his suggestion of revealing the names of reviewers after the final decision, there are some journals experimenting with getting rid of blind review... that's a whole other kettle of fish. Let me just say that revealing the names of tenured faculty but not untenured faculty, as he recommends, would be a nightmare.
As for the "timidity" charge, it's more complicated than Schneiderhan realizes. As he recommends, I have on occasion edited a review to remove a flippant remark or an insulting word, particularly if I think the reviewer may not realize how mean they sound or if they're just frustrated by the quality of the submission. In other instances, though, I choose to leave a mean word intact. For example, once a JPRR reviewer called an author's work "shotty." Of course they meant "shoddy"-- not just mean but incorrect -- and I left it in the review because I thought it would reveal something of the reviewer's state of mind and would allow the author to judge the reviewer's work when deciding how to revise the work for resubmission or submission elsewhere. (Note: that reviewer isn't usually mean.) I generally try to err on the side of NOT editing a review too much because I'd rather have the reviewer see what I saw when making the editorial decision and because I think editing reviews is a slippery slope.
Far more troubling than words like "bordering on uninformed" or "passable" is Schneiderhan's other remark about his mean reviewer:
None of the reviewer's comments were helpful in guiding me on how to make the article better.
This does happen occasionally at JPRR. A reviewer is rushed, frustrated, or just doesn't care for the paper, and doesn't spend enough time explaining what they didn't like and how it could be fixed. But again, this is why we use multiple reviewers: one person's bad day doesn't determine the fate of a manuscript. And let's not forget those reviewers who are too nice. On two occasions I received single-sentence reviews along the lines of "this is good, you should publish it." You can bet that both of those reviews were rescinded just like a too-mean review. In neither instance is the author or the journal served.
I've heard horror stories about mean reviews, and I've seen unfair reviews and just plain bad reviews, but the view from my desk reveals many smart, thoughtful and generous reviewers who should be commended for their service to the field.