Responding to peer review (2 hours)

↵ Back to module homepage

The first thing I recommend you do after receiving and reading any reviewer comments is to close the e-mail and not look at it again for at least 24 hours. Go hiking, play video games, take a nap, pet a cat, or whatever. Just don't think about the paper for 24 hours. After receiving peer review you will inevitably see you something that makes you feel pissed off, and writing an e-mail or revising a paper when you feel pissed off is not going to help you. Give yourself at least 24 hours to cool down.

Once you've finished your 24 hour break, then you can start preparing your next step. If you have a good paper and you're also very lucky, then after your first round of peer review you might get a chance to revise and resubmit your paper. That means you will need to respond to the comments from the peer reviewers. Even if your paper got rejected and you don't have a chance to resubmit it, you should still think about how to respond to the reviewer comments, because you ultimately will be submitting this paper to a different journal, and before doing that you will still need to revise it (after all, you might get the same reviewer at the next journal you submit it to, or you might get a different reviewer who still has the same criticisms as the first reviewers did).

This involves two things: revising the paper itself, and preparing a "response to reviewers" document. If you just send a new manuscript, reviewers might not receive it until many months after their first review, and they may not remember what their concerns the first time around were, and they may not be able to recognize what parts you changed and what parts are the same. Therefore, the "response to reviewers" document is necessary to explain to the reviewers what changes you have made, and why.

When I write a "response to reviewers" document, I usually copy each comment from each reviewer, and below each comment I write my response (including pointing out what I changed in the paper). In other words, I organize the document by reviewer and by comment: all the comments from Reviewer 1, then all the comments from Reviewer 2, etc. Some other people instead organize the document by topic: i.e., they identify the key topics that came up across multiple reviews (sometimes multiple reviewers raise similar comments), and then they respond to each topic.

Sometimes the response to a reviewer's comment is very simple, along the lines of "thanks for your suggestion, we have now done the thing you suggested, it's on page 10". Sometimes it's more complicated; for example, if a reviewer suggests doing something but the author doesn't think it should be done, or thinks the reviewer is wrong, the author may use this response-to-reviewers document to explain why.

In general, my rule of thumb is that almost every reviewer comment should get some corresponding change in the manuscript, even if it's just a minor change. Sometimes a reviewer makes a comment that is objectively wrong; for example, maybe the reviewer criticizes you for only including 50 students in your experiment, but you actually included 100 students in your experiment. Still, even if the reviewer is objectively wrong, you should take some responsibility for it. If the reviewer misunderstood something about your paper, that could mean that you didn't express it clearly in your paper. (Keep in mind, also, that most scientific papers are rarely read or cited; thus, it is likely that your reviewer has read your paper more carefully than anyone else in the world ever will. Therefore, if your reviewer could be confused by your paper, it's likely that other readers may be confused as well.) Thus, even if I think a reviewer is being really stupid, I will usually still try to throw them a bone and make some small change; for example, in the scenario I described above, I might respond by saying something like "In fact, we used 100 students in our experiment, not 50; however, we recognize that this may not have been expressed clearly, so we have revised the description on page 12 to make this clearer." Another good reason to do this is to make reviewers happy; reviewers are human, and most humans have egos, and thus they will like to see that every comment they made caused you to make some change in the paper; it makes the reviewers feel like they are useful.

Continue to the activities below to look at some examples of peer reviews and responses to reviews.

Most peer reviews are confidential, which means it's difficult to see examples of them, although there are starting to be journals that publish peer reviews alongside their articles. Here is one paper of mine that has the peer reviews (and responses to reviews) published publicly alongside it:

So you can understand the context of the peer reviews, let me give you a very brief summary of this paper (you are also welcome to read the whole paper, but it's not necessary for this exercise). This paper is related to the McGurk effect, which is a well-known phonetic illustion that happens when people hear sounds that don't match what they see. For example, if you use video editing software to create a video of a person saying "ga", paired with audio of a person saying "ba", people who watch the video will often hear "da". (You can find many good examples on YouTube.) A few years ago, someone published a paper claiming to show that this illusion does not happen to skilled musicians: i.e., if you show that kind of manipulated video to an experienced musician, they will not be tricked by it. However, we believed that paper had some serious methodological flaws. Thus, we carried out a very similar experiment, with what we considered to be some methodological improvements, and found a different result (the McGurk effect happened to both musicians and non-musicians). We submitted this paper to a "registered reports" format journal (see the "Publishing models and types of journals" module for more information about registered reports). Not surprisingly, the author of the paper we criticize was one of the reviewers (indeed, we listed her as one of the suggested reviewers), and there were two other reviewers as well.

If you go to the "Supplementary materials" section, you can click a link to see all the peer reviews and responses. The way they are organized is weird; the first half of the document is all the review comments (from the first round of review, then the second round of review, and then the third), and the second half of the document is all our responses (first round, second round, etc.). Thus, our responses to a given review are like 20 pages later than that same review. That's just how this journal uploaded the PDFs, I guess.

Remember that I mentioned previously that there are different ways to respond to a review comment. Sometimes you just say something like "thanks for the comment, we did the thing you asked", and sometimes you argue with the reviewer a lot. Looking at the reviewer comments and our responses to the comments, find one place where we directly did the thing the reviewer asked, and find one place where we challenged the reviewer. You don't need to read or understand the entire discussion, as it includes a lot of technical stuff; you can just skim to find examples of these two different kinds of responses.

The review mentioned in the previous question is actually probably the most combative peer review I've ever been involved in. One of the reviewers was somewhat hostile, and my responses to that reviewer were also pretty snarky and hostile (probably unnecessarily so). Being hostile to reviewers and implying that they're stupid is not something I would recommend; in fact I'm probably pretty lucky this paper still got accepted.

In the response-to-reviewers letter, find one response that is good and constructive, and one response that is more hostile than necessary.

When you have finished these activities, continue to the next section of the module: "Different peer review models".


by Stephen Politzer-Ahles. Last modified on 2021-05-04. CC-BY-4.0.