The first thing I recommend you do after receiving and reading any reviewer comments is to close the e-mail and not
look at it again for at least 24 hours. Go hiking, play video games, take a nap, pet a cat, or whatever. Just don't think about the
paper for 24 hours. After receiving peer review you will inevitably see you something that makes you feel pissed off, and writing an
e-mail or revising a paper when you feel pissed off is not going to help you. Give yourself at least 24 hours to cool down.
Once you've finished your 24 hour break, then you can start preparing your next step. If you have a good paper and
you're also very lucky, then after your first round of peer review you might get a chance to revise and resubmit your paper. That
means you will need to respond to the comments from the peer reviewers. Even if your paper got rejected and you don't have a chance
to resubmit it, you should still think about how to respond to the reviewer comments, because you ultimately will be submitting this
paper to a different journal, and before doing that you will still need to revise it (after all, you might get the same reviewer at
the next journal you submit it to, or you might get a different reviewer who still has the same criticisms as the first reviewers
did).
This involves two things: revising the paper itself, and preparing a "response to reviewers" document. If you just
send a new manuscript, reviewers might not receive it until many months after their first review, and they may not remember what
their concerns the first time around were, and they may not be able to recognize what parts you changed and what parts are the same.
Therefore, the "response to reviewers" document is necessary to explain to the reviewers what changes you have made, and why.
When I write a "response to reviewers" document, I usually copy each comment from each reviewer, and below each
comment I write my response (including pointing out what I changed in the paper). In other words, I organize the document by
reviewer and by comment: all the comments from Reviewer 1, then all the comments from Reviewer 2, etc. Some other people instead
organize the document by topic: i.e., they identify the key topics that came up across multiple reviews (sometimes multiple
reviewers raise similar comments), and then they respond to each topic.
Sometimes the response to a reviewer's comment is very simple, along the lines of "thanks for your suggestion, we
have now done the thing you suggested, it's on page 10". Sometimes it's more complicated; for example, if a reviewer suggests doing
something but the author doesn't think it should be done, or thinks the reviewer is wrong, the author may use this
response-to-reviewers document to explain why.
In general, my rule of thumb is that almost every reviewer comment should get some corresponding change in the
manuscript, even if it's just a minor change. Sometimes a reviewer makes a comment that is objectively wrong; for example, maybe
the reviewer criticizes you for only including 50 students in your experiment, but you actually included 100 students in your
experiment. Still, even if the reviewer is objectively wrong, you should take some responsibility for it. If the reviewer
misunderstood something about your paper, that could mean that you didn't express it clearly in your paper. (Keep in mind, also,
that most scientific papers are rarely read or cited; thus, it is likely that your reviewer has read your paper more carefully
than anyone else in the world ever will. Therefore, if your reviewer could be confused by your paper, it's likely that other
readers may be confused as well.) Thus, even if I think a reviewer is being really stupid, I will usually still try to throw them
a bone and make some small change; for example, in the scenario I described above, I might respond by saying something like "In
fact, we used 100 students in our experiment, not 50; however, we recognize that this may not have been expressed clearly, so we
have revised the description on page 12 to make this clearer." Another good reason to do this is to make reviewers happy; reviewers
are human, and most humans have egos, and thus they will like to see that every comment they made caused you to make some change in
the paper; it makes the reviewers feel like they are useful.
Continue to the activities below to look at some examples of peer reviews and responses to reviews.
Most peer reviews are confidential, which means it's difficult to see examples of them, although there are starting
to be journals that publish peer reviews alongside their articles. Here is one paper of mine that has the peer
reviews (and responses to reviews) published publicly alongside it:
So you can understand the context of the peer reviews, let me give you a very brief summary of this paper (you are
also welcome to read the whole paper, but it's not necessary for this exercise). This paper is related to the
McGurk effect, which is a well-known phonetic illustion that happens when people hear sounds that don't match what
they see. For example, if you use video editing software to create a video of a person saying "ga", paired with
audio of a person saying "ba", people who watch the video will often hear "da". (You can find many good examples
on YouTube.) A few years ago, someone published a paper claiming to show that this illusion does not happen to
skilled musicians: i.e., if you show that kind of manipulated video to an experienced musician, they will not be
tricked by it. However, we believed that paper had some serious methodological flaws. Thus, we carried out a very
similar experiment, with what we considered to be some methodological improvements, and found a different result
(the McGurk effect happened to both musicians and non-musicians). We submitted this paper to a "registered reports"
format journal (see the "Publishing models and types of journals" module for more information about registered
reports). Not surprisingly, the author of the paper we criticize was one of the reviewers (indeed, we listed her as
one of the suggested reviewers), and there were two other reviewers as well.
If you go to the "Supplementary materials" section, you can click a link to see all the peer reviews and responses.
The way they are organized is weird; the first half of the document is all the review comments (from the first
round of review, then the second round of review, and then the third), and the second half of the document is all
our responses (first round, second round, etc.). Thus, our responses to a given review are like 20 pages later
than that same review. That's just how this journal uploaded the PDFs, I guess.
Remember that I mentioned previously that there are different ways to respond to a review comment. Sometimes you just
say something like "thanks for the comment, we did the thing you asked", and sometimes you argue with the reviewer
a lot. Looking at the reviewer comments and our responses to the comments, find one place where we directly did the
thing the reviewer asked, and find one place where we challenged the reviewer. You don't need to read or understand
the entire discussion, as it includes a lot of technical stuff; you can just skim to find examples of these two
different kinds of responses.
The review mentioned in the previous question is actually probably the most combative peer review I've ever been
involved in. One of the reviewers was somewhat hostile, and my responses to that reviewer were also pretty snarky
and hostile (probably unnecessarily so). Being hostile to reviewers and implying that they're stupid is not
something I would recommend; in fact I'm probably pretty lucky this paper still got accepted.
In the response-to-reviewers letter, find one response that is good and constructive, and one response that is more
hostile than necessary.
When you have finished these activities, continue to the next section of the module:
"Different peer review models".