Basics of peer review

Basics of peer review (1 hour)

When you submit a paper to a journal, it generally gets checked by administrative staff (to make sure that it meets the basic rules—e.g., it's written in the proper format, it's within the word limit, etc.), and then may get quickly checked by the editor of the journal (to see if it fits within the journal's scope). Some papers get immediately rejected or sent back to the author at this stage, either because they don't meet the formatting requirements, or because the editor deems the paper has little chance of getting published. If papers don't get rejected at this stage, they advance to peer review. The following things happen during peer review:

The editor will invite various people, who are experts in one or more of the areas covered in your paper, to read and review your paper. Lots of people are busy or have some other reason they can't review your paper. Therefore, the editor may spend a long time just to look for reviewers. At most journals, a paper will reviewed by 2-3 reviewers; however, the editor might contact, and get turned down by, 10-20 people before they find 2-3 who are able to review. These reviewers will generally be people who have published papers on the same or similar topics as your paper. However, since much research is interdisciplinary, a reviewer may have expertise in only some aspects of your paper. For example, if you use an advanced statistical method to examine a novel problem in second language learning, you might get one reviewer who is an expert in second language learning but not in statistics, and another reviewer who is an expert in statistics but not in second language learning.

The reviewers will take time to read your paper and write a recommendation to the editor. The amount of time taken differs across journals and across fields—some journals give reviewers a deadline of 2 weeks to complete their review, some give 2 months, etc. Some reviewers are very punctual, and some reviewers always submit their reviews late.

The editor will receive the reviews and make a decision on your paper. There are several possible outcomes:

Accept outright: this basically never happens.
Minor revisions: The editor asks for you to make some minor changes to the paper. At this point, it is likely that the paper will be accepted. It is very rare to receive this outcome on your first submission; it usually happens after one or more rounds of revision (see below).
Major revisions: The editor asks for you to make some major changes to the paper. "Major revision" usually is something that requires new research (e.g., collecting more data, running a second study, or doing some complicated new analysis), unlike "minor revision" which is usually just changing the writing within the paper. Some journals use the term "revise & resubmit" or "reject & resubmit" instead of "major revision". These are all the same thing: they all mean that your paper is not accepted now but you're getting a chance to revise it and get it reviewed again.
Reject outright: The editor rejects your paper and you do not get a chance to resubmit it to this journal.

"Major revision" and "reject outright" are by far the most common outcomes the first time a given paper is submitted.
Note that peer review is not a vote. Each reviewer does not just make a recommendation, but also makes comments about the paper. The editor will weigh the comments according to a subjective judgment of their seriousness. Thus it's possible, for example, to get two reviewers recommending "accept" and one reviewer recommending "reject" and the editor may still choose to reject, if the negative reviewer's comments were more well-justified and more serious than the positive reviewers' comments.

Along with the decision, the editor will usually send the comments that the reviewers wrote. (There are a few journals that don't do this, and only give you a summary instead; but those journals are rare in linguistics and psychology.) Reviewers differ greatly in their style. Some of the differences you may see between reviewers include the following:

Level of detail: Some reviewers write very brief or even lazy reviews (like "looks pretty good to me"), and some write very detailed, pages-long reviews.
Explanations and justifications: Some reviewers just make comments without explaining them much (like "you should use this different analysis"), while others offer lots of justification (like "the problem with your current analysis is.... therefore, you should use this other analysis because...").
Focus: Some reviewers believe that their main responsibility is to make a recommendation to the editor, and thus they don't offer much feedback for you directly (and they may refer to you as "the author" instead of "you"; i.e., they may say "the authors claim..." instead of "you claim..."). Other reviewers believe that their main responsibility is to give you feedback and help improve the paper, and thus they may offer more explanation for their comments, and they may address their comments to you rather than to the editor.
Organization: Some reviewers organize their review in a particular format (I like to include a brief summary of the paper—to let the authors see if I understood the paper properly or if I got the wrong idea, which might mean they need to make things clearer—followed by Major Comments, followed by Minor Comments; sometimes I include further subdivisions, like Major Comments About Methodology vs. Major Conceptual Concerns, Minor Comments About Statistics vs. Minor Comments About Writing, etc.), and some reviewers just put all their comments in a list.

Continue to the activities below to investigate more information about peer review.

How long does peer review usually take?

This is hard to estimate. As described above, there are many sources of variation. It may take editors more or less time to find reviewers in the first place. Once they find reviewers, some journals have a very short timeline for the reviewers to submit their reviews, and some have a longer timeline. Some reviewers follow that timeline, whereas some always miss the deadline. Some editors are very proactive in chasing down late reviewers, and some are not. For that reason, it's very hard to predict how long any given review may take.

That being said, different fields have different standards. In some fields it is typical for review to take 2-3 months (there will, of course, be a lot of variation around that average), and in others it is typical for review to take a year.

Investigate how long peer review typically takes in your field. You can try searching websites like https://scirev.org or https://www.letpub.com, which compile average peer review times for many journals. Or you can ask around among professors, post-docs, or other students who have published some papers, and who may have a good idea how long things usually take.

How long can you expect a typical peer review to take in your discipline?

When you submit a paper, you are often asked to suggest some reviewers. To get an idea whom to invite to review your paper, journal editors will look at your suggestions, as well as your bibliography, and they may do their own literature search. They may not invite all of the people you suggest; indeed, most editors specifically aim to get some reviews from the people you suggested and some reviews from the people you did not suggest. For example, if there is someone who has published many papers in the same topics as yours and you did not suggest them, the editor may invite them anyway. If your paper strongly criticizes a particular previous paper, the editor might invite the author of that paper to be your reviewer, even if you did not suggest them.

Generally, if you suggest a reviewer who is a very big name in the field (like Chomsky or something), it is unlikely that your paper will be reviewed by them; those people are simply too busy, and get too many requests to review, so they don't have time to review every paper that is sent to them. It is possible your paper might get reviewed by one of their students or peers (if people don't have time to review, they often offer the editor some names of other people who could do it).

Think about your own current or proposed doctoral research. Imagine we are three or four years in the future and you have finished your study, written a paper about it, and are about to submit it to a journal. Who might you suggest to review your papers? List 3-5 names here.

When you have finished these activities, continue to the next section of the module: "Responding to peer review".

by Stephen Politzer-Ahles. Last modified on 2021-05-04. CC-BY-4.0.