Why early-career researchers should step up to the peer-review plate

The process of peer review provides a key service to the scientific enterprise.Credit: Getty

Seyedali Mirjalili had just received his master’s degree in computer science in 2011 when he received his first invitation to peer review a paper.

“I had no idea what the e-mail was about, to be honest,” recalls Mirjalili, then attending the University of Technology Malaysia in Johor Bahru. He accepted anyway, and asked the journal editor for guidance about completing the review. He hasn’t stopped reviewing since. “The more I did it, the better I got,” says Mirjalili, now a computer scientist at Torrens University Australia in Brisbane. He currently reviews one or two papers a day, and sits on the editorial board of several journals. To manage the workload, he sticks to papers that fit his expertise and therefore don’t require any further research on his part.

Reviewers contribute a key service to peer-reviewed science, catching errors or problems, helping authors to improve their work and even rejecting shoddy research. Scientists who review also benefit: they see original research before it is made public, helping them to stay up to date in their fields, and gain insight into the review process so that they can improve their own submissions.

In some ways, it is also a thankless task. “It’s unseen labour,” says Rebeccah Lijek, a molecular biologist and peer-review scholar at Mount Holyoke College in South Hadley, Massachusetts. “When you have a lot on your plate, it’s the kind of thing that can drop down on your priority list.”

The result is a system in which one-fifth of researchers contribute up to 94% of reviews1. “The literature has exploded, and there are vastly more papers than a handful of people can handle,” says Randy Schekman, a cell biologist at the University of California, Berkeley, and founding editor of the journal eLife. The COVID-19 crisis has exacerbated the problem, with a proliferation of preprints awaiting attention. According to the website RetractionWatch, more than 200 published papers relating to the pandemic have been retracted (see go.nature.com/3rhwnbj).

Journals and editors are eager to add a diverse range of scientists to their talent pools, including early- career researchers from around the world. “It’s important to get different perspectives,” says Schekman. “Senior people might have a broad view of a subject, but not be versed in the details.” Yet junior researchers can be invisible to editors because they haven’t published much or lack an online presence.

For early-career scientists, there are ways to get noticed and to learn to review. Peer-review training has not been a standard component of postgraduate curricula, but it is possible to gain know-how from a mentor or from online courses offered by several organizations, including Nature Masterclasses (part of Springer Nature, which publishes Nature). However, novice reviewers must take care to understand what’s expected and what constitutes a conflict of interest, as well as how the open peer-review movement — which seeks to make reviews and publishing decisions more transparent — might affect the anonymity of their comments.

“Reliable reviewers are always in high demand,” says Thereza Soares, a chemist at the University of São Paulo in Brazil and an editor for the Journal of Chemical Information and Modeling.

Peer pressure

Despite the need for diversity, peer review has an ongoing problem in this area. Publons, a service that gives reviewers public credit for their work through their online profiles, analysed this aspect in 2018. It found that researchers in certain nations, including several in the global south, contribute fewer reviews than do those in much of Europe, North America and Japan2. The report estimated that women are also likely to be under-represented in peer review. And when a group of researchers analysed the peer-reviewer pool in Frontiers journals, they found that women were under-represented, that male editors were more likely to appoint male reviewers and that female editors more often invited female reviewers3. Junior scientists made up just 3% of the invited reviewers in that sample.

The effects of this older, predominantly male reviewer pool trickle down into manuscript acceptance rates, according to a paper posted on the preprint server bioRxiv (and thus not yet peer reviewed itself)4. Information scientist Cassidy Sugimoto at the Georgia Institute of Technology in Atlanta and her colleagues analysed thousands of eLife submissions, and found that women, as well as researchers who were not from North America or Europe, were less likely to be editors, reviewers and authors. When editors and reviewers happened to match the gender or geography of authors, acceptance rates went up.

However, not all studies follow this pattern. The authors of a 2021 paper covering 145 journals found that papers by women were favourably reviewed overall, particularly in biomedicine, health sciences and social sciences5. Acceptance rates for manuscripts with a higher proportion of female authors were greater than for those with mostly male authorship in biomedicine, health sciences and physical sciences.

Kathrin Rousk, an ecologist at the University of Copenhagen, sometimes feels that even when she is invited to review, her opinions aren’t taken seriously because she is relatively young and female. At times, she says, when she has expressed serious reservations about a manuscript or even recommended it be rejected, the paper was nonetheless approved without many revisions. “I get the impression that the opinion doesn’t weigh as much as those of the senior male colleagues,” she says. “It’s just my feeling.”

Even the tone of reviews matters and can influence the diversity of science overall. According to a 2019 survey of more than 1,000 researchers, 58% of respondents had received a review that was unprofessional in content or tone6. Women, non-binary people and people of colour were more likely than white men to report a drop in self-confidence and disruptions to their publication rate or career advancement as a result.

As a graduate student at Pennsylvania State University in State College in 2015, Carolyn Trietsch received a harsh review of her first paper. In describing the textured back of a particular wasp, she mistakenly used the word alveolate, which means honeycombed, instead of foveolate, meaning pitted. One reviewer was irate. “He basically wrote a three-page rant,” recalls Trietsch, who now coordinates the development of multidisciplinary research proposals at the university. Trietsch was able to laugh it off — and fixed the offending vocabulary — but other scientists might not be so resilient.

“Personal attacks are never a good look” for reviewers, says Lijek. That is, reviewers should focus on the science, not the scientist.

Yet, Lijek says, the reviewer pool is probably more diverse than it seems because of the practice of ghostwriting, in which junior scientists pen reviews on a senior researcher’s behalf. Lijek and her colleagues surveyed postdocs and other early-career researchers, and found that about half had ghostwritten a review. This happened even though more than 80% of respondents thought ghostwriting without credit was unethical7.

Kathrin Rousk in the lab

Ecologist Kathrin Rousk feels her opinions in reviews are sometimes overlooked.Credit: Sidsel Kretzschmer Henriksen

The practice also means that editors can’t check for any conflicts of interest between reviewers and authors, says Lijek’s colleague, Gary McDowell, who is chief executive and founder of the research-policy consultancy firm Lightoller in Chicago, Illinois. Furthermore, ghostwriting means that unnamed reviewers don’t receive boosts to their profiles that might lead to further review opportunities.

Rousk sometimes asks postdocs or graduate students in her group to collaborate with her on reviews, so they learn how to do it. But she’s careful to ensure the journal editor is comfortable with that approach, and to be transparent about who has written the review.

Junior researchers have plenty to offer the peer-review process, McDowell says. In fact, studies find that the best reviews often come from younger scientists810. “Early-career people write very thoughtful, long reviews,” says McDowell. One advantage of including reviewers who are not faculty members is that they’re often more familiar with current experimental methods and equipment. When McDowell was a postdoc at Tufts University in Boston, Massachusetts, his supervisor looked to him to unravel technical details of papers under review and credited him when journal policy allowed.

A foot in the door

There are no hard guidelines for who’s ready to accept their first solo review assignment, although individual journals do sometimes specify reviewer criteria. Critical-thinking skills and expertise in the subject matter are crucial.

Early-career researchers should tell their supervisors they’d like to help with a future review, or express interest to other academics who are journal editors. Scientists can also reach out to editors by e-mail or at conferences. Many will appreciate the contact, says Mirjalili: “They love it.”

Trietsch received her first review invitation a few years after publishing her first paper, when she was midway through her PhD in entomology. She’d taken a professional-development course, but peer review hadn’t been covered. “I just wasn’t sure how to get started, what I should focus on, or not,” she recalls. But help was at hand; Trietsch asked a professor in her department for guidance.

“Experience in peer review doesn’t receive much focus as part of academic training,” says Benjamin Mudrak, a senior author-product manager at the American Chemical Society (ACS) who is based in Durham, North Carolina, and oversees the ACS ‘Reviewer Lab’ training course on peer review. Launched in 2017, the course offers six free modules on how to understand the peer-review process, assess a paper and write a clear, useful review. On completion, scientists can be flagged as a graduate in the ACS reviewer database.

Training and mentoring are also available through the free Web of Science Academy. “It provides the solid principles in academic peer review,” such as how to evaluate the abstract and how to determine whether the literature review is sufficiently comprehensive, says Diaa Ahmedien, a new-media arts researcher at Helwan University in Cairo.

He took the course, then called Publons Academy, in 2017 on completing his PhD at the University of Bern. For the final examination, he selected three published articles to review, and then a professor in his field testified to Publons that his work was satisfactory. As a result, his Publons profile lists him as an academy graduate. (That ‘test’ has been replaced with the Web of Science module ‘Co-reviewing with a mentor’, but still results in credit on the Publons site.) Since then, Ahmedien has completed more than 240 reviews and has become an associate editor of Humanities and Social Sciences Communications, published by Springer Nature.

The Genetics Society of America (GSA) also offers an online peer-review course and mentorship programme to early-career researchers from around the world. After training, participants enter the reviewer pool at the journal Genetics. The course runs annually and is likely to open its next application cycle in the next few months, according to GSA communications manager Jacqueline Treboschi.

The proliferation of preprints, too, provides opportunities to practise reviewing. Researchers with expertise in various fields related to the COVID-19 pandemic, from vaccinology and statistics to history, can volunteer with Rapid Reviews: COVID-19, published by MIT Press in Cambridge, Massachusetts, and edited by a team at the University of California, Berkeley. The project solicits quick-turnaround reviews for COVID-19 articles on preprint servers, such as medRxiv and bioRxiv, that are deemed by the network of reviewers to be important.

Fortunately, peer-review skills are easy to build. Ahmedien says junior researchers are likely to get a feel for the process after completing three or four reviews.

But once researchers get into the reviewer pool, another problem can arise: a deluge of requests. When Sugimoto was an assistant professor, she reviewed about one paper a month, but as her career advanced, the rate of invitations increased. “It’s not uncommon for me to get hundreds of requests per year,” says Sugimoto, now a tenured faculty member. “At a certain point, you just start saying no.”

To avoid burnout, “researchers should try to set realistic goals of how many manuscripts they can review,” says Julia Vilstrup Mouatt, head of the Web of Science Academy, who is based in Auckland, New Zealand. As a rule of thumb, some researchers try to review three papers for every one they publish.

Hazards of reviewing

Indeed, there are times when it’s best to decline or withdraw from a review opportunity — if a researcher lacks relevant expertise, for example. Avoiding conflicts of interest is also key. Such a conflict would arise if the would-be reviewer had collaborated with or worked in the same department as any author in recent years, or if the reviewer would receive financial or other benefits from the paper or review. Researchers should also avoid reviewing papers authored by friends or family members. “My colleagues, my friends, a deep personal relationship — this is a conflict of interest,” says Ahmedien. A more casual acquaintance, however, is not.

Mirjalili says it’s not acceptable to contact authors during the review process, even to ask questions or clarify a point. Instead, he says, “Get in touch with the journal”. Editors will want to track queries, and direct discussions between reviewers and authors could break confidentiality and make the review unusable.

Confidentiality policies vary by journal. The advantage of confidentiality, says Rousk, is that reviewers will be less biased by any previous knowledge of that author, their demographic or geographical location.

Aiming to improve transparency, some journals might identify the authors and reviewers to each other, or even publish reviews alongside the final paper. Such ‘open review’ has been growing in popularity over the past five years, with nearly 80% of medical and scientific journals using it at least occasionally11. One advantage is that open review is expected to make referees consider the scientific issues at hand more carefully. Open reviews tend to be shorter and nicer, says Sugimoto.

Most often, says Schekman, the reviewer will know the authors’ identities, but the authors will not know the reviewers’ — although, he adds, “they usually try to guess”.

Nature reviewers are told the names of manuscript authors, and the journal acknowledges reviewers by name in the published paper with the reviewer’s consent. In February 2020, Nature also began offering authors the option to publish anonymous reviews and author responses alongside papers.

For junior reviewers, having their name linked to a review can boost visibility. But there can be downsides. A senior scientist might take criticism poorly, and in some cases could even later damage the early-career scientist’s chances of employment, promotion or publishing their own work. “That is something that worries me intensely about the move towards open peer review,” says Sugimoto.

One option is not to take on a review under such open conditions. And it’s fine to withdraw after accepting a review offer, if issues arise. Reviewers can contact the journal editor, outside of their formal review, to share any concerns about the paper or their role in the process, says McDowell. After all, he says, editors should be eager to hear about any issues: they don’t want to approve a paper that will be panned on social media or even retracted.

That quality control, after all, is the point of the peer-review enterprise. “We do it,” says Lijek, “because we want science to be better.”