Peer Review

Abstract

Many date editorial peer review to the 1752 Royal Society of London’s use of a “Committee on Papers” to oversee the review of text for publication in the journal Philosophical Transactions. Initially, peer review was created to help editors decide what to publish. In the twentieth century it evolved into a system in which qualified peers not only judge publication merit but also evaluate the quality of scientific work including grant applications, conference proposals, books, and academic personnel actions. Today, it is the major tool in scientific self-regulation. It is often undertaken double ‘blinded’ so that reviewers do not know the names of those they review and vice versa. Peer reviewers names for undertaking specific tasks are often expected to be confidential.

Many date editorial peer review to the 1752 Royal Society of London’s use of a “Committee on Papers” to oversee the review of text for publication in the journal Philosophical Transactions. Initially, peer review was created to help editors decide what to publish. In the twentieth century it evolved into a system in which qualified peers not only judge publication merit but also evaluate the quality of scientific work including grant applications, conference proposals, books, and academic personnel actions. Today, it is the major tool in scientific self-regulation. It is often undertaken double ‘blinded’ so that reviewers do not know the names of those they review and vice versa. Peer reviewers names for undertaking specific tasks are often expected to be confidential.

Reviews can be open, single-blind (reviewer knows author but not vice versa), or double-blind (neither knows the other). Post-publication review is now common, although the mechanisms by which it accomplished are fragmented. PubMed Commons (https://www.ncbi.nlm.nih.gov/pubmedcommons), in which comments are attached to an article’s PubMed record, is one such mechanism for post peer-review commentary. So are journals that utilize the format of target articles with extensive commentaries.

Complaints about peer review include erroneous rejection of important findings, unreliability in the detection of errors and fraud, intellectual plagiarism by reviewers, purposeful delay and undisclosed conflict of interest when reviewers and authors compete for the same funds or publications. Poor agreement among reviewers is seen as both a weakness and as a strength in bringing diverse perspectives to bear. Several kinds of reviewer bias have been noted: confirmation bias in which current beliefs are affirmed rather than challenged, publication bias for positive rather than negative outcomes or replications, bias against certain kinds of methodology (qualitative studies), and embargoing clinically important findings until all peer review is completed. (Manchikanti et al. 2015).

Two studies of peer review are helpful. A review of papers submitted to Annals of Internal Medicine, British Medical Journal, and Lancet concluded that peer review added value by filtering out submissions of poor quality but had problems dealing with exceptional or unconventional papers published later in other journals (Siler et al. 2015). A study in the social sciences found reviewers made considerable useful contributions to manuscript revision, particularly of interpretations of findings (Strang and Siler 2015).

Peer review is a prime duty of being part of a scientific community and enforcing norms of research integrity . Peer review fraud has been uncovered and dealt with. In 2015, Springer retracted 64 articles from ten different journals in which an individual invented fake email addresses and reviewed his own manuscripts (Haug 2015). Peer review will continue to be a major form of quality control in science but reviewers must disclose conflicts of interest and describe any limitations in their ability to undertake peer review to those making requests.

Advice: Expect that peer review will be imperfect but know that you can always learn from reviewers’ comments . Address them directly and explicitly when you revise a manuscript or grant application for resubmission.

Sometimes reviewer comments mean that your manuscript or application is a mismatch with a journal or funding source so you should find other alternatives. Mentors should spend time explaining how to do peer review, and if they do not, mentees should ask before undertaking peer review work.

8.1 Let’s Make Peer Review Scientific

Drummond Rennie

Rennie, D. Let’s make peer review scientific. Nature 535, 31–33 (2016). © 2016 Macmillan Publishers Limited. All rights reserved. An imprint of SpringerNature.

Illustration reproduced courtesy of David Parkins.

8.2 A Stronger Post-Publication Culture Is Needed for Better Science

Hilda Bastian

Bastian H (2014) A Stronger Post-Publication Culture Is Needed for Better Science. PLoS Med 11(12): e1001772. doi:https://doi.org/10.1371/journal.pmed.1001772

8.3 Reviewing Post-Publication Peer Review

Paul Knoepfler

Knoepfler, P. Reviewing post-publication peer review. Trends in Genetics 31(3), 221–223 (2015). © 2015 Elsevier Ltd. All rights reserved.

Reprinted with permission from Elsevier.