{"id":10791,"date":"2017-11-17T20:00:18","date_gmt":"2017-11-17T14:00:18","guid":{"rendered":"https:\/\/www.enago.com\/academy\/?p=10791"},"modified":"2022-10-26T09:14:06","modified_gmt":"2022-10-26T03:14:06","slug":"peer-review-biased","status":"publish","type":"post","link":"https:\/\/www.enago.com\/academy\/peer-review-biased\/","title":{"rendered":"Is Peer Review Biased?"},"content":{"rendered":"

Academic publishing relies on peer review<\/a> to ensure research quality. Is there bias in peer review? And if so, how does it affect academic research? A recent study<\/a> examined the peer review process using simulations. Some of its findings may surprise you. Professor Justin Esarey, the author of the study, set up many simulations based on various peer review processes. In each case, he assumed that there would be three peer reviewers.<\/p>\n

Studying the Peer Review Process<\/h2>\n

In the first simulation, he assumed an agreement among all peer reviewers and the editor for a paper accepted for publication. Then in the second, he assumed that if the majority of reviewers agreed that the paper should be accepted then it would be published. In the third, he assumed that the majority had to include the editor\u2019s vote. Finally, there was a model where the editor made a decision based on the average reviewer report. In this model, the editor ignored the peer reviewers\u2019 votes to accept or reject the paper.<\/p>\n

 <\/p>\n

\"\"<\/a><\/p>\n

Esarey did these four simulations twice. The first time, he used the conditions listed above. The second time he assumed that the editor would reject some papers without sending them to reviewers. In this second round of simulations, the editor rejected any paper whose quality was lower than the median paper in the population.<\/p>\n

The simulation also included readers of the journal. How would readers rank the papers accepted for publication in each case? He did a simulation involving 50,000 papers and 500 journal readers. He set the paper acceptance rate at 10%. Esarey then used a simulation to work out how readers would rate the published papers. In every case, readers ranked the papers as being in the 80th<\/sup> percentile. About 12% of papers that were not rejected by an editor and that the majority of reviewers agreed should be published were viewed as being of poor quality by readers.<\/p>\n

The Value of Editorial Intervention<\/h2>\n

Esarey\u2019s study<\/a> highlights the need for good editors. He found that the system that published papers of the highest quality used peer review but the final decision was left to the editor. The peer review reports summarized the strengths and weaknesses of the paper. Moreover, they did not include a recommendation for accepting or rejecting the manuscript.<\/p>\n

Under these conditions, only 6% of readers felt that papers in the journal were in the 65th<\/sup> percentile. If editors rejected half the papers before sending the rest to peer review, only 1% of readers felt the papers were in the 65th<\/sup> percentile. The best system is similar to ones used by journals where reviewers are asked to submit a qualitative review of the paper. In this case, reviewers do not share their opinion on the publication of the paper.<\/p>\n

In the simulations, Esarey found that peer review led to the acceptance of better papers. This shows that peer review has more value than randomly choosing which papers to publish. Papers of the highest quality\u00a0became most likely\u00a0for publication. The decision to publish a paper in the 80th<\/sup> percentile largely\u00a0depended\u00a0on luck. Publishing papers of this quality had a likelihood similar to a coin toss. This was also true for papers in the 85th<\/sup> percentile where the editor decided to reject some manuscripts without peer review.<\/p>\n

Other Forms of Bias<\/h2>\n

There are other forms of bias<\/a> that can affect peer review. Women and minorities tend to have more difficulty in funding, publication, or promotion. Andreas Neef published a study<\/a> showing women were underrepresented as editors, reviewers, and authors. This underrepresentation remained even after making adjustments for the lower participation of women in science generally.<\/p>\n

The study involved 9,000 editors and 43,000 reviewers from Frontiers journals. There was an improvement over time. Based on current trends, it would take until 2027 for female authors to become no longer underrepresented. Similarly, it would take until 2034 and 2042 before women became no longer underrepresented among reviewers and editors. Additionally, changing this situation requires helping people to identify and work against their biases.<\/p>\n

It is undeniable that peer review adds value to academic publishing. This makes it important to reduce bias in peer review. In order to effectively judge research quality, peer review reports should not include the reviewer\u2019s opinion on acceptance or rejection of the paper. The editor should take this final decision and judge academic research\u00a0by virtue of\u00a0its merits. For this\u00a0reason, anti-bias training\u00a0becomes\u00a0essential. Making these changes should ensure the publication of only the best research.<\/p>\n

Have you encountered bias during a peer review? Share your thoughts with us in the comments below!<\/p>\n

\n\t\t<\/a>\n\t\t<\/div>