Synlett, an international chemistry journal published by Thieme, has successfully tried a new form of peer review aimed to make manuscript assessment faster and fairer. The journal’s editor in chief, Benjamin List of the Max Planck Institute for Coal Research in Germany, and his PhD student Denis Höfler have called the new method “intelligent crowd” peer review. This method allows a large number of approved referees to comment on manuscripts online.
The researchers developed this as an alternative to the traditional peer review system, used by most academic journals. According to List, the current procedure is sloppy and slow; one of his main criticisms is the limited number of referees—usually two or three—to evaluate the quality of a paper.
In the new approach, many peers can access and comment on manuscripts online within a short period, which makes the method potentially faster and more reliable. In the test case with Synlett, the editors selected 100 reviewers based on the recommendation from the editorial board and the researchers who volunteered to participate. These referees were then allowed to comment on ten submitted papers.
The Synlett team believes that the new system is more effective and can remove biased refereeing. The trial took place in 2016, with all the participating manuscripts attracting qualified comments and discussions from the crowd within a few days. This helped the editors to make faster and fairer decisions. All the comments were anonymous. This was important to ensure that the reports were straightforward and critical whenever necessary. Although this can be true, blind peer review and double-blind peer review also have disadvantages.
Other online approaches, such as post-publication peer review, work in a similar way. However, in those cases, anybody from the public can anonymously post a comment on a given manuscript. In that case, abusive and unqualified reviews cannot be completely excluded. Whereas, in intelligent crowd peer review, the editors know all the referees, so above mentioned problem can be avoided.
The publisher and Synlett’s editorial team are happy with the outcome of the trial. According to the editors of the chemistry journal, both referees and authors are satisfied with the results. Nine out of the ten manuscripts taking part in the study were finally accepted for publication. The journal would like to extend the new peer review system to all the papers it handles.