How Should Peer Reviewers Be Evaluated?
Peer Review: A Forgotten Service?
The peer review process in academic journal publishing only seems to get attention when something goes wrong. From the perspective of an anxious author awaiting an answer on the submission of his or her research paper, any delay must be the fault of the peer reviewers taking too long to turn the review around. If a journal is forced to retract a research paper, the first assumption is that something was missed in the peer review, even if it turns out that the entire study was fabricated.
These common assumptions bring into question how a process that forms such an integral part of a highly regarded system based on professionalism and trust can be held in such low esteem? If a peer review can be the last line of defense against a potentially fraudulent study, why do reviewers remain unpaid? Why is the peer review process allowed to take the blame for a sclerotic publishing practice that can take up to two years from submission to publication? Are researchers supposed to believe that all of that time was peer review?
Exclusivity on Demand
Just when you have developed a picture of non-descript reviewers locked away in dungeons with ever increasing piles of articles and research papers to review, journal editors can decide to rapidly escalate the perceived value of a peer reviewer by requiring an exclusive contract that forbids that reviewer from working for any other journals. In addition, when an outside peer review service starts to submit articles for their authors with an accompanying peer review by suitably qualified experts, those same journal editors will immediately reserve the right to request a separate internal review by their “specialist” peer reviewers who are more closely familiar with the journal’s unique processes.
A Different Model
As long as the perceived cachet of peer reviewing for a prestigious journal adds points to a researcher’s resume, this system is likely to persist, but the model appears to be changing. Some companies are now offering to take the “headache” of peer review away from the already overworked journal editors, but at this point the compensation to the peer reviewers is only a modest honorarium rather than a market rate for the work performed.
Other journals are being launched with compensation for peer reviewers to build consistency of service. In January 2015, Collabra, an open access online journal launched with an Article Processing Charge (APC) of $875, of which $250 is placed into a “research community fund” to pay reviewers and editors.
Pay for Performance
Once the dam of payment for peer review services is fully breached, editors will face a new set of problems. Should senior or more experience reviewers be paid more? Will a flat rate per review promote rushed reviews as reviewers try to boost their paychecks? Will journals need to introduce mystery shopper programs to submit rigged articles to test the expertise of their reviewers on a regular basis?
Better still, develop a training program where the work of new reviewers is overseen by more experience members of your team. Whatever options are selected, treating peer reviewers as valued members of the journal editorial team rather than unpaid and highly dedicated volunteers will be a significant improvement for the industry as a whole.