Why double-blind peer review may not be as anonymous as you think: Implications for fairness and bias

A common assumption in academic publishing is that a double-blind peer review process reliably hides author identities and so reduces bias. Yet evidence and recent experiments show that anonymization is imperfect: only a small fraction of authors chose double-blind review in a large publisher study, and reviewers or algorithms can often re-identify authors from manuscripts and metadata. This matters because imperfect anonymity can preserve or even obscure sources of bias, undermining fairness in editorial decisions. This article explains what double-blind review intends to do, how anonymity breaks down in practice, the consequences for fairness, and practical steps authors, reviewers, and editors can take. Key empirical findings and recommended actions follow.

What is double-blind peer review — definition and purpose

• Definition: Double-blind peer review is a model in which reviewers do not know the authors’ identities and authors do not know reviewers’ identities. The objective is to reduce conscious and unconscious biases tied to author name, gender, affiliation, or seniority.
• When it’s used: Many journals and conferences adopt it selectively (authors may be given the option), and implementation varies by publisher and discipline.

Why anonymity breaks down — common failure modes

In practice, several predictable mechanisms reveal or allow inference of author identity:

• Metadata and file properties: Document metadata (MS Word properties, PDF creator fields) often carries author names or institutional information unless stripped. Some journals require authors to remove these fields, but checks may be inconsistent.
• Self-citations and internal references: Authors frequently cite their earlier work. Even when written in the third person, unique combinations of prior results or phrasing can identify a research group.
• Highly specialized topics and small communities: In niche fields, reviewers may know who is working on a problem and can infer authors from the topic, methods, or datasets.
• Public presence: preprints, talks, and code repositories: When authors post preprints (arXiv, bioRxiv), share code, or present preliminary results at workshops, reviewers who follow the literature can match submissions to public records.
• Writing style and reproducible signals (and algorithmic attribution): Human expertise can often guess an author. Empirical work also shows automated methods can succeed: transformer-based models have achieved high authorship-attribution accuracy in controlled settings (up to ~73% in some arXiv subsets), demonstrating that text and bibliography patterns are strong signals.
• Reviewer behavior and bidding patterns: When reviewers have access to author information (single-blind setups), they may bid differently; controlled experiments show reviewers favor papers from famous authors or top institutions, suggesting that identity information affects decisions when available.

Evidence from studies and audits — what the data show

• Uptake and outcomes: An analysis of 128,454 submissions to 25 Nature-branded journals (2015–2017) found only ~12% of authors opted for double-blind review, and double-blind submissions experienced less favorable editorial outcomes on average. This suggests both selection effects (who chooses double-blind) and systemic differences in outcomes. (arxiv.org)
• Anonymization effectiveness: A conference-focused study of anonymization practices found that 74–90% of reviews contained no correct author guess, indicating most guesses were wrong; however, experienced reviewers were more likely to guess and expert reviewers were a persistent source of identification attempts. This paints a nuanced picture: many papers remain effectively anonymous, but a meaningful minority are identifiable.
• Algorithmic threats: Recent machine-learning work shows that automated authorship attribution can be surprisingly effective, particularly when training data are large and the candidate set is limited. Such tools create a new challenge for maintaining anonymity.

Implications for fairness and bias

• Residual bias risk: If reviewers can (accurately or inaccurately) infer identity, biases tied to institution prestige, nationality, gender, or seniority can still influence decisions. Controlled experiments found that when identity is visible, reviewers favor well-known authors and institutions—an effect that can affect acceptance odds.
• Selection and signaling effects: Authors who choose double-blind (or are unable to remove identifying traces) may differ systematically from those who do not—this complicates simple comparisons of acceptance rates by review model. The observed lower success of double-blind papers in some datasets may reflect selection bias (who chooses the option) rather than inferiority of the review model itself.
• Unequal protection: Double-blind review may offer stronger protection for early-career researchers in larger fields but less protection in small, tightly connected subfields or where preprints are pervasive.

Practical steps: what authors, reviewers, and editors can do

Authors — how to minimize identifiability

• Prepare two versions of your manuscript where required: a fully anonymized version for review and a non-anonymized version for administrative files. Follow journal guidelines for self-citation wording.
• Remove file metadata before submission (File → Properties → remove personal information; export to PDF after sanitizing).
• Avoid author-identifying language in acknowledgments, dataset descriptions, acknowledgements, or provenance statements; if necessary, place provenance details in a cover letter for editors.
• If you post preprints, consider timing (for initial submission vs. post-acceptance) and whether you want to preserve double-blind integrity. If preprints are essential, declare them to editors.
• Tips checklist: sanitize metadata; redact acknowledgments; phrase self-citations in third person; submit separate title page.

Reviewers — how to preserve fairness when identity is suspected

• Declare conflicts or recuse yourself if you recognize the work and have a conflict. If recognition is partial (e.g., you suspect the group), inform the editor rather than guessing publicly in comments.
• Don’t sleuth: Review on merits. Focus assessments on methods, data, and reproducibility rather than perceived pedigree.

Editors and publishers — policy and technical changes

• Implement automated metadata checks (strip file properties at upload) and provide clear author instructions and templates for anonymized submissions.
• Train editorial staff to verify that anonymization has been applied correctly and flag submissions that cannot realistically be blinded.
• Consider mixed models: double-blind during initial review, with identity revealed only at appeal or revision, or transparent review models where reviews are signed post-acceptance. Recent trials on reviewer-anonymity in discussions indicate policy choices influence reviewer behavior and perceived safety.

How is double-blind different from other models — quick comparison

• Single-blind: reviewers know authors; faster to administer but more exposure to pedigree bias.
• Double-blind: hides identities for both sides; reduces some sources of bias but is vulnerable to the failure modes described above.
• Open review: identities are disclosed (sometimes with published reviews); increases transparency but changes incentives and may deter frank critique.

Key takeaways — what to do next

• Understand limitations: double-blind review reduces but does not eliminate identification risk.
• Take concrete steps: sanitize metadata, rephrase self-citations, and be transparent with editors about preprints.
• For editors: implement automated checks and reviewer training; monitor outcomes to detect selection biases.
• Consider broader reforms: pairing anonymization with editorial oversight, reproducibility checks, and transparent policies will deliver the best balance between fairness and accountability.