The Results of Manuscript Screening Using Gen AI

At the Society for Scholarly Publishing’s Annual Meeting this year, I presented a poster that showcased our approach to solving a specific problem our client was having: too many poor -quality submissions that weren’t meeting specifications.

The growing volume of submissions, both to preprints and journals, presents a significant challenge for maintaining consistency and throughput in research integrity screening. Human screeners perform high-quality evaluations, but as submissions volumes expand, there is a need for scalable systems that surface potential integrity concerns as signals for human review.

We designed a solution that was scalable and reliable, enabling faster screening without compromising quality. This approach applies a three-tiered screening framework combining LLMs, machine learning, and rule-based logic, to analyse manuscripts across five key categories: scope validation, structural completeness, content sens

In the video below, I talk about the outcomes - 96% accuracy and 86% recall in a human annotated set. LLM-based screening effectively surfaced integrity concerns, while a conservative design prioritised flagging issues for human review, reducing missed cases. These results demonstrate real-world value, and the system's modular architecture enables adaptation across publishing contexts, from preprint servers to traditional journals.

At the Society for Scholarly Publishing’s Annual Meeting this year, I presented a poster that showcased our approach to solving a specific problem our client was having: too many poor -quality submissions that weren’t meeting specifications.

The growing volume of submissions, both to preprints and journals, presents a significant challenge for maintaining consistency and throughput in research integrity screening. Human screeners perform high-quality evaluations, but as submissions volumes expand, there is a need for scalable systems that surface potential integrity concerns as signals for human review.

We designed a solution that was scalable and reliable, enabling faster screening without compromising quality. This approach applies a three-tiered screening framework combining LLMs, machine learning, and rule-based logic, to analyse manuscripts across five key categories: scope validation, structural completeness, content sensitivity, privacy and ethics, and compliance.

In the video below, I talk about the outcomes - 96% accuracy and 86% recall in a human annotated set. LLM-based screening effectively surfaced integrity concerns, while a conservative design prioritised flagging issues for human review, reducing missed cases. These results demonstrate real-world value, and the system's modular architecture enables adaptation across publishing contexts, from preprint servers to traditional journals.

To learn more about how the product can support your manuscript screening, get in touch: publishers@enago.com