Researchers report that an artificial intelligence model for estimating the replicability of scientific studies based on the articles' text and reported statistics, such as effect sizes, performed comparably to prediction markets and surveys, the most accurate methods currently used for estimating replicability, and did so without significant evidence of bias when tested on hundreds of social science studies that had been manually replicated; such a model can scale easily and be used to prioritize manual replication efforts or for self-assessment by authors.
Article #19-09046: “Estimating the deep replicability of scientific findings using human and artificial intelligence,” by Yang Yang, Wu Youyou, and Brian Uzzi.
MEDIA CONTACT: Molly Lynch, Kellogg School of Management, Northwestern University, Evanston, IL; tel: 773-505-9719; e-mail: molly@lynchgrouponline.com
###
Proceedings of the National Academy of Sciences