Published in Nature , this scientific scrutiny led by Abel Brodeur, a professor of economics at the University of Ottawa , came alongside a massive seven-year international project assessing whether studies held up over time, particularly those without a universal scientific score. That study – the Systematizing Confidence in Open Research and Evidence (SCORE) project – reviewed nearly 4,000 social-science papers to find researchers couldn’t replicate the results in half of the studies they tested.
Brodeur’s work, however, provided more encouraging results.
Optimistic Findings
Brodeur employed a dual strategy. His team scrutinized papers at single-day events between 2022 and 2023. He ultimately reviewed 110 articles to discover 85 percent were computationally reproducible, an encouraging score amid a burgeoning climate of public distrust in scientific findings.
“We now have systematic, large-scale evidence on how reliable social science research really is,” explains Brodeur, founder of the Institute for Replication at the University of Ottawa. “The direct impact of this will see encouragement for higher research standards like better coding, data sharing, and general transparency so errors can be identified before they influence policy. Indirect impacts should lead to an improvement of trust in science by showing transparency and self-correction to help policymakers rely on more robust evidence.”
Improved disclosure methods
This first large-scale, coordinated effort to systematically reproduce and replicate results from top economics and political science papers found improved norms for disclosure of data and code for Brodeur’ studies compared to SCORE’s, whose papers spanned 2009 to 2018. Brodeur believes reproducing the original authors’ work should become routine and not exceptional.
“Future research should aim to draw conclusions about reproducibility and replicability more broadly by reproducing and replicating a random sample of papers from journals that do and do not have a data availability policy,” he says. “Our project has the potential to advance science and improve equity issues. The posting of data and code and its re-analysis are likely to advance science not only through course correction but also through learning and understanding new approaches more quickly.”
Leading journals should be leaning into open science and data sharing moving forward with so much public policy and everyday decisions relying on academic research. Reviewing results with the aid of open-source software can also help level the playing field, according to Brodeur.
“This would allow researchers from lower-level universities, those in developing nations, and others who cannot afford expensive licenses to learn from elite scholars,” he says. “The scale of this ongoing project has the potential to change research norms and researchers’ behavior.”
Nature
Data/statistical analysis
Not applicable
Reproducibility and robustness of economics and political science research
1-Apr-2026