Science requires data, and survey research is one important means of gathering it. Surveys provide a scientific way of acquiring information that is used to inform policy decisions, guide political campaigns, clarify the needs of stakeholders, enhance customer service, help society understand itself, and improve the quality of life in the United States.
In recent years, concerns have been raised about growing rates of refusal to participate in surveys, as well as about inaccurate forecasts in high-profile elections, polls with contradictory findings, the declining trust in government and media institutions that fund such research, and skepticism fueled by political polarization. “Although polling is not irredeemably broken,” the authors of a new article write, “changes in technology and society create challenges that, if not addressed well, can threaten the quality of election polls and other important surveys on topics such as the economy.”
In this article, published in the journal PNAS Nexus , 20 experts from diverse fields – including academia, science, government, nonprofits, and the private sector – offer a dozen recommendations to improve the accuracy and trustworthiness of surveys.
The authors’ recommendations aim to better align the practices of survey research with three scientific norms: transparency , clarity , and correcting the record . Among the recommendations are that surveyors and survey researchers should:
“If survey panelists’ responses have been potentially biased by responses to earlier surveys or questionnaires, researchers and readers need to know that,” said co-author Kathleen Hall Jamieson, director of the Annenberg Public Policy Center of the University of Pennsylvania.
The authors also recommend the creation of a publicly available, professionally curated archive of identified technical problems in polling tools and their remedies. And they recommend that professional organizations and universities develop and disseminate a guide to survey research for use in high school courses as well as a brief “users guide” designed for laypeople and journalists who do not have training in statistics or survey research.
“Around the world, survey research produces vital information,” said co-author Arthur Lupia, a distinguished professor at the University of Michigan. “These new recommendations have the potential to increase the quality of future surveys for generations to come.”
See the full list of recommendations online .
The recommendations emerged from a virtual two-day retreat in November 2021 convened by Marcia McNutt, president of the National Academy of Sciences , and cohosted by The Annenberg Foundation Trust at Sunnylands and the Annenberg Public Policy Center (APPC). The sessions were coordinated by Lupia and Jamieson, who also is the Sunnylands program director. Among the participants were the authors of this paper, who include past and current editors of major academic journals, past presidents of the American Political Science Association and the American Association for Public Opinion Research (AAPOR), a past director of the U.S. Census Bureau, scholars who have led some of the nation’s largest university-based election surveys, and individuals responsible for the creation and maintenance of major government and nonprofit survey datasets.
Read more about this retreat and other NAS-Sunnylands-APPC retreats on APPC's website.
The authors of the PNAS Nexus article are:
“Protecting the integrity of survey research” was published online as an open access article in PNAS Nexus on March 28, 2023. DOI: 10.1093/pnasnexus/pgad049.
The Annenberg Public Policy Center was established in 1993 to educate the public and policy makers about communication’s role in advancing public understanding of political, science, and health issues at the local, state, and federal levels.
PNAS Nexus
Commentary/editorial
Protecting the integrity of survey research
28-Mar-2023
Competing interest statement: In addition to his Stanford University affiliation, Doug Rivers is Chief Scientist at YouGov. David Dutwin is senior vice president for NORC at the University of Chicago, a nonpartisan survey and social research organization, and was president of AAPOR in 2018-19. Gary Langer is founder and president of a for-profit company that provides survey research design, management and analysis services to nonprofits, foundations, businesses, and government agencies, as well as a sister company that provides knowledge management software to survey practitioners. Langer also is vice chair of the Roper Center for Public Opinion Research and lead author of its transparency policy, implemented in 2018. Marcia K. McNutt is the President of the National Academy of Sciences. The other authors report no competing interests.