Nav: Home

Do think-tanks matter? A UBC professor says 'think again'

July 22, 2016

A UBC professor is suggesting government policy makers and advisors need to do a re-think when it comes to giving validity to reports coming across their desks.

Carey Doberstein, an assistant professor of political science at UBC's Okanagan campus, recently published an experimental study of public sector workers and determined that many give a written report or study purported to be from a university more credibility than one from a think-tank or advocacy group.

Doberstein conducted a randomized controlled survey experiment involving British Columbia public service staff, asking them to read and assess the credibility of various policy studies. For half of the respondents, the authorship of the studies was randomly switched but the content remained the same. Doberstein then compared the average credibility assessments between the control and experimental groups.

"There were systematic and at times extraordinarily large differences between the credibility assessments provided by these policy professionals on precisely the same policy studies, when the only part I changed was the label of who wrote it," said Doberstein. "Irrespective of the content and just by virtue of presenting it as written by an academic, the report suddenly becomes more credible in the eyes of bureaucrats."

The results surprised him, in part due to the magnitude of the differences observed. For one report, originally authored by the Fraser Institute, the credibility skyrocketed among study participants when they read the same document thinking it came from a university academic.

Another policy study, this time written by a university economist, received very high credibility assessments in the control group. But when authorship was changed to be purportedly written by the Canadian Centre for Policy Alternatives think-tank, its credibility plummeted dramatically.

"Put simply, the think-tank affiliation was a significant drag on the perceived credibility of their report and analysis," said Doberstein. The same was true for reports said to be written by research-based advocacy groups.

Some may interpret this finding positively," he said. "That analysts in government are skeptical of reports or studies that emerge from think tanks or advocacy organizations offering analysis and conclusions that tend to align with the organization's obvious ideological position."

Yet Doberstein says having a report's credibility increase simply by changing the name of the source is concerning as it can appear that policy-relevant research contained within its pages is being ignored by government policy advisors.

"We expect public servants to objectively examine the research evidence available to them," he said. "However, it seems many are taking shortcuts, and in essence giving academics a free pass."

And while this study examined the biases among policymakers in BC, Doberstein notes similar results were observed his subsequent replication experiment involving provincial policy analysts in Ontario, Saskatchewan and Newfoundland.

Doberstein's study was recently published in Policy Studies Journal.
-end-


University of British Columbia

Related Research Articles:

More Research News and Research Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: Reinvention
Change is hard, but it's also an opportunity to discover and reimagine what you thought you knew. From our economy, to music, to even ourselves–this hour TED speakers explore the power of reinvention. Guests include OK Go lead singer Damian Kulash Jr., former college gymnastics coach Valorie Kondos Field, Stockton Mayor Michael Tubbs, and entrepreneur Nick Hanauer.
Now Playing: Science for the People

#562 Superbug to Bedside
By now we're all good and scared about antibiotic resistance, one of the many things coming to get us all. But there's good news, sort of. News antibiotics are coming out! How do they get tested? What does that kind of a trial look like and how does it happen? Host Bethany Brookeshire talks with Matt McCarthy, author of "Superbugs: The Race to Stop an Epidemic", about the ins and outs of testing a new antibiotic in the hospital.
Now Playing: Radiolab

Dispatch 6: Strange Times
Covid has disrupted the most basic routines of our days and nights. But in the middle of a conversation about how to fight the virus, we find a place impervious to the stalled plans and frenetic demands of the outside world. It's a very different kind of front line, where urgent work means moving slow, and time is marked out in tiny pre-planned steps. Then, on a walk through the woods, we consider how the tempo of our lives affects our minds and discover how the beats of biology shape our bodies. This episode was produced with help from Molly Webster and Tracie Hunte. Support Radiolab today at Radiolab.org/donate.