When Solid Data Isn’t Enough

It takes more than accurate information to bridge the gap between evidence and policy.

I’ll never forget an incident early in my career, working in the Texas legislature, when a pair of academics testified about their statistical analysis for improving the funding distribution for school bus grants to school districts across the state. They used regression analyses to demonstrate a more efficient and fairer way to allocate the aid. But a state legislator was enraged. He asked how he was supposed to explain “this statistical mumbo-jumbo” (actually, his language was more colorful) to parents in his school district when they complained about why their kids couldn’t ride a bus to school.

It’s not just an academic vs. policymaker divide. In looking back, the financial information generated by the 1990 Chief Financial Officers Act and the performance information generated by the 1993 Government Performance and Results Act have had little impact on policy decisions, especially in Congress. The lesson: having technical experts in agencies creating a supply of information is not enough; there needs to be a demand for it, as well.

Up until recently, academics and scientists also felt that creating a supply and improving the quality of studies and research would be enough for policymakers to pay more attention to their work. Their goal was to reduce uncertainty, so legislators and others could make more informed decisions.

Bridging a Culture Gap 

A new pair of articles in a recent issue of Public Administration Review shows that reducing uncertainty isn’t enough, any more than it has been for financial or performance information.

In the first article, researchers Paul Cairney, Kathryn Oliver and Adam Wellstead say:

“Improving the supply of evidence helps reduce scientific and policymaker uncertainty.  However, policymakers also combine their beliefs with limited evidence to reduce ambiguity” when choosing a course of action to solve a problem. The researchers, therefore, explored whether there are solutions to closing the “evidence-policy” gap and “maximize the uptake of scientific evidence within government.”

They argue that academics and scientists need to also factor in coalition building and persuasion. After all, lawmakers want “to reduce ambiguity by establishing a dominant way to ‘frame’ policy problems.” That is, having a point of view, or an ideological solution, simplifies decision making for them. As a result, the researchers say it is important for advocates of scientific evidence to understand this and engage in the process of persuasion, and “combine evidence with simple stories to exploit the emotional and ideological biases of policy makers.”

There are differences between academic and political cultures: language and jargon, longer scientific timescales, low incentives to engage, differing perceptions of scientific knowledge.  Scientists like to constantly challenge evidence to ensure it is robust; policymakers feel a need to create policy certainty and “reconcile evidence with well-established beliefs,” note the researchers.

The researchers think that a potential solution to this culture gap is to use the successful experiences of health and environmental scientists, who have learned to package their evidence in ways that are easy to understand. Oftentimes, they “foster networks with policy makers and encourage academic-practitioner workshops; use intermediaries such as ‘knowledge brokers;’ recognize the importance of timing; and encourage policy maker skills or better government understanding of problems.” The article’s authors found that scientific knowledge only helps to reduce uncertainty. It does not address other dimensions that policy makers consider, “such as the inability to completely separate empirical facts from human values and ways of thinking.”

Using the Tools of Political Policy Experts

The researchers suggest that academics and scientists adopt the tools of political theorists, where “The focus of policy theory is on the links between evidence, persuasion, and framing (in the wider context of a tendency for certain beliefs to dominate discussion).”

They offer the following four strategies to increase demand for scientific evidence:

  • Combine facts with emotional appeals.
  • Tell simple stories that are easy to understand.
  • Interpret new evidence through the lens of the preexisting beliefs.
  • Offer policy solutions that are feasible.

They say that scientists have a difficult choice to make: maintain an image of academic objectivity, or engage in advocacy to increase their impact. “Without such advocacy, policymakers will rely on highly biased and persuasive stories from vested interests rather than struggle through inaccessible and equivocal scientific papers,” they wrote.

In conclusion, the article’s authors recommend academics and scientists make efforts to:

  • Promote networks between communities of scientists and policymakers to allow both sides to understand how the other camp operates and thinks.
  • Reward academics for doing relevant research and engage them in agenda setting.
  • Use generalist knowledge brokers who direct questions appropriately and have a good relationship with both policymakers and academics.
  • Measure the practical value of academic research on policy.

A Shared Responsibility 

In the second article, Max Arinder, an experienced government executive formerly with the Mississippi Joint Legislative Committee on Performance Evaluation and Expenditure Review, says that getting policy makers to use scientific evidence when making decisions isn’t a one-sided responsibility. Both sides need to be open. He suggests there also needs to be a culture shift in policymaking institutions themselves. “Policymakers often balance evidence, emotions, and beliefs in making a decision,” he said. The question is how to create an optimum balance. “Constituent opinion is a strong voice and will certainly be a part of any legislative arena. But how well informed is that voice?”

Arinder noted that “Just as constituent opinion is important in policymaking, [a sound technical staff] is also an important element in determining the policy culture of a legislature.”

If would help if policymakers developed “a transparent, evidence-informed process that increases public knowledge,” he said. Arinder offers an example: In 2015, Mississippi legislative leaders created a system to review requests for new programs and funding using an “evidence screen.” He said this shows that it’s possible to “make research-sensitive strategies a more important part of our core value system in the selection and funding of policy initiatives and programs.” The Pew Charitable Trusts is sponsoring similar efforts to expand the use of empirical evidence in other state legislatures.

Both articles show how all stakeholders need to engage to shape an environment where the demand for evidence becomes part of the policymaking culture. While decisions may still be based on other considerations, the supporting data will be available and transparent, and hopefully create a better-informed electorate.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.