Shutterstock.com

Building a Culture of Informed Decision Making

A new law aims to institutionalize an evidence-driven culture in agencies. Implementation won’t be easy.

A recent article by Ed O’Brien, an associate professor at the University of Chicago’s Booth School of Business, observes that “people assume they can and will use more information to make their decisions than they actually do, according to the research.”

While this is distressing news to those who promote the use of evidence in decision making, it doesn’t have to be that way. There is an effort to actually change the culture within federal agencies to become more evidence-based in decision making. The recently signed Foundations for Evidence-Based Policymaking Act of 2018 creates new momentum for this culture change.

This new law has three distinct sections:

  • The first section creates a new capacity for, and governance of, evidence building activities and the use of data. For example, it requires agencies to designate an agency evaluation officer and create a career path or professional occupational job series for evaluators. It also requires agencies to develop evidence-building plans (sometimes called learning agendas) to address key questions policymakers want answered.
  • The second section focuses on increasing access to government data. It requires agencies to designate chief data officers, conduct inventories of the data they have, and generally make their data open by default.
  • The third section strengthens privacy and confidentiality of agency data by codifying several existing administrative practices to protect against inappropriate disclosure of personal information. But it also encourages data sharing by shifting assumptions from “no access without explicit authorization” to a “presumption of access unless prohibited.”

Shortly after the law was signed, the National Academy of Public Administration hosted a panel to discuss the new law’s implementation. What follows are highlights of that conversation.

Key Implementation Challenges

Of course, the primary locus of implementation of the new law will be at the agency and program levels, but having the top institutions aligned will be important to sustain implementation over a period of years. Experts highlight two key challenges:

  • The first is identifying the big policy questions that need to be asked by agencies.  Agencies have to define what they want to know and where the data is located that can answer these questions. The existing annual reviews of agency strategic objectives could be one venue for identifying questions, but some of the questions will reach across agency boundaries and will need cross-agency advocates to raise and answer them.
  • A second challenge is coordinating implementation across the various cross-agency management councils (such as the President’s Management Council, the Program Improvement Council, the Chief Information Officers Council, etc.) and program-level executives in the White House Office of Management and Budget as well as within the agencies. Typically, each has different perspectives and portfolios as to what the priorities should be for a common research agenda. Historically, there has been too much infighting over the control of data and turf to craft such agendas, and a new governance structure is needed at multiple levels to provide necessary coordination.

Parallel Centers of Influence

To align the top-level institutions of government around a common agenda and to create an evidence-based decision-making culture, there are three centers of influence that should be leveraged to ensure the law actually changes the culture and doesn’t just become another compliance exercise:

  • OMB must play a key role to connect people within agencies across their traditional silos and help ensure the agencies ask questions that matter. OMB is best positioned to spur a new learning culture and establish the necessary supporting systems required to develop and bring evidence to bear at all stages of policy development and implementation. For example, OMB could use the currently ongoing development of the Federal Data Strategy to help bridge institutional silos. It has done this successfully in the past: the Bush Administration’s Program Assessment Rating Tool was a good example of how to bridge the internal OMB silos.
  • Congress should reinforce the importance of the law through appropriations committee report language and other means. Authorizing committees could ask agencies how they are identifying the most important questions for which the agencies are seeking answers. Congress also should address overlapping committee jurisdictions to reduce redundancy. This has precedent. A similar coordination effort was undertaken in the mid-1990s in the House via a set of special cross-committee task forces that assessed agencies’ first strategic plans developed under the Government Performance and Results Act.
  • Outside stakeholders could help agencies focus on asking the right questions and develop the best evaluation techniques. These include state and local governments; academics and the research community, and private industry. Advocacy groups have a role as well, providing outside pressure to help sustain momentum (this happened with the implementation of the DATA Act).  

Anchoring Change Within Agencies

The agencies are where the work must be done to ensure the law does not become just another set of requirements to implement. If OMB and Congress truly want to institutionalize an evidence-driven culture in agencies, efforts will be needed to connect evidence and data to the management of actual programs, including at the state and local levels. A key piece of this will be the development of agency-level data governance that defines and leverages the roles of the newly-designated agency evaluation officers and chief data officers.

One approach might be for a third party to create a public scorecard for each agency to assess progress and allow agencies to compare themselves with their peers.  The right metrics would capture not just procedural changes but also the successful application of evidence to policy. A possible example is a scorecard developed by the nonprofit Results for America. In that scorecard, the Labor Department is one of the pioneering agencies that has made significant progress in embedding the use of evaluation and evidence into its decision-making process. Scorecards have been used in other program areas to draw attention to an initiative and create a sense of urgency for continued implementation momentum.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.