House Speaker Paul Ryan, R-Wis., and Sen. Patty Murray, D-Wash. (left), have long championed evidence-based policy. In this 2015 photo, Ryan signs legislation changing how schools are evaluated. Susan Walsh/AP
Congress Wants to Boost Agency Use of Evidence in Program Evaluation
Improving data to measure outcomes is not a pretext to cut spending, panelists say.
Congress is readying legislation to implement some recent recommendations made by the Commission on Evidence-Based Policymaking in an effort to improve data quality, not to simply target programs for elimination, officials said on Thursday.
The bipartisan movement to sharpen and better circulate data for use in federal program evaluation is up against agency cultures that are sometimes reluctant to share data and have legitimate concerns about protecting privacy, a key statistician and a congressional staffer said during a management panel at Fedstival, a conference put on by Government Executive Media Group.
House Speaker Paul Ryan, R-Wis., and Sen. Patty Murray, D-Wash., co-sponsored the legislation that established the commission in 2016. The new new legislation aims “to create a database with a lot of breadth and depth,” said Nancy Potok, chief statistician of the United States and head of the statistical and science policy branch at the Office of Management and Budget, who served on the commission.
“Not a lot of people in government have a background in program evaluation,” Potok said, citing one of several hurdles managers face in basing decisions on data and evidence.
Hence, even in times of tight budgets, agencies should set up an “independent evaluation office that is not biased toward what the outcomes should be,” Potok said, citing programs as varied as the Housing and Urban Development Department’s subsidized low-income housing and Medicaid health services. Doing so would put agencies in a better position to evaluate results.
Congress is growing more interested in evidence-based evaluations, stressed Ted McCann, the assistant to House Speaker Paul Ryan for policy. “A lot of time Congress doesn’t create clear outcomes, and often at the congressional level we can’t find out,” he said. “The agencies just don’t have the data, access is limited and agencies have different interpretations of their ability to share data.”
For example, the Bureau of Labor Statistics and the Census Bureau are prohibited by law from sharing information on businesses and individuals, Potok noted, and violations can result in prison time or a $500,000 fine. But leaving agencies to resort to studies and random control trials to evaluate programs is expensive. The better approach is to seek opportunities to standardize practices to allow the collection of “information and data that tell a story,” she said. That means bringing together such officials as the chief performance officer, chief information officer, chief evaluation officer and chief data officer to create a strategy.
The goal of the commission and the coming legislation is “not to see what doesn’t work so we can eliminate it,” said McCann, “it’s how do we get the best bang for the buck” for taxpayers, which may generate more support for programs, he said.
In the past, Potok said, agencies have been confused about the differences between performance and outcomes measures, and too often relied on inputs such as “we pushed X number of dollars out the door.”
Agencies are mission-driven and people are dedicated to their programs, Potok added. So rather than viewing evidence-based evaluation as outsiders “looking over their shoulder, shouldn’t they want to know that the program is doing what it’s supposed to do?”