Report charts new path for improving program performance
Efforts have focused too heavily on achieving narrow targets and earning good marks on OMB assessments, author says.
After 16 years of using performance data to better manage the federal bureaucracy, there is still no comprehensive way for the public and Congress to track how well the government is functioning, according to a new report from the IBM Center for The Business of Government.
"You would have hoped the [Obama transition] performance teams could have come in and looked at performance reports that would have let them know how kids' health had improved or how water quality had improved -- that they would look at performance trends on the matters that agencies were trying to improve," said Shelley Metzenbaum, author of the report, during an interview. "But that's not what the performance summaries do."
The Bush administration measured agencies' quality of work through achievement of 1993 Government Performance and Results Act goals, the Program Assessment Rating Tool and grades on stoplight-style score cards.
While useful for agency managers and program-level staff, these systems were not particularly valuable as a public reporting mechanism, said Metzenbaum, director of the Edward J. Collins Jr. Center for Public Management at the University of Massachusetts Boston's McCormack Graduate School of Policy Studies.
"There was no feedback mechanism to let me know that if I did something … to get to [the highest score card rating of] green, that fewer people were hungry or the air quality was cleaner or that fewer people had died in mines," she said.
The grading system, she said, focused too much on publicly embarrassing and disciplining managers of failing programs and not enough on improving outcomes and developing ways to solve problems.
Managers also failed to use performance data properly, according to Metzenbaum.
"You want people to use the data diagnostically not unlike a doctor would," she said. "The system did not encourage that. It focused more about what was on paper, which was important, but not on goal-focused, data-driven discussion."
Metzenbaum interviewed congressional appropriations committee staffers, officials with four Cabinet agencies and public interest group representatives to identify the most useful aspects of the past two governmentwide performance initiatives.
While some criticized PART and GPRA for fostering a narrow view of performance and creating unnecessary paperwork, many credited the programs for increasing transparency and accountability.
"GPRA and PART genuinely raised awareness, causing us to think differently," one agency official said. "They helped us broaden our performance measures and think about key indicators that describe program success."
But, congressional staff frequently expressed concern that agencies left out relevant information, particularly budget data, from documents. Many also had trouble understanding the paper performance reports or the online PART reviews.
"We are the primary audience for this information, yet we cannot figure it out, because the stuff is so massive and it is buried in hundreds, maybe thousands, of pages that we end up throwing out," one congressional staffer told Metzenbaum.
"We do not appropriate by goals," another Hill aide observed. "We appropriate by program. We want them to tell us information by specific program area. I wish I had the resources to go out and figure out if [this program] works. But I don't and this does not tell me what I need to know."
Metzenbaum offered 22 recommendations for the new administration. Among them, she suggested that Obama:
- Identify clear presidential and Cabinet priorities, assigning responsibility for them and meeting at least quarterly with Cabinet members to assess progress;
- Run goal-focused, data-driven meetings on his priority targets;
- Direct the new chief performance officer to encourage increased analysis of performance and other relevant data pertaining to presidential, cross-agency, agency and program goals.
- Direct agencies and programs to set targets and track specific real-world performance trends;
- Revise PART by shifting the emphasis to performance improvement and eliminating the ratings;
- Redesign the government's federal performance Web site to make it easier to find performance trends, targets and other related information.
- Revise performance trends and targets to reflect the new administration's priorities;
- Create agency-specific, Web-based performance portals, accessible through agency home pages;
- Identify key audiences for federal performance information, determining their needs and establishing priorities.