Agencies get advice on measuring performance

Federal agencies can find ways to evaluate their programs, even when measurable performance data are hard to come by, according to a recently released study by the General Accounting Office.

In 1993, Congress passed the Government Performance and Results Act, requiring agencies to submit five-year strategic plans, annual performance plans and annual performance reports showing whether goals were met.

But many agencies, particularly regulatory agencies, have problems developing performance measures that demonstrate how goals set in strategic plans are met. Congress has criticized some agencies for having vague plans that measure activities, such as contract awards, rather than the impact of those activities.

According to the report, "Program Evaluation: Studies Helped Agencies Measure or Explain Program Performance"(GGD-00-204), in-depth program evaluations allow agencies to demonstrate the broader impact of their efforts by helping program managers develop valid and reliable performance data. GAO reviewed eight agencies that had used different evaluation methods to determine if their programs were working well.

Surveying program participants to learn how they had been affected by the program is one example of how to use program evaluations, GAO found.

For example, one program within the Department of Health and Human Services aims to improve health care in underserved areas. To measure its effectiveness, HHS performs national surveys of the health centers funded under this program. The surveys reveal who is visiting the centers and the treatment they receive. HHS plans to repeat these surveys every five years to gain long-term data on the program's goals.

Evaluations help agencies improve their measurement of program performance or at least understand how it might be improved, the report said.

But performance data alone don't explain why things work or don't work, GAO said. When the Veterans Affairs Department surveyed veterans on their use of education benefits under the Montgomery GI Bill, they had to rely on supplemental interviews with recruiters and school officials to find out how improving things like communication, payment schedules and certification procedures could impact the program's effectiveness.

"The agencies whose evaluations we studied demonstrated creative ways of integrating the results of different forms of program assessment to deepen understanding of how well their programs were working," the report said.