Can We Focus on What Works?

Evidence-based program management has to be better than this constant triage of failures.

Can we shift the conversation in Washington from “waste, fraud, and abuse” to “what works and let’s fund it” instead?

I attended a recent Senate hearing on wasteful spending in the federal government, and some of the witnesses pointed to examples such as the legislative requirement that the Defense Department ship coal to Germany to heat American bases there. Others pointed to failures of large-scale computer projects and the dozens of programs on the Government Accountability Office’s High Risk List.

While many of the examples were seen as shocking, there was little conversation about focusing on what works and expanding those programs.

Interestingly, there is a movement underway across the U.S. to do just that. There are advocacy groups, foundations, states and localities promoting the idea of “let’s find out what works and fund it.” Some call this “evidence-based government,” “Moneyball government,” or “pay for success.” The federal government has dipped its toes in the water, a well, with several pilot programs in various agencies and bipartisan legislation pending in Congress.

The hot, new thing that has captured the imaginations of many policy wonks is called “Pay for Success,” or in some circles, “social impact bonds.”

In 2010, the British government launched an innovative funding scheme, which it called social impact bonds, where private sector investors committed funding upfront to pay for improved social outcomes that result in public sector savings. The investors were repaid by the government only when the outcomes were determined to have been achieved.

This funding scheme has attracted substantial attention in the U.S. where it and many variations are being piloted.

What is “Pay for Success?” According to the Urban Institute, PFS is a type of performance-based contracting used to support the delivery of targeted, high-impact preventive social services, in which intervention at an early stage can reduce the need for higher-cost services in the future.

For example, experts believe that preventing asthma attacks among at-risk children reduces emergency room visits and hospitalization, which are more costly than preventive services. When the government pays for preventive services, it hopes to lower its costs. In the past, however, there was no guarantee that this investment would work. Under a PFS program, the government would pay only if the desired set of specified outcomes are achieved. Thus, the government only “pays for success.” If the program fails, private sector or foundation investors bear the costs, instead of the government. This creates a strong incentive for someone—other than the government—to watch to ensure the program is making progress.

While the concept is compelling and there is substantial public and philanthropic interest, the evidence of its success so far is not clear, largely because the concept is so new. According to the Nonprofit Finance Fund, there are seven PFS projects underway in the U.S. These projects are anticipated to operate over a 3- to 7-year period before final payments are made—and the oldest is only two years old. However, over 75 are under development.

The Potential

There are several reasons the PFS concept has captured the imagination of many policy advocates on a bipartisan basis. It creates a focus on outcomes—not programs, policies or activities. It creates access to new sources of funding, outside of constrained government budgets. And it offers strong accountability for outcomes, in that government only pays for what works, thus shifting the risk of program failure to other stakeholders.

The PFS approach also allows resources/interventions to be applied more flexibly to the root causes of problems wherever they are in the policy system. This is especially important when the root cause falls within the purview of a different agency than that of the outcome.

For example, in the Baltimore Green & Healthy Homes initiative, reducing hospital emergency admissions due to asthma attacks triggered by unhealthy home environments benefits the health care system by reducing costs. However, the Medicaid program—which would benefit from lower costs—typically cannot reimburse preventive services, such as home carpet cleaning, asbestos removal or re-housing. The PFS approach creates a new way to circumvent these programmatic restrictions by focusing on the “purchase” of an outcome–no emergency room visits by the asthmatic children—that in turn results in savings for the broader governmental system (and increases the quality of life for the affected children).

The Challenges

Because the concept is so new, there is no single, proven approach for structuring PFS projects. While this offers broad opportunities for innovation, projects to date have been administratively complex arrangements that are not yet scalable or necessarily repeatable. The administrative complexity in developing, implementing and evaluating PFS initiatives has been costly in terms of time and money. To date, it is unclear whether these costs are higher or lower than the amount of savings to be achieved.

There are lessons that can be learned from other public-private partnership initiatives—the development of the state health information exchanges, the “All Payers Claims Database,” and even public-private infrastructure and technology projects. In each of these cases, there are complexities that arise around governance (especially when government cedes operational control over projects to the private or nonprofit sectors) and accountability (when there is too great an emphasis on processes versus results).

The use of contracts, service level agreements, interim progress payments and scorecards, and the careful selection and monitoring of projects help ensure service providers are not “skimming” the target population for only those who are easiest to serve. However, these can be complicated legal agreements to craft.

Nevertheless, even with these administrative challenges and a steep learning curve, a focus on success rather than on avoiding failures has to be a more productive approach to solving public problems.

(Image via ptf01/