Here’s how to fix the Pentagon’s Cost Assessment and Program Evaluation office

When the House Armed Services Committee reported its massive 2024 National Defense Authorization Act by an overwhelming 58 to 1 vote, it adopted a quick, simple solution to a research problem. The Pentagon’s Office of Cost Assessment and Program Evaluation, or CAPE, had failed to endorse the congressionally mandated plan of building more amphibious warships on a quicker schedule for the Marines, something President Biden’s fiscal 2024 defense budget request had also eschewed. The House committee’s solution: abolish the agency, fire its director, and give the work to someone else, presumably more servile to the committee, in another Pentagon office.

It’s a bit awkward for a congressional committee to reach across to a separate but equal branch of government to tell people what to think, but the abolition is a recommendation that the House Armed Services Committee can make stick, given Congress’ exclusive constitutional control of funding—if the rest of the House and the Senate agree.

Instead, others in Congress should reject the House Committee’s arrogance and instead fundamentally revitalize CAPE, which—in truth—has not distinguished itself with the kind of work that continuing, even deepening, Pentagon pathologies demand. As discussed below, the fixes must go to the heart of the operation, involving the people and bureaucratic status—perhaps even location—of CAPE.

In the past, firing people and abolishing agencies for thinking what lawmakers in power believe is the wrong thing was a time-honored congressional reaction to unwanted research. In 1995, Congress abolished its own Office of Technology Assessment as one of Newt Gingrich’s so-called reforms of congressional spending.

Not coincidentally, the technology assessment office had failed to match the enthusiasm of most Republicans for a national missile defense. Not long after that, as Sen. Pete Domenici’s, R-N.M., national security analyst on the Senate Budget Committee, I was tasked to resolve a request in a letter from several Republican senators to fire a team of analysts at the Congressional Budget Office who had assessed the cost of a national missile defense significantly higher than the advocates wanted to admit to. As chairman of the Budget Committee, Domenici was in a position to make life miserable for CBO if it did not comply. To his credit, Domenici refused, even though he supported the missile defense. He knew that making CBO the plaything of congressional whims would mean the death of CBO’s credibility.

However, the problem at CAPE goes beyond the House Armed Services Committee’s crude behavior. CAPE was created by Sens. John McCain, R-Ariz. and Carl Levin, D-Mich., in the 2009 Weapon System Acquisition Reform Act to replace the previous Office of Program Analysis and Evaluation. PA&E, as it was called, had a mixed record. Under some of its directors and managers, such as Russell Murray and Thomas Christie, in the 1970s and 80s, it had a reputation for extraordinary work. But there was no serious objection to its being reorganized into CAPE as its work had declined from the office’s heyday and clearly needed revitalization. Unfortunately, no revival took place. As revealed in a 2018 RAND study, CAPE’s cost analysis was methodologically weak and persistently wrong. The organization was never the center of the badly needed, rigorous analysis of Pentagon problems that had sometimes characterized the earlier PA&E. Note, for instance, the absence of a single example of compelling analysis in a commentary by some former directors of CAPE arguing against the HASC’s recommended abolition.

Some in the Senate Armed Services Committee have serious concerns about the nature and quality of CAPE’s work. These apprehensions are described in the press as a need to “tweak” the organization, perhaps to create in it a “Competitive Analysis Cell” to provoke superior work.

The need to fix CAPE should be obvious. The question is, how?

As an assistant director in GAO’s Program Evaluation and Methodology Division, or PEMD, I had the privilege of working in an organization that had the characteristics that those interested to revitalize CAPE should foster there. Guided by an incisive and habitually frank director Eleanor Chelimsky, PEMD specialized in how to design breakthrough evaluations and then went on to perform them. In PEMD’s defense section, we executed work that others in GAO said could not be done, and once we did it, our GAO critics said our work couldn’t be right, because the Pentagon didn’t like it. When our comprehensive evaluation of the air war in Desert Storm used a unique data-based synthesis to prove the performance of many high-cost aircraft and munitions was preposterously overstated, our GAO critics argued that the proof against us was that Air Force officials told them we were wrong. The data the Air Force itself had collected, scores of pilots from the war, and hundreds of after-action and other reports all said otherwise.

Those GAO critics rarely ventured beyond a pathetically weak “this is what officials told us” methodology, and they always tolerated the Pentagon’s selective release of documents to them. To do otherwise would foster unhappy “relations with the agency,” they argued: a revealing proposition

Happily, in my opinion, more recent GAO work has shown some improvement.

The primary lessons of this history are that for better analysis of cost and programs you must have:

●      Aggressive and fearless leadership dedicated to the proposition that staff are expected to peel the onion of the research object down to the inner core. And they should expect support from management when others try to obstruct that work in any way;

●      Unrestrained organizational independence to deny others the opportunity to filter, alter, or squelch research. This includes access to all relevant documentation and outside advice and assistance, when needed, to fully achieve selected evaluation objectives, as well as the right to distribute the completed research to all relevant parties, and;

●      A highly trained professional staff that has demonstrated real evaluation or auditing skill and is free of affiliations that could compromise work. This means staff who do not have an interest, expressed or implied, toward defense corporations, and who will not be compromised by a current or future professional affiliation. This would mean that an active-duty member of a military service might be inappropriate to assess a program of his/her own service, but evaluating another service’s program might be an effective use of expertise. It could also mean that a retired military expert is preferable to an active-duty one.

Others may have other important suggestions for effective, uncompromised, independent research. If parts of the existing CAPE organization—from the bottom to top—cannot epitomize these characteristics, the need for real change should be obvious. Legislative directives to “tweak” CAPE should have these characteristics very much in mind. If the Pentagon won’t fully cooperate with a CAPE remake, the Senate Armed Services Committee “tweakers” may want to consider moving the cost analysis and program evaluation function to a newly created division in GAO or some other fully independent construct.

There is a caveat, however. Even exemplary leadership with an extraordinary staff and unfettered evaluation power are not a guarantee of ultimate success. PEMD had a long record of making bureaucratic enemies inside GAO by telling other divisions their work was inferior and by doing far better. They struck back. First, our director was fired, and when her successors proved equally supportive of work others did not want done, GAO management abolished the division.

The House Armed Services Committee campaign to punish CAPE for being insufficiently servile is congressional crudeness and arrogance at their worst. However, the work in the Senate to “tweak” CAPE into being a more successful and independent voice for uncompromised research is to be praised. The road to that end is not an easy one, and there is no guarantee that there will not be dead evaluation bodies at the end of the street. But it is an important journey nonetheless.

Winslow Wheeler was an evaluator then assistant director in GAO for nine years.  In addition, he spent 22 years working for Republican and Democratic senators and then 13 years at the Center for Defense Information, ultimately as director.