GovernmentExecutive.com

Industry groups advocated limiting the use of a certain method of measuring contractor performance in comments submitted to the councils responsible for the Federal Acquisition Regulation.

The method, known as earned-value management, relies on principles from calculus to measure contractors' performance. It compares expected cost, actual cost and work completed, and involves extra data collection both before and after an award is made.

The Office of Management and Budget requires agencies to use the technique in large contracts for development projects. The proposed rule standardizes how it should be done.

The Information Technology Association of America and Professional Services Council, which represent contractors, said the rule, if left unchanged, would impose costs on small businesses that would prevent them from competing, and make sensitive information available to competitors.

"Right now, these small companies do not have the funds or the capital to put up the money or personnel to do these reviews in advance of the award," said Trey Hodgkins, director of defense programs at ITAA.

He said the rule should be changed to require agencies to reimburse contractors for their extra efforts related to EVM. "Without that, small businesses can't compete," he said. "We're asking agencies to absorb the costs for the data they're asking for," he said.

Hodgkins also said he was concerned that auditing rights in the proposed rule were too expansive. "The concern is that information that is proprietary could unfortunately be deemed to be less than proprietary by someone who is auditing, and inadvertently disclose it to competitors," he said.

Agency comments generally supported the new rule while acknowledging the cost issue. Dave Platts, a manager for earned-value management at the Defense Contract Management Agency, said standardizing the use of the technique was a "great" idea. He said the pre-award data collection may be costly, but said it "may provide a good return on investment."

Charles Hurley, an earned-value management advocate at DCMA, said agencies should not be allowed to conduct certain intensive reviews prior to award. "I believe it is unreasonable to require [contractors] to prepare for and support a [review] prior to contract award because it places an additional cost burden on offerers and also on the government agency program managers," he said.

He also said that performing reviews for large weapon programs, for example, could delay the award of contracts for much-needed supplies.

The Contract Services Association, another industry group, urged the FAR councils to hold a public hearing or issue a second proposed rule after considering the public comments. "It would be counterproductive to use [earned-valued management] in a 'gotcha' mentality," the group said.

The group also said the performance measurement technique "raises the bar in terms of providing visibility into whether a program is on target with respect to cost, schedule and technical performance."

COMMENTS

• There seem to two main objections to the proposed rule. The first deals with the need to have (and be able to demonstrate that you have) a system that is in compliance with the EVMS requirement prior to award. If we accept the premise that EVMS will be required during the performance of the contract, it would appear to be a high risk to assume that the system will be developed after award without impacting the likelihood of successful performance. In the alternate, the system may be created but too late to actually be used in managing the project, but still provided to “meet the requirement,” giving us “two sets of books”. If the government is convinced that EVMS is a necessary part of successful programs, it would seem prudent for contractors to begin adapting their systems to the requirement, making the pre-award response of minimal impact. As Wayne Abba points out, there is a reasonable breadth of solutions that will meet the EVMS criteria and the needs of the individual companies. The second objection, to having the IBR prior to award, would seem to be more valid. The key is the nature of the pre-award IBR. In other jurisdictions, notably Australia and Great Britain, the focus of the initial IBR is to confirm the existence of high level WBS, integrated schedules and resource estimates. The objective is to determine that, at the top level, there is commonality of understanding between the government and the contractor of the scope of work and the major elements of it and that the proposal is based on a good understanding of the project objectives.. After award, when the contractor has extended the project planning to a level needed to manage the program, a detailed IBR is accomplished to allow both parties to agree on the details of the effort. It may be of value to consider the benefits that these other jurisdictions have seen form this approach and determine its value in US contracts. Tom McCann Posted August 11, 2005 2:24 PM
• There appears to be little objection from industry concerning the merits of forward planning techniques such as Earned Value Management. The predominant objection from industry appears to be that the audit rights imposed by the new rule are too expansive and may result in inadvertent disclosure of proprietary information. Earned Value management reviews have been conducted in the defense industry since the mid-60s. I am not personally aware of a single case concerning the disclosure of proprietary information arising from these reviews. On the other hand, there has been considerable number of instances where Government reviewers have been accused of over zealous interpretation of the standards resulting in excessive delays and costing to the programs being reviewed. If the application of these requirements is to work in civilian agencies it is necessary to adopt review methodologies that avoid the shortcomings of the past. This is particularly true since civilian programs are much more diverse in both size and nature than those of past defense programs. Data integrity reviews should be the responsibility of the client, are much less intrusive and can be conducted at arms length. System reviews should be responsibility of the supplier. Supplier system reviewers must focus both on the integrity of performance data generated from these systems as well as adherence to standards. They must truly embrace the scalability aspects of the relevant standards and tailor the requirements to meet the individual programs at hand. Industry should assume the responsibility of reviewing and maintaining their own systems through internal or third party assurance regimes so long as the integrated data has integrity. There already exists a professional third party assurance regime for Earned Value Management Systems that is obligated to protect proprietary information and will provide both industry and governmental organisations with reliable system assessment which, in turn, results in confidence in the integrated cost, schedule and performance data. Lloyd L Carter Posted August 9, 2005 9:51 PM
• Your story about the proposed rule on earned value management explains that EVM "relies on calculus to measure contractors' performance" and suggests it is too costly for small businesses. EVM brings to program and contract management greater accountability for resources than other techniques, but the marginal costs are more than balanced by better insight into contract cost, schedule and technical performance and by more reliable estimates of cost at completion. Calculus? Hardly. EVM calculations are based on simple ratios - the amount (measured in hours, dollars or any measurable unit) of work accomplished vs. the amount planned for the same work and the amount of work accomplished vs. its actual cost. And what is the marginal cost of generating the data? The EVM standard says "Different companies must ... establish and apply a management system that suits their management style and business environment. The system must, first and foremost, meet company needs and good business practices." And "EVMS scalability is viewed as a spectrum employing the principles of EVMS as fundamental to all programs and the EVMS guidelines as applicable to large and/or high risk programs, allowing any program regardless of size and complexity to realize the benefits of earned value management." In other words, EVM is not "one size fits all" - so companies that are concerned about the cost should take a closer look. They might discover, as have many companies around the world, that investing in EVM makes them more competitive. The proposed rule concerning "pre-award data collection" is not EVM per se, but a planning and risk reduction technique known as the integrated baseline review. An IBR indeed might result in more comprehensive planning and communication with the customer before and immediately after contract award, but I would be hard-pressed to find the downside. Most government agencies are woefully poor at planning. Spending more front-end time in planning pays off in contract execution. In use since the 1960s at DoD, EVM proved its value for programs spanning the management spectrum - large and small; cost and fixed price; hardware and software; maintenance and services; contracts and in-house. Wave after wave of defense acquisition "reformers" examined EVM and reaffirmed it. The proposed FAR rule is the logical next step. Wayne Abba Posted June 23, 2005 11:21 PM