GovernmentExecutive.com

Return to Article: Industry, agencies critique method of measuring performance

• 11549
  

  There seem to two main objections to the proposed rule. The first deals with the need to have (and be able to demonstrate that you have) a system that is in compliance with the EVMS requirement prior to award. If we accept the premise that EVMS will be required during the performance of the contract, it would appear to be a high risk to assume that the system will be developed after award without impacting the likelihood of successful performance. In the alternate, the system may be created but too late to actually be used in managing the project, but still provided to "meet the requirement," giving us "two sets of books". If the government is convinced that EVMS is a necessary part of successful programs, it would seem prudent for contractors to begin adapting their systems to the requirement, making the pre-award response of minimal impact. As Wayne Abba points out, there is a reasonable breadth of solutions that will meet the EVMS criteria and the needs of the individual companies. The second objection, to having the IBR prior to award, would seem to be more valid. The key is the nature of the pre-award IBR. In other jurisdictions, notably Australia and Great Britain, the focus of the initial IBR is to confirm the existence of high level WBS, integrated schedules and resource estimates. The objective is to determine that, at the top level, there is commonality of understanding between the government and the contractor of the scope of work and the major elements of it and that the proposal is based on a good understanding of the project objectives.. After award, when the contractor has extended the project planning to a level needed to manage the program, a detailed IBR is accomplished to allow both parties to agree on the details of the effort. It may be of value to consider the benefits that these other jurisdictions have seen form this approach and determine its value in US contracts.

  Tom McCann Posted August 11, 2005 2:24 PM
• 11502
  

  There appears to be little objection from industry concerning the merits of forward planning techniques such as Earned Value Management. The predominant objection from industry appears to be that the audit rights imposed by the new rule are too expansive and may result in inadvertent disclosure of proprietary information. Earned Value management reviews have been conducted in the defense industry since the mid-60s. I am not personally aware of a single case concerning the disclosure of proprietary information arising from these reviews. On the other hand, there has been considerable number of instances where Government reviewers have been accused of over zealous interpretation of the standards resulting in excessive delays and costing to the programs being reviewed. If the application of these requirements is to work in civilian agencies it is necessary to adopt review methodologies that avoid the shortcomings of the past. This is particularly true since civilian programs are much more diverse in both size and nature than those of past defense programs. Data integrity reviews should be the responsibility of the client, are much less intrusive and can be conducted at arms length. System reviews should be responsibility of the supplier. Supplier system reviewers must focus both on the integrity of performance data generated from these systems as well as adherence to standards. They must truly embrace the scalability aspects of the relevant standards and tailor the requirements to meet the individual programs at hand. Industry should assume the responsibility of reviewing and maintaining their own systems through internal or third party assurance regimes so long as the integrated data has integrity. There already exists a professional third party assurance regime for Earned Value Management Systems that is obligated to protect proprietary information and will provide both industry and governmental organisations with reliable system assessment which, in turn, results in confidence in the integrated cost, schedule and performance data.

  Lloyd L Carter Posted August 9, 2005 9:51 PM
• 10583
  

  Your story about the proposed rule on earned value management explains that EVM "relies on calculus to measure contractors' performance" and suggests it is too costly for small businesses. EVM brings to program and contract management greater accountability for resources than other techniques, but the marginal costs are more than balanced by better insight into contract cost, schedule and technical performance and by more reliable estimates of cost at completion. Calculus? Hardly. EVM calculations are based on simple ratios - the amount (measured in hours, dollars or any measurable unit) of work accomplished vs. the amount planned for the same work and the amount of work accomplished vs. its actual cost. And what is the marginal cost of generating the data? The EVM standard says "Different companies must ... establish and apply a management system that suits their management style and business environment. The system must, first and foremost, meet company needs and good business practices." And "EVMS scalability is viewed as a spectrum employing the principles of EVMS as fundamental to all programs and the EVMS guidelines as applicable to large and/or high risk programs, allowing any program regardless of size and complexity to realize the benefits of earned value management." In other words, EVM is not "one size fits all" - so companies that are concerned about the cost should take a closer look. They might discover, as have many companies around the world, that investing in EVM makes them more competitive. The proposed rule concerning "pre-award data collection" is not EVM per se, but a planning and risk reduction technique known as the integrated baseline review. An IBR indeed might result in more comprehensive planning and communication with the customer before and immediately after contract award, but I would be hard-pressed to find the downside. Most government agencies are woefully poor at planning. Spending more front-end time in planning pays off in contract execution. In use since the 1960s at DoD, EVM proved its value for programs spanning the management spectrum - large and small; cost and fixed price; hardware and software; maintenance and services; contracts and in-house. Wave after wave of defense acquisition "reformers" examined EVM and reaffirmed it. The proposed FAR rule is the logical next step.

  Wayne Abba Posted June 23, 2005 11:21 PM