Watchdog gives mixed report on FBI case management project

Schedule and cost estimates for the FBI's upgrade of information technology infrastructure could be unreliable if the bureau fails to expand its acquisition handbook to implement certain best practices, according to a new report from the Government Accountability Office.

While the FBI largely is using good methods to manage its major IT overhaul, known as Sentinel, certain gaps could cause disparities between expected and actual progress, the report (GAO-07-912) stated.

Sentinel is meant to replace the failed Virtual Case File project and would allow agents to manage their work electronically.

In the report, Randolph Hite, director of information technology architecture and systems issues at GAO, praised the bureau for following best practices when soliciting interest from contractors and making an award, and for adequately defining expectations for the prime contractor. Hite also wrote that the FBI has "established and is following effective processes to proactively identify and mitigate program risks before they have a chance to become actual cost, schedule or performance problems."

But Hite identified areas where the FBI's failure to follow key acquisition practices could hinder the program.

One of several threats to the contract scheduling cited is the fact that the FBI did not indicate that extra time must be set aside for "high risk activities." The report stated that best practices include buffer or reserve time within a contract for certain elements that could take more time than originally scheduled, but the FBI did not include this reserve time in guidelines for Sentinel.

The report stated that until this omission is remedied, "schedule estimates for FBI IT programs, like Sentinel, will be of questionable reliability, and thus the risk of Sentinel's actual performance not tracking closely to plans is increased."

The initial phase of Sentinel faced delays, and Hite wrote that the limitations of the FBI's existing scheduling procedures may have contributed.

Similar problems exist on the cost estimate side. According to the report, the bureau's guidelines fail to incorporate historical data on actual costs and estimates for comparable programs. Additionally, they do not put in place ways to judge the credibility of cost estimates by analyzing any potential uncertainty or bias.

In responding to a draft of the report, officials from the FBI's Office of the Chief Information Officer agreed that certain scheduling and cost estimate best practices were not being implemented and need to be put in place as soon as possible. In a letter to Hite, Zalmai Azmi, the FBI's CIO, said the bureau was planning to update its IT acquisition handbook before this fiscal year ends Sept. 30.

The FBI disagreed with the report in the area of performance metrics, however. The report criticized the bureau for failing to monitor management support contractors with the standards used to evaluate prime contractor performance.

"By not implementing this practice, GAO believes that the FBI's monitoring does not adequately ensure that Sentinel support contractors are performing important program management functions effectively and efficiently."

Bureau officials disagreed, saying the existing oversight is sufficient. Azmi said the Sentinel Program Management Office clearly defines performance expectations for its support contractors, and a large part of their work requires written products, which are constantly reviewed by government supervisors.