Further Complications

Complexity is innate to software, and complex things fail. Complex systems interact in unexpected ways. The more they're asked to do and the larger they get, the more likely unanticipated consequences will pop up.

The question isn't whether or not software projects will sometimes fail. They will. A conservative estimate of the cost of federal software failures pegs the annual value at $3 billion, according to Robert Charette, president of ITABHI Corp., a Spotsylvania, Va., risk-management consultancy.

Take the failure of an Internal Revenue Service electronic fraud detection system, for example. The IRS worked for five years to replace its old system with new Web-based software. Was the failure of the Web-based software a function of its essence, or was it an accident?

The tax agency had assurances from contractor Computer Sciences Corp. of El Segundo, Calif., that the new system would be online by January 2006. It wasn't. The IRS had no contingency plan and already had shut down its original system, leaving the 2006 tax season wide open for cheats. As a result, according to a Treasury Department inspector general for tax administration report (TIGTA 2006-20-108), tax dodgers may have gotten away with $318.3 million in improper refunds. The IRS had invested $20.5 million in the Web-based system through last April. When auditors investigated, they found that developers were unfamiliar with user needs, teams failed to coordinate with each other (so when one team made a change affecting others, the other teams wouldn't know about it, requiring a software fix), there was excessive turnover among CSC and IRS employees, and CSC repeatedly missed intermediate deadlines while still promising to complete the project on time.

"If anything could go wrong in a system-development-type atmosphere, it went wrong," says Margaret E. Begg, a Treasury tax administration assistant IG.

What happened at the IRS was neither an accident nor an example of software's complex essence, says Charette. It "doesn't even deserve the name 'failure,' " he says. "It's a blunder."

No governance process was in place to resolve difficulties and, despite a critical need for the software to perform, risks surrounding the project piled up mostly unattended, the IG found. When, before the project's collapse, the IRS director of refund crimes requested an independent study to scrutinize whether CSC would be able to deliver on its promises, he was informed that "the time needed to resolve 'a few existing glitches' " would prevent the study from starting until after the final deadline. At another point, when an IRS employee found that CSC grossly exaggerated the work it had completed, the IRS "did not pursue the matter," IG investigators found.

CSC refused to comment, citing contractual obligations. Asked about specific problems cited by auditors, the IRS responded with a statement that it is "taking a number of steps to ensure the continued integrity of this system" and has revamped its governance structure so "significant risks and issues are identified early on and elevated to the appropriate executive level for attention."

A blunder is classic and boring, the stuff of basic project management, Charette says. It's made up of avoidable execution mistakes. Proper management of a software project includes building into the design enough wiggle room to absorb a reasonable number of mistakes.

Blunders are avoidable, but also unlikely to disappear. Institutional pressures conspire to ensure that, despite having blundered many times, the federal government likely will blunder again. Getting a project funded often means overpromising results while downplaying costs. Managers have an incentive to make their projects look good, no matter their true status, often in the hope that problems will clear up before anybody else can notice. And, of course, deception is self-reinforcing and self-perpetuating; once it starts, it spreads. The problem is systemic: Industry proposes unrealistic costs and understates risks, while government agencies blithely accept them. "CSC led them to believe that [the fraud system] would be operational, right up until the end, when in fact, they were unable to deliver," notes Begg.

In August 2005, IRS managers began to realize they had a serious problem. The agency started holding biweekly meetings with contractors, and in early December, executives started meeting daily. During the project's last couple of months, technical staffers were sending out two or three daily voicemail updates.

Finally, on April 19, 2006, the IRS decided it had had enough. The April deadline for most Americans to file their personal tax returns had come and gone with no electronic fraud detection system in place. The agency halted further development work and said it would restore the old system, the one that in 2001 it had said needed replacement. IRS says the old system will be working in time for the 2007 tax season.