January 10, 2013
The Federal Communications Commission slammed Verizon Communications Inc. for network and generator failures that knocked out emergency 9-1-1 services in Northern Virginia in the wake of a fast-moving thunderstorm in June 2012 and questioned whether the company has taken step to beef up its backup power systems nationwide in a report released yesterday.
FCC chairman Julius Genachowski said, “These failures are unacceptable and the FCC will do whatever is necessary to ensure the reliability of 9-1-1.” He said the outages resulted from “a number of preventable system failures [that] caused major disruptions to communications providers’ networks connecting to 9-1-1 call centers during and shortly after the storm. As a result, 9-1-1 was partially or completely unavailable to millions of Americans -- in some instances, for several days.”
FCC also suggested that the Internet-based Next Generation 9-1-1 system, under development since 2006, could have alleviated the outages experienced in the wake of the June storm, which crippled communications in Midwest and Mid-Atlantic States. The Next Generation system, FCC said, provides greater redundancy and reliability than today’s 9-1-1 system.
The agency soon will launch rulemaking to improve the reliability of existing 9-1-1 networks and prevent failures like those experienced in June, Genachowski said, adding that FCC would accelerate the Commission’s Next Generation projects, expected to cost $2 billion.
The June storm took down public service answering points -- call centers that receive 9-1-1 calls and dispatches -- from Arlington, Fairfax and Prince William counties due to loss of commercial power and the failure of generators in Verizon’s Arlington and Fairfax central office, which resulted in the failure of network and 9-1-1 switches, the report said.
The storm “exposed significant vulnerabilities” in Verizon’s infrastructure, including a “significant number of backup generator failures,” the report said.
The company’s Arlington central office used two backup generators required to work in tandem. One failed because air had entered the fuel system, and the second then overloaded and automatically shut down. The FCC report said this contravenes “best practices” for emergency power. The company also had not conducted routine testing of the generators in that facility for a year, the agency said.
A similar generator setup in Fairfax led to a series “multiple cascading failures” that combined to cause outages at all four Northern Virginia call centers, but Verizon did not inform Fairfax emergency officials of the true extent of the problem, the report said.
According to Fairfax County officials, Verizon also did not provide adequate notification of the existence and scope of problems in its network affecting 9-1-1 service.
Instead, “Verizon sent a cryptic email to designated Fairfax County staff saying that the Arlington central office was without power or backup battery/generator. The references to Arlington suggested that 9-1-1 service was affected only in Arlington County. Without a corresponding phone call explaining the situation and the email, Fairfax County’s [call center] staff continued with their normal operations, unaware that incoming 9-1-1 call service from Verizon was about to rapidly deteriorate,” the report said.
The Virginia Corporation Commission, in an appendix to the FCC report, said Verizon’s “workforce was not timely dispatched, prepared, or trained to recognize or correct the Critical conditions from the June 29 [storm].”
Verizon, the report said, committed to take a number of steps to improve its network in the Washington area including perform audits of “mission critical” backup power systems, update monthly preventive maintenance procedures for backup power, and provide immediate high-level notification of power problems and redundant network paths.
FCC said while these are appropriate steps they are not yet sufficient based on the information it has received form the company. Verizon “has much more to do nationwide” including changing generator configurations that plagued the Arlington office, the report noted. Verizon also has not made a commitment to maintain backup power for a specific length of time nor has the company funded improvements or provided a timeline for them, the report said.
(Image via Cheryl Casey/Shutterstock.com)
January 10, 2013