911 emergency systems were disrupted for more than an hour on Monday in cities in 14 US states. The interruptions have led many news agencies to speculate that the problem is related Microsoft‘s Azure a web services platform that at the time was also struggling with widespread disruption. However, many sources tell KrebsOnSecurity that the problems with the 911 stem from some kind of technical speed. Inside and Lumen, two companies that together handle 911 calls for a wide area of the United States.
On the afternoon of Monday, Sept. 28, several states, including Arizona, California, Colorado, Delaware, Florida, Illinois, Indiana, Minnesota, Nevada, North Carolina, North Dakota, Ohio, Pennsylvania and Washington, reported 911 interruptions. settlements .
Numerous news reports have suggested that the outages may have been related to a continuous service outage at Microsoft. But a spokesman for the software giant told KrebsOnSecurity, “we have seen no indication that the disruption of the 911 in several countries is the result of yesterday’s disruption of the Azure service.”
Inquiries to emergency expedition centers in several of the cities affected by the 911 disruption point to a different source: Omaha, Nebraska-based, Intrado – until last year, known as Western safety communication – provider of 911 and emergency communication infrastructure, systems and services to telecommunications companies and public safety agencies throughout the country.
Intrado did not respond to numerous requests for comment. But according to officials in Henderson County, North Carolina, who had their 911 failures yesterday, Intrado said the outage was the result of a problem with an unspecified service provider.
“On September 28, 2020, at 4:30 PM MT, our 911 service provider complied with internal conditions for their network, which led to an impact on the delivery of 911 calls,” said an Intrado statement issued to county officials. . “The impact was mitigated and the service was restored and confirmed for operation by 17:47 MT. Our service provider is currently working to determine the root cause. “
The service provider mentioned in Intrado’s statement appears to be Lumen, a communications company and provider of 911, which until recently was known as CenturyLink Inc. A look at the company’s status page shows that many Lumen systems suffered complete or partial service disruptions on Monday, including its private and internal cloud networks and the network of control systems.
In a statement to KrebsOnSecurity, Lumen blamed the problem on Intrado.
“At approximately 16:30 MT, some Lumen customers were affected by a vendor-related event that affected 911 services in AZ, CO, NC, ND, MN, SD and UT,” the statement said. “The service was restored in less than an hour and all 911 traffic is being routed right now. The supplier’s partner is in the process of investigating the event. “
It may not be a coincidence that both companies are already operating under new names, as this would hardly be the first problem between the two to disrupt 911 access for a large number of Americans.
In 2019, Intrado / West and CenturyLink agreed to pay $ 575,000 to settle an investigation by Federal Communications Commission (FCC) in a break from August 2018, which lasted 65 minutes. The FCC found that the accident was the result of a West Safety technician who confused a change in the configuration of the company’s 911 routing network.
On April 6, 2014, about 11 million people in the United States were disconnected from 911 services for eight hours due to a “completely preventable” software bug related to Intrado systems. The incident affected 81 call centers, with emergency services not operating in all of Washington and parts of North Carolina, South Carolina, Pennsylvania, California, Minnesota and Florida.
According to a 2014 Washington Post story for follow-up investigation and a report published by the FCC, this issue involves a problem with the way Intrado’s automated system assigns a unique identification code to each incoming call before passing it to the appropriate “call center.” public safety response, ”Or PSAP.
“On April 9, the software responsible for assigning the codes reached the maximum limit,” explained The Post. “The meter has literally stopped counting 40 million calls. As a result, the routing system stopped accepting new calls, leading to a difficulty and a series of cascading failures elsewhere in the 911’s infrastructure. “
Combining the duration of the outage in 2014, the FCC found that the Intrado server responsible for categorizing and tracking service outages classified them as “low-level” incidents that were never marked for manual review by humans.
The FCC eventually fined Intrado and CenturyLink $ 17.4 million for the state’s interruption in 2014. An FCC spokesman declined to comment on Monday’s interruption, but said the agency was investigating the incident.
Tags: 911 Interruption, Azure, CenturyLink, FCC, Federal Communications Commission, Intrado, Microsoft, The Washington Post, West Corporation
This entry was posted on Tuesday, September 29th, 2020 at 6:26 pm and is filed under Little Sun. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a comment. Ping is currently not allowed.