If you were affected by yesterday’s nationwide Telstra outage, you now have something to blame: a software fault.
In a statement provided to Kotaku Australia, the company said the “initial cause of the disruption” was a software fault that started a chain reaction on the network. The fault began at approximately 1000 AEST yesterday, hitting Telstra’s 4G data and voice networks, which ultimately created congestion on the 3G network.
The root cause of the software fault is still unknown at the time of writing, although full service was restored just before 1400 AEST.
Starting from approximately 10:00am yesterday morning some key network equipment failed causing a disruption to 4G voice and data services nationally. The impact was widespread and with a large number of customers dropping back to 3G there was significant disruption to 3G voice and data services as demand exceeded the capacity of our 3G network.
We have identified that the initial cause of the disruption was a software fault which triggered multiple elements across the network to fail. The network is designed to switch onto standby hardware which it did. Following the failover however, a further fault caused an interruption which impacted 4G connections. There is redundancy built into these systems but this did not operate as intended.
By around 11:40am we had successfully isolated the impacted network infrastructure and services began to be progressively restored. It took about two hours for full service to be restored after that, although our team was working longer with some government and enterprise customers to help them with their systems recovery.
We are still investigating the root cause of the software fault. We are working closely with our technology vendors on the specific element of software which triggered the issue.
Our teams have worked around the clock to restore services and to investigate why the redundancy in our network did not prevent customer impact, for which we are deeply sorry.
Leave a Reply