Facebook service has been restored after an outage lasting almost six hours hit the company's services on Monday.
In a statement, Facebook's vice president of infrastructure, Santosh Janardhan, said its services including Instagram, WhatsApp and Facebook itself went down after a faulty router configuration change.
Internet outage tracker DownDetector said the Facebook fault was "one of the largest ever tracked," adding that it was "an extremely impactful event".
Facebook said that it did not believe the outage was the result of an external attack.
"We also have no evidence that user data was compromised as a result of this downtime," Janardhan wrote.
What took Facebook down?
As the outage continued, cybersecurity experts noticed that the BGP - "Border Gateway Protocol" - routes into Facebook's network had been withdrawn.
BGP is the mechanism that routes data between networks on the internet. If a BCP route is missing, it effectively renders that network inaccessible from the outside.
"With those withdrawals, Facebook and its sites had effectively disconnected themselves from the internet," web infrastructure service Cloudflare wrote in a blog about the incident.
Columbia University computer scientist Steven Bellovin speculated on Twitter about the situation Facebook faced, saying "if they can't reach their border routers from inside the company, they're in for a world of hurt: people will have to physically go to these data centers and reconfigure things by hand".
Facebook's statement on the incident, while not explicitly naming BGP errors as the culprit for the outage, said that "configuration changes on the backbone routers that coordinate network traffic between our data centers caused issues that interrupted this communication...bringing our services to a halt".
Security expert John Bambenek told the Associated Press that the way online infrastructure was built made failures like this inevitable.
"The reality is the internet is kind of held together by defective duct tape and bubble gum. So it's going to fail. The only surprising thing for somebody like me who's done it so long is that it works at this scale at all, in the first place," he said.
'A huge awakening'
Facebook's sheer scale meant the six-hour outage had effects that went far beyond simple social media browsing.
Knitwear seller Kendall Ross told the Associated Press that his business relied on Instagram to advertise its products and that the service going down had led to a loss of sales.
"The outage today is frustrating financially," he said. "It’s also a huge awakening that social media controls so much of my success in business".
"The reason we're talking about it is kind of the radical size and monopoly power that Facebook has," Bambenek said, adding that the company "really reaches its tentacles into not just our society, but many societies".
Communication risks
Hacker and cybersecurity expert Rachel Tobec told AP that many people's reliance on Facebook, WhatsApp, and Instagram to communicate could have left them vulnerable to criminals taking advantage of Facebook's downtime.
"They don’t know how to contact the people in their lives without it,” she said. "They’re more susceptible to social engineering because they’re so desperate to communicate".
Tobac said that previous outages had seen incidents of people clicking malicious links in an attempt to restore their social media access, thereby exposing their personal data.
Facebook-owned messaging service WhatsApp was also taken down by the service outage, leaving users unable to make calls and send or receive messages.
Users who rely on Facebook to log in to services unrelated to the company, as well as websites and apps using Facebook's advertising network were also hit by the outage.
Locked out
In its statement, Facebook also confirmed that the outage had been prolonged by the company's reliance on its own servers for basic functions like internal communications and even access to offices and data centres.
It was widely reported on Monday that Facebook employees had been shut out of company buildings by "smart" internet-connected door locks that relied on Facebook servers to work.
"The underlying cause of this outage also impacted many of the internal tools and systems we use in our day-to-day operations, complicating our attempts to quickly diagnose and resolve the problem," Janardhan said.
Facebook's reputation damaged
The service outage is another blow to Facebook, whose reputation has already been damaged by a series of allegations and leaked documents in recent weeks.
Frances Haugen, a former Facebook product engineer, leaked numerous internal research documents that stoked concerns over the damage the company was allegedly doing to children's mental health.
Haugen further accused the company of "(choosing) profit over the safety" of its users, in an interview broadcast by CBS on Sunday.
On Wall Street, Facebook's share price, already down at the beginning of the session, accelerated its losses on Monday and fell by nearly 6 per cent, shedding more than $50 billion (€43 billion) in value.
