my tweets are severely limited by my lack of understanding of what I am doing.
Facebook, Instagram and WhatsApp are down. Appears to be DNS related.
Afficher cette discussion
For bonus points at least one of Facebook's BGP routes has been withdrawn 
Roll back those changes!
-
-
This one looks like a pretty epic configuration error, Facebook basically don't exist on the internet right now. Even their authoritative name server ranges have been BGP withdrawn.
Afficher cette discussion -
By not having BGP announcements for your DNS name servers, DNS falls apart = nobody can find you on the internet. Same with WhatsApp btw. Facebook have basically deplatformed themselves from their own platform.
Afficher cette discussion -
For those asking - more often than not with this kind of thing, it's not a cyberattack(tm), it's somebody made a boo-boo.
Afficher cette discussion -
Also heard this. Facebook have lost their LAN/WAN due to networking woes so there are a ton of knock on impacts. It’s basically a core network failure for them (which I’ve been through at other companies, not pretty, will take hours for recovery).https://twitter.com/sheeraf/status/1445099150316503057 …
Afficher cette discussion -
Also fun fact, Facebook runs a custom network stack at least in part. Eg they have their own BGP implementation, which lets them make real time updates.
Afficher cette discussion -
The Facebook outage has another major impact: lots of mobile apps constantly poll Facebook in the background = everybody is being slammed who runs large scale DNS, so knock on impacts elsewhere the long this goes on. https://twitter.com/jgrahamc/status/1445066136547217413 …
Ce Tweet est indisponible.Afficher cette discussion -
As
@atoonk correctly points out, Facebook are reintroducing BGP routes now. I can get DNS requests through to Facebook. Most services are erroring out, so now they enter the recovery phase of trying to get everything working together again.pic.twitter.com/FzR4BQ8TKZ
Afficher cette discussion -
I can reach Facebook, WhatsApp, Instagram and Oculus Quest successfully in the UK now.
It will take time across ISPs and countries, and I imagine some turbulence as devices reconnect etc.Afficher cette discussion -
Also this one was a loss of core networking across Facebook’s production network, and not a global cyber war!!1! I think people forget but Facebook had a 14 hour outage 2 years ago, this one was shorter. Shoutout to the people who debugged and fixed it
Afficher cette discussion -
Also anytime I post a thread like this somebody turns up a week later and says ‘well it’s working for me!’, so I look forward to seeing that tweet.
Afficher cette discussion -
Facebook Engineering blog is out. Core network failure from config change. https://engineering.fb.com/2021/10/04/networking-traffic/outage/ …pic.twitter.com/b076U3v3T2
Afficher cette discussion -
Facebook have issued an RCA for this. It backs up my thread. One engineer issued a command, which took down the company due to a cascading set of issues which took down DNS and internal systems. Don’t punish the engineer, improve the processes. https://engineering.fb.com/2021/10/05/networking-traffic/outage-details/ …pic.twitter.com/ohKneIwFOL
Afficher cette discussion
Fin de la conversation
Nouvelle conversation -
Le chargement semble prendre du temps.
Twitter est peut-être en surcapacité ou rencontre momentanément un incident. Réessayez ou rendez-vous sur la page Twitter Status pour plus d'informations.