En plus de souci sur SBG, nous avons le souci sur le réseau optique en Europe qui interconnecte RBX et GRA avec les POP. Il est down (!!).
Show this thread
Nous avons un souci d'alimentation de SBG1/SBG4. Les 2 arrivées électriques EDF sont down (!!) et les 2 chaines de groupes électrogènes se sont mis en défaut (!!!). L’ensemble de 4 arrivées elec n'alimentent plus la salle de routage. Nous sommes tous sur le problème.
-
-
-
SBG: ERDF is trying to find out the default. 2 separated 20kV lines are down. We are trying to restart 2 generators A+B for SBG1/SG4. 2 others generators A+B work in SBG2. 1 routing room is in SBG1, the second in SBG2. Both are down.
#MurphyShow this thread -
-
RBX: all optical links 100G from RBX to TH2, GSW, LDN, BRU, FRA, AMS are down.
Show this thread -
SBG: 2 routing rooms are UP powered by gen. EDRF still down. ETA: 15min RBX: the DB of the optical node in RBX is corrupted. All links are still down. We upload the backup. ETA: 30min
Show this thread -
-
-
-
SBG: ERDF repared 1 line 20KV. the second is still down. All Gens are UP. 2 routing rooms coming UP. SBG2 will be UP in 15-20min (boot time). SBG1/SBG4: 1h-2h
Show this thread -
SBG: 2 routing rooms are UP. we are restarting the servers in SBG1/SBG4.
Show this thread -
SBG: elec is UP. all racks are UP. we are checking which services/customers is still impacted.
Show this thread -
Résumé des 2 incidents qui ont impacté
#Ovh ce matin.#OVHGate http://travaux.ovh.net/?do=details&id=28256 …pic.twitter.com/ELdJEzBvqn
Show this thread -
-
Résumé Incident de RBX http://travaux.ovh.net/?do=details&id=28244 …pic.twitter.com/Qclv8bYMb9
Show this thread -
Comment by Octave - Thursday, 09 November 2017, 17:53PM Il reste encore en panne: - 2100 serveurs dédié - 1500 instances PCI - 25000 VPS - 300 hosts PCC http://travaux.ovh.net/?do=details&id=28247 …
Show this thread -
Résumé de la panne à SBG http://travaux.ovh.net/?do=details&id=28247 …pic.twitter.com/LELnBbSnkB
Show this thread -
all the details about SBG http://travaux.ovh.net/?do=details&id=28247 …pic.twitter.com/vTs9T1nvuh
Show this thread -
SBG: 100% of the infrastructure has been UP since yesterday 11h00. Since yesterday 11h00, we are working on the hardware issues: we are remplacing broken servers with the spares. http://travaux.ovh.net/?do=details&id=28247 …
Show this thread -
Port-mortem Incident RBX http://travaux.ovh.net/?do=details&id=28244 …pic.twitter.com/5rdWQqTJfc
Show this thread -
Le suivi SBG Comment by Octave (oles) - Saturday, 11 November 2017, 20:10PM http://travaux.ovh.net/?do=details&id=28247 …pic.twitter.com/fdrwYL1LUV
Show this thread -
We've finished to fix the last PCI/VPS hosts. If your VPS/PCI is still down, please DM me the tech deatils (IP/Name of VPS/PCI) On a fini de fixer le hardware du dernier host PCI/VPS. Si votre VPS/PCI est encore down, merci de m'envoyer DM avec les infos tech (IP/Nom)
Show this thread -
today, we have 11 teams x 3 persons each to fix the last hardware issues.
Show this thread -
Les cellules haute tension 20kV de SBG1 avec les cellules motorisées et l'automate NSM qui n'a pas marché.
#Ovhpic.twitter.com/p4iPeahrB1
Show this thread -
Les groupes électrogènes de SBG1 (2x2MVA), SBG2 (2x1.4MVA) et SBG3 (2x1.4MVA)
#Ovhpic.twitter.com/Yx0DoD1LZZ
Show this thread -
Les navettes déposent à SBG les équipes fraiches qui viennent de Roubaix et on récupère ceux qui ont tout donné.
#Ovhpic.twitter.com/EEZfZ8oi12
Show this thread -
-
This night, we've fixed the last servers with the hardware issue. All serveurs are UP. Our apologies. Cette nuit, nous avons fixé les derniers serveurs avec le problème hardware. Tous les serveurs sont UP. Toutes nos excuses.
Show this thread -
SBG: >50000 servers. Usually, there is 1-2% of the servers that don't restart after a brutal electic shutdown, because of hardware issues. We had 0.84% of servers that we fully changed, new hardware. It took us <4 days last server.
Show this thread -
We continue to work on the servers in SBG: rebuild spare stock, stock of servers in SBG, rebuild the servers in SBG3 (we took lot of pieces from the new servers prepared for SBG3), clean-up the operations, ..
Show this thread -
Incident is closed. If you have any issue, please DM me or contact the support. We have the software teams that will help you to fix your setup issues (firewall, etc).
Show this thread - 1 more reply
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.