RANT -- Remember that scene from that show where the guy goes to open a drawer but it's broken, so he goes into the garage to get a tool, but the light is burnt out, etc. etc.
Well over the past few days, I've been dealing with the IT version of that. First it started with the
Conversation
Reddit ingest breaking. So I sat down at my workstation to diagnose the problem and the keyboard and mouse didn't work. So I replaced the batteries but they still didn't work -- come to find out a half hour later than all the USB ports were dead. So I couldn't use that computer.
1
5
So then I go into the server room, pull a spare workstation NUC from the rack, switch the NVMe drives so I can boot into the same OS -- but I pulled the wrong nuc and took down another ingest that was running. So while fixing that, I logged into the switch and noticed a ton of
Replying to
CRC errors on one of the ports. So I literally picked up that nuc, rotated it 90 degrees and the CRC errors went away (crosstalk in the cable?). While all this is going on, Hetzner has an outage that take out of of the other ingests that is running, so then I have to stop what I
1
6
am doing to fix the Reddit ingest, log into Hetzner and spend a few hours fixing THAT problem. Then as I'm fixing that problem, one of the SSD drives in one of the other servers decided it was a great time to just die -- so then I had to take apart that server and fix it.
1
5
So anyone in IT probably remembers a week where they felt that everything just broke around them and they were playing wack a mole trying to fix things faster than they were breaking.
I need a tylenol.
1
9