I fear leaving the office today. Not because I don’t want to go home. In fact, quite the opposite. There isn’t anywhere, right now, that I’d rather be. Since yesterday afternoon, work projects in general have not been going smoothly.
At approximately 2030 yesterday, I started being paged about an error on one of our production web servers. After getting access to a computer, I managed to establish the issue was a full disk. This concerned me. Corporate office should have spotted this issue on the 24th when the server started complaining, but didn’t. 20 minutes after the initial warning, I’d cleared 11GB of log files up (IIS), and I’d yet to receive any notifications from corporate office that anything was wrong.
At roughly the same time, a customer of mine messaged me frantically telling me that their remote offices were down, and they couldn’t get anything working, and had no idea why. He also proceeded to tell me that he had another 24 hour consulting company looking at it. Now consider that I just do small stuff for them, offer advice, get them out of tight spots, etc, and usually leave the rest to them. This would be constituted as a “tight spot” as its the end of the month, which means the remote offices will be closing sales data. The configs looked okay, but the eigrp wasn’t working properly. So I checked the logs, and noticed something unusual.
00:41:28: IP-EIGRP: Neighbor 192.168.252.5 not on common subnet for Serial0/0
This was most odd, they should be right… I start looking a little closer, and ask if he’d swapped the cables around at any point. Yes, he sure had. The day before, they were also doing some router work, and had one site crash entirely, and thought it might be the local card (ended up being a DSU issue on remote site). They’d changed the IPs in the config to match environments, but apparently hadn’t saved it. When they reset the router, it reverted back to the old config, and as such, the networks were wrong. This ended up being an easy fix (swap cables).
Then starts today… I’ve been up since about 0500 this morning as I had a fire inspection at the office. This involved a fire marshal testing our security system, to ensure it unlocked the doors properly on the event of a fire. Last thing we want is to trap people behind a glass door in a blazing fire… though… thinking about it… never mind, I can get in trouble for that kind of thinking.
So I got to the office, bright an early for the 7am test. I had to arrive a little earlier as the security company had to re-enable the mag-locks on the front door. This is where today’s troubles started. First thing I noticed when walking in was that one of our doors was left unlocked. Frustrated is not the way to start a day. I turn around, and go back to working on the front door, at which, they are having issues. Apparently, the mag-lock doesn’t want to re-enable. Arghhhhh… After a while of playing with wires, testing, and cursing, and putting everything back to how it was, it started working again.
After performing some basic tests to make sure the fire system releases the door, and all is well. Just as the engineer wanders around to make sure the other doors are okay, he walks into the door that was left unlocked last night, and if flies open… The lock on that door clicks, and doesn’t bolt shut. Not good. So we spend another 45 minutes playing with that lock, and figure it to be a lose cable, and tape it up. I should have known this wasn’t going to last, as about 25 minutes ago, the door is broken again sighs.
Shortly after the engineer had left, and I sat down, I received an email reminding me of work I should have had finished by this morning, which I had planned on doing last night. So I jumped on the server to start (and finish) the build out, to find the server hadn’t been reimaged like originally stated, and had simply had the IP changed…
And back to the subject line. I fear leaving the office as I will probably have one, if not all, of the following happen:
- Get stuck in elevator going downstairs
- Get hit by a car in the car park
- Reverse into another car
- Get run off the road
- Slam fingers in door of car
- Trip and break something
I’m off in search of some bubble wrap.