Well, things could be worse.
We’ve pretty much got our whole network under control now.. the ongoing problem mentioned last post was finally figured out by Cisco support. It turns out it was a
bug undocumented feature in IOS dealing with how they learn MAC addresses.
There was also another network problem we got resolved yesterday that was causing general slowness on web and mail servers. It’s complicated (i.e. I don’t understand it exactly myself), but in the end we took a distribution switch out of the network and that fixed it.
We still have one open ticket with Cisco for our core routers having some HSRP problems. It doesn’t seem like that’s having any real effect on our network, but we want it fixed!
We are also installing two new Ciscos to offload the BGP duties from the core routers so they’ll just have to handle switching. This set-up should be able to handle about 300% more traffic than our entire network now pushes at peak times!
Thanks to these network problems being resolved, we’ve also begun re-deploying in Alchemy, who at least didn’t have the second power outage.
We’re also still in the process of getting real UPS power on our network cabinet, plus our internal databases and a few internal servers. Basically, everything that keeps all the customer mail, web, database, and file servers from coming right back up quickly should there ever be another outage.
So, um.. that’s how it stands now! We hope this will all soon be nothing more than a long bad dream (that was real).