|Crikey! Actual work!
||[Feb. 16th, 2005|08:13 pm]
I started work on Monday. My head is still spinning. Lots to learn really quick. Some quick, ill-conceived impressions after three days:
-Great people to work with
-Fun projects (building new NOC, major fiber backbone work)
-Bureaucracy somewhat quick (like, heavy cream instead of molasses)
-Acceptable upper management support
-Fairly familiar with most software & hardware used
Not quite as good parts:
-Not the newest systems 
-Not the most reliable systems
-No real standardization of... anything really.
-Massively overworked technical staff
-No processes to deal with common issues
-[redacted, but really bad]
-[even worse... no worse... okay, now imagine a steamroller. yup, that's it]
Needless to say, my job now is the [redacted] bit. On the plus side, I live in the server room where no one can get to me. On the minus side, I have 20 seconds to fight for survival before the halon discharge.
 - this could qualify for the understatement of the year award.
Bill has always been,...methodical. Some people appreciate that.
want me to send you an oxygen tank/mask in case you get locked in the server room?
Yeah, I'm rather introverted, which tends to mean I think a lot about what I'm going to be saying/typing.
As for the oxygen mask, halon actually doesn't directly kill you if you're in the room when it discharges. Halon interrupts the exothermic chemical reaction known as "fire" through other methods than starving said fire of oxygen. I have no idea as to the specifics.
Since Halon works at an atmospheric concentration of about 6%, you can still breathe (reasonably) fine after a halon dump. The halon dump process is the unpleasant bit.
First there's the horn, which is ungodly loud.
Then the ventilation shuts off and the doors close (airtight seals!).
After 20 seconds of horn, the EPO triggers the UPS to shut down, killing all power to the room.
Immediately following that, the dump happens. 300 pounds of halon 1301 (gaseous flooding agent) discharges in about 10 seconds. If any part of your body is within about 3 feet of one of the dump nozzles, you just got a severe case of frostbite. Very loud, blows papers and other light things around the room and causes ceiling tiles to explode out of their frame from the sudden overpressure. The room becomes opaque as the moisture in the room (kept at a RH of ~55%) condenses into fog. Parts of the room close to the dump nozzles have frost on them. The room is now pretty fucking cold.
Assuming there was no fire, then you get a headache and you breathe a bit faster (which you'd be doing anyway at this point in your terror).
Assuming there was a fire, the halon that interacted with it breaks down into toxic (to humans) byproducts which shouldn't be inhaled. It's a very sharp acidic odor, apparently.
Either way, that 200 pounds of halon has been dumped to the atmosphere, and eventually will contribute to the destruction of the ozone layer. Lord knows how much it costs for 200 pounds of replacement halon nowadays, since it hasn't been manufactured since 1994, but $20K sounds about right, not counting all the other system bits that need checking on after a dump, and the technical guy who still knows how to service a halon system from 1984.
There are detection elements and manual dump elements, also a halon_abort switch and an emergency_power_off switch.
Some systems use FM200 or CO2, instead of halon. I believe those are used at concentrations of ~25% to smother fires; therefore you don't want to be in the room, since there really isn't a suitable concentration of oxygen to breathe. Basically it's like being on Mt. Everest, with all those oxygen molecules being wayyy to far apart to capture enough in each breath.
2005-02-17 02:16 pm (UTC)
I just went thru the "safety training" class for access to our data center - it was great! If you hear an alarm, run like hell, and try to get out before the FM-200 goes off. Don't eat, drink, or mess with things that you don't own. Nice and simple. We even got to watch a nice sales video from the FM-200 people to see what it would look like if it discharged... course, the demo room they showed was like 1/100th of the size of our DC... but hey.
2005-02-17 02:16 pm (UTC)
Oh... and FM-200 is actually less smothering than Halon - you're better off in a room with that than Halon. At least, that's what the sales video said. =D
Good to know. Then it was just the CO2 systems that actually can kill you.
We have a FM-200 handheld extinguisher; guess I'll read the labels the next time I'm waiting on my computer.
2005-02-17 09:44 pm (UTC)
The guy who taught the class for us said that you can actually be in the room when the FM-200 goes off, and all it will do is disorient you (for the same reasons as Halon - the fog, the sudden lack of oxygen, etc), but that breathing wouldn't be all that impaired. Then again... he probably got his info from the video... =)
2005-02-17 09:56 pm (UTC)
Hm - I just grabbed it in like 10 seconds. Wierd.
It's similar to the one I saw... but it was actually a sales movie, like a 5 or 7 minute demo tape - not just video of them showing it going off. They don't have the alarms or anything in that tape, tho... that's really half of the fun!
Worked in a building with a Massive main room with a mix Halon/FM system. It had what was called a "Dead-Man's Switch" (remember that, it comes in to play later). It had the old-school system too.
So here's the drill. The system was set to *not* go off automatically. Instead a set of big flashing red lights would go off. Then an operator is supposed to go to the "Dead-Man's Switch" and pull the two big red handles down. When asked what next we were told to leave through the door next to the two big red handles. One itty bitty problem. The two big red handles did a few things.
1) Release the halon/FM (Halon for the downstairs and FM upstairs)
2) Seal all the Chiller Vents and all the external vents to the rooms
3) Seal all adjoining doors.
Including the one you were supposed to calmly walk out of.
So basically it was suicide to pull those handles. Needless to say we complained, nothing happened, the system stayed. I left a few years back and heard that they finally fixed it two years ago (the one door is now left operational for 60 seconds).