On Fri, 3 Dec 2010 11:45:12 -0600, Darth Keller 
<[email protected]> wrote:

<snip>
You're now on battery power and have 23 minutes to power everything off as 
gracefully as possible.  
</snip>


I, unfortunately, have lived it.  Minus the 23 minutes.

We had a normal power bump to our building.  Happens once in awhile, right?  
Lights go off for a second and back on, PC reboots, you've recovered from 
that sudden shock it gives you.

So you decide to go into the computer room, "just to check", like you always 
do...

You enter to a dark room with only emergency lights on.  Dead silence...  Well, 
almost dead, but dead compared to what the room usually sounds like.

You walk through the computer room, through the spotty darkness.  Wait - 
you hear something!  Something is running!  You hear fans!  You walk towards 
the sound...  It's your disk drives thank goodness!  You remember they have 
their own internal battery backup!  And within minutes you hear them slowly 
spin down to silence.  Now what?  What do I do?


The Scenario:  The building took a power bump.  The UPS's had not been 
maintained by building services appropriately.  They note the power hit, 
switch from local to battery.  Only, the batteries fail.  

Building services manually switches the UPS's over to local, you check all the 
breakers.  You have management standing over your shoulder asking when 
everything will be back up again.  You don't know - you don't know if there 
was any data corruption.  You don't know if there will be a hardware failure.  
You just. Don't. Know.

If anyone else knows that feeling, you know what I'm talking about, half 
wondering if you should do "something" or go blind, I hear ya.  You wonder if 
you are going to work for a week straight trying to fix things.  You wonder 
where that copy of your resume is.  

Luckily we did not suffer any issues.  Apparently the disk drives were able to 
de-stage all the data from cache to disk before it's battery ran out.  
Everything else, 3480's, 3590's, 3745, 9672, all dead.

Luckily (again) we are a somewhat small community, and you know your IBM 
CE personally.  You hunt and fish with him, know his family, invited to their 
kids weddings.  You call him up directly and tell him to get his ___ over here 
pronto.

He shows up and starts powering up everything.  The disk drives are already 
spinning and alive.  Everything initializes.  The 3480 auto mounts click in 
succession. You have finally POR'ed the CPU.  You IPL.  DB2 and IMS do their 
backout.  You restart batch jobs that were running.

We may have expirenced a christmas miracle, I don't know.  Just know I 
thanked the big man upstairs that night when I went to bed.

But to answer your question Darth, no, we have no real "T minus what the 
heck to do" sort of document created.  Absolutely should.

Hope you have enjoyed Friday story time!

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Reply via email to