By the third time, I had a pretty solid thought that it was my action. And, yes the ultimate cause was the update. It was quickly repaired. The need to rework the somewhat extensive JES2 Wylbur mods with each release was causing the site significant delays on system upgrades and was one reason we eventually dropped Wylbur. I missed it.
> -----Original Message----- > From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On > Behalf Of Seymour J Metz > Sent: Thursday, March 6, 2025 4:38 AM > To: IBM-MAIN@LISTSERV.UA.EDU > Subject: Re: Stupid outages you caused (was: Cost of an outage) > > Is it fair to say that you cause the outage? I would attribute it to the bad > update. > > -- > Shmuel (Seymour J.) Metz > http://mason.gmu.edu/~smetz3 > עַם יִשְׂרָאֵל חַי > נֵ֣צַח יִשְׂרָאֵ֔ל לֹ֥א יְשַׁקֵּ֖ר > > > > ________________________________________ > From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> on > behalf of Dave Gibney <000006fb76de82cb-dmarc- > requ...@listserv.ua.edu> > Sent: Wednesday, March 5, 2025 8:29 PM > To: IBM-MAIN@LISTSERV.UA.EDU > Subject: Re: Stupid outages you caused (was: Cost of an outage) > > External Message: Use Caution > > > Very early in my career, I had a Wylbur Exec that dropped a TYPRUN=PRINT > jobcard as line 0.0 and submitted it. At that time, a line printer was in the > same > room as my desk. > Apparently, they updated Wylbur, or the JES2 mods, I never learned which. > One day, I did my print command and Wylbur crashed. It came back up, and I > resumed my work, issued the command again. I probably shouldn't have done > it the third time. > They left it down until they came to my "office" and asked me not to do that. > > Apparently, the update did not account for a Wylbur file with line number > zero. > > > -----Original Message----- > > From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On > > Behalf Of Seymour J Metz > > Sent: Wednesday, March 5, 2025 3:42 PM > > To: IBM-MAIN@LISTSERV.UA.EDU > > Subject: Re: Stupid outages you caused (was: Cost of an outage) > > > > Welll, this may seem penny ante and not nearly dramatic enough, but I once > > type EXEC CMS ERASE when I meant to type ERASE CMS EXEC. It was the > > fastest PA1 in the West and a very red face. No permanent damage, and > > nobody pointing at me laughing, but I was still embarrassed. > > > > -- > > Shmuel (Seymour J.) Metz > > http://mason.gmu.edu/~smetz3 > > עַם יִשְׂרָאֵל חַי > > נֵ֣צַח יִשְׂרָאֵ֔ל לֹ֥א יְשַׁקֵּ֖ר > > > > > > > > ________________________________________ > > From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> on > > behalf of Phil Smith III <li...@akphs.com> > > Sent: Wednesday, March 5, 2025 6:01 PM > > To: IBM-MAIN@LISTSERV.UA.EDU > > Subject: Stupid outages you caused (was: Cost of an outage) > > > > External Message: Use Caution > > > > > > Rupert Reynolds wrote about taking down a system by compressing a PDS. > > What stories can y'all share about times you or someone you worked with > > took down a system in a way that made you SMH afterward? > > > > I'll start with a couple of VM stories: > > > > Back at University of Waterloo, we had four systems running VM/SP in an > SSI > > configuration (think "Sysplex", only less so) with 20,000 students using the > > system (among other things). We had a service virtual machine (an SVM; > think > > "STC") named PRIV that would accept commands via SMSG (think "TELL"), > > validate the issuer and command against a table, and issue the command (or > > not) depending on whether they were authorized. This was nice, and had > > granularity so, for example, BOB could recycle some SVMs but not others, or > > could force off specific users. > > > > I was doing some enhancements to PRIV and logged onto it. Hmm, how to > > take it down? I know: SMSG * SHUTDOWN > > > > Then I waited. And waited. And all of a sudden an operator came barreling > out > > of the Red Room yelling, "System A just shut itself down?!" > > > > Oops. Nothing I've written since has accepted SHUTDOWN as a command, > so > > as not to tempt anyone. > > > > > > Years later, at my first vendor, I was testing a product for possible > > acquisition. > > This was in the early days of VM/XA SP, which was notoriously unreliable at > > that stage in its development (at one point the service for it overflowed a > tape, > > necessitating some quick work on IBM's part because nobody had ever > > considered that a possibility). > > > > Because the possible acquisition was a Big Secret, I went down to our > > (unstaffed) toy data center to work. I fired up the product and the system > > crashed; not unusual for VM/XA SP, so I went over and started bringing it > back > > up. About halfway through, the other two developers came down to see if > > they needed to do anything. I let them finish the process, and as soon as I > got > > a logo on my terminal, I logged back on and fired up the product again. And > it > > crashed again instantly. They both turned around and said, "What did you > > do?" and I had to come clean! Turned out the product was mucking with low > > core, ick. > > > > > > Last one isn't my fault, from 15 years later. I was at Linuxcare, where we > were > > doing Linux provisioning under z/VM. One of our guys was onsite at a bank > > doing a trial install and needed some disk space. He was really a Linux guy, > not > > a VM guy, but had mucked around on our MP3000, so he [thought he] > knew > > what to do: he found a free volume, attached it, and formatted it. Oops: > z/OS > > had had plans for that data, and folks were NOT happy when they realized > > what he'd done. Of course it was at least partly their fault for having > > left him > > alone on a production system on a privileged ID. > > > > This was on a Friday and I was off that day because I was having knee > surgery. > > I got a call late that evening from our CEO saying, "You need to be in > > Chicago > > first thing Monday morning". So early Monday I flew to ORD and took a cab > to > > an Embassy Suites and spent the day there working, waiting for a call to go > > do...something. Finally I got one late in the day saying "Nevermind, go > home". > > I guess they found enough of a backup and didn't want to have to discuss > who > > screwed up worse. > > > > > > What have YOU done that you wouldn't want on your resume? > > > > ---------------------------------------------------------------------- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > > > > > ---------------------------------------------------------------------- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
