Obviously the part about killing was only poor joke, but there is some
sense hidden in.
I mean it is good idea to talk to person who did it. Not to punish, but
talk and explain and hear his/her explanations.
For RACF admin it is quite obvious the security model should be somehow
checked. Again the person can have good explanation of current state of
protection.
Regarding traces - It is a little bit hard to test, especially without
access to mainframe ;-) but I guess SMF80 can be written just before
system freeze. Note it is RACF security check - it happens BEFORE the
command is interpreted by the system. Simpler example: when you issue
CANCEL CICSABC and you don't have such started task, you first will be
checked by RACF (and maybe rejected) and then the command is really
issued, and you will get answer like "there is no such started task to
cancel".
BTW: I imagined what would happen after such case on production...
--
Radoslaw Skorupka
(looking for new job)
Lodz, Poland
W dniu 24.03.2021 o 14:58, Pommier, Rex pisze:
I'm going to agree with *most* of it. I don't like the part about killing the RACF
admin. I'm not the one who initially set up the OPERCMDS security but I missed the fact
the QUIESCE command wasn't set as "don't let anybody use". Hari-kari is not on
my bucket list. :-)
On to Radoslaw's comment about logging - it is logged, after the fact. QUIESCE
does exactly that - it stops the LPAR in its tracks. Do not pass Go, do not
collect $200. No z/OS logging at the time it happens. IBM hardware support
found and reported the wait state back to us from some hardware logs that were
forwarded to them from our CE. The z/OS logging takes place after the PSW
restart from the HMC occurs and yes, it shows the console or user that executed
the command. However in our case since the LPAR stopped in the middle of the
day and we had managers breathing down our necks to get the system back up we
didn't have time to properly diagnose until after the fact - which included an
IPL which in turn did not allow the logging of the quiesce command to take
place.
Rex
-----Original Message-----
From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On Behalf Of
Carmen Vitullo
Sent: Wednesday, March 24, 2021 8:07 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [External] Re: z14 HMC log information
agree 100%, when I tested the command on my sandbox system I see my ID in the
syslog as the culprit :) if done from a console, then the console name is shown.
Carmen Vitullo
-----Original Message-----
From: Radoslaw <r.skoru...@hotmail.com>
To: IBM-MAIN <IBM-MAIN@LISTSERV.UA.EDU>
Date: Wednesday, 24 March 2021 8:01 AM CDT
Subject: Re: z14 HMC log information
IMHO there should be a trace in a syslog. Maybe that part of syslog is somehow
lost.
And it would be good idea to have SMF record for that command. I don't know
about console commands, but SMF80 could be cut if you take care about
AUDIT(ALL(READ)) in advance. I mean RACF profile.
Then find the person and kill him. Don't forget to kill RACF admin who forgot
to protect the command. Kill'em all. ;-)
Seriously: it is a symptom of bad security model.
--
Radoslaw Skorupka
(looking for new job)
Lodz, Poland
W dniu 24.03.2021 o 13:54, Pommier, Rex pisze:
Hi Carmen,
We use the quiesce parameter of the reset command as well. I'm thinking that's what
happened here as well, but we have no idea "whodunit". We had 1 operator on
duty that day and he was off doing something else when it happened so the command made it
into the system via an SDSF-like console session.
Rex
-----Original Message-----
From: IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> On
Behalf Of Carmen Vitullo
Sent: Wednesday, March 24, 2021 7:43 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [External] Re: z14 HMC log information
There is a queisce command to put a job to sleep, and I think that's what my
operator was thinking he was doing, for a runaway job we do use automation to
queisce a job and send us an alert.
from SDSF there's a quiesce line command RQ it translates to RESET
jobname,QUIESCE
Carmen Vitullo
-----Original Message-----
From: Radoslaw <r.skoru...@hotmail.com>
To: IBM-MAIN <IBM-MAIN@LISTSERV.UA.EDU>
Date: Wednesday, 24 March 2021 5:35 AM CDT
Subject: Re: z14 HMC log information
IMHO not really.
By freezing the system you disrupt all the online processing and you loose
capability to find out what's wrong.
I had to do with CPU hogging tasks and spool 100% filled by some ugly job. In
both cases the very first thing to fix it was to logon to the system.
--
Radoslaw Skorupka
(looking for new job)
Lodz, Poland
W dniu 24.03.2021 o 03:30, kekronbekron pisze:
It's useful when you want to say "STOP IT!" to run-away CPU or spool jobs.
Will buy you some time to investigate.
- KB
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, March 23, 2021 7:07 PM, Pommier, Rex <rpomm...@sfgmembers.com>
wrote:
Hey Carmen,
At least you had an operator tell you what they did. :-) Once we determined what had
happened (after a painful middle-of-the-day IPL) the only response I got back from
anybody was "I didn't do anything like that!" But now that I know what that
command does and how it works I doubt I'll ever forget this one.
Rex
-----Original Message-----
From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On
Behalf Of Carmen Vitullo
Sent: Tuesday, March 23, 2021 7:47 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: [External] Re: z14 HMC log information
WOW! this takes me back 20 years when an operator wanted to quiesce
a job and issues that command from the console he called my told me
what he did, I researched what that command does, seemed to me it
was like on the old system consoles O2 and O3 IIRC one was a PSW
stop and one a restart, I found the restart just for grins and it
worked, been a long time since I've hear tell someone had that same
experience great find
Carmen Vitullo
-----Original Message-----
From: Rex rpomm...@sfgmembers.com
To: IBM-MAIN IBM-MAIN@LISTSERV.UA.EDU
Date: Monday, 22 March 2021 5:18 PM CDT
Subject: Re: [External] Re: z14 HMC log information
OK, this is embarrassing. We found the problem - but don't know the
culprit. We discovered the quiesce command wasn't locked down and
somebody somewhere keyed it in. The reason we don't know who/where
is because our SDSF-like product had the capability to also do a
quiesce and since we didn't know that's what caused it, we IPLed the
LPAR. I did some research after the fact to determine just how
quiesce works (been working on these things for 30 years and never
had a reason to even play with quiesce). Now I know - and I know how
to make the machine go again, picking up where it stopped. PSW
Restart from the HMC doesn't do what I consider a "restart" but a
"take the brakes off and let it run". :-)
IBM hardware support found the wait state and informed us of it.
Thanks everybody for the bandwidth and suggestions.
Rex
-----Original Message-----
From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On
Behalf Of kekronbekron
Sent: Wednesday, March 17, 2021 8:50 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [External] Re: z14 HMC log information
Please do share when you find out!
I wonder if an LPAR with full BCPii authority over the box can silently
query/log information from the HMC for monitoring, i.e., actions occuring not
via BCPii itself, but just accessing HMC logs.
- KB
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, March 17, 2021 6:17 PM, Carmen Vitullo cvitu...@hughes.net wrote:
so the only other thing I can think of is if you are in a sysplex and your SFM
policy took the system out of the plex, XCF may have not seen a heartbeat in a
long enough time it partitioned the system from the plex, I've had this happen
before, you should see some IXC messages if this happened.
when this happened to me there was no indication anyone did
anything the system was just removed from the plex
Carmen Vitullo
-----Original Message-----
From: Rex rpomm...@sfgmembers.com
To: IBM-MAIN IBM-MAIN@LISTSERV.UA.EDU
Date: Tuesday, 16 March 2021 4:33 PM CDT
Subject: Re: z14 HMC log information Nothing out of the ordinary in
the HMC logs.
-----Original Message-----
From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On
Behalf Of Carmen Vitullo
Sent: Tuesday, March 16, 2021 1:10 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [External] Re: z14 HMC log information so it looks like
one place to check is from the HMC, /console actions/View control
tasks performed, this may get you what you need another place is
from console actions, View console events
Carmen Vitullo
-----Original Message-----
From: Rex rpomm...@sfgmembers.com
To: IBM-MAIN IBM-MAIN@LISTSERV.UA.EDU
Date: Tuesday, 16 March 2021 12:51 PM CDT
Subject: Re: z14 HMC log information I found the log - nothing in
it. Heading off to IBM.
Rex
-----Original Message-----
From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On
Behalf Of Pommier, Rex
Sent: Tuesday, March 16, 2021 12:24 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [External] z14 HMC log information Hi all, Probably a
simple question but I'm not having any luck looking for it so I'm asking here.
Is there a centralized place on a z14 HMC to show activity occurring, LPAR
activation/deactivation, pretty much anything that happens on an HMC or on a z. We just
took a hit where one of our LPARs and we're not seeing anything. No hardware messages, no
dumps, no nothing. The syslog on the LPAR shows everything running as normal then it just
stopped, 15 minutes later are the IPL starting messages. It almost looks like somebody
hit the "deactivate" button on the HMC so I'm looking for any log info from a
hardware point of view.
Thanks,
Rex
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
