Hi Hermann, I happened to read along this conversation and was just solving this issue today. I added this part to the epilog script to make it work:
# Add job report to stdout
StdOut=$(/usr/bin/scontrol show job=$SLURM_JOB_ID | /usr/bin/grep StdOut |
/usr/bin/xargs | /usr/bin/awk 'BEGIN { FS = "=" } ; { print $2 }')
NODELIST=($(/usr/bin/scontrol show hostnames))
# Only add to StdOut file if it exists and if we are the first node
if [ "$(/usr/bin/hostname -s)" = "${NODELIST[0]}" -a ! -z "${StdOut}" ]
then
echo "################################# JOB REPORT
##################################" >> $StdOut
/usr/bin/seff $SLURM_JOB_ID >> $StdOut
echo
"###############################################################################"
>> $StdOut
fi
Contrary to what it says in the slurm docs
https://slurm.schedmd.com/prolog_epilog.html
<https://slurm.schedmd.com/prolog_epilog.html> I was not able to use the env
var SLURM_JOB_STDOUT, so I had to fetch it via scontrol. In addition I had to
make sure it is only called by the „leading“ node as the epilog script will be
called by ALL nodes of a multinode job and they would all call seff and clutter
up the output. Last thing was to check if StdOut is not of length zero (i.e. it
exists). Interactive jobs would otherwise cause the node to drain.
Maybe this helps.
Kind regards
Sebastian
PS: goslmailer looks quite nice with its recommendations! Will definitely look
into it.
--
Westfälische Wilhelms-Universität (WWU) Münster
WWU IT
Sebastian Potthoff (eScience / HPC)
> Am 15.09.2022 um 18:07 schrieb Hermann Schwärzler
> <hermann.schwaerz...@uibk.ac.at>:
>
> Hi Ole,
>
> On 9/15/22 5:21 PM, Ole Holm Nielsen wrote:
>> On 15-09-2022 16:08, Hermann Schwärzler wrote:
>>> Just out of curiosity: how do you insert the output of seff into the
>>> out-file of a job?
>> Use the "smail" tool from the slurm-contribs RPM and set this in slurm.conf:
>> MailProg=/usr/bin/smail
>
> Maybe I am missing something but from what I can tell smail sends an email
> and does *not* change or append to the .out file of a job...
>
> Regards,
> Hermann
>
smime.p7s
Description: S/MIME cryptographic signature
