Hi Paul, agreed. Actually, we even have
AccountingStoreFlags=job_comment,job_script,job_env set in slurm.conf. Although we didn't observe any issues so far in regular operation with about 10000 jobs per day, this did clearly affect database conversion time during major Slurm version updates. Therefore, as always, one must weigh the pros and cons. There is no such thing as a free lunch ... Best regards Jürgen * Paul Edmon via slurm-users <slurm-users@lists.schedmd.com> [240807 08:55]: > Warning on that one, it can eat up a ton of database space (depending on > size of environment, uniqueness of environment between jobs, and number of > jobs). We had it on and it nearly ran us out of space on our database host. > That said the data can be really useful depending on the situation. > > -Paul Edmon- > > On 8/7/2024 8:51 AM, Juergen Salk via slurm-users wrote: > > Hi Steffen, > > > > not sure if this is what you are looking for, but with > > `AccountingStoreFlags=job_env´ > > set in slurm.conf, the batch job environment will be stored in the > > accounting database and can later be retrieved with `sacct -j <jobid> > > --env-vars´ > > command. > > > > We find this quite useful for debugging purposes. > > > > Best regards > > Jürgen > > > > * Steffen Grunewald via slurm-users <slurm-users@lists.schedmd.com> [240807 > > 14:08]: > > > Hello everyone, > > > > > > I've grepped the manual pages and crawled the 'net, but couldn't find any > > > answer to the following problem: > > > > > > I can see that the ctld keeps a record of it below /var/spool/slurm - as > > > long as the job is running or waiting (and shown by "squeue") - and that > > > this stores the environment that contains SLURM_SUBMIT_HOST > > > - but this information seems to be lost when the job finishes. > > > > > > Is there a way to find out what the value of SLURM_SUBMIT_HOST was? > > > I'd be interested in a few more env variables, but this one should be > > > sufficient for a start... > > > > > > Is "sacct" just lacking a job field, or is this info indeed dropped and > > > not stored in the DB? > > > > > > > > > Thanks, > > > Steffen > > > > > > > > > -- > > > Steffen Grunewald, Cluster Administrator > > > Max Planck Institute for Gravitational Physics (Albert Einstein Institute) > > > Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany > > > ~~~ > > > Fon: +49-331-567 7274 > > > Mail: steffen.grunewald(at)aei.mpg.de > > > ~~~ > > > > > > -- > > > slurm-users mailing list -- slurm-users@lists.schedmd.com > > > To unsubscribe send an email to slurm-users-le...@lists.schedmd.com > > > > -- > slurm-users mailing list -- slurm-users@lists.schedmd.com > To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
smime.p7s
Description: S/MIME cryptographic signature
-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-le...@lists.schedmd.com
