I get it working with a slurm Prolog, but because Prolog runs on every job, it creates a TMPDIR every time it slurm runs a job on any node.
Ideally, I’d like to create the TMPDIR ONLY if the job requested –tmp. I’ve only perused the code a bit, to see how it works, but does this Spank plugin only create a TMPDIR with all jobs, or only if a job requests –tmp? Erik -- Erik Ellestad Wynton Cluster SysAdmin UCSF From: slurm-users <slurm-users-boun...@lists.schedmd.com> On Behalf Of Greg Wickham Sent: Monday, May 18, 2020 11:59 PM To: Slurm User Community List <slurm-users@lists.schedmd.com> Subject: Re: [slurm-users] Reset TMPDIR for All Jobs Hi Erik, We use a private fork of https://github.com/hpc2n/spank-private-tmp<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_hpc2n_spank-2Dprivate-2Dtmp&d=DwMGaQ&c=iORugZls2LlYyCAZRB3XLg&r=Ct3zEADMmPgyUYfpHDJQWaWsE9mNbEHEhGxpsYoThbE&m=z5SoPrb3-WhcufGXKtxYDuOeJKDLyoBXvnVUERecyh8&s=hD4WqyRTUkz00jF4NHFZwprPai2j3OaROq7PsOD1Vxg&e=> It has worked quite well for us - jobs (or steps) don’t share a /tmp and during the prolog all files created for the job/step are deleted. Users absolutely cannot see each others temporary files so there’s no issue even if they happen to have a hard corded path ie: /tmp/myfiles.txt -Greg On 12 May 2020, at 18:40, Ellestad, Erik <erik.elles...@ucsf.edu<mailto:erik.elles...@ucsf.edu>> wrote: I was wanted to set TMPDIR from /tmp to a per job directory I create in local /scratch/$SLURM_JOB_ID (for example) This bug suggests I should be able to do this in a task-prolog. https://bugs.schedmd.com/show_bug.cgi?id=2664<https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.schedmd.com_show-5Fbug.cgi-3Fid-3D2664&d=DwMGaQ&c=iORugZls2LlYyCAZRB3XLg&r=Ct3zEADMmPgyUYfpHDJQWaWsE9mNbEHEhGxpsYoThbE&m=z5SoPrb3-WhcufGXKtxYDuOeJKDLyoBXvnVUERecyh8&s=0beKr3hsB8aMTqZ0pXDEggBEIVjhxrvwXlJoFLmobi8&e=> However adding the following to task-prolog doesn’t seem to affect the variables the job script is running with. unset TMPDIR export TMPDIR=/scratch/$SLURM_JOB_ID It does work if it is done in the job script, rather than the task-prolog. Am I missing something? Erik -- Erik Ellestad Wynton Cluster SysAdmin UCSF
