If you check the applicable code in src/slurmd/slurmstepd/task.c, TMPDIR is set
to "/tmp" if it's not already set in the job environment and then TMPDIR is
created if permissible. It's your responsibility to set TMPDIR -- e.g. we have
a plugin we wrote (autotmp) to set TMPDIR to per-job and per-step paths
according to the job id.
> On Nov 21, 2018, at 10:33 , Michael Gutteridge <michael.gutteri...@gmail.com>
> wrote:
>
>
> I don't think that's a bug. As far as I've ever known, TmpFS is only used to
> tell slurmd where to look for available space (reported as TmpDisk for the
> node). The manpage only indicates that, not any additional functionality.
> We set TMPDIR in a task prolog:
>
> #!/bin/bash
> echo "export TMPDIR=/loc/scratch/${SLURM_JOB_ID}"
> echo "export SCRATCH_LOCAL=/loc/scratch/${SLURM_JOB_ID}"
> echo "export SCRATCH=/net/scratch/${SLURM_JOB_ID}"
>
> - Michael
>
>
> On Wed, Nov 21, 2018 at 6:52 AM Shenglong Wang <s...@nyu.edu> wrote:
> We have TMPDIR setup inside prolog file. Hope users do not have absolute path
> /tmp inside their scripts.
>
> #!/bin/bash
>
> SLURM_BIN="/opt/slurm/bin"
>
> SLURM_job_tmp=/state/partition1/job-${SLURM_JOB_ID}
>
> mkdir -m 700 -p $SLURM_job_tmp
> chown $SLURM_JOB_USER $SLURM_job_tmp
>
> echo "export SLURM_JOBTMP=$SLURM_job_tmp"
> echo "export SLURM_JOB_TMP=$SLURM_job_tmp"
> echo "export SLURM_JOB_TMPDIR=$SLURM_job_tmp"
> echo "export TMPDIR=$SLURM_job_tmp”
>
> Best.
> Shenglong
>
>> On Nov 21, 2018, at 9:44 AM, Roger Moye <rm...@quantlab.com> wrote:
>>
>> We are having the exact same problem with $TMPDIR. I wonder if a bug has
>> crept in? I spoke to the SchedMD guys at SC18 last week and they were not
>> aware of a bug but since more than one person is having this difficulty
>> something must be wrong somewhere.
>>
>> -Roger
>>
>> From: slurm-users [mailto:slurm-users-boun...@lists.schedmd.com] On Behalf
>> Of Douglas Duckworth
>> Sent: Wednesday, November 21, 2018 7:38 AM
>> To: slurm-users@lists.schedmd.com
>> Subject: [slurm-users] $TMPDIR does not honor "TmpFS"
>>
>> Hi
>>
>> We are setting TmpFS=/scratchLocal in /etc/slurm/slurm.conf on nodes and
>> controller. However $TMPDIR value seems to be /tmp not /scratchLocal. As a
>> result users are writing to /tmp which we do not want.
>>
>> We are not setting $TMPDIR anywhere else such as /etc/profile.d nor do users
>> have it defined in their ~/.bashrc or ~/.bash_profile.
>>
>> We do not see any error messages anywhere which could indicate why the
>> default value of /tmp overrides our value of of TmpFS.
>>
>> As I understand prolog scripts can change this value though, if that's the
>> case, then what's the purpose of setting TmpFS in /etc/slurm/slurm.conf?
>>
>>
>> Thanks,
>>
>> Douglas Duckworth, MSc, LFCS
>> HPC System Administrator
>> Scientific Computing Unit
>> Weill Cornell Medicine
>> 1300 York Avenue
>> New York, NY 10065
>> E: d...@med.cornell.edu
>> O: 212-746-6305
>> F: 212-746-8690
>>
>> -----------------------------------------------------------------------------------
>>
>> The information in this communication and any attachment is confidential and
>> intended solely for the attention and use of the named addressee(s). All
>> information and opinions expressed herein are subject to change without
>> notice. This communication is not to be construed as an offer to sell or the
>> solicitation of an offer to buy any security. Any such offer or solicitation
>> can only be made by means of the delivery of a confidential private offering
>> memorandum (which should be carefully reviewed for a complete description of
>> investment strategies and risks). Any reliance one may place on the accuracy
>> or validity of this information is at their own risk. Past performance is
>> not necessarily indicative of the future results of an investment. All
>> figures are estimated and unaudited unless otherwise noted. If you are not
>> the intended recipient, or a person responsible for delivering this to the
>> intended recipient, you are not authorized to and must not disclose, copy,
>> distribute, or retain this message or any part of it. In this case, please
>> notify the sender immediately at 713-333-5440
>>
>
