I found it is a bug in slurm 17.11.7, if I run the same command in 17.02, it can be replaced, the below is the command run under slurm 17.02 [root@head ~]# export CUDA_VISIBLE_DEVICES=0,1 [root@head ~]# srun -N1 -n1 --nodelist=head --export=CUDA_VISIBLE_DEVICES=NoDevFiles,ALL env|grep CUDA CUDA_VISIBLE_DEVICES=NoDevFiles CUDA_HOME=/usr/local/cuda [root@head ~]#
From: Chaofeng Zhang Sent: Friday, August 31, 2018 12:13 AM To: Slurm User Community List <slurm-users@lists.schedmd.com> Subject: Whether I can replace value of the variable when use srun export CUDA_VISIBLE_DEVICES=0,1 srun -N1 -n1 --nodelist=head --export=CUDA_VISIBLE_DEVICES=NoDevFiles,ALL env|grep CUDA The srun result is CUDA_VISIBLE_DEVICES=0,1, how could I replace CUDA_VISIBLE_DEVICES with NoDevFiles. Thanks.
