[slurm-users] Re: Slurm fails before nvidia-smi command

2024-07-29 Thread Aziz Ogutlu via slurm-users
Jeff *From:* Aziz Ogutlu via slurm-users *Sent:* Monday, July 29, 2024 3:23 AM *To:* slurm-us...@schedmd.com *Subject:* [slurm-users] Slurm fails before nvidia-smi command Hi there all, We have Dell server with 2 x Nvidia H100 and running slurm on it. After r

[slurm-users] Slurm fails before nvidia-smi command

2024-07-29 Thread Aziz Ogutlu via slurm-users
Hi there all, We have Dell server with 2 x Nvidia H100 and running slurm on it. After restart server if we do not write nvidia-smi command slurm fails. When we run nvidia-smi && systemctl restart slurmd && systemctl restart slurmctld , slurm queue begins. Do you have any idea about this error