Could also review the node log in /varlog/slurm/ . Often sinfo -lR will tell you the cause, fro example mem not matching the config.
Doug On Thu, May 25, 2023 at 5:32 AM Ole Holm Nielsen <ole.h.niel...@fysik.dtu.dk> wrote: > On 5/25/23 13:59, Roger Mason wrote: > > slurm 20.02.7 on FreeBSD. > > Uh, that's old! > > > I have a couple of nodes stuck in the drain state. I have tried > > > > scontrol update nodename=node012 state=down reason="stuck in drain state" > > scontrol update nodename=node012 state=resume > > > > without success. > > > > I then tried > > > > /usr/local/sbin/slurmctld -c > > scontrol update nodename=node012 state=idle > > > > also without success. > > > > Is there some other method I can use to get these nodes back up? > > What's the output of "scontrol show node node012"? > > /Ole > >
