Re: [slurm-users] Two gpu types on one node: gres/gpu count reported lower than configured (1 < 5)

2023-10-17 Thread Gregor Hagelueken
Dear Feng, That worked! Thank you! Cheers Gregor Sent from my iPhone. > Am 16.10.2023 um 17:05 schrieb Feng Zhang : > > Try > > scontrol update NodeName=heimdall state=DOWN Reason="gpu issue" > > and then > > scontrol update NodeName=heimdall state=RESUME > > to see if it will work. Probab

[slurm-users] Do we have IRC (EoM)

2023-10-17 Thread Styrk, Daryl

Re: [slurm-users] Correct way to do logrotation

2023-10-17 Thread Timony, Mick
Schedmd has docs about how to do this at: https://slurm.schedmd.com/slurm.conf.html#SECTION_LOGGING Our config at HMS looks like this: /var/log/slurm/slurmctld.log { create 0640 slurm root daily dateext nocompress notifempty rotate 10 sharedscripts postrotate /bin/pkill -x

Re: [slurm-users] Correct way to do logrotation

2023-10-17 Thread Groner, Rob
Thanks for doing that, as I did not see this original message, and I also am having to look at configuring our log for rotation. We once accidentally turned on debug5 and didn't notice until other things started failing because the drive was full...from that ONE file. I did find this conversat

Re: [slurm-users] Correct way to do logrotation

2023-10-17 Thread Davide DelVento
I'd be interested in this too, and I'm reposting only because the message was flagged as both "dangerous email" and "spam", so people may not have seen it (hopefully my reply will not suffer the same downfall...) On Mon, Oct 16, 2023 at 3:26 AM Taras Shapovalov wrote: > Hello, > > In the past it

Re: [slurm-users] Site factor plugin example?

2023-10-17 Thread Loris Bennett
Christopher Samuel writes: > On 10/13/23 10:10, Angel de Vicente wrote: > >> But, in any case, I would still be interested in a site factor >> plugin example, because I might revisit this in the future. > > I don't know if you saw, but there is a skeleton example in the Slurm > sources: > > src/p