[slurm-users] Inaccurate Preemption Notification?

2023-04-24 Thread Jason Simms
Hello all, A user received an email from Slurm that one of his jobs was preempted. Normally when a job is preempted, the logs will show something like this: [2023-03-30T08:19:16.535] [25538.batch] error: *** JOB 25538 ON node07 CANCELLED AT 2023-03-30T08:19:16 DUE TO PREEMPTION *** [2023-03-30T08

Re: [slurm-users] Terminating Jobs based on GrpTRESMins

2023-04-24 Thread Hoot Thompson
See below…... > On Apr 24, 2023, at 1:55 PM, Ole Holm Nielsen > wrote: > > On 24-04-2023 18:33, Hoot Thompson wrote: >> In my reading of the Slurm documentation, it seems that exceeding the limits >> set in GrpTRESMins should result in terminating a running job. However, in >> testing this, T

Re: [slurm-users] Terminating Jobs based on GrpTRESMins

2023-04-24 Thread Ole Holm Nielsen
On 24-04-2023 18:33, Hoot Thompson wrote: In my reading of the Slurm documentation, it seems that exceeding the limits set in GrpTRESMins should result in terminating a running job. However, in testing this, The ‘current value’ of the GrpTRESMins only updates upon job completion and is not upda

[slurm-users] Terminating Jobs based on GrpTRESMins

2023-04-24 Thread Hoot Thompson
In my reading of the Slurm documentation, it seems that exceeding the limits set in GrpTRESMins should result in terminating a running job. However, in testing this, The ‘current value’ of the GrpTRESMins only updates upon job completion and is not updated as the job progresses. Therefore jobs a

Re: [slurm-users] Migration of slurm communication network / Steps / how to

2023-04-24 Thread Ole Holm Nielsen
On 4/24/23 08:56, Purvesh Parmar wrote: Thank you.. will try this and get back. Any other step being missed here for migration? I don't know if any steps are missing, because I never tried moving a cluster like you want to do. /Ole On Mon, 24 Apr 2023 at 12:08, Ole Holm Nielsen