I tried setting a 5 minute time limit on some low resource jobs, and one hour on high resource jobs, but my 5 minute jobs are still waiting behind the hourlong jobs.
Can you suggest some combination of time limits that would work here? On Fri, Nov 1, 2019 at 11:08 AM c b <breedthoughts....@gmail.com> wrote: > On my low resource jobs I'm setting the time to 1 hour, and on my large > ones I'm setting time=unlimited. > > Is the unlimited part the problem? I have that setting because in my > cluster there are some machines that come in and out during the day via > reservations, and I want to keep these larger jobs from running on those > machines. > > > > > > On Fri, Nov 1, 2019 at 10:56 AM Burian, John < > john.bur...@nationwidechildrens.org> wrote: > >> Are you setting realistic job run times (sbatch –t <time>)? >> >> >> >> Slurm won’t backfill low priority jobs (with low resource requirements) >> in front of a high priority job (blocked waiting on high resource >> requirements) if it thinks the low priority jobs will delay the eventual >> start of the high priority job. If all jobs are submitted with the same job >> run time, then Slurm will never backfill, because as far as Slurm knows, >> the low priority jobs will take longer to finish than just waiting for the >> current running jobs to finish. >> >> >> >> John >> >> >> >> >> >> *From: *slurm-users <slurm-users-boun...@lists.schedmd.com> on behalf of >> c b <breedthoughts....@gmail.com> >> *Reply-To: *Slurm User Community List <slurm-users@lists.schedmd.com> >> *Date: *Friday, November 1, 2019 at 10:30 AM >> *To: *"slurm-users@lists.schedmd.com" <slurm-users@lists.schedmd.com> >> *Subject: *[slurm-users] job priority keeping resources from being used? >> >> >> >> [WARNING: External Email - Use Caution] >> >> >> >> Hi, >> >> >> >> Apologies for the weird subject line...I don't know how else to describe >> what I'm seeing. >> >> >> >> Suppose my cluster has machines with 8 cores each. I have many large >> high priority jobs that each require 6 cores, so each machine in my cluster >> runs one of each of these jobs at a time. However, I also have lots of >> small jobs that each require one core, and these jobs have low priority so >> in my queue they are behind all my large jobs. >> >> >> >> In theory, these small jobs could slip in and run alongside the large >> jobs, but I'm not seeing that happen. So my machines have two cores >> sitting idle when they could be doing work. How do I configure slurm to >> run these jobs better? >> >> >> >> thanks for any help. >> >> >> >
