For this kind of short time jobs, maybe they are try to use GNU parallel. Ask a few compute nodes from job scheduler, use parallel to run many jobs at the same time. parallel will launch new jobs when there are CPU cores free.
Best, Shenglong > On Aug 19, 2016, at 5:47 AM, Gokcehan Kara <[email protected]> wrote: > > Hello, > > We have a small cluster consisting of ~40 machines in our university for > running MD simulations. One of our users is working on a project where she > needs to submit many jobs of about 5 min lengths. To my understanding, she is > doing some calculations in a script to find some parameters and submit 4 or 5 > of these jobs at the same time in the script. And then rinse and repeat. So > the problem is that she says after a while (~3000 jobs) jobs start to take > some time before scheduled. We thought it might be related to some slurm > settings about fairness or such but couldn't find anything about it. Is there > a way to disable this waiting time? > > Thanks in advance, > Gokcehan
