Hi Alfredo?,
Beyond what is already suggested, I have used the following script to run a set
number of jobs simultaneously within a batch script.
for thing in ${arrayOfThings[@]}; do echo $thing; done | (
srun -J JobName xargs -I{} --max-procs ${SLURM_JOB_CPUS_PER_NODE} bash -c '{
someCommand -args -and -options {}
}'
)
Where {} is $thing
In operation
here?<https://github.com/shawarden/Fastq2VCF/blob/master/slurm-scripts/blockalign.sl#L215>
Cheers,
Sam
________________________________
Sam Hawarden
Assistant Research Fellow
Pathology Department
Dunedin School of Medicine
sam.hawarden(at)otago.ac.nz
________________________________
From: slurm-users <slurm-users-boun...@lists.schedmd.com> on behalf of Carlos
Fenoy <mini...@gmail.com>
Sent: Thursday, 20 December 2018 04:59
To: Slurm User Community List
Subject: Re: [slurm-users] requesting resources and afterwards launch an array
of calculations
Hi Alfredo,
You can have a look at using https://github.com/eth-cscs/GREASY . It was
developed before array-jobs were supported in slurm and it will do exactly what
you want.
Regards,
Carlos
On Wed, Dec 19, 2018 at 3:33 PM Alfredo Quevedo
<maquevedo....@gmail.com<mailto:maquevedo....@gmail.com>> wrote:
thank you Michael for the feedback, my scenario is the following: I want
to run a job array of (lets say) 30 jobs. So I setted the slurm input as
follows:
#SBATCH --array=1-104%30
#SBATCH --ntasks=1
however only 4 jobs within the array are launched at a time due to the
allowed max number of jobs as setted in the slurm configuration (4). As
a workaround to the issued, the sysadmin suggested me to request the
resources, and afterwards distribute the resources asigned into a
multiple set of single CPU task. I believe that with the solution you
mentioned only 30 (out of the 104) jobs will be finished?
thanks
Alfredo
El 19/12/2018 a las 11:15, Renfro, Michael escribió:
> Literal job arrays are built into Slurm:
> https://slurm.schedmd.com/job_array.html
>
> Alternatively, if you wanted to allocate a set of CPUs for a parallel task,
> and then run a set of single-CPU tasks in the same job, something like:
>
> #!/bin/bash
> #SBATCH --ntasks=30
> srun --ntasks=${SLURM_NTASKS} hostname
>
> is one way of doing it. If that's not what you're looking for, some other
> details would be needed.
>
--
--
Carles Fenoy
