Thank you Michael/Willy/Jeffrey - these example scripts and the template
were very helpful.
On Wed, Jul 15, 2020 at 5:24 PM Jeffrey T Frey wrote:
> On our HPC systems we have a lot of users attempting to organize job
> arrays for varying purposes -- parameter scans, SSMD (Single-Script,
> Multip
On our HPC systems we have a lot of users attempting to organize job arrays for
varying purposes -- parameter scans, SSMD (Single-Script, Multiple Datasets).
We eventually wrote an abstract utility to try to help them with the process:
https://github.com/jtfrey/job-templating-tool
May be of
Michael answered your question but since I completed my response just as
it came through I guess I will still send it. The only thing I would
reiterate over Michael's response is you can set the slurm array id's as
desired so you don't necessarily have to do the subtract 1 add 1 to
determine array
If the 500 parameters happened to be filenames, you could do adapt like
(appropriated from somewhere else, but I can’t find the reference quickly:
=
#!/bin/bash
# get count of files in this directory
NUMFILES=$(ls -1 *.inp | wc -l)
# subtract 1 as we have to use zero-based indexing (first e
I'm trying to run an embarrassingly parallel experiment, with 500+ tasks
that all differ in one parameter. e.g.:
job 1 - script.py foo
job 2 - script.py bar
job 3 - script.py baz
and so on.
This seems like a case where having a slurm array hold all of these jobs
would help, so I could just submi
