Pretty sure Slurm doesn’t support dynamic spawn - we don’t support it when direct launched using srun. It is only supported when launched by mpiexec
Might change in future releases, assuming Slurm has or adds the support > On Feb 26, 2015, at 5:26 AM, Lev Givon <l...@columbia.edu> wrote: > > I've been using OpenMPI 1.8.4 manually built on Ubuntu 14.04.2 against the PMI > libraries provided by the stock SLURM 2.6.5 Ubuntu packages. Although I am > able > to successfully run MPI jobs that use MPI_Comm_spawn via mpi4py 1.3.1 (also > manually built against OpenMPI 1.8.4) to dynamically create processes when I > launch those jobs via mpiexec directly, I can't seem to get SLURM to start > them > (I am able to use SLURM to successfully start jobs with a fixed number of > processes, however). For example, attempting to run a job that spawns more > than > one process with > > srun -n 1 python myprogram.py > > results in the following error: > > [huxley:24037] [[5176,1],0] ORTE_ERROR_LOG: Not available in file dpm_orte.c > at line 1100 > [huxley:24037] *** An error occurred in MPI_Comm_spawn > [huxley:24037] *** reported by process [339214337,0] > [huxley:24037] *** on communicator MPI_COMM_SELF > [huxley:24037] *** MPI_ERR_UNKNOWN: unknown error > [huxley:24037] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will > now abort, > [huxley:24037] *** and potentially your MPI job) > > Running the same program with > > mpiexec -np 1 python myprogram.py > > works properly. > > Has anyone successfully used SLURM (possibly a more recent version than 2.6.5) > to submit spawning OpenMPI jobs? If so, what might be causing the above error? > -- > Lev Givon > Bionet Group | Neurokernel Project > http://www.columbia.edu/~lev/ > http://lebedov.github.io/ > http://neurokernel.github.io/ > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/02/26403.php
