Ralph,
Thank your very much for your response. I'll pass this along to my
users. Sounds lie we might need to do some testing of our own. We're
still using Slurm 15.08, but planning to upgrade to 17.11 soon, so it
sounds like we'll get some performance benefits from doing so.
Prentice
On 12/18/2017 08:12 PM, r...@open-mpi.org wrote:
We have had reports of applications running faster when executing under OMPI’s
mpiexec versus when started by srun. Reasons aren’t entirely clear, but are
likely related to differences in mapping/binding options (OMPI provides a very
large range compared to srun) and optimization flags provided by mpiexec that
are specific to OMPI.
OMPI uses PMIx for wireup support (starting with the v2.x series), which
provides a faster startup than other PMI implementations. However, that is also
available with Slurm starting with the 16.05 release, and some further
PMIx-based launch optimizations were recently added to the Slurm 17.11 release.
So I would expect that launch via srun with the latest Slurm release and PMIx
would be faster than mpiexec - though that still leaves the faster execution
reports to consider.
HTH
Ralph
On Dec 18, 2017, at 2:18 PM, Prentice Bisbal <pbis...@pppl.gov> wrote:
Greeting OpenMPI users and devs!
We use OpenMPI with Slurm as our scheduler, and a user has asked me this:
should they use mpiexec/mpirun or srun to start their MPI jobs through Slurm?
My inclination is to use mpiexec, since that is the only method that's
(somewhat) defined in the MPI standard and therefore the most portable, and the
examples in the OpenMPI FAQ use mpirun. However, the Slurm documentation on the
schedmd website say to use srun with the --mpi=pmi option. (See links below)
What are the pros/cons of using these two methods, other than the portability
issue I already mentioned? Does srun+pmi use a different method to wire up the
connections? Some things I read online seem to indicate that. If slurm was
built with PMI support, and OpenMPI was built with Slurm support, does it
really make any difference?
https://www.open-mpi.org/faq/?category=slurm
https://slurm.schedmd.com/mpi_guide.html#open_mpi
--
Prentice
_______________________________________________
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users
_______________________________________________
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users
_______________________________________________
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users
