Re: [OMPI users] Help Getting Started with Open MPI and PMIx and UCX

2019-01-23 Thread Matt Thompson
gt; > > _MAC > > > > *From:* users [mailto:users-boun...@lists.open-mpi.org] *On Behalf Of *Matt > Thompson > *Sent:* Tuesday, January 22, 2019 6:04 AM > *To:* Open MPI Users > *Subject:* Re: [OMPI users] Help Getting Started with Open MPI and PMIx > and UCX > >

Re: [OMPI users] Help Getting Started with Open MPI and PMIx and UCX

2019-01-22 Thread Cabral, Matias A
: Open MPI Users Subject: Re: [OMPI users] Help Getting Started with Open MPI and PMIx and UCX Well, By turning off UCX compilation per Howard, things get a bit better in that something happens! It's not a good something, as it seems to die with an infiniband error. As this is an Omnipath s

Re: [OMPI users] Help Getting Started with Open MPI and PMIx and UCX

2019-01-22 Thread Matt Thompson
Well, By turning off UCX compilation per Howard, things get a bit better in that something happens! It's not a good something, as it seems to die with an infiniband error. As this is an Omnipath system, is OpenMPI perhaps seeing libverbs somewhere and compiling it in? To wit: (1006)(master) $ mpi

Re: [OMPI users] Help Getting Started with Open MPI and PMIx and UCX

2019-01-20 Thread Howard Pritchard
Hi Matt Definitely do not include the ucx option for an omnipath cluster. Actually if you accidentally installed ucx in it’s default location use on the system Switch to this config option —with-ucx=no Otherwise you will hit https://github.com/openucx/ucx/issues/750 Howard Gilles Gouaillard

Re: [OMPI users] Help Getting Started with Open MPI and PMIx and UCX

2019-01-19 Thread Gilles Gouaillardet
Matt, There are two ways of using PMIx - if you use mpirun, then the MPI app (e.g. the PMIx client) will talk to mpirun and orted daemons (e.g. the PMIx server) - if you use SLURM srun, then the MPI app will directly talk to the PMIx server provided by SLURM. (note you might have to srun --mpi=pm

Re: [OMPI users] Help Getting Started with Open MPI and PMIx and UCX

2019-01-18 Thread Cabral, Matias A
Getting Started with Open MPI and PMIx and UCX On Fri, Jan 18, 2019 at 1:13 PM Jeff Squyres (jsquyres) via users mailto:users@lists.open-mpi.org>> wrote: On Jan 18, 2019, at 12:43 PM, Matt Thompson mailto:fort...@gmail.com>> wrote: > > With some help, I managed to build an Open M

Re: [OMPI users] Help Getting Started with Open MPI and PMIx and UCX

2019-01-18 Thread Matt Thompson
On Fri, Jan 18, 2019 at 1:13 PM Jeff Squyres (jsquyres) via users < users@lists.open-mpi.org> wrote: > On Jan 18, 2019, at 12:43 PM, Matt Thompson wrote: > > > > With some help, I managed to build an Open MPI 4.0.0 with: > > We can discuss each of these params to let you know what they are. > > >

Re: [OMPI users] Help Getting Started with Open MPI and PMIx and UCX

2019-01-18 Thread Jeff Squyres (jsquyres) via users
On Jan 18, 2019, at 12:43 PM, Matt Thompson wrote: > > With some help, I managed to build an Open MPI 4.0.0 with: We can discuss each of these params to let you know what they are. > ./configure --disable-wrapper-rpath --disable-wrapper-runpath Did you have a reason for disabling these? They'

Re: [OMPI users] Help Getting Started with Open MPI and PMIx and UCX

2019-01-18 Thread Matt Thompson
All, With some help, I managed to build an Open MPI 4.0.0 with: ./configure --disable-wrapper-rpath --disable-wrapper-runpath --with-psm2 --with-slurm --enable-mpi1-compatibility --with-ucx --with-pmix=/usr/nlocal/pmix/2.1 --with-libevent=/usr CC=icc CXX=icpc FC=ifort The MPI 1 is because I need