[OMPI users] UCX and MPI_THREAD_MULTIPLE

2019-08-23 Thread Paul Edmon via users
I have a code using MPI_THREAD_MULTIPLE along with MPI-RMA that I'm using OpenMPI 4.0.1.  Since 4.0.1 requires UCX I have it installed with MT on (1.6.0 build).  The thing is that the code keeps stalling out when I go above a couple of nodes.  UCX is new to our environment as previously we have

Re: [OMPI users] UCX and MPI_THREAD_MULTIPLE

2019-08-23 Thread Jeffrey Layton via users
Adding the UCX list to this thread. On Fri, Aug 23, 2019 at 7:35 PM Paul Edmon via users < users@lists.open-mpi.org> wrote: > I have a code using MPI_THREAD_MULTIPLE along with MPI-RMA that I'm > using OpenMPI 4.0.1. Since 4.0.1 requires UCX I have it installed with > MT on (1.6.0 build). The

Re: [OMPI users] UCX and MPI_THREAD_MULTIPLE

2019-08-23 Thread Joshua Ladd via users
Paul, Can you provide a repro and command line, please. Also, what network hardware are you using? Josh On Fri, Aug 23, 2019 at 3:35 PM Paul Edmon via users < users@lists.open-mpi.org> wrote: > I have a code using MPI_THREAD_MULTIPLE along with MPI-RMA that I'm > using OpenMPI 4.0.1. Since 4.0

Re: [OMPI users] UCX and MPI_THREAD_MULTIPLE

2019-08-23 Thread Paul Edmon via users
Sure.  The code I'm using is the latest version of Wombat (https://bitbucket.org/pmendygral/wombat-public/wiki/Home , I'm using an unreleased updated version as I know the devs).  I'm using OMP_THREAD_NUM=12 and the command line is: mpirun -np 16 --hostfile hosts ./wombat Where the host file

Re: [OMPI users] UCX and MPI_THREAD_MULTIPLE

2019-08-23 Thread Paul Edmon via users
I forgot to include that we have not rebuilt this OpenMPI 4.0.1 against 1.6.0 of UCX but rather 1.5.1.  When we upgraded to 1.6.0 everything seemed to be working for OpenMPI when we swapped the UCX version with out recompiling (at least in normal rank level MPI as we had to do the upgrade to UC