Couple of things stand out. You should remove the following configure options:
--enable-mpi-thread-multiple
--with-threads
--enable-heterogeneous
Thread multiple is not ready yet in OMPI (and openib doesn't support threaded
operations anyway), and the support for hetero systems really isn't work
In fact, I also have tried to configure the OpenMPI with this:
./configure --with-sge --with-openib --enable-mpi-thread-multiple
--with-threads --with-hwloc --enable-heterogeneous --disable-vt
--enable-openib-dynamic-sl --prefix=/home/jescudero/opt/openmpi
And the problem is still present
El
--mca btl_openib_ib_path_record_**service_level 1 flag controls openib btl,
you need to remove --mca mtl mxm from command line.
Have you compiled OpenMPI with rhel6.4 inbox ofed driver? AFAIK, the MOFED
2.x does not have XRC and you mentioned "--enable-openib-connectx-xrc" flag
in configure.
O
I have a 16-node Mellanox cluster built with Mellanox ConnectX3 cards.
Recently I have updated the MLNX_OFED to the 2.0.5 version. The reason
of this e-mail to the OpenMPI users list is that I am not able to run
MPI applications using the service levels (SLs) feature of the OpenMPI
driver.
Cu
