We found with 1.8.[1,2] that is you compile with --with-mxm --with-cuda --disable-dlopen
OMPI will compile install and run, but if you run disabling mxm (say to debug
something)
mpirun --mca mtl ^mxm ./a.out
You will get a notice saying that you cannot have enable cuda with disable
dlopen, and the job dies.
Is there any reason this shouldn't be a configure time error? Rather than a
runtime error?
Note we have moved to --disable-mca-dso to get the result we want (less iops to
the server hosting the install)
Our full configure was:
PREFIX=/home/software/rhel6/openmpi-1.8.2/gcc-4.8.0
MXM=/opt/mellanox/mxm
FCA=/opt/mellanox/fca
KNEM=/opt/knem-1.1.1.90mlnx
CUDA=/home/software/rhel6/cuda/5.5/cuda
COMPILERS='CC=gcc CXX=g++ FC=gfortran'
./configure \
--prefix=$PREFIX \
--mandir=${PREFIX}/man \
--disable-dlopen \
--enable-shared \
--enable-java \
--enable-mpi-java \
--with-cuda=$CUDA \
--with-hwloc=internal \
--with-verbs \
--with-psm \
--with-tm=/usr/local/torque \
--with-fca=$FCA \
--with-mxm=$MXM \
--with-knem=$KNEM \
--with-io-romio-flags='--with-file-system=testfs+ufs+nfs+lustre' \
$COMPILERS
Brock Palen
www.umich.edu/~brockp
CAEN Advanced Computing
XSEDE Campus Champion
bro...@umich.edu
(734)936-1985
signature.asc
Description: Message signed with OpenPGP using GPGMail
